NOTE: Gallup has STOPPED publishing their Unemployment Numbers effective in July 2017. This includes U-3, Underemployment (U-6), and Good Jobs Index (Payroll to Population Rate).
From 2010-2017, the Gallup Survey people both generated numbers to help us understand the employment/unemployment situation. Unfortunately, they often presented a different picture from the numbers generated by the U.S. Bureau of Labor Statistics. Typically the BLS data presented a rosier picture than the independently surveyed Gallup numbers.
Often in the summer, the gap between the BLS and the Gallup closed. In July 2016, the BLS and the Gallup numbers came in identical. Early in 2017 the gap widened to 1.3% in April but has narrowed again to 0.6% in June.
For June 2017 Gallup said unadjusted U-3 unemployment was 5.1% down from 5.2% in May and 5.4% in April while the BLS said it was 4.5% up from 4.1% in May and April. So the spread is currently 0.6%.
Whose Unemployment Numbers are Right?
There has been some talk about “full employment” in the media of late and if we look at the Unemployment numbers created by the U.S. Bureau of Labor Statistics (BLS) we might get that impression. If we look at the Employment numbers rather than the Unemployment Rate we see a significant increase but it hasn’t been until recent months that the increase has actually outpaced the growth in the population.
We’ve looked at Employment vs. Unemployment on other pages to see how they compare and we’ve looked at U-6 (total labor force including those who’ve given up looking) vs. U-3 (those who are still actively looking). The U-3 unemployment rate is the commonly quoted one. But the one problem is that all that data comes from the government. If they are fudging the numbers how would we know? Unless as we’ve noted before there are inconsistencies between the Unemployment and Employment Charts. But we do have an alternative source of information.
BLS vs Gallup Unemployment Comparison
In an effort to determine the True Unemployment Rate the Gallup survey people began doing their own survey on unemployment rates back in 2010. So we can compare their results with the results the BLS publishes.
In this first chart, we have BLS U-3 Unemployment rates (both Seasonally Adjusted and Unadjusted) along with the Gallup Unadjusted Unemployment rate. In this series, Gallup is using similar criteria to the U-3 so we can compare apples to apples. Note that in a few months in Summer 2013 and the Summer of 2014 the red line converged with the green line but mostly beginning in 2013 the BLS has been severely low-balling Unemployment numbers compared to the independent Gallup numbers.
In a perfect world with unbiased information, the two lines should be identical. In the real world, you might expect some minor variations but they should track closely. Lately, however, they have diverged drastically once again. So for most of 2015 and through the first half of 2016 , there was a major divergence between what the BLS was saying and what Gallup was saying.
Click Chart for larger image
In the beginning of this jumble, it is hard to determine whether the BLS numbers are providing the True Unemployment Rate or not. We can see that the Gallup numbers start out higher at the beginning and are also higher frequently throughout but there are also a few points where they are actually lower so it is hard to determine whether the BLS is fudging or not. But after 2013 the BLS numbers appear to get further and further from the Gallup numbers.
Some people have questioned why I consider the Gallup numbers “reality” and not the other way around.
For Calculating Unemployment the BLS says they interview “60,000 different households statistically calculated to represent the entire country.” However, they actually only contact about 15,000 of these households in any given month and then using statistical modeling they estimate the U.S. unemployment rate from this data sample. “The households in the pool are rotated to limit the burden on any specific family.”
Gallup on the other hand, creates weekly results which reflect “30-day rolling averages ending each Sunday, based on telephone interviews with approximately 30,000 adults.” So Gallup numbers are more current, have a larger sample size and lack a motivation for bias. In fact, since they sell their data, their motivation is to provide the most accurate data possible to their customers.
According to Shadowstats, the government is really underestimating unemployment by even more than our numbers suggest since ” long-term discouraged workers were defined out of official existence in 1994.” The new U-6 numbers only include short-term discouraged workers.
Next, let’s look at only the Unadjusted Unemployment Rate. I always prefer to see the data before the BLS “adjusts” it for “seasonal” reasons. So we will look at the BLS and the Gallup Unadjusted numbers for Unemployment.
True UnAdjusted Unemployment Numbers according to BLS and Gallup
Click Chart for larger image
In this chart, we can see that the Gallup numbers are generally higher than the BLS numbers but in the beginning, they were occasionally lower as well, so it was difficult to tell if the data is significantly different. They appear to track pretty well until July of 2013 when they start diverging drastically. The BLS numbers seem to take a break from reality as they continue to fall while Gallup numbers actually move higher. But then in July and August of 2014, the numbers become very close… as they should be. But reality doesn’t last long and in September the BLS got back to their old tricks and went down sharply while Gallup rose and went in the opposite direction. By January 2015 the gap returned to one full percentage point. In July of 2016 the BLS numbers and the Gallup numbers converged once again. But they diverged in March of 2017.
Let’s look at the actual data.
| BLS Unadj. | Gallup Unadj. | Diff. | Gallup Higher | BLS Higher | |
| Jan-2010 | 10.6% | 10.7% | 0.1% | 0.1% | |
| Feb-2010 | 10.4% | 10.8% | 0.4% | 0.4% | |
| Mar-2010 | 10.2% | 10.4% | 0.2% | 0.2% | |
| Apr-2010 | 9.5% | 10.1% | 0.6% | 0.6% | |
| May-2010 | 9.3% | 9.5% | 0.2% | 0.2% | |
| Jun-2010 | 9.6% | 9.2% | -0.4% | -0.4% | |
| Jul-2010 | 9.7% | 9.0% | -0.7% | -0.7% | |
| Aug-2010 | 9.5% | 9.2% | -0.3% | -0.3% | |
| Sep-2010 | 9.2% | 9.6% | 0.4% | 0.4% | |
| Oct-2010 | 9.0% | 9.8% | 0.8% | 0.8% | |
| Nov-2010 | 9.3% | 9.2% | -0.1% | -0.1% | |
| Dec-2010 | 9.1% | 9.2% | 0.1% | 0.1% | |
| Jan-2011 | 9.8% | 9.6% | -0.2% | -0.2% | |
| Feb-2011 | 9.5% | 10.1% | 0.6% | 0.6% | |
| Mar-2011 | 9.2% | 10.1% | 0.9% | 0.9% | |
| Apr-2011 | 8.7% | 9.6% | 0.9% | 0.9% | |
| May-2011 | 8.7% | 9.3% | 0.6% | 0.6% | |
| Jun-2011 | 9.3% | 8.8% | -0.5% | -0.5% | |
| Jul-2011 | 9.3% | 9.0% | -0.3% | -0.3% | |
| Aug-2011 | 9.1% | 9.0% | -0.1% | -0.1% | |
| Sep-2011 | 8.8% | 8.8% | 0.0% | — | — |
| Oct-2011 | 8.5% | 8.4% | -0.1% | -0.1% | |
| Nov-2011 | 8.2% | 8.5% | 0.3% | 0.3% | |
| Dec-2011 | 8.3% | 8.6% | 0.3% | 0.3% | |
| Jan-2012 | 8.8% | 8.4% | -0.4% | -0.4% | |
| Feb-2012 | 8.7% | 9.0% | 0.3% | 0.3% | |
| Mar-2012 | 8.4% | 8.8% | 0.4% | 0.4% | |
| Apr-2012 | 7.7% | 8.3% | 0.6% | 0.6% | |
| May-2012 | 7.9% | 8.0% | 0.1% | 0.1% | |
| June-2012 | 8.4% | 8.0% | -0.4% | -0.4% | |
| Aug-2012 | 8.6% | 8.2% | -0.4% | -0.4% | |
| Sept-2012 | 7.6% | 7.9% | 0.3% | ||
| Oct-2012 | 7.5% | 7.0% | -0.5% | -0.5% | |
| Nov-2012 | 7.4% | 7.8% | 0.4% | 0.4% | |
| Dec-2012 | 7.6% | 7.7% | 0.1% | 0.1% | |
| Jan-2013 | 8.5% | 7.8% | -0.7% | -0.7% | |
| Feb-2013 | 8.1% | 8.0% | -0.1% | -0.1% | |
| Mar-2013 | 7.6% | 8.0% | 0.4% | 0.4% | |
| Apr-2013 | 7.1% | 7.4% | 0.3% | 0.3% | |
| May-2013 | 7.3% | 7.9% | 0.6% | 0.6% | |
| June-2013 | 7.8% | 7.9% | 0.1% | 0.1% | |
| July-2013 | 7.7% | 7.8% | 0.1% | 0.1% | |
| Aug-2013 | 7.3% | 8.7% | 1.4% | 1.4% | |
| Sep-2013 | 7.0% | 7.7% | 0.7% | 0.7% | |
| Oct-2013 | 7.0% | 7.3% | 0.3% | 0.3% | |
| Nov-2013 | 6.6% | 8.2% | 1.6% | 1.6% | |
| Dec-2013 | 6.5% | 7.4% | 0.9% | 0.9% | |
| Jan-2014 | 7.0% | 8.6% | 1.6% | 1.6% | |
| Feb-2014 | 7.0% | 8.0% | 1.0% | 1.0% | |
| Mar-2014 | 6.8% | 7.5% | 0.7% | 0.7% | |
| Apr-2014 | 5.9% | 7.1% | 1.2% | 1.2% | |
| May-2014 | 6.1% | 7.0% | 0.9% | 0.9% | |
| Jun-2014 | 6.3% | 6.8% | 0.5% | 0.5% | |
| Jul-2014 | 6.5% | 6.4% | 0.1% | -0.1% | |
| Aug-2014 | 6.3% | 6.3% | 0% | — | — |
| Sep-2014 | 5.7% | 6.6% | 0.9% | 0.9% | |
| Oct-2014 | 5.5% | 6.2% | 0.7% | 0.7% | |
| Nov-2014 | 5.5% | 6.2% | 0.7% | 0.7% | |
| Dec- 2014 | 5.4% | 5.8% | 0.4% | 0.4% | |
| Jan-2015 | 6.1% | 7.1% | 1.0% | 1.0% | |
| Feb-2015 | 5.8% | 6.7% | 0.9% | 0.9% | |
| Mar-2015 | 5.6% | 6.5% | 0.9% | 0.9% | |
| Apr-2015 | 5.1% | 5.9% | 0.8% | 0.8% | |
| May-2015 | 5.3% | 6.2% | 0.9% | 0.9% | |
| Jun-2015 | 5.5% | 5.9% | 0.4% | 0.4% | |
| Jul-2015 | 5.6% | 6.2% | 0.6% | 0.6% | |
| Aug-2015 | 5.2% | 6.3% | 1.1% | 1.1% | |
| Sep-2015 | 4.9% | 6.3% | 1.4% | 1.4% | |
| Oct-2015 | 4.8% | 5.6% | 0.8% | 0.8% | |
| Nov-2015 | 4.8% | 5.6% | 0.8% | 0.8% | |
| Dec-2015 | 4.8% | 5.6% | 0.8% | 0.8% | |
| Jan-2016 | 5.3% | 5.5% | 0.2% | 0.2% | |
| Feb-2016 | 5.2% | 6.1% | 0.9% | 0.9% | |
| Mar-2016 | 5.1% | 6.1% | 1.0% | 1.0% | |
| Apr-2016 | 4.7% | 5.2% | 0.5% | 0.5% | |
| May-2016 | 4.5% | 5.5% | 1.0% | 1.0% | |
| Jun-2016 | 5.1% | 5.4% | 0.3% | 0.3% | |
| Jul-2016 | 5.1% | 5.1% | 0.0% | — | — |
| Aug-2016 | 5.0% | 5.4% | 0.4% | 0.4% | |
| Sep-2016 | 4.8% | 5.3% | 0.5% | 0.5% | |
| Oct-2016 | 4.7% | 5.1% | 0.4% | 0.4% | |
| Nov-2016 | 4.4% | 4.9% | 0.5% | 0.5% | |
| Dec-2016 | 4.5% | 5.1% | 0.6% | 0.6% | |
| Jan-2017 | 5.1% | 5.8% | 0.7% | 0.7% | |
| Feb-2017 | 4.9% | 5.5% | 0.6% | ||
| Mar-2017 | 4.6% | 5.7% | 1.1% | 1.1% | |
| Apr-2017 | 4.1% | 5.4% | 1.3% | 1.3% | |
| May-2017 | 4.1% | 5.2% | 1.1% | 1.1% | |
| Jun-2017 | 4.5% | 5.1% | 0.6% | 0.6% | |
| BLS Unadj. | Gallup Unadj. | Sum 40.3% | Ave. 0.6529% |
Ave. -0.318% |
In the fourth column marked “Difference” I’ve subtracted the BLS number from the Gallup number. This will result in a positive number if the Gallup number is higher and a negative number if the BLS number is higher. Theoretically, if the data collection methods are equal and the difference in the numbers is just based on random gathering differences, the Gallup number should be higher 50% of the time and the BLS number should be higher 50% of the time. Also, the amount of difference between the numbers should be equal.
So What are the Results?
First of all, we find that out of 90 data pairs the BLS number was higher only 17 times and the Gallup number was higher much more often at 70 times and only three times were the results the same. This definitely sounds like the BLS numbers are lower for some reason other than random chance. Next, we look at the average variation and we see that when the BLS is higher the average variation from the Gallup numbers is only 0.318% but when Gallup is higher the average difference is 0.6529%.
So not only do the Gallup numbers come out higher more often, the amount of difference is higher as well. From the table, we can see that when you add up all the negatives with all the positives the difference is 40.3 percentage points. I’ve been accused of including this “nonsense number” even though it doesn’t really mean anything. But if the methods were equivalent you would expect the positives to cancel out the negatives and the total would be zero. So the fact that this number is “non-zero” does indicate that this is an ongoing situation with the BLS continually underestimating the Unemployment rate compared to the Gallup numbers.
There is one major difference in the calculation of the BLS numbers and the Gallup numbers and that is the age Gallup considers employment for those 18 and up while BLS considers those 16 and up. This could account for the difference in the calculations except that the unemployment rate among teenagers is way above that of the general populace so rather than explain it, it actually makes the BLS numbers even further off base.
The True Unemployment Rate– Conclusion
Although up until recently the difference wasn’t massive it does appear that the BLS data is biased to the low side compared to the independently surveyed Gallup numbers. The average amount the BLS numbers come out below the Gallup numbers is roughly 0.6373% and 77% of the time the BLS numbers come out lower so in other words if the BLS says the unemployment rate is 5.0% approximately 75% of the time Gallup would say the True Unemployment Rate was really 5.64%. But on several occasions recently the difference was significantly more than 1%.
| Date | Difference |
| August 2013 | 1.4% |
| November 2013 | 1.6% |
| January 2014 | 1.6% |
| April 2014 | 1.2% |
| August 2015 | 1.1% |
| September 2015 | 1.4% |
| March 2016 | 1.0% |
| May 2016 | 1.0% |
| March 2017 | 1.1% |
| April 2017 | 1.3% |
But this does not take into consideration the other major problem that most people cite when they are concerned about the true unemployment rate and that is all the people who have stopped looking for a job. For more information on the people who have stopped looking, you need to look at the U-6 unemployment rate. See: What is U-6 Unemployment? Also recently the FED and various economists are saying that we are at or near “full-employment” see Is the U.S. Really at “Full Employment”?
There is also some evidence that a factor like Obamacare is causing a shift in the number of part-time employees (reducing the number of hours worked per employee) so the number of part-time workers necessary is increasing. See Unemployment, Part-time Workers and Obamacare, See also Labor Force Participation Rate.
For more information on the true unemployment rate see Employment vs. UnEmployment.
See Also: Job Growth Stalls, Labor Participation at 38-Year Low
Unemployment and Employment Charts
Source: US Bureau of Labor Statistics and Gallup Pollsters.
Does the Gallup Poll Provide the True Unemployment Rate?
At least twice you say, “The BLS numbers seem to take a break from reality…” where what is really happening is that the Gallup numbers and the BLS numbers are diverging. You seem to equate “reality” with the Gallup results. Why would you do that? Do you have some insight you have not shared with the reader that convincingly concludes that Gallup is always right? And that their data variance is lower than BLS? It looks to me that their variance month-to-month is much higher than BLS, especially in the periods where the results diverge. That indicates to me a process out of control.
Hal,
You are correct my personal bias is to accept the independently derived Gallup numbers over the Government sanctioned numbers for a couple of reasons. First is the data collection method. Gallup numbers are collected consistently over the month and updated more frequently resulting in a “leading” effect as the month nears the end. And secondly just like I don’t trust Pharmaceutical companies to provide fair and unbiased test results when they have an economic incentive to get certain results, the government has an incentive to underestimate unemployment (overestimate the number of people happily employed). Gallup on the other hand has an economic incentive to get the numbers right. I find it amazing that people tend to distrust government numbers when it comes to inflation (where they can observe the results personally) but tend to trust the unemployment numbers when the results are more widely disbursed and more difficult to discern personally.
I a, guessing that you aren’t a statistician because this analysis is comical. The cumulative difference of 31% is a meaningless pseudo statistic. Time series analysis might be a more accurate tool it looks as if the Gallup numbers are lagging the BLS numbers and therefore are the less accurate ones. For example, on January 2014 the Gallup number is 160 basis points higher, but by Aug 2014 they are identical. Why? Because the Gallup number dropped dramatically while the BLS number declined modestly. So the BLS number was more accurate. Every time the Gallup number is 140-160 basis points higher, and en converges a few months later, it always converges because the Gallup number drops more dramatically. If you did a time series analysis, this would become more obvious.
Actually Deborah, if you knew as much as you think you do you would realize that the “Pseudo statistic” is not meaningless at all. Although the actual quantity is insignificant, the fact that it is “non-zero” means a lot. It indicates that the Gallup number is consistently higher than the BLS number (or that the BLS is consistently underestimating unemployment.)
If the variations between the two were truly based on random gathering differences the “convergences” that you talk about should not be convergences at all but should actually be periods where the Gallup numbers were lower than the BLS numbers. Thus converting that “pseudo statistic” you so easily disregard into a zero. Plus since Gallup bases its numbers on a 30-day rolling average, the numbers are actually more up to date than the BLS monthly numbers. If the BLS numbers were more current your argument might have some validity but in a down-trending environment a rapid decline indicates that the Gallup numbers are simply leading the BLS numbers.
You seem to assume that it’s the BLS numbers that are “biased,” but you never justify this assumption. How do you know Gallup isn’t biased? How do we know YOU aren’t biased?
Good Question Ken. If we have two sets of numbers that supposedly measure the same thing and they provide two different sets of answers there are a few obvious conclusions. The first is that due to sampling or other random statistical reasons they aren’t really significantly different. But we found that the Gallup numbers are consistently significantly higher so we can eliminate this as a possibility. So there is obviously some sort of bias involved. The key question is whose numbers are biased?
Am I biased? Well maybe. Have I cherry picked data? Have I misrepresented any data-set? In this case, whether I’m biased or not really doesn’t matter since I am reporting their entire numbers and you can easily check the original sources against this article.
So that leaves it up to you to decide whose numbers are biased. The logical way to do that is to ask yourself who has the most to gain by fudging the numbers… a company whose reputation and revenue is based on the accuracy of their numbers and polls or the government whose political fortunes can be enhanced by rosy numbers.
Next if you look at our other articles you can see that we have also compared the BLS’ own numbers against themselves in our Employment vs Unemployment Chart and shown that they don’t add up. Thus providing additional evidence that it is the BLS numbers that are suspect. Plus we have provided Shadowstats corroborating evidence that it is the BLS that is doing the fudging.
Most people have no problem accepting that the inflation numbers provided by the BLS are biased since they see the results every week at the store and the government benefits financially by underestimating inflation but for some reason these same people trust the unemployment numbers. Why is that?
The gov’t bureaus always slant. A close acquaintance of mine (considered one of the top 10 economists in the US, and #1 in his specific field), says every incoming administration can cherry pick data points and assemblage to make their 1st year look 5% better than the previous year, even if nothing changes. If Gallup has been doing this analysis for a longer period than you have covered, would be interested in seeing comparisons going back to 2000. This would illustrate if the separation between Gallup and gov’t figures is based on a specific regime, or that divergence is the way it is always skewed, no matter who is in power. Would in fact be interesting to see if one party plays fast and loose with numbers more than another on a consistent basis, and you have perhaps found one way to ascertain that.
Unfortunately, Gallup just started tracking it so we have all the data they have collected. But way back I was at a dinner with James Dale Davidson (who at the time was the President of the American Taxpayers Union) and he told a story about meeting with Bill Clinton shortly after he was elected and before he took office. Davidson said Clinton asked him if he could think of a way that the BLS numbers could be fudged. Davidson remarked that he was amazed that Clinton hadn’t even taken office yet and was already thinking of ways to rig the system.
John,
Unfortunately, Gallup numbers only go back to 2010.
It is very difficult to ascertain the exact unemployment rate by just relying on U.S. Bureau of Labor Statistics (BLS) report. The uncertainties in right unemployment and employment figure have led the independent Gallup survey people to find out the True Unemployment Rate. The result shows that average Gallup numbers are 1% higher than BLS report, which is to the low side. Therefore, it is always wise to compare both Gallup survey and BLS report to determine true unemployment rate across the country.
I think people these days rely too much on “facts and figures” to guide their lives. Fact is that your personal situation and employment chances depend much more on the reality of your town/workplace and much less on what the national statistics say or even what the unemployment is within your state….
Johnny,
You are absolutely right. If you are without a job it is much easier to find a job in a state with 5% unemployment than in a state with 10%. But once you have a job, if you get your work done faster so you can help others it will put you in the top 25% of job performers so they would have to lay-off 3/4 of the workers before your job was in jeopardy. Thus your individual likelihood of getting laid-off is virtually non-existent and the likelihood of getting a promotion is much greater. So doing a good job goes a long way toward job security.
I don’t think the U.S. government would intentionally “fudge” the unemployment numbers, although certain local municipalities might provide the U.S. gov’t with funny numbers.
I wish that were true. I also wish NSA wasn’t spying on Journalists, the IRS wasn’t targeting certain groups, and the data presented by Snowden didn’t exist. But unfortunately if we take off the rose colored glasses and look at the actual data presented in this and other articles and compare them with other datasets from both the BLS and other sources we find that the BLS data looks awful fishy.