Cherry-picking data is often times (and somewhat inexplicably) not even classified as a fallacy. Thus for example, my favorite “go to” source on all things fallacious, the Fallacy Files, does not list it anywhere in its otherwise quite comprehensive encyclopedia. (Gary Curtis, over at Fallacy Files, and I exchanged some cheerful emails, and the simple answer as of this writing is that it simply hadn’t occurred to Gary to add an entry on the subject. He hopes to post something on his weblog in the not too distant future, and once it is up I’ll link to it HERE. [UPDATE: Gary Curtis has now posted his comments at Fallacy FIles.])
There are reasons why cherry-picking data might not properly qualify as a fallacy. For one thing, it is, in many cases, less of an error than it is a deliberate attempt to misrepresent and deceive. However, such maneuvers (which, in the technical literature, are sometimes known as “lying”) are possible in many other well recognized fallacies. For example, the straw man argument can as easily be a product of malevolent misdirection as of honest, but ill-considered error. The same is true of various statistical errors such as the Texas Sharpshooter Fallacy (seeing a correlation where there is none) or other probabilistic fallacies. Human beings have absolutely terrible intuitions when it comes to probability and statistics, and only a vanishingly small group of them have ever taken the trouble to learn even the most superficial gloss of the subjects. So disingenuous con artists can easily fool large numbers of people by tossing out impressive looking numbers or dramatic sounding “facts”. Yet these things are also considered fallacies, even though they they are open to, and even invite, calculated misuse. So this is not an adequate argument against including cherry-picking amongst the logical fallacies.
There are psychological reasons why one might engage in cherry-picking without being calculatedly dishonest. The most important of these is likely to be the well-known habit of “confirmation bias.” We are all subject to various cognitive and emotional biases that can effect how we perceive and evaluate information. When not guarded against, these can lead us to prejudicially view evidence in a manner that “confirms” our own preconceptions, while ignoring evidence that mitigates against those notions. It is for reasons such as confirmation bias that we cannot presume to judge the underlying psychological reasons which might lead a person to cherry-pick data in a biased and/or non-representative fashion.
Some examples might be useful here. After I’ve gone through a few examples, I’ll finish on some suggestions regarding the taxonomical placement of cherry-picking within the broader analysis of informal fallacies. I’ll start the examples with very broad strokes.
One of the commonest forms of (gross) cherry-picking that we find, especially in social media, relates to the denial of Anthropogenic Global Warming (AGW). Study after study is produced showing that the global temperatures are, as a matter of statistical fact, warming. Yet, time after time, the initial response is, “well, it was cool here today.” (Or “this month,” “or this season” … ) This is hands down one of the most singularly grotesque piece of vacuous twaddle that anyone could ever imagine producing. Does it really need to be said – again! – that there is a reason why it is called GLOBAL warming, and not “your back yard” warming? Well, evidently the answer to this question is, “yes,” because this generic response to these meticulous scientific studies is constantly tossed about, as though the mere repetition of such nonsense was sufficient to grant these responses even the abstract possibility of cogency. We see such idiocy in the highest chambers of Federal office, where Senator Inhofe from Oklahoma brought a snow ball to the floor of the Senate and declared this evidence that AGW was a hoax.
This is cherry-picking in its most galactically out-of-control form. A single data point is chosen (a cool day here, now; a snow ball on the Senate floor) and held up as a conclusive refutation to the staggering mountains of evidence, derived through numerous independent lines of inquiry, that have established the reality of AGW. A more careful – likely enough, calculatedly deceitful – act of cherry-picking was the case of the illegally obtained emails from the Hadley Climate Center in East Anglia University. Words and phrases from those emails were cherry-picked out of context, and then presented as though they were evidence of an ongoing conspiracy to suppress “The Truth.” The histrionics that denialists manufactured around these emails have been repeatedly demonstrated to be total bunk. But the damage was done. The cherry-picked statements were sufficient to reinforce the confirmation bias of the credulous and the ideological in their conviction that AGW was just a grand fraud to undermine capitalism and democracy. It has taken years to regain any momentum in the arena of policy and public will to implement any meaningful action on climate change.
Now, clearly cherry-picking is not something that is limited to just and only the science of AGW. Indeed, any situation that involves a large number of data-points in complex (and, frequently enough, statistical) relational structure, will be open to having those data-points cherry-picked. Thus, for example, Fox News was recently accused of cherry-picking data regarding the increase of the minimum wage in the Seattle area. The network presented its information in a way that seemed to indicate that the increase in the minimum wage was leading to a decline in business and employment in the restaurant industry. However, the claim as presented was faulty on multiple levels. For one thing, the decline was less than 1% of total employment in this particular business sector, and not so substantial that it would stand out from ordinary business cycles in what is, after all, a highly volatile industry. For another, the decline was only measured in the city of Seattle itself, and not in the greater Seattle-Tacoma metro area, where the wage increase is occurring (hence, the decline was chosen from a non-representative sample). Thirdly, overall employment throughout that greater metro area continues to rise. Moreover, other studies indicate no connection between this slight drop in the restaurant industry and the wage increase – indeed, many restaurateurs indicate that not only are they unconcerned about the increase, but they are looking to expand their businesses.
(As a side note, we might add that here we see the tendency for informal fallacies to come in clusters or clumps. By asserting a causal connection between the slight decline in restaurant employment and the wage increase, Fox may also be guilty of committing a post hoc, ergo propter hoc fallacy.)
How then might we classify cherry-picking within the broader taxonomy of fallacies? As noted above, there seem to be connections between cherry-picking and various statistical errors as well as confirmation bias. But these connections do not seem to be such that one might subsume one of the related items under the heading of the other as a specialized case. A more appropriate move would be classify cherry-picking under the very general heading of the ignoratio elenchi, which is to say, the red herring fallacy. A red herring is committed when the premises do not logically lead to the conclusion that is presented, yet the fault is not traceable to an exclusively linguistic problem such as ambiguity. In the case of cherry-picked data, the information will typically be clear enough. The problem, rather, is that the data is not genuinely representative of some relational whole (where, again, “representative” need not be specifically or exclusively statistical in nature.)
Two last points might be made here. First, this is, perhaps, not as satisfying a classification as one might wish for, as the red herring embraces a very large family of logical errors. The second is that I may be overly delicate here in separating cherry-picking from explicitly statistical errors. Perhaps, with further consideration, it might be argued that there simply are no genuinely non-statistical acts of cherry-picking. I am not invested in the answer to this question one way or the other, and so have elected to take the broadest perspective possible.