<< News Scan | Main | News Scan >>

The Morphing Factoid

| 1 Comment
WaPo Fact Checker Glenn Kessler examines a claim made by Attorney General Eric Holder over four years ago.  "Disturbingly, intimate partner homicide is the leading cause of death for African American women ages 15 to 45."  That would indeed be disturbing if it were true.  It is not.  Not even close.

Why is The Fact Checker focusing on a statement made four years ago? This assertion by the attorney general is an interesting case of a game of telephone being played with a factoid, in which the original statistic has become lost from its moorings.
There was a BJS report in 1998 that discussed the issue but did not say what it has since been cited for.  An article in the American Journal of Public Health miscited the study as saying homicide (not necessarily intimate partner homicide) was the leading cause of death in the demographic group in question.  Then another study published in DoJ's National Institute of Justice Journal miscites it again with the even more stark, and even more wrong, statistic.
Eventually, the factoid ended up in the bloodstream of information about intimate partner homicide. For instance, a month before Holder gave his speech, the University of Minnesota Institute on Domestic Violence in the African American community displayed the fact on its Web site: "Intimate partner homicide is the leading cause of death for African-American women ages 15 to 45." The 2003 NIJ Journal article is listed as the source.
*                                   *                                 *

Perry [the person who called this to the Kessler's attention] called for Four Pinocchios, and if this request had been made shortly after Holder gave the speech, we might have been so inclined. (Note: The Fact Checker did not exist in 2009.) But regular readers know that we generally do not go far back in time to award Pinocchios, especially if the offending sentence has not been recently repeated.

The Justice Department should have been quicker to fix its Web site. But, given that officials have conceded the factoid is not valid, and plan to make that clear on Justice's Web site, we see no reason to award Pinocchios at this time. There's certainly a lesson here: When a fact sounds too startling to be true, always go back to the original source. Secondary sources can sometimes lead you astray.

Kessler is absolutely right about the need to go back to the original source.  Many times, in writing legal briefs, we have picked up a citation saying that a precedent says such-and-such but upon checking the original found out that it did not really say that.  The same holds true in empirical research.

Also, I can appreciate Kessler's reluctance to award retroactive Pinocchios.  I have been known to advocate limits on retroactivity myself on occasion.

As a corollary to the need to go back to the original source, we need to be extremely wary of factoids that "end[] up in the bloodstream" of public discourse on a subject.  The most notorious, of course, is the claim that 130+ former death row inmates have been "proved innocent" or "exonerated."  We have completely debunked that, see, e.g., this post, yet it continues to be repeated, and few in the press seem interested in checking it.

1 Comment

Another Woozle Exposed, but still not fully corrected.

The Woozle Effect (False Evidence By Citation) was coined by Beverly Hougton in 1979 (paper: "Review of research on women abuse." annual meeting of the American Society of Criminology, Philadelphia. Nov 1979)

Donald Dutton made a fascinating observation in his 2006 book "Rethinking Domestic Abuse":

"Woozles are usually not simply a matter of authentic misreporting. They also reveal a desire to read into the data an a priori position that is really not there, what Bacon calls "idols of the theatre". ... All the data reporting mistakes I have found in the literature, without exception, were made in the direction of supporting feminist preconceptions." - page 29.

Leave a comment

Monthly Archives