"We have a new study on car accidents. We have no data on whether the driver was drunk, how fast he was going, or what the weather was, but our study shows...." Need we read any further? It is highly likely that this hypothetical study does not show diddly squat. If you have no data on the primary causal factors, coming up with anything useful on the factors you did study is unlikely.
Yet that is exactly the situation with a new study on race and executions getting much attention today. In the Newsweek story noted in today's News Scan, interviewer Eve Conant asks lead author David Jacobs about the nonracial factors that might affect who is executed:
[Conant]: Or the nature of the crime? Or was it simply race?
[Jacobs]: We don't have much data on the nature of the crime.
But Supreme Court regulations require a state to come up with aggravating and mitigating factors for capital cases. Aggravating factors might include, say, the killing of a child or torturing a victim. Mitigating factors might include the age of the offender or their childhood experience, whether they were abused, etc.Yes indeed, the Supreme Court does require explicit consideration of aggravating and mitigating factors because they are important. So why aren't the circumstances of the crime included in the study? Well, they are difficult and expensive to gather. You can't just download them anywhere. But if you are comparing two groups that differ in race, how do you know that a difference in treatment is the result of racial discrimination and not legitimate differences in the groups if you have no data on the legitimate variables? You don't. A good example is the Paternoster study in Maryland, discussed in my 2003 article here. Statewide data appeared to indicate a race-of-victim effect such as other studies have purported to find, but that effect slipped beneath the statistical radar when the jurisdiction of prosecution was added to the model. Local democracy and jury of the vicinage, not racism, were the reasons.
It is the nature of social science that a researcher cannot control, or even have data on, all the factors that go into human behavior. People are harder to study than electrons in this way. With a large enough sample size, studies can draw conclusions despite uncontrolled factors on the assumption that these factors average out and are approximately equal between the groups being compared. The probabilities that we see computed in this way are valid if and only if the uncontrolled factors are independent of the factors used to form the two groups. If your groups are fruit grown in New England and fruit grown in Florida, can you assume the two groups contain equal proportions of apples and oranges? Of course not. Then you can't compare them as if they do.
The Newsweek story proclaims, "A new study finds that blacks on death row convicted of killing whites are more likely to be executed than whites who kill minorities." On a quick initial read, I did not find in the published study any result regarding whites killing minorities. In the "Findings" section, the study says "blacks convicted of killing whites are more likely to be executed than other death row offenders." That is quite a different statement, because the vast majority of murders by white perpetrators are of white victims. The white-killing-black combination is quite rare.
Putting aside for a moment the issues of other minorities, which were a small portion of the database, if black-killing-white murders compared to black-black and white-white murders result in more death sentences being imposed, as prior studies indicate, or in more of them being carried out, as this one claims, can we jump to the conclusion that racism and politics are the reasons, as this study does? No, because the confounding variables -- the perfectly legitimate factors regarding the crime -- cannot be assumed to be independent of the variables in the study.
Cross-racial crimes are more often between strangers. They are more often the consequence of a premeditated design to commit a violent crime and less often the unplanned result of a sudden argument. A result based on the assumption that cross-racial and intra-racial killings as groups are equivalent on the legitimate factors is an invalid result. These factors are obviously important at trial, but they are also important on appeal. The complexity of trials and the Supreme Court's constant churning of the rules governing the penalty phase make it nearly inevitable that some events at trial will be deemed "error" in an appeal or collateral review conducted years later. The question of whether that error requires reversal will be influenced by the facts of the case. Without any effort to control for legitimate reasons why cases may reach different results, this study tells us little or nothing.
But learning the truth may not have been the purpose. Reading the article, a person familiar with the death penalty debate cannot help but be struck by how hotly disputed propositions are assumed or stated without support and how studies supporting a particular finding are cited while contrary authorities are omitted. Here are a couple of examples.
"Although studies repeatedly show that victim race is the most important determinant of death sentences...." (Italics added.) Really? No citation is given for this remarkable statement. What study shows that victim race is more important than the circumstances of the crime in determining the sentence? None that I am aware of. The Baldus studies in Georgia and Nebraska do not find that, and neither does the Paternoster study in Maryland. Further, both the RAND study and Paternoster study showed that race of the victim was not even statistically significant, much less "the most important," in some models. The report of the special master appointed by the New Jersey Supreme Court, a court extremely hostile to capital punishment, similarly refutes the above statement.
"Such views about the efficacy of incapacitation and deterrence provide the basis for empirically dubious conservative claims that a few executions will protect many innocent victims from criminal brutality." To support this statement a footnote selectively cites the studies finding no deterrence and simply omits the ones that do find deterrence.
These are not statements one would expect to find in objective research seeking the truth. These are statements one would expect to find in an advocacy document with a purpose of supporting a particular position.