<< Psychotherapy for Sex Offenders | Main | No Quick and Dirty Deals >>

How Good is the Neuroscience?

One of the most pervasive and enduring problems with all of the fancy fMRI studies that are done are the very small sample sizes used.  It's not unusual to have 20, 15 or even 8 subjects in a study.  Despite these severely underpowered studies, claims abound that neuroscience has shown X to be true with hardly a word mentioned about the significant limitations inherent in these studies. 

The use of small samples is finally getting some attention.  The May issue of Nature Reviews Neuroscience has the article Power failure: why small sample size undermines the reliability of neuroscience.  The abstract:

A study with low statistical power has a reduced chance of detecting a true effect, but it is less well appreciated that low power also reduces the likelihood that a statistically significant result reflects a true effect. Here, we show that the average statistical power of studies in the neurosciences is very low. The consequences of this include overestimates of effect size and low reproducibility of results. There are also ethical dimensions to this problem, as unreliable research is inefficient and wasteful. Improving reproducibility in neuroscience is a key priority and requires attention to well-established but often ignored methodological principles.

It's about time. 

Leave a comment

Monthly Archives