Dead salmon have meaningful brain activity, or how to get scientists to stop using outdated methods
We’ve all seen in our science bad approaches or terminology that get established and are difficult to kill. Chuck Doswell has his pet peeves, I’ve battled against my share: conditional symmetric instability to explain banded precipitation and moisture flux convergence as a diagnostic for severe storms forecasting.
Bennett et al. were awarded the 2012 Ig Nobel in Neuroscience “for demonstrating that brain researchers, by using complicated instruments and simple statistics, can see meaningful brain activity anywhere — even in a dead salmon.”
From the authors’ acceptance speech (Annals of Improbable Research, 18(6), pp. 16-17):
“we…found that up to 40% of papers were using an incorrect statistical approach. And many other people had argued that they should be doing it correctly, but it wasn’t sticking. So we decided: can we use the tools of humor and absurdity to change a scientific field? Nay, to put a dent in the universe. And the truth is that you can. Through a dead salmon, you can get the number of people who use the incorrect statistic under 10%. So we’ve had a real impact.”
Bennett et al. 2010: Neural Correlates of Interspecies Perspective Taking in the Post-Mortem Atlantic Salmon: An Argument For Proper Multiple Comparisons Correction. Journal of Serendipitous and Unexpected Results, 1(1): 1-5.
Abstract:
With the extreme dimensionality of functional neuroimaging data comes extreme risk for false positives. Across the 130,000 voxels in a typical fMRI volume the probability of at least one false positive is almost certain. Proper correction for multiple comparisons should be completed during the analysis of these datasets, but is often ignored by investigators. To highlight the danger of this practice we completed an fMRI scanning session with a post-mortem Atlantic Salmon as the subject. The salmon was shown the same social perspective-taking task that was later administered to a group of human subjects. Statistics that were uncorrected for multiple comparisons showed active voxel clusters in the salmon’s brain cavity and spinal column. Statistics controlling for the family-wise error rate (FWER) and false discovery rate (FDR) both indicated that no active voxels were present, even at relaxed statistical thresholds. We argue that relying on standard statistical thresholds (p < 0.001) and low minimum cluster sizes (k > 8) is an ineffective control for multiple comparisons. We further argue that the vast majority of fMRI studies should be utilizing proper multiple comparisons correction as standard practice when thresholding their data.
Read Scientific American’s blog post about this study.
