Title: The false promises of MRI in cognitive neuroscience.
Abstract: Most published brain imaging studies contain errors ; the complexity of the data is not captured by any adequate model ; the logic of statistics used in data analysis, null hypothesis significance testing, is not appropriate anyway. For those reasons, a revision of brain imaging software was called for (Hupé 2015, Frontiers in Neuroscience). Since then, the authoritative American Statistical Association published (at last !) a statement banning p-values from valid scientific reasoning (Wasserstein & Lazar 2016, The American Statistician). This concerns the whole scientific production, not only brain imaging studies. Fortunately, when frequentist models are correctly applied to the data, misleading interpretations of p-valuescan be easily corrected by computing confidence intervals (e.g., Hupé 2016, Science). However, for fMRI, a recent study confirmed empirically that statistical procedures based on parametric models were unable to control the rate of false positives : Eklund, Nichols & Knutsson (2016, PNAS) suggested that up to 80% of so-called “significant” activations revealed by those analyses could be false positives, even when performed without errors and using state-of-the art analysis. How are we going to solve this major scientific crisis ? Eklund et al. advocated that non-parametric procedures (permutation tests) allowed the correct control of the risk of false positives. However, to my knowledge, permutation tests do not allow any inference on physiological measures (that is, the computation of confidence intervals of parameters interpretable within a theoretical model). Trying to rely on statistical tricks highlights a general problem in cognitive neuroscience, where the measure tends to be confounded with what is measured (akin to mythic thinking in the sense of Barthes). It seems that the understanding and validity is not questioned anymore of what psychologists used to call “hypothetical constructs” (MacCorquodale & Meehl, 1948, Psychological Review). Many if not most attempts of “brain behavior correlations” face the difficulty to construct relevant measures, that is, measures constructed based on solid theories. Instead, the very lack of any solid theory (we have no idea of the neural code of thoughts) is replaced by the computation of averages of what is measurable, without questioning what the mean means. Claude Bernard had yet warned that the only possible science of living systems may be a science of the individual (Introduction à l’étude de la médecine expérimentale, 1865). I would therefore suggest that cognitive neuroscience has not been short of technology but epistemology. Without reconsidering what the valid objects to measure are, we may continue to “avancer à l’aveuglette, dans un empirisme dénué de sens », as already diagnosed in 1998 by the great molecular biologist Michel Djerzinski. I shall illustrate my claim with a couple of popular results of cognitive neuroscience, which are in fact brain myths. But maybe in our post-truth world it does not matter anymore if story-telling has taken the place of science (or was there ever anything but story-telling ?).
