Neuropsychological impairments that everybody has

Robert McIntosh and Sergio Della Sala on quality science and 'sonic attacks' in Cuba, previewing a forthcoming letter in the Journal of Neurology.

In recent months, several US employees working at the US Embassy in Havana have reported subjective complaints including headaches, weakness and fatigue, vertigo, hearing loss, insomnia, memory problems and loss of concentration. They invariably linked these symptoms to acoustic stimuli of varying intensity and duration, experienced whilst in Havana. The story, which has precipitated an ongoing diplomatic standoff between the US and Cuba, has been branded by the media as evidence of a malign ‘sonic attack'.

The reports of these symptoms are largely anecdotal, but apparently stronger objective data have now emerged, in a high-profile paper with a supporting editorial, in the Journal of the American Medical Association. The paper reports on 21 people, all US diplomats previously working in Havana, concluding that they exhibit neurological manifestations of unknown origin, which resemble mild cerebral concussion, yet with no evidence of brain injury, and possibly related to obscure acoustic stimuli. The evidence for this extraordinary claim rests in large part on the extensive neuropsychological testing of six of these 21 people, with significant cognitive impairments reported in all cases.

This article has attracted a lot of attention, but remarkably little scrutiny. A closer reading of the paper shows that the neuropsychological evidence presented in the Supplementary Material (eTable 2) is almost unbelievably flimsy. The six people assessed with the neuropsychological battery were tested with 37 tests split across 10 cognitive domains. In each domain, the person was considered impaired if their score fell below the 40th (fortieth) percentile in at least one of the tests. This threshold for impairment is inexplicably high.

By definition, 40% of people assessed on any test should score at or below the 40th percentile. Considering that each participant was given 37 tests, without any statistical correction for the number of tests administered, it seems unlikely that anyone would escape with a clean bill of cognitive health (for further analysis, see Della Sala & Cubelli, in press). As we have reported in a letter to the Journal of Neurology, we tested this intuition using a simple simulation. We substituted the authors’ raw data for randomly-generated test scores, and applied their bizarre diagnostic criteria (Further details of this simulation, including analysis code and graphical output, are available at https://osf.io/5b4sq/). Across a thousand repetitions of our simulation, the cognitive symptoms of this syndrome proved to have a worrying lack of specificity: everybody is affected.

It is hard to understand how claims like this, based on the most blatant p-hacking, could pass any meaningful peer-review process. The pseudo-scientific approach taken implies that one may define arbitrarily liberal cut-offs to make one’s case, and interpret the outcomes as if they were clinically valid. We should be much more worried about reputational damage to neuropsychology, and psychology in general, than about any sinister new sonic weapons.

Robert D McIntosh, PhD and Sergio Della Sala, MD, PhD;

Human Cognitive Neuroscience, Psychology, University of Edinburgh, Edinburgh, UK

Reference

Della Sala, S., & Cubelli, R. (2018). Alleged “sonic attack” supported by poor Neuropsychology. Cortex, in press.

Della Sala, S. & McIntosh, R. D. (2018). Cognitive impairments that everybody hasJournal of Neurology.

BPS Members can discuss this article

Already a member? Or Create an account

Not a member? Find out about becoming a member or subscriber

Comments

Comment By:

 

Prof.Maan A.Bari Qasem Saleh

Depart. Of Behavioral Sciences

Faculty of Medicine

University of Aden-Yemen

Department Of Psychiatry

College of Medicine

University of Imam Abdulrahman Bin Faisal,KSA

 

 

I read with great interest the article of Robert D. McClntosh and Sergio Della Sala posted in July2018 issue of The Psychologist discussing "Neuropsychological impairments that everybody has"(p-2).

I think it is an article that shakes a warning bell for the state of psychology from the risk of sliding behind the conflict of interests and politics.
I think the article follows a solid experimental scientific method in the re-testing "Replication". This method has many problems and saved a large human sciences with the difficulty of re-testing to verify the accuracy of the results that have been referred to by magazines and papers such as a paper of “Rejena Notzo(2014)statistical Mistakes published at Nature.

The use of scientific standards, for example, the statistical value of "P", has become critical observations of many scientists because of excessive blind generalization and manipulation. Researchers believe that this has affected the credibility of research results in psychology and social sciences in general.

It is important to draw attention to previous experiences that have been implicated by the American Psychological Association (APA) and have exceeded the scientific and ethical standards for the applications of psychology in practice and as demanded by institutions. I mean here, a number of well-known American psychologists who have been involved in justifying the use of torture techniques in Guantánamo prison as a technique for obtaining information. This was what the annual conference of APA stands against in a critical and frank manner not to be repeated in August 2015. (See www.apa.org)

I am not here against benefiting from the applications of psychology in the military and security field. We cannot deny the progress they have made. I also understand the commitment and discipline of the strict rules of work in these military and security institutions, especially in the circumstances of war and acute crises. But we should take into consideration the extreme and dangerous politics that use the techniques and scientific methods in a perverse manner to harm the reputation of science, and we are still traumatized by the approaches and techniques of intense pressure practiced in psychology in the last century; for example, in the era of Stalin and Hitler during the Soviet Union and Nazi regime, respectively. We found in the same issue of July some comments looking at the experiences of Stanley Miligram and the crisis of ethical and methodological standards he followed (p-23).

However, there is a question whether psychology, like other sciences, is subject to systematic and technical errors. As theories in chemistry, physics and biology have changed, as well as their applied techniques in the evolution of science, would it be better for psychology to change through practical techniques to withstand challenges and obstacles that could impede its development?

Once again, the issue of the role of individuals in science, professional figures, academic institutions, psychological associations and psycho-periodicals remains to be discussed in the creation of dialogue and critical debate based on solid scientific research in favor of filtering psychology of impurities and narrow calculations.
Thanks to Robert and Della Sala for the effort and scientific perseverance they put into replicating the scientific experience to correct the code of ethics and save the face of psychology.