My lost finding
Academic research can be a tough, messy business. Add in the wonderful complexity of the people that are usually the subject matter of psychology and it's little wonder that a career can be a path of wrong turnings, forking paths and dead ends. The 'replication crisis' of recent years has renewed attention on this unseen or unloved data: what's in the file drawer, down the back of the sofa, or out there but underappreciated? For want of a better word, what's the significance of it all?
We want to hear from you about your 'lost finding'. Perhaps you've never shared it with anyone. Perhaps it was an 'ugly' initial finding which you're prepared to admit you transformed into something beautiful before unleashing it on the world (the 'chrysalis effect'). Perhaps you published it only to see it lie dormant before something woke it (a 'sleeping beauty' paper). Perhaps it's still dormant, unloved, and you can't work out why.
Speaking personally, there's a whole load of data I collected while at Glasgow Caledonian University which never got written up because I jumped ship to this job. And there's a published finding on how children, boys in particular, underplay or hide their academic effort, which I've always felt deserved as much attention as my bullying and social cognition stuff.
We'll be calling for contributions via @psychmag on Twitter using #MyLostFinding - but you can also email me with your thoughts on [email protected]. No more than 400 words, written in a style that's going to engage and inform our large and diverse audience of professional psychologists. We're hoping to collect enough of these to publish a selection in The Psychologist, which goes to 50,000+ members of the British Psychological Society each month. Can an exploration of what has been lost reveal something about the best way forward?
Dr Jon Sutton
Thanks to Dr Tom Stafford from the University of Sheffield for getting the ball rolling!
'Our paper fell through a disciplinary crack'
Hidden between pages 267 to 272 of Behavioural Brain Research (volume 243) is the paper I’m proudest of. While I love it to an unreasonable level, it is a result which was difficult to publish and hasn’t had much attention since publication either.
The reason I’m so proud of paper is it represents the kind of science I aspire to – a carefully crafted behavioural experiment which directly verifies a prediction, a prediction based in computational analysis of neural circuits.
The neural circuits are those underlying reward and habit learning, circuits which rely on that famous neurotransmitter dopamine. Pete Redgrave, head of our neuroscience group, and Kevin Gurney, professor of computational neuroscience, had a theory that dopamine signals could support learning new actions (contrary to the dominant theory, which is that dopamine signals the value of events).
The key to our experiment was the realisation that signals that first trigger dopamine release come from a subcortical area called the colliculus, and the colliculus only gets some visual information from the eye. In particular, it is insensitive to colour changes which affect the short-wave photoreceptors. What this means you can design a stimulus that is invisible to the colliculus (or at least invisible until the information arrives via visual cortex, which can detect such changes).
In practice, this stimulus looks like a kind of mauve, but it has to be tailored for each individual, since everyone has a different distribution of colour photoreceptors. This takes about an hour of delicate psychophysical measurement, before you even start the main experiment. Fortunately we worked with Martin Thirkettle who had the expertise – and patience – to implement all this.
After months and months of arduous testing (for Martin and the participants) we had a result, and it turned out just like the theory predicted – action acquisition is slowed if the information that signals a correct actions is delivered via one of these stimuli specifically crafted to match the insensitivity of the colliculus.
It’s a dramatic confirmation of a prediction which requires both a computational analysis and a detailed knowledge of the specific neural circuits involved. The understanding of colour perception and psychophysics builds on over one hundred years of experimental psychology. It was a true multidisciplinary team effort, and funded by the EU incidentally.
We were excited to try and publish, but that excitement turned sour as we clocked up rejection after rejection. I learnt that the work you put in doesn’t automatically translate into kudos from other people. I could see how difficult it had been to run the experiments, and how neat the result was, but to many editors it obviously seemed a bit…meh. 'The actual results are rather meagre', said one reviewer. Meagre?! Years of planning and experiment design, months and months of testing! And meagre!? One journal found us two reviewers, one of whom said they could review the psychophysics, but didn’t know about the neuroscience; the other said they could review the neuroscience but not the psychophysics. They couldn’t put the pieces together, and neither could the editor, so our paper fell through a disciplinary crack. With hindsight, I only blame ourselves for this – the rejections made us hone and hone the presentation of the result, so that not only was what we did clear, but what we thought it meant.
Dr Tom Stafford
University of Sheffield
Thirkettle, M., Walton, T., Shah, A., Gurney, K., Redgrave, P., & Stafford, T. (2013).The path to learning: Action acquisition is impaired when visual reinforcement signals must first access cortex. Behavioural Brain Research, 243, 267–272. doi:10.1016/j.bbr.2013.01.023
(no paywall link http://eprints.whiterose.ac.uk/83640/1/Thirkettle2013_postprint.pdf)
Lost, but in a good cause
There is something relentless about the grant-driven research business these days. The very idea that some findings will not be published does not fit its business model. What if we had a research culture that borrows from creative play? Here ideas flow constantly, and even if they seem brilliant when shiny and new, only few are good enough to keep. It’s unlikely that all research findings are worth talking about. If in doubt, the filing drawer might be a good place for them.
Do you, like me, hardly manage to read the abstracts let alone the published papers in your field? Do you find that it is rare to read something that makes you sit up and listen? Do you feel like having to constantly sift through tedious dross when you read the literature? If so, I am sure it will have occurred to you that some of your own findings might be part of the dross.
Here I would like to make a radical suggestion. When there is overproduction of goods, there often is a reward for ‘set-aside’. It sounds drastic, and does not fit a frugal model of the market. In our current research culture, I doubt if we have a frugal model, but we pretend we do. In grant applications, we justify our research as not only worthwhile, but also of crucial importance, and in pressing need to be done.
You know where this is leading: I suggest applications should be allowed to be less pretentious and more tentative instead. If so, it would be worth rewarding researchers to keep a proportion of their results in the file drawer. This would have the pleasing effect of a less cluttered literature. There could be a prize, not for the most productive researcher, but for the most discerning.
In research, you need hindsight to know what the best ideas and the most robust findings are. Keep a record of your unpublished results by all means. Mull over them, unearth them when they suddenly make sense, and share them when asked. My only concern is this: is your file drawer big enough?
- Uta Frith DBE FRS FBA FmedSci is Emeritus Professor of Cognitive Development at UCL Institute of Cognitive Neuroscience and President British Science Association. Find much more in our archive and at
It's April 1996 and I'm nearly four years into what's shaping up to be a five-year PhD. My nerve-racking qualifying exam behind me, I've advanced to doctoral candidacy and have about two-thirds of my data in the bag. Down the street from my lab, cafés and surf shops spill into wide, white sands and then the Pacific Ocean. It's sunny in California but the uncertainties and self-doubts of graduate school cast a grey pall.
My elder brother is autistic, unable to speak, assaulted by his unmodulated senses, and often unable to get control of his thinking or behaviour. My niece has been born a few weeks ago; I'm anxious about her but haven't mentioned to my sister. (The diagnosis will come a couple of years hence.) All this is why I'm here: one and the same ponderous attention to detail and regularity has made me a scientist and my brother autistic, and I want to map the developmental watershed between us so that we can steer people across it.
I'm developing a new way to measure the speed with which people shift attention across the visual field. My data show that attention enhances a flash-evoked brain electrical rhythm, like an amplifier: the signal grows as attention enters the contralateral hemifield, then shrinks when attention shifts away. During a late night I have stared at the statistics and experienced that moment of transcendent discovery that all fortunate young scientists have: at that instant I knew something nobody else in the world did. I have mentioned this result to the head of an EEG lab up the hill, and found out that his postdoc has been pursuing it too. The race is on.
Being a computer scientist originally, since our programmer left I've been taking up the slack. It seems everyone has some kind of small computational task I can help with, and everybody's feverishly preparing posters or slides for the Cognitive Neuroscience Society. My own poster is the last to be finished; I work all night, leave for the airport directly from the lab, give a successful presentation, then arrive back at the lab directly from the airport without sleep. Anxious and exhausted, I'm irritable and my social perspective-taking, which never has been great in the first place, goes out the window: I get into a stupid argument with another student and get myself expelled from the lab. All my data are on tapes in the lab and I have no access to them.
After I'm gone, the lab hires a programmer. After many months I get my data back but the result already has been published, by the other group, through the 'Communicated by' route in PNAS (Morgan et al., 1996). For a year I don't touch science. In the end I send my confirmatory paper to Cognitive Brain Research (Belmonte, 1998).
I will eventually finish a PhD elsewhere, with a different project, and go on to publish more findings that will influence my field. But this was my first big result, and its loss still stings. I now try to remember that students need time to be students, that people under pressure often are not themselves, and that everybody can benefit when high-stress academic workplaces support rather than exclude those with mental health needs.
- Matthew K. Belmonte is at The Com DEALL Trust, Bangalore; Centre for Autism at the University of Reading, and Reader in Psychology at Nottingham Trent University.
Morgan, S.T., Hansen, J.C. & Hillyard, S.A. (1996). Selective attention to stimulus location modulates the steady-state visual evoked potential. Proceedings of the National Academy of Sciences of the United States of America, 93(10), 4770-4774. http://www.pnas.org/content/93/10/4770
Belmonte, M.K. (1998). Shifts of visual spatial attention modulate a steady-state visual evoked potential. Cognitive Brain Research, 6(4), 295-307. https://doi.org/10.1016/S0926-6410(98)00007-X
BPS Members can discuss this article
Already a member? Or Create an account
Not a member? Find out about becoming a member or subscriber