My lost finding
Academic research can be a tough, messy business. Add in the wonderful complexity of the people that are usually the subject matter of psychology and it's little wonder that a career can be a path of wrong turnings, forking paths and dead ends. The 'replication crisis' of recent years has renewed attention on this unseen or unloved data: what's in the file drawer, down the back of the sofa, or out there but underappreciated? For want of a better word, what's the significance of it all?
We want to hear from you about your 'lost finding'. Perhaps you've never shared it with anyone. Perhaps it was an 'ugly' initial finding which you're prepared to admit you transformed into something beautiful before unleashing it on the world (the 'chrysalis effect'). Perhaps you published it only to see it lie dormant before something woke it (a 'sleeping beauty' paper). Perhaps it's still dormant, unloved, and you can't work out why.
Speaking personally, there's a whole load of data I collected while at Glasgow Caledonian University which never got written up because I jumped ship to this job. And there's a published finding on how children, boys in particular, underplay or hide their academic effort, which I've always felt deserved as much attention as my bullying and social cognition stuff.
We'll be calling for contributions via @psychmag on Twitter using #MyLostFinding - but you can also email me with your thoughts on [email protected]. No more than 400 words, written in a style that's going to engage and inform our large and diverse audience of professional psychologists. We're hoping to collect enough of these to publish a selection in The Psychologist, which goes to 50,000+ members of the British Psychological Society each month. Can an exploration of what has been lost reveal something about the best way forward?
Dr Jon Sutton
Thanks to Dr Tom Stafford from the University of Sheffield for getting the ball rolling!
'Our paper fell through a disciplinary crack'
Hidden between pages 267 to 272 of Behavioural Brain Research (volume 243) is the paper I’m proudest of. While I love it to an unreasonable level, it is a result which was difficult to publish and hasn’t had much attention since publication either.
The reason I’m so proud of paper is it represents the kind of science I aspire to – a carefully crafted behavioural experiment which directly verifies a prediction, a prediction based in computational analysis of neural circuits.
The neural circuits are those underlying reward and habit learning, circuits which rely on that famous neurotransmitter dopamine. Pete Redgrave, head of our neuroscience group, and Kevin Gurney, professor of computational neuroscience, had a theory that dopamine signals could support learning new actions (contrary to the dominant theory, which is that dopamine signals the value of events).
The key to our experiment was the realisation that signals that first trigger dopamine release come from a subcortical area called the colliculus, and the colliculus only gets some visual information from the eye. In particular, it is insensitive to colour changes which affect the short-wave photoreceptors. What this means you can design a stimulus that is invisible to the colliculus (or at least invisible until the information arrives via visual cortex, which can detect such changes).
In practice, this stimulus looks like a kind of mauve, but it has to be tailored for each individual, since everyone has a different distribution of colour photoreceptors. This takes about an hour of delicate psychophysical measurement, before you even start the main experiment. Fortunately we worked with Martin Thirkettle who had the expertise – and patience – to implement all this.
After months and months of arduous testing (for Martin and the participants) we had a result, and it turned out just like the theory predicted – action acquisition is slowed if the information that signals a correct actions is delivered via one of these stimuli specifically crafted to match the insensitivity of the colliculus.
It’s a dramatic confirmation of a prediction which requires both a computational analysis and a detailed knowledge of the specific neural circuits involved. The understanding of colour perception and psychophysics builds on over one hundred years of experimental psychology. It was a true multidisciplinary team effort, and funded by the EU incidentally.
We were excited to try and publish, but that excitement turned sour as we clocked up rejection after rejection. I learnt that the work you put in doesn’t automatically translate into kudos from other people. I could see how difficult it had been to run the experiments, and how neat the result was, but to many editors it obviously seemed a bit…meh. 'The actual results are rather meagre', said one reviewer. Meagre?! Years of planning and experiment design, months and months of testing! And meagre!? One journal found us two reviewers, one of whom said they could review the psychophysics, but didn’t know about the neuroscience; the other said they could review the neuroscience but not the psychophysics. They couldn’t put the pieces together, and neither could the editor, so our paper fell through a disciplinary crack. With hindsight, I only blame ourselves for this – the rejections made us hone and hone the presentation of the result, so that not only was what we did clear, but what we thought it meant.
Dr Tom Stafford
University of Sheffield
Thirkettle, M., Walton, T., Shah, A., Gurney, K., Redgrave, P., & Stafford, T. (2013).The path to learning: Action acquisition is impaired when visual reinforcement signals must first access cortex. Behavioural Brain Research, 243, 267–272. doi:10.1016/j.bbr.2013.01.023
(no paywall link http://eprints.whiterose.ac.uk/83640/1/Thirkettle2013_postprint.pdf)
Lost, but in a good cause
There is something relentless about the grant-driven research business these days. The very idea that some findings will not be published does not fit its business model. What if we had a research culture that borrows from creative play? Here ideas flow constantly, and even if they seem brilliant when shiny and new, only few are good enough to keep. It’s unlikely that all research findings are worth talking about. If in doubt, the filing drawer might be a good place for them.
Do you, like me, hardly manage to read the abstracts let alone the published papers in your field? Do you find that it is rare to read something that makes you sit up and listen? Do you feel like having to constantly sift through tedious dross when you read the literature? If so, I am sure it will have occurred to you that some of your own findings might be part of the dross.
Here I would like to make a radical suggestion. When there is overproduction of goods, there often is a reward for ‘set-aside’. It sounds drastic, and does not fit a frugal model of the market. In our current research culture, I doubt if we have a frugal model, but we pretend we do. In grant applications, we justify our research as not only worthwhile, but also of crucial importance, and in pressing need to be done.
You know where this is leading: I suggest applications should be allowed to be less pretentious and more tentative instead. If so, it would be worth rewarding researchers to keep a proportion of their results in the file drawer. This would have the pleasing effect of a less cluttered literature. There could be a prize, not for the most productive researcher, but for the most discerning.
In research, you need hindsight to know what the best ideas and the most robust findings are. Keep a record of your unpublished results by all means. Mull over them, unearth them when they suddenly make sense, and share them when asked. My only concern is this: is your file drawer big enough?
- Uta Frith DBE FRS FBA FmedSci is Emeritus Professor of Cognitive Development at UCL Institute of Cognitive Neuroscience and President British Science Association. Find much more in our archive and at
BPS Members can discuss this article
Already a member? Or Create an account
Not a member? Find out about becoming a member or subscriber