A peculiar prevalence of p just below .05
In the August issue of Quarterly Journal of Experimental Psychology
Like a tired boxer at the Olympic Games, the reputation of psychological science has just taken another punch to the gut. After a series of fraud scandals in social psychology and a US survey
that revealed the widespread use of questionable research practices, a paper published this month finds that an unusually large number of psychology findings are reported as ‘just significant’ in statistical terms.
The pattern of results could be indicative of dubious research practices, in which researchers nudge their results towards significance, for example by excluding troublesome outliers or adding new participants. Or it could reflect a selective publication bias in the discipline – an obsession with reporting results that have the magic stamp of statistical significance. Most likely it reflects a combination of both these influences. On a positive note, psychology, perhaps more than any other branch of science, is showing an admirable desire and ability to police itself and to raise its own standards.
E.J. Masicampo at Wake Forest University, USA, and David Lalande at Université du Québec à Chicoutimi, analysed 12 months of issues, July 2007 to August 2008, from three highly regarded psychology journals – the Journal of Experimental Psychology: General; Journal of Personality and Social Psychology; and Psychological Science.In psychology, a common practice is to determine how probable (p) it is that the observed results in a study could have been obtained if the null hypothesis were true (the null hypothesis usually being that the treatment or intervention has no effect). The convention is to consider a probability of less than five per cent (p < .05) as an indication that the treatment or intervention really did have an influence; the null hypothesis can be rejected (this procedure is known as null hypothesis significance testing).
From the 36 journal issues Masicampo and Lalande identified 3627 reported p values between .01 to .10 and their method was to see how evenly the p values were spread across that range (only studies that reported a precise figure were included). To avoida bias in their approach, they counted the number of p values falling into ‘buckets’ of different size, either .01, .005, .0025 or .00125 across the range.
The spread of p values between .01 and .10 followed an exponential curve – from .10 to .01 the number of p values increased gradually. But here’s the key finding – there was a glaring bump in the distribution between .045 and .050. The number of p values falling in this range was ‘much greater’ than you'd expect based on the frequency of p values falling elsewhere in the distribution. In other words, an uncanny abundance of reported results just sneaked into the region of statistical significance.
‘Biases linked to achieving statistical significance appear to have a measurable impact on the research publication process,’ the researchers said.
The same general pattern was found regardless of whether Masicampo and Lalande analysed results from just one journal or all of them together, and mostly regardless of the size of the distribution buckets they looked at. Of course, there’s a chance the intent behind their investigations could have biased their analyses in some way. To check this, a research assistant completely blind to the study aims analysed p values from one of the journals – the same result was found.
Masicampo and Lalande said their findings pointed to the need to educate researchers about the proper interpretation of null hypothesis significance testing and the value of alternative approaches, such as reporting effect sizes and confidence intervals. ‘[T]he field may benefit from practices aimed at counteracting the single-minded drive toward achieving statistical significance,’ they said.
When sales staff smile everyone wins
In the Journal of Applied Psychology
Serving customers with a smile must be tough if you’re not in the mood. In the end, though, sales employees who are more smiley may end up reaping the benefit. A new study has looked at the way an employee’s positive emotion infects their customers, and how this in turn feeds back to the employee, boosting their own mood.
Eugene Kim and David
Yoon observed 117 interactions between staff and customers at clothing and accessory stores at a large shopping mall in Seoul, South Korea. The emotional behaviour of the employees was observed, then the way their customers responded, and finally, right afterwards, both employee and customer were quizzed about their mood and personality.
The more positive the employee, the more positive the customer tended to be. Moreover, employees who were more positive tended to be in a better mood afterwards, an association that was fully explained by the positive emotions displayed by the customer. In other words, smiley and polite staff initiated a virtuous interactive circle in which customers tended to respond in kind, thus benefiting the worker’s own mood.
Of course, not all customers are made equal. Kim and Young found that customers who scored lower in agreeableness and lower in emotional stability were more influenced by the positive emotion of the staff. More agreeable customers would be friendly anyway and highly stable customers are less prone to outside influences on their emotions.
A weakness of the study is that the researchers didn’t assess staff mood at the outset, prior to each customer interaction. Though unlikely, they admitted this means that they couldn’t completely rule out the possibility that interaction had nothing to do with the results – that an employee’s mood at the outset had simply affected both their own emotional display, the customer’s response and their own mood at the end.
Notwithstanding the need for more longitudinal research, Kim and Yoon said a key message for managers was to see customers as ‘coproducers of a positive service interaction’. As well as ‘recruiting and hiring employees who are adept at displaying positive emotions’, they said that managers should also consider reminding customers of the part they have to play by saying thank you and being civil.
Cheeky pictures suggest psychologists identify with the arts
In the PLoS One
What does it matter which side of your face you show when you’re having your photograph taken? A team of scientists say that it reflects how much you see yourself as emotional and arty or rational and scientific. Owen Churches and his colleagues analysed the personal webpages belonging to 5829 English-language university academics around the world. They found that engineers, mathematicians and chemists more often posed with their right cheek; English lit. dons and psychologists with their left. ‘[M]ost academic psychologists, who may have entered the profession during its arts-oriented past, perceive themselves as being more akin to arts academics than scientists,’ said the authors.
The researchers made their observations after choosing 30 universities at random from the 200 listed by the Times Higher Education World University Rankings for 2010 to 2011. When they found scholarly departments where the convention was for academics to present a photo of themselves, they went on to analyse all academic photos to see which cheek was visible. Straight-on photos were ignored. That left 3168 photos for analysis. Consistent with previous research there was a strong effect of sex – women more often pose with their left cheek showing. But the differences between the arts and science academics held even after controlling for this confound. The contrast also survived an analysis that excluded any photos that looked like they’d been taken by a professional photographer.
These new findings build on past research that’s shown the left side of the face is perceived as more emotionally expressive than the right; that emotionally expressive people are more likely to pose with the left cheek showing; that, historically, people have tended to pose more often with their left side showing, but older portraits of scientists, in contrast, show them posing more often with their right cheek; and that viewers tend to guess that an unknown academic posing with their right cheek is a scientist, whilst guessing that left-cheek posers are arts scholars. These findings, Churches and his team explained, ‘suggest a difference in the inward role of the two cerebral hemispheres in the creation and analysis of the emotional display’.
In the current study, the general pattern of cheek posing and academic affiliation broke down when it came to fine arts and performing arts – they showed no bias for posing with their left side. The researchers speculated this may be because of their expert knowledge of the history of portraiture.
Critics may wonder about the researchers’ interpretation that the posing position of psychologists suggests they identify with the arts. This seems quite a leap from the data that’s available. It’s also worth noting that there’s a huge amount of variation within each academic discipline in posing position. Even among male engineers, for example, nearly 40 per cent posed with their left side facing the camera.
‘Academics be warned,’ the researchers concluded.
‘We present ourselves to our students and colleagues in our profile pictures and the way we do so may reveal more about ourselves than we think.’
The material in this section is taken from the Society’s Research Digest blog at www.researchdigest.org.uk/blog, and is written by its editor Dr Christian Jarrett. Visit the blog for full coverage including references and links, additional current reports, an archive, comment and more.
Also see the Society’s Occupational Digest, at www.occdigest.org.uk.
Subscribe by RSS or e-mail at
Become a fan at
Follow the Digest editor at
BPS Members can discuss this article
Already a member? Or Create an account
Not a member? Find out about becoming a member or subscriber