Psychologist logo

Coin-flipping cowboys

Suzy J. Styles has a story about the wild west of lucky cowboys, and picking your team after the rodeo is over. If you've been following the open science/replication debate, you may find some of the key characters familiar… 

08 November 2018

It’s Rodeo day, and the small town of Luckville has gathered at Long Tail Ranch. There is crepe-paper bunting along the side of the barn where the coin-flipping competition is held. The competition coin is normal, evenly weighted, and has a picture of a ‘head’ on only one side. It’s pretty clear that if there’s nothing wrong with the coin, and none of the cowboys are cheating, the typical response will be 50 heads. That’s what chance tells us, right? But not all of the cowboys will get exactly 50. Some will get 49, others 52, or 43 – their scores could be distributed anywhere from 0 to 100. If we want to look at the distribution, we can ask all the cowboys to line up in a barn according to how many ‘heads’ they got. There are numbers on the floorboards, and we’ll get them to stand at the number for their score. We’ll get a big bunch of cowboys in the middle of the barn, near the 50% mark, and fewer and fewer cowboys out near the walls. We can call this muster of cowboys a ‘normal’ distribution. Let’s look at some examples…

Plain Jones – He’s standing right in the middle of the group because he scored exactly 50 heads. That’s some pretty average flipping. No wonder they call him ‘Plain Jones’ since he’s so ordinary.

Lucky Tom – He’s one of the four cowboys furthest from the middle of the barn – he got a ‘head’ almost every time he flipped. The probability of getting a score like that is less than 5% so he sure is a lucky cowboy!

Maggots – This guy is right at the end of the barn because he didn’t get any heads at all. It’s super unlikely, since less than 5 cowboys got that score. His p-value is therefore less than 5%. Rotten luck, Maggots!

If we had to elect a single cowboy to represent the group, Plain Jones would be a good cowboy to vote for, since he is representative of the cowboys in this barn. But what if we accidentally ended up with Maggots instead? Or Lucky Tom?

Both Lucky Tom and Maggots have scores that are very different from the group. This can be reported as a probability (p) of less than 5%, or p < .05 . In the social sciences, it’s common to call a difference like this significant  since there is less than 5% chance of getting a result like Maggots.

So is a ‘significant’ difference special? We’ve seen from our coin flipping example that cowboys will be scattered all the way from one end of the barn to the other. It’s very unlikely that any individual cowboy will end up right at the barn door, but 5% of cowboys will get those extremely unlikely scores, just by chance. They will simply be lucky cowboys  (or unlucky cowboys – sorry Maggots!). Since 5 out of every 100 cowboys will be extreme, that’s the same as 1 in 20. In the Wild West of data science, if you run 20 tests, one of them will be significant  just by chance. 1 in 20 findings is a lucky cowboy .

Let’s see what happens in the barn on competition day…

The punters of Luckville

Nathaniel H Testing thinks black hats make cowboys lucky. He can test his hypothesis at the rodeo. If the Black Hats flip significantly more heads than the White Hats, it will mean that Black Hats are luckier. His neighbour, Nellie D Null  thinks his idea is ridiculous, and that coin flipping is a simple game of chance. As far as she is concerned, no special clothing or cowboy characteristics are going to change the outcome. Nathaniel places his bet, Nellie does not. After all, her prediction is the null hypothesis – she will simply wait for Nathaniel to fail.

Sean P Hacker places a bet for the Leather Chaps to beat the Woolies. Nellie gives him a stern look. Last year, he placed a bet for The Moustaches. When the game was over he complained about the way they measured facial hair, and had half of the team disqualified from the competition. Interesting how the moustaches he excluded were all the unlucky ones…

Josie Branching-Lane loves this game since there are so many options to choose from. She writes out a betting slip for Lassos to beat Guns, and another for Moustaches to beat Baldies. But what about the Spurs? Or the Neckerchiefs? They sound like good options too. All together Josie writes out twenty different betting slips.

Since she can’t decide which is the best option, she doesn’t lay down any money. She keeps hold of all her betting slips since she doesn’t want to miss out on the fun.

Jack Harker writes for the New Rancher. He thinks the light-reflecting properties of white hats help to keep cowboys’ heads cooler, and being cool under pressure will help with a cowboy’s luck. He picks up a betting slip to include in his newspaper article, but he doesn’t bother filling it out since he won’t be placing a bet.

Cherie Picker writes for the Breeders’ Digest. She has a regular column where she profiles lucky cowboys across the state. She has a hunch that cowboys with white hats are luckier, since there are always lots of white hats among the winners. She travels from rodeo to rodeo watching coin-flipping competitions, and deciding which cowboys she will profile. She features a lot of white hats.

Incidentally, her cousin owns a hat making company that makes a popular hat called the Bright Diamond. Her cousin gave her the carriage she drives between rodeos. 

The luckiest cowboys

The cowboys flip their coin. The Black Hats beat the White Hats, and all hell breaks loose...

Josie Branching-Lane shouts “I knew it I knew it!” holding up one of her betting slips, and filing away the rest in her saddlebag. “Can I place my bet now? See it’s exactly what I predicted!”

“Madam,” replies the book keeper, “you can’t place your bet after the coin-flips are in! For all I know you might have 20 different betting slips stuffed down your drawers, and this one may be nothing better than chance!”

Josie is annoyed, but she plans to tell all her friends she picked the winner anyway.

Sean P Hacker mutters something about chap-width, and starts asking the judges if some of the cowboys ought to be replaced because their chaps are “not the regulation kind”. The book keeper is not amused, and Nellie helps her escort Sean from the barn. He skulks away planning his letter to the editor. He’ll tell the ‘true’ story later, explaining why he thinks some cowboys should be excluded, and how that means the Leather Chaps won.

Jack Harker is surprised by the result, and pauses for a moment to think. The result doesn’t match what he previously thought about the heating properties of hats. “Of course!” he declares, “The cooling properties of white hats are bad for luck,  because cowboys need ‘hot hands’ for lucky coin flips!” He jots down Black Hats on his unfilled betting slip, and thinks, ‘This’ll make a great article in the New Rancher’.

Cherie Picker dashes forward to get an exclusive with one of the cowboys. She dives straight for Lucky Jim, his Bright Diamond tilted down, shading his dark eyes against the sun.

“Congratulation son!” she exudes, “You sure are a lucky cowboy!”

“Lady,” he replies “I don’t know you from a trough of sheep dip. And my team didn’t win.”

“Oh pish” she says “Who cares about the team result! You flipped the highest score out of all of these cowboys – how would you like to be famous?”

Nathaniel H Testing shows his betting slip to Nellie D Null, with a slight smile.

“Significant” he says. “I registered it and everything”.

“Well done dear,” she replies, taking his arm, “but you know you could just be lucky cowboy…”

“What do I have to do to convince you?” he asks, handing his slip to the book keeper, for verification.

“Well,” say Nellie gently “you could always ask the cowboys to flip their coins again…”

“I think you’ll find one test is enough for me to award the prize” say the book keeper, “he placed his bet fair and square before the flipping competition.” She opens the safe and starts counting out the prize money.

Nellie is unconvinced. She’s happy that Testing won the competition today, but without repeating the test, there’s no way to know if Nathaniel is just a lucky cowboy.

She plans to make him place the same bet next year…

An unwelcome intrusion

“Whoa there!” comes a deep voice from the back of the barn “Hold your horses before you hand over that money - I demand a recount” It’s Rick Orr, self-appointed Sherriff of the rodeo.

“Oh not you again Rick,” says Cherie Picker, “Why are you always trying to ruin everything?"

"You only do this to make yourself seem important” snaps the bookie, slamming the safe door, “You’re nothing but a low-down data thug!”

“Call me what you like Ma’am,” Rick Orr replies, “I simply uphold the law. Now. Let’s see the records of all the coin flips. Are all of these cowboys standing in the right place?”

Rick grabs the stack of score sheets and stalks along the barn. He stops in front of each cowboy, asks their name, and looks up the tally of their coin flips. He’s checking if they are standing in the right place.

“Charlie Broke-Eyes – two steps to the left” he snaps, “Justin Beaver – one step to the right. Spider… Spider… Nope. Stay where you are.” And so on – all the way down the line. “Lucky Tom – it says here you that you flipped your coin yesterday. Were there any judges present when you did it? No? That’s not good cowboy behaviour, young‘un!” he says, “We’ll have to kick you out of this year’s league.”

There’s a gasp from the audience.

“Are you accusing this fine young man of cheating!?” says Cherie, in a huff.

“I made no such accusation Ma’am,” says Rick Or, popping a stalk of wheat between his teeth, and furrowing his bushy brows. “We just can’t confirm quite what happened, so we can’t include the score. No accusation, just an honest accounting of gaps.”

Lucky Tom spits, and swaggers away from the lineup. Cherie Picker bustles him away to a nearby hay bale, and pulls out her notebook.

“Right then,” says Rick. “We spotted some mistakes, and some of those coin flips should not have been counted. Let’s see the results. Will we reproduce the original finding?”

The judges consult. The Black Hats are still the winners. The cowboys yip and holler.

“Right then you news people,” says Rick Orr to Jack Harker and Cherie Picker, “I trust your stories will include the correct scores?”

“Thanks for reviewing the scores before I published my story,” says Jack, “I have more confidence in the result knowing someone else has checked it."

Cherie Picker laughs, tearing a page of dense shorthand from her notebook. She hands it to a scruffy boy who bolts out the door and leaps onto the back of a startled pony.

“You’re too late,” says Cherie Picker “I’ve already sent my story about Lucky Tom to the telegraph office – It’ll go to print in the morning”

“But Lucky Tom flipped without a judge in the room!” says Nellie, shocked, “He was excluded from the league.”

“Well that’s as may be, but what’s published is published. It was true at the time of writing.”

“You could call the story back – retract it?”

“Not on your Nellie, Nell.” Says Cherie, “My pay-check depends on stories like this one.”

“How about a correction after the article runs, Cherie?” says Rick.

“But I think he is lucky. Who’s to say he isn’t?” says Cherie.

“We simply don’t know how lucky he is.” Says Nell.

“Well I haven’t seen any evidence that he isn’t Lucky,” says Cherie, stalking out of the barn, “My story will go ahead as planned.”

In which representation matters

“Well, I sure am glad that review is over,” says Nathaniel, turning back to the book keeper. “I was worried for a moment there, but since the Black Hats are still the winners, I’ll just collect my prize money and go.” He turns to Nell, “Now I know that black hats are luckier, I’ll be sure to only employ them on my ranch in the future.”

“You know what, son,” says a booming voice from outside the barn doors. It’s Tester’s Dad, Mr Meht A Tester . “Before you do anything stupid about our employment policies, we need to look at more than just one flipping test. Let’s do a little poking around the Rodeo records. When we look at all of those dang records at once, we’ll really be able to find out whether these black hats make a difference, or whether Nellie here is right, and you’re nothing but today’s lucky cowboy.”

*BAM*

The barn door slams open. There’s a lick of cold wind, and a sinister silhouette standing in the doorway.

“Listen up you salt-licking bronco-humps!” comes a piercing voice, “whatever you think you know about luck is horse-chaff.” It’s Peggy Lim from the Texas Cowgirl Collective. Pig-tailed. Hatless. Five feet and three inches of sinew and brine. Five-time lasso champion, and a whip cracker of such precision she could snuff out a cigarette at 20 paces while it was held in your mouth. So long as you didn’t wink at her. Seriously, don’t wink at her while she’s working. Wink, and you might lose an eye… If you’re lucky.

“This Black Hat bull-tripe only goes as far as the walls of this barn. And let me tell you, it’s pretty weird in here,” says Peggy. “Have you noticed how similar all of these cowboys are? They’re all men between the ages of 18 and 23, they’re all right handed, and you haven’t got a single flipping cowgirl!”

There’s an awkward pause.

“But um..” says Plain Jones twisting the ends of his neckerchief in his fingers “it’s a coin-flipping cowboy competition, it says so on the sign”, pointing to the handpainted banner above the bunting. “My Mammy made that sign when we started the competition 20 years ago.”

“Holy Handkerchiefs,” says Peggy. “Just because that’s how you’ve done it before doesn’t mean you have to keep doing it the same. Do cowgirls not have thumbs? Can we not flip?”

“But um…” says Plain Jones, “we thought the competition would be better if we reduced unnecessary sources of difference, like whether you flip with your right or your left hand… and whether um… it’s a ~special time~ of the month for some of the competitors…”

“What the tuck-box, Jones? Special time? I hope you mean like payday right? Because if you mean ~something else~ you might have to spell it out for me...” Jones opens his mouth, sees Peggy’s hand resting on her the butt of her whip, closes it again.

“Oh check your gingham at the door, Little Miss,” says Plain Jones, from the middle of the barn. “It’s only a bit of fun. What’s all this fuss about?”

“I’ll tell you what there is to be angry about. Mr Moneybags over here just said wants to employ back hats from now on. On the basis of this kind of test... A test that didn’t include any cowgirls… And most cowgirls I know don’t even wear hats! So how the book-ends do you think we’re ever going to get jobs? We can never win a competition we’re not allowed to enter.” She says “If you want people to take this flipping seriously, then it’s your responsibility to make sure the comp includes all the competitors!”

“How about this Peg,” says Jack, “in my New Rancher article, I’ll be sure to say the cowboys in the comp aren’t representative of the general cow-care community.”

“Well that certainly is a right-on way to start, friend!” says Peggy.

“And we’ll ask the mayor of Luckville to change the competition rules next year. We should let everyone flip if they want to” says Maggots.

“Yee Haw,” says Left-Hand Luke, from the back of the barn. “I’ve wanted to flip in the competition my whole dang life!”

“And as for you Mr Meht A Tester,” says Nell, “your score checking will be better if you make sure to include coin flips from a wider range of people.”

“I do believe you’re right!” says Tester Senior, scribbling in his notebook. “That’s just what I’ll do.”

“Well knock me down if this ain’t the best darn barnload of cow-folks I ever did have the pleasure of meetin’!” says Peggy. “Who’dda thought you’d all be so open to improving your flipping methodologies!”

- Suzy J Styles is an Assistant Professor of Psychology at Nanyang Technological University in Singapore, and the Director of the Brain, Language and Intersensory Perception Lab (BLIP Lab). She is a developmental psycholinguist, whose research investigates relationships between language and perception.

Suzy has talked about coin-flipping cowboys with her junior research collaborators for the last five years. This is the first time the cowboys’ full story has emerged.

Find Suzy on Twitter.

Y'all want some extra cowboy vocab? Download the PDF.