I recently watched
the movie Green Book. There’s a scene where Tony Lip is proud of his
ability to BS his way through different situations. It reminded me of an
article published last month by the Institute of Labor Economics. The three authors
are academics: John Jerrim, Phil Parker and Nikki Shure. The title and abstract
are shown below.
Yes, people study
BS. Per the typical academic article, it provides a history of previous BS
studies. There are many ‘over’-statement definitions in the field.
Over-confidence. Over-claiming. Over-estimation. Over-placement.
Over-precision. You find what you might expect: “participants are more likely
to bullshit when they believe they can get away with it, and are less likely to
bullshit when they know they will be held accountable for the responses they
provide”. There’s even a Bullshit Receptivity scale.
Compared to
previous studies, the present one has a much larger sample size (40,000+
respondents from nine Anglophone countries) because it uses data from PISA
2012. That’s how I stumbled on this report, since I follow educational
developments and I’ve looked at the latest PISA results with respect to scientific inquiry. PISA is aimed at 15-year-olds and measures skills in
mathematics, reading and science. Since PISA 2012 had an additional focus on math,
there were extra questionnaire items that allowed researchers to crunch data
for the present study.
Classifying students’
propensity for BS was based on a list of 16 items where they rated whether they
had ‘never heard of it’ up to ‘know it well, understand the concept’ on a
five-point scale. Some items on this list were nonsense. Students also answered
Likert-scale questions that provided measures of (1) self-efficacy, (2)
self-belief in problem-solving ability, (3) self-reported popularity at school,
and (4) self-reported measures of perseverance. The self-efficacy questions
were aimed at gauging how comfortable students were with a math-related task,
such as ‘calculating how many square metres of tiles you need to cover a floor’
or ‘finding the actual distance between two places on a map with a 1:10,000
scale’ or ‘solving an equation like 2(x+3) = (x+3)(x-3). And of course, there
are the actual PISA results in mathematics so we know how well students
actually performed.
The results are
interesting. Teenagers in the U.S. and Canada are more likely to be BS-ers than
in Britain (England, Ireland, N. Ireland, Scotland, Wales) or Australia and New
Zealand. Along gender lines, boys tend to be BSers more than girls; effect
sizes were quite large in this area for Britain and smaller for North America.
Effect sizes were also significant comparing socio-economic status (a standard
PISA measure) – teenagers from higher socio-economic status showed a larger
preponderance for BS.
Not surprisingly,
BSers show higher self-efficacy scores and have higher self-views of their
problem-solving abilities relative to how they actually performed on PISA. They
also viewed themselves as more likely to persevere when confronted with a
difficult task (unclear if they would actually do so) and tend to believe they
are more popular at school. The effect sizes are smaller in the latter case
compared to the former. When given ‘approach to problem solving’ tasks, they
tend to pick conventional ‘safe’ answers rather than try to seem less
knowledgeable (again, unclear if this is what they would actually do).
The authors
acknowledge limitations of their study. BS in one area (math) may not overlap
in a simple way with other areas. The data is a one-time snapshot; BS behavior
likely changes over time within an individual. Also, the “social and labor
market outcomes” and implications of BSers remains unclear. Does it help? Does
it hurt? Don’t know.
Do BSers reflect
on their self-efficacy on any related traits? Do they recognize their own BS?
Or does repetition inure one to that fact such that you start believing in your
own imagined abilities? I don’t know. There’s certainly self-help advice out
there in the ‘fake it until you make it’ variety. One thing I try to get across
to students is how to avoid self-deception in thinking you know more than you
actually do when preparing for exams. As a result, I’ve included more
metacognitive exercises and introduced annotated self-grading. Perhaps I am in an
area where it’s much more difficult to BS on an exam. You know it, or you
don’t. And it shows up clearly.
One useful
reminder to myself: Domain knowledge and critical thinking in one area doesn’t necessarily extend to other areas, particularly if they are less proximate. I might know a lot about chemistry, or maybe my particular subfield of
chemistry, but I should be wary of the temptation to pontificate in other
areas. When you’re in a profession where you’re constantly viewed (by students)
as an authoritative source, it’s easy to slip into professing BS, and not
realize it.