Knowledge and Overconfidence

A little knowledge can be a dangerous thing.

 

This aphorism seems obvious, but can it be quantified? There’s an interesting study published in the journal Nature Human Behavior (2023, vol. 7, pp. 1490-1501). The title is provocative: “Intermediate levels of scientific knowledge are associated with overconfidence and negative attitudes towards science”. Ugh. Probably most of the general population has some (albeit limited) knowledge. And thanks to the internet and more widespread dissemination of information (which may be true, false, and shades of grey), more people than ever before fall into the category of having a little knowledge.

 

Maybe it’s not so bad in chemistry? Anecdotally, when I meet someone new who doesn’t know what I do for a living, the most common response translates as “I was bad at chemistry and didn’t understand anything”. But that’s what most people might say if they met someone with an expertise in a particular area. It’s a social response that doesn’t necessarily reflect a deep-seated feeling about competence (or lack thereof) in science.

 

The study looked at three large scale surveys covering almost 100,000 respondents across Europe and the U.S. over a 30-year timespan. Often, the way to measure confidence was to provide a “don’t know” answer option, but this has several drawbacks. The authors address this by redoing the analysis, and acknowledge that there are always compounding factors whatever technique you use to measure confidence. (I do like the emoji approach!) They also report on two newer but smaller scale studies that mitigate some of the issues in the previous surveys. I was at first skeptical about whether their conclusions were warranted, but after reading the article in its entirety, I think they’ve been careful in pointing out the caveats of their work. And I think their conclusions might be relatively robust.

 

The questions that test knowledge are interesting. Here are some True/False/Dunno statements that relate to chemistry.

·      The oxygen we breathe comes from plants

·      Radioactive milk can be made safe by boiling it

·      Electrons are smaller than atoms

The attitudinal questions are also interesting. Here are a few.

·      Thanks to science and technology, there will be more opportunities for the future generations.

·      For me, in my daily life, it is not important to know about science.

·      Because of their knowledge, scientific researchers have a power that makes them dangerous.

 

But on to the results. Confidence does increase with knowledge. But it increases faster than knowledge before tapering off at the end. Thus participants who answered very few questions with true or false said dunno a lot. At the other end, participants who answered many questions correctly and had very few wrong answers also used dunno a bit more (in relative proportion). Those who got a bunch of wrong and some right answers had relatively fewer dunno responses. This group also had the most negative attitude towards science based on the attitudinal surveys. I’m probably not explaining this very well (because the variables aren’t completely independent), so I recommend looking at the bar charts and graphs in the actual paper. (It might be paywalled; sorry.)

 

At first glance, some results seem “obvious”. But that’s our hindsight projecting in to give ourselves a boost. It does make sense that someone who was willing to say dunno and really didn’t know much indeed had lower knowledge confidence. Someone who knows a lot and encounters a “tricky” question might acknowledge dunno – confident in what one knows and doesn’t. Both these have more accurate metacognition on their actual state of knowledge. The overconfident person – a little knowledge being a dangerous thing – has poorer metacognition, over-estimating self-knowledge and getting lots of stuff wrong.

 

Maybe this is simply part of the learning process. When you learn something new for the first time, you might learn it superficially. You have some grasp but don’t really know it well, and you simply don’t have enough knowledge to assess how well you actually know it. You automatically try and connect this new piece of knowledge to other things, correctly or incorrectly, perhaps randomly, because your overall knowledge base isn’t wide. When you actually have a deeper grasp, your metacognition is sharpened and as you encounter new information, you can better evaluate how it fits into your wider knowledge base. (I’m assuming that knowledge here means “factually correct” scientific knowledge.) Anecdotally, in my chemistry class, the very weakest students know that they don’t know anything, and are not surprised by their exam results. The strongest students actually underestimate their ability slightly (but this might just be social playing-down their ability). It’s the middle-to-weak group that don’t have a good sense of where they stand. “I studied really hard and felt I knew the material” is a common refrain

 

As to attitude towards science, the results surprised me a little. I thought that those with the least knowledge would have the most negative attitude towards science, but that’s not the case. It turns out that this group tended to be rather ‘neutral’ towards science overall. The ones with the strongest negative opinions tend to think they know more about science than they actually do. This makes things especially difficult for science communicators to the public. Trying to simplify the science so that it is easier to grasp or digest could “offer a false sense of knowledge to the public, leading to overconfidence and less support, further reinforcing the negative cycle.” It gets worse: the overconfident are “more resistant to new information, especially if it contradicts their certainty, creating a negative reinforcement loop”. And because the surveys examined covered a wide timespan, there seems to be a higher correlation between overconfidence and negative attitude as the internet has grown to be a key information source for most people.

 

It's with a little despair that I ponder this information. I suppose that I should just keep calm and carry on doing my part to convey scientific knowledge as accurately as possible, clear up misconceptions, and try to sharpen student metacognition. After several years of self-tests in G-Chem, I stopped doing them because I felt it wasn’t helping the lower end of the class, at least in the current format I am using. Possibly some tweaking could fix the problem but I will need to ponder this a bit more.