Science Fictions

You might lose faith in science after reading Science Fictions by Stuart Ritchie. That would be the OPPOSITE reaction to what Ritchie desires, and he makes this clear in the book. But when you read about the problems plaguing how results are communicated in shades or with increasing hype, and consider the perverse incentives in today’s scientific currency, it’s enough to make one increasingly skeptical about the whole enterprise.

 

Ritchie’s area of expertise is in the quantitative social sciences (particularly in social and behavioral psychology) where the problems are the most egregious, and He has many nightmarish and depressing examples of bad actors. But medical research, fundamental physics, and even biochemistry take a share of hits in Science Fictions. While there are stories of outright fraud, the book also explores inherent biases, motivations, unconsciously blinkered views, and the increasing role played in hype – the need to get noticed in an increasing deluge of scientific “breakthroughs”. Many of the vignettes relate to the misuse and massaging of statistical data; you might have heard of the p-hacking scandals and the “replication crisis” in the behavioral sciences.

 

“Hype” is the chapter that I found most interesting. The opening vignette was very familiar to me: the exciting press release of GFAJ-1, a microorganism in the harsh environment of Mono Lake, that supposedly survives and grows on arsenic rather than phosphorus, possibly even incorporating it into genetics and metabolism. Ten years ago, I was in a packed session at a conference where the lead scientist was assailed by critics, still in the early days before subsequent attempts to replicate the work led to different conclusions that were nowhere as exciting as the hyped press release. It was a circus, and not in a good way. (My origins-of-life class read the paper and discussed this issue last semester.)

 

Why is there scientific hype? And why is it getting worse? Here’s an illuminating paragraph from that chapter:

 

All this spin serves the same ultimate purpose as exaggeration in press releases and books: scientists want to emphasize the impressiveness and ‘impact’ of their work because impressive and impactful work is what attracts grants, publications and plaudits. The problem is that this can create a feedback loop: the hype nurtures and expectation of straightforward, simple stories on the part of funders, publishers and the public, meaning that scientists must dumb down and prettify their work even more in order to maintain interest and continue to receive this funding.

 

That’s not me. Or so I think, because my research isn’t going to be soaked up by mass media anytime soon. And I don’t think the reviewers of my grant applications and my papers aren’t in the hype machine. But academia has its peculiar and perverse incentives, which Ritchie discusses in detail – a story that is familiar to us in the business. To be “successful” be it in publication-wise, funding-wise, or prestige-wise, is like running in a hamster wheel that’s increasing in speed. If you don’t keep running faster, you’ll fall out of the virtuous cycle, which in reality is a vicious cycle.

 

I’ve personally wondered about my continuing complicity in this system. Now that I’ve reached full professor-hood, I can no longer get promoted (at my current institution) unless I want to go into administration – which I don’t consider a promotion because the fancy title and a higher salary do not, in my opinion, offset the frustrating and ridiculous parts of those jobs. (I can speak from personal experience. I’ve done these because it’s important service and if I’m clearly needed in such a position. Otherwise I’m happy to bow out, and have done so more than once.) I find myself edging away from it, and falling away from the trappings of what it means to be “successful”, and therefore awards and accolades won’t be coming my way, nor will I be headhunted by some other institution looking to burnish their credentials. I still publish research results in the usual channels because I want to give my students a leg-up into a (bleakly) increasing competitive future, but otherwise I’ve mostly exited the ratrace.

 

Finding our way out of a perverse system – and it is a complicated system that “takes on a life of its own” (to use an apt phrase) – will be very challenging. The closing chapters of Science Fictions describes some steps that have been taken to combat fraud, bias, negligence, and hype. But Ritchie also discusses why these measures are limited and may, in themselves, set up new perverse incentives that replace the old ones. Goodhart’s Law keeps coming into play: When a measure becomes a target, it ceases to be a good measure.

 

Or maybe it’s all just an ever-moving target. Science Fictions is a welcome addition to this discourse. I recommend reading it even though parts of the story can be depressing to many of us, the rank-and-file scientists stuck in a system of perverse incentives. It certainly made me think a little more carefully about my biases, how I frame hypotheses in my research projects, and how I write my papers. I was already familiar with the issues in the behavioral sciences since I read that literature on a regular basis too. But it was a good reminder to me not to be lazy when making an argument, and to catch myself from using post-hoc rationalizations. And that it’s worth doing good science!