Interestingness Filter

Tim Harford is an economist in England.  He writes serious (and interesting) essays on economics, and a rather tongue-in-cheek advice column called Dear Economist.

This article, When it comes to research, we live in interesting times, make some really interesting points about popular science sources, and about more serious systematic reviews.  It’s a short article and worth clicking through.

The bottom line is that, for most of us, the information we have access to has already been filtered multiple times, with biases for availability, popularity, political correctness, and most of all, interestingness.

Quite apart from the fact that nobody wants to read all the evidence, there is a deep problem with the way evidence is selected throughout academia. Even a studiously impartial literature review will be biased towards published results. Many findings are never published because they just aren’t very intriguing. Alas, boring or disappointing evidence is still evidence. It is dangerous to discard it, but let’s not blame Malcolm Gladwell just because he doesn’t stick it on page one.

There’s a hierarchy of evidence here. The systemic review tries to track down unpublished research as well as what makes it into the journals. A less careful review will often be biased towards results that are interesting. A peer-reviewed article presents a single result, while a popular social-science book will highlight a series of results that tell a tale. The final selection mechanism is the reader, who will half-remember some findings and forget the rest.

Those of us who tell ourselves we are curious about the world are actually swimming in “evidence” that has been filtered again and again in favour of interestingness. It’s a heady and perhaps toxic brew, but we shouldn’t blame popularisers alone for our choice to dive in.

Comments are closed.