An interesting critique of Kahneman
Jason Collins has an interesting critique of Daniel Kahneman. It will, of course, be of most interest to those who have read Kahneman. Collins begins:
A bit over four years ago I wrote a glowing review of Daniel Kahneman’s Thinking, Fast and Slow. I described it as a “magnificent book” and “one of the best books I have read”. I praised the way Kahneman threaded his story around the System 1 / System 2 dichotomy, and the coherence provided by prospect theory.
What a difference four years makes. I will still describe Thinking, Fast and Slow as an excellent book – possibly the best behavioural science book available. But during that time a combination of my learning path and additional research in the behavioural sciences has led me to see Thinking, Fast and Slow as a book with many flaws.
First, there is the list of studies that simply haven’t held up through the “replication crisis” of the last few years. The first substantive chapter of Thinking, Fast and Slow is on priming, so many of these studies are right up the front. These include the Florida effect, money priming, the idea that making a test harder to read can increase test results, and ego depletion (I touch on each of these in my recent talk at the Sydney Behavioural Economics and Behavioural Science Meetup).
It’s understandable that Kahneman was somewhat caught out by the replication crisis that has enveloped this literature. But what does not sit so well was the confidence with which Kahneman made his claims. For example, he wrote:
When I describe priming studies to audiences, the reaction is often disbelief . . . The idea you should focus on, however, is that disbelief is not an option. The results are not made up, nor are they statistical flukes. You have no choice but to accept that the major conclusions of these studies are true.
I am surprised at the blind spot I had when first reading it – Kahneman’s overconfidence didn’t register with me.
As I was also, Kahneman is a fan of the hot hand studies. Someone who believes in the hot hand believes that a sportsperson such as a basketball player is more likely to make a shot if they made their previous one. Kahneman wrote:
The hot hand is entirely in the eye of the beholders, who are consistently too quick to perceive order and causality in randomness. The hot hand is a massive and widespread cognitive illusion. [Could the same be said about much of the priming literature?]
The public reaction to this research is part of the story. The finding was picked up by the press because of its surprising conclusion, and the general response was disbelief. When the celebrated coach of the Boston Celtics, Red Auerbach, heard of Gilovich and his study, he responded, “Who is this guy? So he makes a study. I couldn’t care less.” The tendency to see patterns in randomness is overwhelming – certainly more impressive than a guy making a study.
And now it seems there is a hot hand. The finding that there was no hot hand the consequence of a statistical error (also covered in my recent talk). The disbelief was appropriate, and Auerbach did himself a favour by ignoring the study.
As I’ve picked on Dan Ariely for the way he talks about organ donation rates, here’s Kahneman on that same point: . . .