Blind to their own blindness October 29, 2011Posted by Ezra Resnick in Economics, Reason.
In an excerpt from his new book, Thinking, Fast and Slow, Daniel Kahneman discusses “the illusion of validity”: not only do people (even professionals) make confident predictions in situations where they really don’t have enough information to do so; they continue to feel and act as if their predictions are valid even when they have been made aware that their past predictions performed little better than random guesses.
We are prone to think that the world is more regular and predictable than it really is, because our memory automatically and continuously maintains a story about what is going on, and because the rules of memory tend to make that story as coherent as possible and to suppress alternatives. Fast thinking is not prone to doubt.
The confidence we experience as we make a judgment is not a reasoned evaluation of the probability that it is right. Confidence is a feeling, one determined mostly by the coherence of the story and by the ease with which it comes to mind, even when the evidence for the story is sparse and unreliable. The bias toward coherence favors overconfidence. An individual who expresses high confidence probably has a good story, which may or may not be true.
One area which is especially prone to unfounded confidence is the stock market. Kahneman recounts how, in preparation for an invited talk at a firm of financial advisers, he analyzed their investment outcomes over a period of eight years. The firm naturally considered its advisers to be skilled professionals, and awarded annual bonuses based on performance. But Kahneman found that the year-to-year correlation in the ranking of advisers was basically zero — the kind of results you would expect from a dice-rolling contest. There was no long term consistency that would indicate differences in ability among advisers; the firm was rewarding luck as if it were skill. And of course, they continued to do so even after Kahneman presented his findings.
This doesn’t mean we should distrust all professionals. According to Kahneman, it is possible to develop true expertise in fields that provide good feedback on mistakes in a sufficiently regular environment, like medicine. But in general, we should not take expressions of high confidence at face value:
people come up with coherent stories and confident predictions even when they know little or nothing. Overconfidence arises because people are often blind to their own blindness.
So, now that you know about the illusion of validity, will you avoid it? Probably not. Kahneman predicts:
The confidence you will experience in your future judgments will not be diminished by what you just read, even if you believe every word.