Â鶹ÒùÔº

The late psychologist Daniel Kahneman once said if he could wave a magic wand and get rid of any bias, he would choose overconfidence. However, overconfidence is here to stay. In a in Psychological Science, researchers examined this bias among tournament chess players, to investigate if overconfidence is prevalent in an environment that should discourage it.

Chess is a fitting test bed for overconfidence research. "Both sides have access to the same universe of possibilities on the board in front of them," said Patrick Heck, a researcher at the Consumer Financial Protection Bureau's Office of Research and the study's lead author who spoke to the Observer in his personal capacity.

Heck and colleagues describe chess as an environment that should be inhospitable to overconfidence.

"We measured overconfidence in a domain where people receive regular, accurate, precise, and public information about their skill—a domain where people should have no reason to express excess confidence in their abilities other than genuinely overoptimistic beliefs," he explained.

To understand players' levels of confidence, the researchers surveyed about 3,000 chess players from three different chess tournaments and organizations and asked them to rate themselves. Those ratings were then compared with Elo ratings, a system for rating and ranking players, to assess a player's actual skill. They found that the players with the lowest Elo ratings (i.e. the less skilled players) showed the greatest overestimation of their abilities. Those with higher Elo ratings were more accurate.

One of the most surprising findings, Heck said, was the sheer magnitude of overconfidence, a difference of 89 Elo points—that difference objectively predicts victory on average for the higher rated player, Heck explained.

"If I was matched against someone who is 89 points higher than me—that's the overconfidence effect size—I would not feel good about that match," Heck said. "I would not feel good about being able to consistently win."

The findings of this study build upon a psychological phenomenon called the Dunning–Kruger effect, proposed in 1999. APS Fellow David Dunning, a professor at the University of Michigan and one of the researchers who first described the effect, said, "Those lacking expertise lack the expertise they need to recognize that they lack expertise." In other words, the less skilled don't know what they don't know, which can make them more likely to overestimate their ability in certain areas and situations.

The findings of the chess study illustrate this concept clearly: Those who had greater expertise were better at estimating their ability because they were dealing with fewer unknown unknowns than those who didn't.

Dunning and Carmen Sanchez, an assistant professor at the University of Illinois Urbana-Champaign, also study confidence and how it relates to skill level. Their study presented the "beginner's bubble" hypothesis. Briefly, those with no knowledge and experience in an area avoid overconfidence, but after just a little learning or experience develop a striking degree of it, until their skill level catches up with their confidence later.

Both studies demonstrate how prevalent overconfidence can be in everyday life. Sanchez pointed out that overconfidence can be especially concerning in the field of medicine and finances. She explained that a doctor who would be the least susceptible to the beginner's bubble is one with a lot of experience or one fresh out of medical school.

"The medical school doctor or the resident is probably a fine doctor because when they don't know something, they're going to go ask somebody," she said. "And the doctor with a lot of experience, they're going to know the answer."

Examples of overconfidence that apply to this research are not hard to find, and Heck suggests the concept may be a ubiquitous aspect of human experience. "We're sort of getting at this idea that overconfidence is or can be a universal or core part of human psychology," Heck said.

In Dunning's view, overconfidence may be unavoidable, but it can be managed if one is aware.

"My take is that overconfidence is actually inevitable if you think about it," he said. "Confidence is good, but confidence has to be managed before it becomes overconfidence that's going to hurt you."