Don't know enough to know that they don't know

By Matt Young

Some of our more recent trolls have reminded me of an article, Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments, by Justin Kruger and David Dunning of Cornell University. Briefly, Kruger and Dunning demonstrated that college students who scored in the lowest quartile on several tests grossly overestimated their own abilities compared to everyone else’s, probably because they did not know enough to know that they did not know. Oddly, students in the highest quartile slightly underestimated their own abilities.

Figure 1 is a graph redrawn from the paper by Kruger and Dunning. It plots the students’ predicted scores on a particular test on the x-axis versus their actual scores on the y-axis. On average, 66 % of the students thought they were better than average at performing that test; in reality, only 50 % of the students could have been better than average. More interestingly, the students in the lowest quartile estimated their abilities at around the 60th percentile, whereas in fact they should have got the 12.5th percentile. By contrast, the students in the 4th quartile estimated their abilities at around the 75th percentile, whereas in fact they should have got the 87.5th percentile. Kruger and Dunning provide 3 more graphs that pertain to other tests, but they all look roughly the same. There is more, but I won’t bother you with it; let it suffice to say that the least competent students vastly overestimated their own abilities. Remind you of anyone?

UnskilledGraph_600.jpg
Figure 1. Students’ actual scores on a certain test versus their predicted scores. The weaker students substantially overestimate their abilities compared to everyone else’s, whereas the stronger students slightly underestimate theirs. [After Kruger and Dunning’s Figure 1.]

While researching this article, I discovered that this phenomenon is now known as the Dunning-Kruger effect. According to the Wikipedia article (the link), the effect may be specific to the United States or at least more pronounced in the United States. If I had to guess, I would blame the self-esteem movement.

The Wikipedia article further notes that 94 % of college professors rank their own work as above average in comparison to their peers’. Kruger and Dunning were awarded the Ig Nobel Prize in Psychology in 2000. No doubt they think their own work in their field is above average with respect to that of their peers. No doubt you and I think the same about our work.