Don’t know enough to know that they don’t know

| 183 Comments

Some of our more recent trolls have reminded me of an article, Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments, by Justin Kruger and David Dunning of Cornell University. Briefly, Kruger and Dunning demonstrated that college students who scored in the lowest quartile on several tests grossly overestimated their own abilities compared to everyone else’s, probably because they did not know enough to know that they did not know. Oddly, students in the highest quartile slightly underestimated their own abilities.

Figure 1 is a graph redrawn from the paper by Kruger and Dunning. It plots the students’ predicted scores on a particular test on the x-axis versus their actual scores on the y-axis. On average, 66 % of the students thought they were better than average at performing that test; in reality, only 50 % of the students could have been better than average. More interestingly, the students in the lowest quartile estimated their abilities at around the 60th percentile, whereas in fact they should have got the 12.5th percentile. By contrast, the students in the 4th quartile estimated their abilities at around the 75th percentile, whereas in fact they should have got the 87.5th percentile. Kruger and Dunning provide 3 more graphs that pertain to other tests, but they all look roughly the same. There is more, but I won’t bother you with it; let it suffice to say that the least competent students vastly overestimated their own abilities. Remind you of anyone?

UnskilledGraph_600.jpg

Figure 1. Students’ actual scores on a certain test versus their predicted scores. The weaker students substantially overestimate their abilities compared to everyone else’s, whereas the stronger students slightly underestimate theirs. [After Kruger and Dunning’s Figure 1.]

While researching this article, I discovered that this phenomenon is now known as the Dunning-Kruger effect. According to the Wikipedia article (the link), the effect may be specific to the United States or at least more pronounced in the United States. If I had to guess, I would blame the self-esteem movement.

The Wikipedia article further notes that 94 % of college professors rank their own work as above average in comparison to their peers’. Kruger and Dunning were awarded the Ig Nobel Prize in Psychology in 2000. No doubt they think their own work in their field is above average with respect to that of their peers. No doubt you and I think the same about our work.

183 Comments

Little googling here in Finland. Here people are repeating as a fact: 80% of car drivers think they are better drivers than others.

Another recent study (whose source I can’t remember at the moment) found that the people who thought they were the best at multi-tasking were actually the worst, when tested.

Again, US subjects. (A pattern emerging?)

I am intrigued that actual performance results in a straight line linear relationship of a kind best described by a simple linear regression. Shouldn’t surprise me - and it doesn’t - but am especially intrigued with this result. Of course none of the usual PT intellectually-challenged creos would understand this.

A look into Wikipedia for the “Lake Wobegone effect” (“where all the children are above average”) takes one to Illusory superiority.

No wonder ID people truly believe that ID is on the cutting edge of research (what research?) while neo-Darwinism is slowly dying. LOL!

“I am intrigued that actual performance results in a straight line linear relationship of a kind best described by a simple linear regression.”

Since they are plotting the average of the ranked percentile, it will be a dead straight line: the average percentile of the bottom 25% is 12.5%, the next 25% is 37.5%, etc.

And of course, when it comes to evolution, it’s even worse than that. Not only do the know-nothings think they know it all, but they consistently ignore the opinions of the experts. Of course the claim is always that they are “biased”. How hypocritical can you get? They have never even met these people, have no idea what they believe or why they believe it and yet they are willing to completely ignore the very people who have earned the right to an opinion. And of course they never admit their own bias, since that would obviously disqualify them by their own criteria.

You know, after reading this article, I made a concerted effort to self reflect and be honest on my skill levels in a variety of things.

One reason why this kind of thing is recognized is because it’s depressing. It’s almost painful to realize that one is merely average at some tasks and below average for almost everything else. True experts, those that are highly above average are, by definition rare.

In all but one case for every skill/task/knowledge area, there is someone better than you. And 1 out of 2 people that you meet will be better than you at some skill/task/knowledge area.

It is a truly humbling reflective period.

I try to look at things from another perspective. When I see a crap paper published in a high profile journal, I tell myself: “Well there’s at least some chance that they’ll publish my crap.”

Olorin said:

Another recent study (whose source I can’t remember at the moment) found that the people who thought they were the best at multi-tasking were actually the worst, when tested.

It’s either Windows OS or women, right?

in reality, only 50 % of the students could have been better than average.

That’s a little bit sloppy:

Almost every human has more eyes than average…

In defense of Dunning and Kruger, they actually cover this in their discussion, where they speculate (to rather humorous effect) that they might be as incompetent as those they mock, since they feel confident in their findings.

DiEb said:

in reality, only 50 % of the students could have been better than average.

That’s a little bit sloppy:

Almost every human has more eyes than average…

Cue George Carlin…

DS said:

And of course, when it comes to evolution, it’s even worse than that. Not only do the know-nothings think they know it all, but they consistently ignore the opinions of the experts. Of course the claim is always that they are “biased”. How hypocritical can you get? They have never even met these people, have no idea what they believe or why they believe it and yet they are willing to completely ignore the very people who have earned the right to an opinion. And of course they never admit their own bias, since that would obviously disqualify them by their own criteria.

Indeed, for creationists, knowledge=bias.

in reality, only 50 % of the students could have been better than average.

Yeah, you often get a chuckle from people when someone points out that over half of X thinks they are better than average. But there’s no reason that should be funny – for any left skewed distribution, the median is always larger than the mean, and so over half of the population is better than average.

OTOH, right-skewed distributions are usually more common, and then less than half of the population is better than average – the majority is below average.

The Founding Mothers said:

Olorin said:the people who thought they were the best at multi-tasking were actually the worst, when tested.

It’s either Windows OS or women, right?

Hell yeah! There’s just nothing funnier than a little witty misogyny.

…and the horse you rode in on.

I did not think that Dunning and Kruger needed any defense, and I hope no one has misinterpreted my last paragraph. I was essentially making the same joke that Mr. Monotreme cites, but I forgot that the authors had beaten me to it.

I was sort of assuming a symmetrical distribution and therefore blurring the distinction between the mean and the median. Presumably there is a lower limit to how incompetent you can be and still get into Cornell (at least I hope there is). There is, on the other hand, presumably no upper limit, so I would guess that distribution is in reality skewed to the right. If that is so, then, as Professor Theobald points out, the majority of the population is below average, because the highly competent outliers pull the average up without affecting the median. All that means, I suppose, is that the median is the proper statistic to use.

Douglas, you as such a pedant.

We used to use self-assessment where I used to work and I found that if I plotted my staff’s competencies against the self assessment the curve would actually go downward.

I think this is because you farm out the high risk stuff to the competent and farm out the drudge work to the incompetent.

“Ignorance more frequently begets confidence than does knowledge”

This is ubiquitous in humanity. Dunning Krueger go into the condition analogous to other medical syndromes where the patients flat out is in denial about their disabilities. Production and recognition are the same skill set.

if you cant taste sugar, you will have no idea sugar is in a cake, and never bake it right

if you are tone deaf and can’t understand a C# from C you will never sing it right, falsely assume you are correct.

I have studied animation and in college nothing confused us more than people who thought nothing about their work was poor. They worked for a few hours to created imagery/motion that was so pathetic we would all give up during critiques, say nothing, and just let them waste that student loan for 4+ years SAD!

From an evolutionary stand point it makes sense to me, If you know you can do it, you use your brain… and if you can’t at least you’re too dumb to think you can’t, so you try anyway.. pass or fail.

It is probably good that youth has such ignorant confidence, to just act out and make a mark amongst your peers in your early teens is important. Establish dominance and be dead by 30

Sports ad’s center on “just do it” another one I see read’s “smart has the brains but dumb has the balls”

humanity 101 imo.

And that is a linear relationship. I might have expected something else given human nature, but the results are best explained by a simple linear regression:

Divalent said:

“I am intrigued that actual performance results in a straight line linear relationship of a kind best described by a simple linear regression.”

Since they are plotting the average of the ranked percentile, it will be a dead straight line: the average percentile of the bottom 25% is 12.5%, the next 25% is 37.5%, etc.

Except of course in the Journal of Irreproducible Results:

Karen S. said:

No wonder ID people truly believe that ID is on the cutting edge of research (what research?) while neo-Darwinism is slowly dying. LOL!

What is really sad is you see this ‘balls “greater than” brains’ in discussions about evolution and also climate science.

More informed or experienced people give the confidant+stupid people benefit of the doubt, naively assuming they have something to contribute.

In news organizations/reporting it is even worse, as they will give “equal” time to a different point of view, even if that point of view is shared by an inequivalent amount of information or by a small yet cantankerous/vocal group.

OgreMkV said: One reason why this kind of thing is recognized is because it’s depressing. It’s almost painful to realize that one is merely average at some tasks and below average for almost everything else.

Indeed, studies have also shown that depressed people tend to make more accurate self-assessments than non-depressed people.

Blessed are the stupid, for they can’t realize it.

My wife and I agree on this: the OTHER one of us is definitely NOT in the 90% of drivers who are above average.

I wish that Dunning-Krueger were all that underlies creationism, but the sinister aspect of creationism is that it claims that intellectual dishonesty is intellectual honesty, and that appalling lies are shining verities.

We all know this, I’m just stating it.

They are seriously trained to misunderstand evidence, to make arbitrary and absolute lines between “microevolution” and “macroevolution” (quote marks denote the sliding, various, and almost always indefinite notions creationists have about those), and to deny the meaning of evidence that is accepted everywhere else–for instance, when they accept the evidence of common descent in their amorphous “microevolution.”

And this is all supposed to be better than what “godless scientists” believe, hence their tiny “knowledge” constituted almost wholly of lies and/or distortion, becomes “higher knowledge” than are the results of honest education and thought.

Like ignorance, the ignorance of ignorance is something that is, in principle, curable. The counter-“knowledge” and anti-scientific views of the creationist is sometimes curable (not all are ineluctably close-minded) but often is not.

So Dunning-Krueger matters in creationism as well as elsewhere, but the problem is so much worse when false claims and dishonest “interpretations” actually come to be favored over a preference for honest methods and meaningful facts. I don’t mean to detract from the importance of the ignorance that the ignorant have of their ignorance, no question, rather to point out that creationism is so much more resistant than is ordinary ignorance.

Glen Davidson

I find the ‘know nothings’ fall into either the category mentioned here, or the other category:

People who would lose money if their ideas were overturned. These folks are often smart enough to know they are defrauding, yet smart enough to keep pushing their memes.

It’s the second group that worries me.

***Sorry - I ruled the rest of this comment out of order. – Matt***

Karen S. said:

No wonder ID people truly believe that ID is on the cutting edge of research (what research?) while neo-Darwinism is slowly dying. LOL!

I strongly suepect that almost no one truly believes that ID is on the cutting edge of research. The ones peddling it know that they are both avoiding relevant research (apparently aware that it will fail), and playing word games (“we don’t need to connect no stinkin’ dots”) to fool nonscientists. Meanwhile those nonscientists don’t have a clue what research involves, and mostly don’t care, as long as someone tells them what they want to hear. And as I have been saying for years, all they have to be told is that “Darwinism” is “weak” and they fill in the blank to assume that their particular fairy tale - which often contradicts that of their neighbor - is somehow validated. I think of ID not as “creationism lite,” but as “creationism xtreme.”

I call it the Kruger-Dunning effect because it rolls off the tongue easier than Dunning-Kruger (and it’s the author order on the original paper). I also think it is a truly monumental paper with a lot more to it than just the take-home soundbite. It’s really worth reading in full.

One of the *good* things to take from it is that high-performers tended to *under*-estimate their competence. So if you are good at your job, you may be better at it than you think.

Another interesting observation, not often commented on, is that, “Paradoxically, improving the skills of participants, and thus increasing their metacognitive competence, helped them recognize the limitations of their abilities.” This seems obvious, but at least it means that there is some hope that people can be educated about the deficiencies in their knowledge. This is where, I am afraid, I have to agree completely with Glen Davidson. The problem with creationism is not merely ignorance and lack of self-awareness, but wilful construction of anti-rational strategies. This it has in common with vaccine alarmism and 9/11 “Truthers”.

Chris Lawson said: So if you are good at your job, you may be better at it than you think.

I recall hearing of exceptionally good performers at some task who have the fear of not being all that good. In the extreme, that they can have a secret feeling that their work is trivial.

And thus Sarah Palin is explained. Has there ever been a more self-confident ignoramous?

DS said:

Alright Kris, perhaps we have been too hard on you. Allow me to help by trying to reword what you are trying to say.

As I understand it, your main point is that “someone” might be so stupid, ignorant and lazy that they might find some excuse to deny the theory of evolution, despite all of the evidence. “Someone” might refuse to believe in evolution unless you could prove to their satisfaction that science had all of the answers to life, the universe and everything, up to and including the big bang and abiogenesis. And this cannot be allowed because that “someone” might cause some kind of problem for science some day, despite the fact that they are lazy, ignorant and stupid. Do I have that right?

OK. So what should we do about it? Here, I’ll make it a multiple choice question, just to avoid any further misunderstandings:

1) Ignore them and hope they die an early death due to their laziness, stupidity and ignorance.

2) Try to educate them that science does not have all of the answers and that all answers in science are provisional and that if they want certainty they should go somewhere else.

3) Try to educate them that there is ample evidence to demonstrate conclusively that evolution did in fact produce the diversity of life we see on earth, regardless of the validity of any other theories.

If you still refuse to answer and persist in your obnoxious and pointless behavior, I see no reason for anyone to continue responding to you.

I was hoping that you might actually get it but you’re still reading things into what I’ve said and the questions I posed.

And speaking of demanding, your last paragraph is a clear example of ‘or else’ with an insult thrown in too.

I’m off to more productive things, at least for now.

I think I did get it. Good bye for good.

Anyone with a clue knows what I meant.

And THIS is how you respond to the idea of, you know, evidence and detail and dynamic systems and the nature of measurement and all that other useless stuff science wastes its time with.

The only obvious reason that you or anyone else is pissed off by what I’ve said is because you are filled with massive, uncontrolled, reality distorting hatred of creationism, AND anyone who doesn’t kiss your ass simply because you’re on the side of science in your “war”, AND anyone who doesn’t say anything but good things about science.

No need to keep illustrating how creationists think, and how they can’t help but expect everyone else to think. We’ve heard it all before, many times. But there’s always the hope that someone, someday, will reflect on his need to demonize his enemies rather than respond to what they’re saying. You fail, but you’re typical.

(And if you admire what science “really is” as you say, you might construct an experiment to detect, and maybe even measure, the “reality distorting hatred” you think you see. You might even suggest what an intersubjectively valid experiment might look like. That is, if you understand words like “detect” and “measure” and “valid” and “experiment” and such. Care to give it a try?)

About this Entry

This page contains a single entry by Matt Young published on November 21, 2010 12:05 PM.

Mapping fitness: landscapes, topographic maps, and Seattle was the previous entry in this blog.

New Issue of Evolution: Education and Outreach is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.

Categories

Archives

Author Archives

Powered by Movable Type 4.381

Site Meter