General Musing

blaze your trail

Peer assessment and the Dunning-Kruger effect #hr #management

leave a comment »

What about it?

Whilst having discussions on a ‘thinkers’ board that I infrequently-frequently visit, someone mentioned the Dunning-Kruger effect in relation to specific politicians.

So I looked it up and something started to dawn on me.

Of late I have seen quite a number of companies adapting the ‘Google method’ of peer assessment when it comes to hiring new personnel, but for some reason those companies were having rather a decline in technical competence instead of getting the increased benefit of adding more ‘brainpower’.

As I understand it, and as related to my own observations in the peer assessments, the problem lies here in points 2 and 3 of the hypothesis put forth by Kruger and Dunning:

2) Fail to recognize genuine skill in others;

Oftentimes the higher skilled candidate is being dismissed because “he talks about weird things and can’t communicate properly” or variations thereof.

Now what I have seen Human Resources do is not recognizing this problem but rather projecting a form of ‘insecurity’ on the assessing employee ‘He may be afraid of his position’, whereupon they start complementing and ‘securing’ the assessing employee. It would go too far too add Pavlovian conditioning to this story, but it may not be too far from it.

3) Fail to recognize the extremity of their inadequacy;

Dunning has drawn an analogy to someone having an impairment, but I think a much clearer and less insulting explanation can be given by the concepts of Flatland (when read without the Victorian context, that is).

What does that mean for the business?

As for the origin of the inadeqacies, I leave that for the respective physicians and psychologists, but there are a few common mistakes companies make that help in attenuating this effect within their ranks.

The HR departments are not sitting at the table when the peer assessments take place

This has the effect of ‘ganging up’ on a potential candidate; remember that the assessing employees will subconsciously defend their comfort zone, so no fresh blood that dares to challenge even the group of ‘old hands’ will ever be given a good mark.

And in technology, zealotism is stronger than in religion. Mention the wrong editor somewhere and you are classed ‘unqualified’. Mention that you do not have a fascist adherence to Linux/MacOS/Windows/Anythingatall and you are classed ‘incompetent’.

The way to remedy this is to have a properly prepared (as in read up, albeit cursorily, on the subject matter) HR staff sit at the table and support the candidate in matters of confidence and to ‘call off the hounds’ when needed. Also, the HR staff should ask questions like ‘why is this editor thing important in our company?’ so as to prevent the technology policies becoming the pre-conditions for a personal playground of the techs.

Sitting personnel has gotten to the position on merits of ‘employment years’ and ‘being there first’ or ‘helping to set it up’

Often, because ‘in the land of the blind, one-eye is king’, a manager or ‘chief’ of a technical department is someone that has been a long time in a company. This is a tradition that stems from the old ‘Foreman’ habit; a senior gets to lead his peers because he knows very well what the work entails and he knows the peers very well.

But in ICT, I really have to say this, a lot of technically competent people have problems when interacting with the rest of the world. I will even go so far as to say that some of these people are in ICT because of their Rainman-like qualities; they simply are prone to defend against anything that threatens the world they have created in their own mind.

This can be remedied by not giving them decision powers. They should have all the execution powers, or put differently; they should be allowed to decide on ‘how’ to do things, but never ‘what’ to do.

Here then comes a gray area; it is sometimes hard to see when it is a ‘how’ and when it is a ‘what’. But there’s a good rule of thumb for it (this is just a marker, not a hard fact): if it involves anyone outside of the inhouse technical crew, treat it as a ‘what’. All activities done by that technical crew can be treated as ‘how’.

Obviously it never is going to be that simple, but think of this as having a race-horse pulling a gurney; the horse pretty well knows how to run by itself and given practice it even knows how to turn etc. But because the jockey has more information (i.e. the strategy, the strength and endurance of the competition) and has the ability to make judgements on that information (i.e. if the others are conserving energy, if the other horses are at their peak, when to fully go all-out) it must always be the jockey who is in control.

The horse can do things the jockey can not, but the jockey can do things the horse can not. And if the horse decides that it knows the course better than the jockey, the race will be lost most of the time.

The horse does not see it’s inadequacy in decision making, because it can outrun that little jockey even on a bad day…

And now what?

Read more articles on

Well, just because they know more about technology and about the work they do, that does not mean they know more about healthy and proper assessment procedures.

When assessing new personnel, have the tech department set up a kind of exam with a scoring method. That way they can ask anything they want and open questions can be scored ‘double blind’ if wanted (although simply anonymous is usually good enough).

This test can then be sent to an outside consultant or other tech company to verify both the validity of questions and the standard answers.

You can have candidates (give them fair warning though that you are going to do this) take this test and have it objectively scored. This makes for an up-to-date questioning and it also gives the candidate the possibility to defend his/her answers against the scoring because it can be done fully in writing. Sometimes that will yield that the candidate is overqualified for a certain setting. But that leaves the candidates dignity in place and gives the tech department a chance to work on themselves.

The HR staff can assess the social qualities and all other properties after a candidate has gotten through the test.

I sometimes hear that ‘it is hard getting good personnel’, but I do not think so. I think it is hard breaking down the little kingdoms that have come to be and that in an open and social world, there really should be no place for them anymore.

This article is a guest written article, and was originally posted here.

Image source: AttributionNo Derivative WorksSome rights reserved by Sebastian Fritzon

Written by Adrianus Warmenhoven

April 24, 2012 at 11:48 am

Posted in business, risk, science

Tagged with ,

Please Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: