With so much of what we do these days guided by ‘expert’ judgements, I recently found myself reflection on the nature of expertise. Specifically, what exactly does it mean to be an expert?
The nature of expertise
If we were to try and define an expert, we’d probably say something like “somebody who has a broad and deep competence (or is capable of exceptionally high levels of performance) in terms of knowledge, skill and experience through practice and education within a given domain”. Sometimes we even put a spotlight on individuals through ascribing them labels such as guru, virtuoso, master or genius.
We all identify with people who we might consider to be leadership guru’s, master sports people or virtuoso musicians. We listen to them, they guide our thinking.
What do you think an expert is?
- Do they know more about a given subject than anyone else?
- Have they acquired unique skills, capabilities or knowledge that other people can’t emulate?
- What they do is perceived to be leading edge…?
Given ‘experts’ are now ubiquitous in our world, from expert witnesses in court to scientific expertise guiding societal responses to pandemics, should we really accept the notion of expertise without challenge?
Expertise is also highly transient these days, with multi disciplinary teams working on the same issue world-wide, most expertise is simply ‘point in time’ knowledge that ages quickly…
Why might expertise be a problem?
This was a question that Philip Tetlock started to ponder back in 1984. Tetlock started to explore how expertise and good judgement were related. Consequently he assembled a group of 284 ‘experts’ who were all involved in commenting or advising on political and economic matters. They were from a variety of fields, including government officials, professors, journalists, and others, with a broad range of backgrounds. He asked them to make specific, quantifiable forecasts and then waited to see if their forecasts came true.
The results were interesting. Tetlock found that the forecasters were often only slightly more accurate than chance, and usually worse than basic extrapolation algorithms, especially on longer–range forecasts three to five years out.
Tim Harford in his book Adapt (2011), suggests that the expertise wasn’t entirely useless as the expert forecasts were better than a control group of undergraduates. However Harford observes that:
The return on expertise was distinctly limited.
Harford cites that once experts have acquired a broad knowledge of the political world, deeper expertise in a specific field doesn’t seem to help much. For example, predictions about Russia from experts on Russia were no more accurate than predictions about Russia from experts on Canada.
Why might expert insight be self-limiting?
We all know that the vast majority of organisations are metrics driven. There is nothing wrong with the experts – even though the results of Tetlocks experiments might have been disappointing for them. Clearly experts have insights to contribute its just that in the overall scheme of things these insights might only go so far. The real challenge is the complexity of the world that we now live in means that the problems we have to solve go well beyond the limits of expertise in any given field.
Consequently, if you need to resort to expertise to solve a broad and complex problem, ask yourself:
- Is the expert input broad enough to give us the right perspective?
- Do we have sufficient conflicting voices to enable us to robustly challenge both the problem and the advice?
Nothing wrong with specific expertise and it is absolutely relevant – such as getting a legal opinion on a contract dispute for example. Some things to consider:
- Do we have enough dissenting voices? Before Coperniucs postulated that the earth revolved around the sun, the expert view was that the earth was at the center of the universe… and idea you’d find very hard to get anyone to disagree with today but was accepted doctrine amongst experts for centuries.
- Are you giving in to confirmation bias? Simply listening to those views that support your thinking rather than robustly examining all of the evidence for and against.
- Do the ‘experts’ have a vested interest in the outcome? Are you genuinely getting independent un-biased advice, or does your expert have an ‘agenda’. Agenda’s can be anything from ego (self-interest) through to a desire for financial reward or to be in the limelight (see below).
- Is the expertise sufficient to address the complexity of the issue? An observation is that during World War 2, economists were part of Churchill’s body of advisors as he recognised that economic considerations had to be part of addressing the national challenge. Is this something we are incorporating in our thinking about Covid today? Seems as if there field of expertise offering the advice is quite narrow, so perhaps no.
- How current is your experts knowledge?
There are no cookie cutter solutions. The nature of complex problems is such that they are all difficult and distinctly different from each other. In which case the point is not to be pro or anti expertise, it is to be properly critical about how well the advice ‘maps’ to the presenting issue. We may be out of our depth in terms of specific knowledge, but good leaders always instinctively challenge the context and accuracy of what they are being told. Do you do enough of this yourself? What perspectives are you overlooking?
One final thought
Sometimes you do have to work away from something, but this should only be a temporary solution. Otherwise if you don’t know where you are headed “any road will do”. Its quite a prescient question to think about at
It can be as dangerous to make decisions without expert opinion as it can be to make them with expert opinion where you haven’t considered the complexity of the problem.
However, forecasters with the biggest news media profiles in Tetlocks study produced some of the worst forecasts. This work suggests that there is an inverse relationship between fame and accuracy. Now think how much of public policy and societal norms are driven by ‘experts in the public eye’. Make you wonder what else we take for granted…
If you want to explore what you might be taking for granted or to #changesomething, why not give us a call?