Uta: As chair of the Royal Society’s Diversity Committee I have struggled with communicating the biases that enter into decisions made by selection committees. There is a strong commitment to increase diversity at the Royal Society at all levels, but nothing convinces scientists more than evidence. So it seemed a good idea to collect evidence on how group decisions are made. Not being a specialist in this area myself, I pleaded with Dan Bang and Chris Frith to write a review, and here I am asking them about what they found.
The review has taken over a year and refers to 203 publications. It just appeared in the Royal Society journal Open Science.
Dan, was this the first ever comprehensive review of the literature on decision-making in groups?
There are other excellent reviews which we refer to in our paper, but I think ours is the first to draw on computational modelling of decision-making, and to bring together findings from economics, psychology and cognitive neuroscience.
Our article is not a review in the classic sense, we do not calculate the effect size of different group interventions or treatments. Instead, we use a general model of decision-making as our starting point, and then ask how individuals and groups solve each of the relevant computations.
Uta: For some reason the question of how we make decisions has always been of great interest to experimental psychologists. Why?
For experimental psychologists decision-making is a very useful framework for studying all kinds of cognitive processes, particularly those we are not aware of. For example, my perception of the letter A is a decision because I choose this interpretation of what I see instead of any other, such as the number 4. So, decision-making is something we do all the time but are usually unaware of.
Uta: Why has there been a recent upsurge in studies on decision-making?
I think this is due to developments in computational modelling. Computational models allow us to break decision-making down into its component processes, generate predictions for what happens if we tweak one of the processes, and for understanding exactly where decision-making goes wrong or right. I like to think that experimental psychologists have gotten over their fear of maths and have begun to realise what computational approaches can offer.
Your favoured model is Bayesian Decision Theory (BDT). Can you explain what this is, briefly?
BDT describes how to make optimal decisions in an uncertain world. A key aspect is the integration of past experience (prior beliefs) and new evidence (likelihood). It turns out that many decisions, whether made by individuals or in groups, go wrong because of excessive reliance either on past experience or on new evidence.
Is there a mathematical formula telling you how to make a good decision?
BDT provides the formulas for finding the best action. But the calculations required are often intractable, or too complex and time consuming, to be useful in practice. We can think of the task of finding the best action as one of finding the highest peak in a hilly landscape. BDT requires that you know the contours of the entire landscape. An alternative ‘quick and dirty’ strategy is to visit just a few different points in the landscape; this is often sufficient to get a good idea of where the peak might be. This strategy is called sampling. We can sample from our memories of past actions; we can sample from the internet; we can read about the goodness of some of the most obvious actions; we can ask our friends for advice.
When is the right time to make a decision?
Somehow you have to recognise whether you have enough knowledge to make a good decision. Do I have taken enough samples or should I take more? If I stop sampling, I may miss a higher peak. If I go on sampling too long, my decision may come too late.
Uta: In the figure, we see a schematic landscape with different peaks and we see people who are ‘explorers’ or ‘exploiters’. I believe this distinction comes from the study of foraging behaviour in animals. If I know where apples are to be found, I exploit this knowledge by going to that tree. But eventually all the apples will have been eaten, and I need to explore to find a new tree. How does this play out in group decision-making?
Explorers and exploiters point us towards the advantages of group decision-making over individual decision-making. In many animals, such as honeybees, there are individual differences in the drive to explore or to exploit. Most bees in the hive are exploiters who go where they know that nectar bas been found. But about 4% are explorers (scouts) who look for new sources of nectar. The scouts can guide others to new sources of food. Among humans, it is perhaps the risk takers and sensation seekers who are the equivalent of these scouts.
Chris, since this is our ” socialminds” blog, can you explain in what way group decision-making is part of the social mind? Are there processes that you identified that are specifically social, rather than general cognitive processes?
The special social feature of group decision-making is that members of the group interact with one another. For example, we are typically unaware of our own biases, but very sensitive to those of others. By interacting with others we can discover our biases and try to overcome them. We prefer to justify our own immediate solution to a problem rather than considering other options. When we interact with others we may be forced to consider these other options.
Is there a downside to group discussions?
One example is that the biases of one individual might spread through the whole group. Another problem can occur with perceived confidence. When people make decision together, they will often discuss how confident they are in their choice. In many cases, better decisions will be made when the solution of the more confident person is chosen. This is because confidence correlates with competence. But there are exceptions. Group decisions might be dominated by a vociferous and confident individual who is also incompetent.
Some examples immediately spring to mind…
Dan, what do we know now from the research that you reviewed that we didn’t know before? Was there a finding that surprised you? Is it all common sense, really?
Well, common sense is always easier to spot in hindsight! What we have tried to do is to provide an evidence base for common sense.
I think the most surprising challenge for decision-makers is living with and accepting uncertainty. We hate uncertainty. So, even if you make the optimal decision, you still may not get a good result. Then, it’s easy to say: oh, I shouldn’t have taken this particular action, but the other action. But this is hindsight bias! Actually, given what you knew at the time, you did take the best action.
On the other hand, you might sometimes be lucky: you made a poor choice, but through random luck, there was a good outcome. In this case, you might fool yourself into thinking that you actually took the best action and receive praise from your colleagues.
So it was a new insight that choice and outcome have to be kept separate? Are there any practical implications?
Indeed – I think that innovation and exploration is often discouraged because people focus too much on outcomes. Especially in groups, people are too afraid to get it wrong. This is because they know they’ll be evaluated on outcome, and not on a careful consideration of the grounds for their choice. This is one important reason why groups tend to stick with business as usual. We feel much more regret after making an unusual choice that turns out to be wrong. When doing business as usual, others can’t point the finger at us.
Chris, what ideas are ripe for exploring further? What is the next question that the review urges to be studied?
We need to learn more about the role of group leaders. We can think of a group of people with different areas of expertise as a super-brain. The group members are like populations of neurons which perform different functions, but whose output is brought together to make sense of the world. The brain has solved the problem of competition for influence and relies on a central executive system to coordinate information processing. Group leaders need to emulate such an executive system to overcome some of the dangers associated with group decision-making. Here are some practical examples: applying the rule that each member can only speak for a fixed amount of time. Another is pointing out bias where it occurs.
Uta: Here is a puzzle for me. Equality bias is one of the causes you identified as potentially leading to poor decision-making in groups. Do we really assume that all members of our committee are equally good at decision-making? If we have such a bias, how does this square with our irrational tendency to feel that we are better drivers/teachers/parents than other people. Why has it not helped us towards more diversity?
Equality bias means that equal weight is given to all opinions. It is part of the more general belief that others are more similar to us than they really are. Since not all group members are of equal competence, the bias results in too much weight being given to the opinions of less competent members of the group. If anything, this bias should work in favour of diversity. The difficulty for achieving diversity arises because we feel much more comfortable with people who are really like ourselves. Our shared expectations make them easier to communicate with. But this does not necessarily lead us to make better decisions than we would have made by ourselves.
Uta: I am just speculating, but this equality bias makes me think that if we believe, rather egocentrically, that others are more similar to us than they really are, then we get an unpleasant jolt when we notice the discrepancy between assumed and actual similarities. However, the discomfort is worth it when it leads to better decision-making by pooling diverse opinions.
Dan, are you convinced by your review that groups really make better decisions than individuals?
Yes, absolutely. We discuss many benefits in the paper: more accurate knowledge from pooling of knowledge, better inferences from pooling cognitive resources, better coverage of hypothesis space from mix of exploiters and explorers. But checks and balances must be in place for groups to work.
Uta: What are the hottest recommendations for this committee’s work?
One new idea that is gaining strength is the use of lotteries in decision-making. When choosing which projects to support it is usually easy to pick out the good ones and the bad ones. But there will always be a middle range of equally good projects, only some of which can be supported. Selection at this stage could be done by a lottery. This would eliminate the effects of unconscious biases and would eliminate a lot of fruitless discussion.
Uta: The whole point of the review was to give evidence-based recommendations so that selection committees and panels can make better decisions. Therefore, I’m listing some of them here.
- Strive for diversity among group members, both in terms of expertise and background. This will allow you to explore the full hypothesis space and avoid narrowing of ideas.
- Don’t be afraid to weight opinions. Give more weight to the opinion that is based on better knowledge.
- Be sensitive to differences in expressed confidence. Talking time rarely correlates with expertise. Consider allocating a fixed speaking time to each member and enforcing a no-interruption rule, the latter benefitting people who have minority views, or who are in a minority.
- Ensure independence of opinions. For example, do not express a preference for candidates before everyone has discussed them. Opinions are subject to contagion effects.
- Avoid shared information bias. Do not focus discussion on information that everybody is familiar with, such as track record. Sometimes only one member has pertinent information. They may not reveal it as they assume others know too.
- Balance exploration and exploitation. Business as usual is not always the best decision. Changes can be beneficial even though there is risk attached.
- Appoint a meta-champion who keeps a check on process, is aware of the pitfalls and can point out common biases. Promoting independence among opinions is probably the most helpful service that the champion can provide.