Fighting bias with quota and lottery

Before you cry out “quota? – no way!” imagine you have the power to decide which of the wonderful girls in the Genius school can enter the charmed circle. Your first impulse might be to select students who are most like the existing members of the circle, the prototypical candidate. On second thoughts, you might wonder which talents and skills the circle is currently missing out on. It is obvious that a diversity of skills would be most advantageous for its future projects and adventures. Selection is a stressful task, because, if you selected the wrong student, the circle will weaken. At the same time your reputation as a shrewd selector would suffer.

Diversity is a necessity, given the changing demographics in our societies, but it also brings benefits. There is a business case, and there is evidence that science thrives in diverse teams. Both the number of hypotheses and the number of solutions are increased if groups working on a problem have different perspectives and contribute different points of view. A diversity of viewpoints helps us to avoid being stuck in a rut. Clearly, charmed circles, such as academy members, grant holders and award winners, need to be diverse if we have any hope of solving the pressing problems of our times.

Increasing diversity is not easy and the previous post mentions some of the hidden biases that make it an uphill struggle. But we should not feel defensive about this. Remember, our social brain is processing vast amounts of information in split seconds and therefore has short cuts and preference settings. These lead us straight back into the same old choices. If we want greater diversity, we need to interrupt this processing flow. There would never be any change if we didn’t. To become aware of hidden biases is one way to achieve this, but setting quota could be a better way.

Lets talk about Quota

With the charmed circle at Genius school it would make sense to have a quota for missing skills. You could also insist on a quota for boys, in case the school decides to become co-ed in the future. With modern selection panels the same applies. Under-represented minorities have special skills and preferences that go to waste because they are not in the circle. What’s sad is that the existing members may not even realise that this is their loss.

Quota, in theory, can easily be applied, as long as there is a pool of excellent candidates. But there is a snag. In selection for excellence, who would like to think that the reason they were selected was because of quota? This is not exactly bolstering their self-esteem! It is also a reason for feeling denigrated if you are not selected. Some will feel that they should have been selected by rights, but did not have the required minority status. Everyone will feel demeaned.

The situation changes when we take into account the size of the charmed circle. If there is only one winner, quota setting will probably not work. But it can work well, when more candidates have to be attracted. For instance, setting a quota for boys when the school becomes co-ed. New people with different ideas and different skills can be selected as soon as a quota is adopted. Quota here is strictly for the purpose of building capacity. It says ‘we are open for you – even though we weren’t open to you before – please apply’.

It follows that my personal view on quota is twofold: At this point I would probably not endorse quota when the selection is about giving rare awards. In contrast, I would endorse quota when the selection is about capacity building. The most likely scenario here is that there are many candidates and the number of awardees is somewhat flexible. This is different from the typical scenario of rare awards. When there are many excellent candidates, but, for whatever reason their names did not come to mind and they were not put forward, then a strong case for quota can be made. One practical move adopted by some learned societies is to add placeholders for individuals from a desired group (e.g. underrepresented minorities) without touching the historical annual quota for new members. This is capacity building because it means that additional members can be elected as long as they are from an underrepresented group. Clever. Nobody loses, and the institution wins praise for its willingness to change.

Even if there are no set quota, beware that as a selector, you will get eggs thrown at you if you systematically exclude minority members. There has been a “softly, softly” approach that avoids setting quota but strives for gender balance, and this has been quite successful in many areas, where charmed circles have been broken into. Speaker lists at conferences that do not have a reasonable proportion of women are rightly condemned. Academies whose membership remains stubbornly ‘senior white male’ have come in for criticism and even derision. Reputation is tied in with the willingness to modernise, not with continuing old traditions. Fortunately, the desire for a good reputation is such a strong motivation that changes in selection practice follow of their own accord even without the formal need to set quota.

Lets talk about using a lottery element

Instead of making impossibly difficult decisions, why not choose by lottery? This can make sense when the candidates are of equal merit, and in this case, a lottery is the one tool we have to definitely rule out any hidden bias. Juries in civil and criminal courts are selected randomly as a first step in the process. City states and universities have employed lottery procedures (also known as sortition) with the effect of minimising corruption and avoiding conflict. Famous examples include Athens, Venice and Basel. In these historical cases governing elites opened up to individual newcomers, but avoided being taken over chaotically. In addition, if an individual was rejected by lottery their honour was protected.

As a candidate, how would you feel? If you are selected you will feel lucky, but you should not feel big-headed; if your are not selected, you will feel unlucky, but should not feel disparaged. As a selector you would probably feel confused! Your job after all is to discriminate between candidates – the very opposite of leaving selection to chance. UsIng chance seems like abandoning reasoned argument. But sometimes comparisons between candidates are impossibly difficult. Here, bias is likely to gain the upper hand. The trigger-happy unconscious mind will always nudge the dithering conscious mind. Post-hoc justification of intuitive choices is not a triumph of reason.

Here again there is a practical move, already adopted experimentally by some funders of grant applications: a lottery element is used as part of the usual procedure of discussion among panel members. Panels often agree on their top candidates who clearly stand out from the rest. They also usually agree on candidates who are not competitive. This leaves a middle field where there is less certainty. Through discussion consensus can sometimes be reached which allows separating the stronger from the weaker cases in this middle field. But there is often not enough funding for all the strong candidates, and those just below the top tier will be the subject of much discussion which leaves selectors often uneasy about their final choices. This is where a lottery could be the tool we have been waiting for. It is the one tool we have to avoid bias and guarantee fairness.



2 thoughts on “Fighting bias with quota and lottery

  1. Please can you explain exactly what a placeholder system is, and teh crucial difference between it and a quota system?

    I cannot see how it does not suffer from the same problem as a quota system in terms of the winner feeling demeaned because they were selected via a less competitive route.

    What am I missing here?

  2. Thank you, Ros, your helpful comments made me think.

    Why do I not reject quota outright?
    Why should traditional and novel entrants not compete with each other directly ensuring a stringent selection process? Only then would novel entrants not feel demeaned. While this procedure is being used by progressive selection committees, the pace of change has been disappointingly slow. A quota system might speed up the process.

    What’s the difference between the quota and the placeholder system?
    In the case of quota the overall number of places stays the same but a decision has to be made about the right proportion of newly desired members. This is not a trivial matter. It might be aspirational or practical, depending on estimates of the size of the potential pool of novel entrants. Because the overall number remains fixed, there will be losers among the traditional candidates. An increase in numbers is often resisted as this might be perceived as devaluation.

    In the placeholder case, the members of the new type are added on top, and hence the overall number selected each year will become variable. Traditional entrants will not feel they are losing out, since the same number are selected as before. The use of an unspecified number of additional places fits with the idea that the pool size is often unknown and that growth is rarely smooth and linear.

    Of course, in all cases all candidates, traditional or novel, have to meet the required standards. I prefer the placeholder system because it is more flexible while gently changing the traditional selection procedure. Thus, there is less fear of a backlash.

Leave a Reply

Your email address will not be published. Required fields are marked *