Mental Models 4 Life

Fairness in Machine Learning is a topic that I have been wanting to better understand for a little while now, and this blog post summarises what I learned from reading the Fairness and Machine Learning (FML) book by Solon Barocas, Moritz Hardt and Arvind Narayanan available at https://fairmlbook.org these past couple of days. (The book is highly recommended!)

A fundamental pillar of Fairness is the idea of Equality of Opportunity, which says that people with similar ability and ambition should be provided the opportunity / prospect to achieve similar outcomes. As one can expect, there are many nuances to that high-level statement, relating to interpretations of “similar ability and ambition”, “similar outcomes”, and “opportunity” / “prospect”, some of which are described in this article in the Stanford Encyclopedia of Philosophy: https://plato.stanford.edu/entries/equal-opportunity. For our limited purpose, we will adopt the three views described in the FML book:

• Narrow view, where the goal is to ensure that people who are similarly qualified for an opportunity have similar chances of obtaining it;
• Middle view, where the goal is similar to the narrow view but subject to the constraint that current differences in people’s qualifications that are (likely) due to past injustice(s) are proactively considered and adjusted for;
• Broad view, where the goal goes beyond how individual decisions are made and into questions on how we can design entire socio-economic systems to ensure people of equal ability and ambition are able to realise their potential equally well.

The Narrow view captures what many people meant by the term meritocracy, and the Middle and Broad views are what drive diversity, equity and inclusion (DEI) initiatives like affirmative action in workplaces and bigger issues like debt cancellations and reparations. (See, for example, The Case for Reparations by Ta-Nehisi Coates.)

We next look at how these different viewpoints manifest themselves in machine learning problems. There are three well-studied foundational fairness criteria in the supervised learning literature: Independence, Separation, and Sufficiency / Calibration. Suppose the goal is to learn a model from a set of labelled data . Denoting by the sensitive group attributes (e.g. gender, race, disability, etc), and the prediction variable, the Independence criterion says that the learned model has to satisfy , which means that , so knowing the value of A does not change the prediction R in anyway because the formula is equivalent to . In particular, for all , we have so to the extent that the prediction value R is used to determine allocation of resources, there is equality of opportunity between people regardless of their group membership.

Separation is a conditional version of Independence, where we ask that the learned model satisfy, for some variable W that is usually related to Y, the condition , which means or equivalently . In other words, given knowledge of W, the prediction value is not affected by the value of A. W is usually just the target variable Y, so in the case of binary classification problems, we insist that the false positive and false negative rates are both equal across the different sensitive groups. In some cases, we may only insist that the false positive rate or the false negative rate is equal across the different groups, in which case we can set W to be the variable or the variable as appropriate for the application and its social context.

In contrast, the concept of Sufficiency, which is closely related to the concept of calibrated estimations, is defined as the condition , which means , or equivalently . In other words, the probability of the actual outcome depends on the prediction value R and, further, knowing the value of A does not change the outcome probability given R.

The Independence or Separation condition is usually used to enforce the Middle view of equality of opportunity described above, and we can recognise that in the names of formal criteria like error-rate parity and Roemer’s normative criteria. In contrast, the Sufficiency / Calibration condition is usually used to enforce the Narrow view, and we can recognise that in the names of formal criteria like meritocratic fairness and calibration by group.

Importantly, the three fairness criteria are essentially incompatible with each other. In particular, Sufficiency and Separation (which captures Independence as a special case) cannot both be satisfied under most natural conditions. (This is essentially a consequence of elementary properties of conditional independence; see Chapter 3 of the FML book for details.) So we have to pick one in practice, but how do we do that?

In a way, the question of which fairness criterion (between Independence, Separation and Sufficiency) one should adopt in a machine learning problem depends essentially on whether one should / can take a Narrow or Middle / Broad view of equality of opportunity in the intended application, and that obviously is dependent on the social context.

BUT there are some computational considerations, mostly because the Sufficiency / Calibration condition is basically “free”, in the sense that standard (unconstrained) supervised learning, under mild technical conditions on the loss function and the adequacy of the available feature variables, will produce models that satisfy the Sufficiency / Calibration condition approximately, and the more accurate the resultant model is with respect to the ideal Bayesian estimator, the better the model satisfy the Sufficiency / Calibration condition. (See the paper Liu et al, The Implicit Fairness Criterion of Unconstrained Learning for details.)

So I think this means that in cases where the Narrow view and the Middle view are both acceptable / justifiable in an application, one should probably choose the Narrow view by default because you have a much higher chance of solving that problem better given the maturity of standard unconstrained supervised-learning theory and algorithms. And if you have to take the Middle view, it may still make sense to start with the most calibrated model you can learn using unconstrained supervised-learning algorithms, and then apply post-processing rules (like different thresholds on R for different values of A) to get to your desired Separation criterion, rather than start with a machine learning algorithm with the Separation criterion built-in. This is assuming the exact form of the Separation criterion for the intended application requires some trial-and-error to get right, which is probably not an unreasonable assumption when it comes to something as complicated as fairness in the real world.

Hope that helps.