What Is Algorithmic Bias?

Question

What is algorithmic bias, and why has it become one of the defining issues of the AI age? The question arises because algorithms are now involved in decisions of enormous consequence — who gets hired, who gets credit, who gets bail, who gets flagged for surveillance — and the evidence is growing that these systems systematically disadvantage particular groups. The problem is not that algorithms are biased in the way people are biased; it is that they reproduce, amplify, and hide bias behind a screen of mathematical objectivity.

Quick Answer

Algorithmic bias is the systematic unfairness produced by algorithmic systems. It arises when an algorithm's outputs are systematically skewed against certain groups — by race, gender, class, or other protected characteristics — even when the algorithm's designers did not intend it. The sources are threefold: data bias (the training data encodes historical discrimination, so the algorithm learns it), design bias (the choices of what to measure, what to predict, and how to evaluate encode values and assumptions), and deployment bias (the system is used in settings or on populations for which it was not validated). Because these systems are presented as objective, their biases are harder to detect and contest than human biases — and because they operate at scale, their harms are distributed at scale. The philosophical problem beneath the technical one is that fairness is not a single, well-defined quantity: different formal criteria of fairness conflict, and any "fair" algorithm embodies a contested judgment about what fairness requires.

Historical Wisdom

The problem of algorithmic bias has deep roots in the longer history of justice and measurement. The question "what does equal treatment require?" is as old as Aristotle, who distinguished formal equality (treating equals equally) from distributive justice (the contested question of who counts as equal and what they are owed). The modern history of the problem begins with the use of statistical models and scoring systems in the twentieth century: credit scoring, actuarial tables, and standardized testing all raised the question of whether quantitative methods could be fair or merely appeared to be. The civil rights era added the crucial legal concept: disparate impact — a practice can be discriminatory in its effects even without discriminatory intent. Algorithmic bias is the contemporary form of this ancient problem: the arithmetic is new, the question of justice is not.

Philosophical Perspectives

The philosophy of algorithmic fairness has three central topics. The first is the diagnosis of bias. Algorithms are not neutral mirrors of data: the data itself is a social product, shaped by the decisions and discriminations of the past. An algorithm trained on historical hiring data learns historical hiring practices, including their biases; an algorithm that predicts crime from arrest records learns the patterns of policing, not the patterns of crime. The "garbage in, garbage out" framing is too weak: the problem is not dirty data but the fact that the data is a record of a society that is itself unjust, and the algorithm inherits the injustice.

The second is the definition of fairness. The formal literature has produced several precise criteria: demographic parity (the rate of positive outcomes should be equal across groups), equalized odds (the error rates should be equal across groups), and calibration (the scores should mean the same thing across groups). The striking result is an impossibility theorem: these criteria cannot all be satisfied simultaneously except in trivial cases. Fairness is therefore not a quantity to be computed but a value to be chosen — and the choice of a fairness criterion is a moral and political decision, not a technical one. This is why the philosophy of algorithmic fairness is not a sub-discipline of computer science but a branch of the philosophy of justice.

The third is the ethics of automated decision-making. When an algorithm decides, who is responsible for its decisions? The human decision-maker can be questioned, can explain, can be held accountable; the algorithm often cannot. The opacity of machine learning systems — the difficulty of saying why a particular decision was made — creates a new form of accountability gap. The emerging consensus is that automated decisions require transparency, contestability, and oversight: the people affected by algorithmic decisions must be able to understand them, challenge them, and hold someone responsible for them.

Lessons From Thinkers

The algorithmic bias literature yields several lessons. From Cathy O'Neil's Weapons of Math Destruction: the most dangerous algorithms are the opaque, consequential, and poorly regulated ones — "weapons of math destruction" — that entrench inequality while claiming objectivity. From the fairness literature: there is no free lunch — every fairness criterion has costs, and the choice of criterion is a political choice that must be made openly. From the data literature: the past is not a neutral teacher — an algorithm trained on the past will reproduce the past, including its injustices. And from the philosophy of justice: algorithmic fairness is ultimately about power — who builds the systems, who defines their objectives, who benefits from their outputs, and who bears their harms.

Practical Application

Algorithmic bias has direct practical implications for the design and governance of AI systems. In practice, mitigation requires: auditing training data for bias and historical discrimination; testing systems for disparate impact across groups before deployment; documenting the assumptions and objectives built into the system; building transparency and contestability into the decision process; and establishing governance structures that represent the affected communities. Regulatory frameworks — from the EU's AI Act to sectoral rules in hiring, credit, and criminal justice — are beginning to require these measures. For individuals, the practical lesson is to recognize that algorithmic decisions are human decisions in disguise: they encode choices, and those choices can and should be questioned.

Quotes

Cathy O'Neil names the danger in Weapons of Math Destruction: "Models are opinions embedded in mathematics." The fairness literature states the impossibility result plainly: "Fairness is not one thing — it is a contested value, and different definitions conflict." And the practical warning of the field is captured in the observation that "the algorithm is not the problem; the problem is what we ask the algorithm to optimize, and the data we give it."

Related Questions

• What Is the Alignment Problem? — the broader challenge of making AI serve human values.
• What Is Machine Ethics? — how machines should behave toward humans.
• What Is Justice? — the concept algorithmic fairness applies.
• What Is Distributive Justice? — how benefits and burdens should be shared, including by algorithms.