Cognitive Biases · Belief

Algorithmic bias

Also called: Proxy bias, Biased training data, Label bias

A model repeats or widens unfair patterns in its data, often because it predicts a stand-in measure, such as past spending, in place of the real goal, such as need.

Evidence grade: Strong. How we grade evidence

For example

A county uses a model to decide which households get priority for help. The model predicts past service use, and households that had trouble reaching services show low use, so they rank low.

DECIDE BETTER® · Rule 3Data answers your questionEvery data set answers somebody's question. Make sure it's yours, or you end up precisely wrong and only generally right.

A caution

A property of data and models. Ask what the model was trained to predict, and on whom.

Spot it in your week

Three quick questions. Your answers stay on this page.

  1. Did you rely this week on a score or ranking from a model without knowing what it was built to predict?
  2. In a meeting this week, did people treat an automated ranking as neutral?
  3. Did you read a report this week where a stand-in measure replaced the thing the reader actually cared about?

Your DecisionDNA™ can tell you which biases you're most exposed to.

Find your DecisionDNA™

Sources

Related entries

Get the countermeasure

Get the countermeasure and the decisions where this bites, free in the Volley app.

See the Volley app