An interesting spark in the news is the topic of Artificial Intelligence (AI) bias. This is the underlying prejudice or imbalance in data used to create algorithms, ultimately resulting in discrimination and other social consequences. Understanding how to catch these biases has become a necessary skill for data scientists, engineers, analysts, and now the global community, to know.

Here are Three Real-Life Examples of AI Bias:

  • Discrimination in US healthcare - In 2019, researchers found an algorithm used on more than 200 million people in US hospitals more heavily predicted white patients over Black patients would need follow-up care. Race itself was not used in the model learning, but healthcare cost history was a variable. The healthcare cost summarizes how many healthcare needs an individual would have. For various reasons, Black patients incurred lower healthcare costs than white patients with the same conditions.
  • The COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) system used in US court systems to predict the likelihood of a repeat offender - Due to the data used, the model chosen, and the process of creating the algorithm, the model predicted twice as many false positives for recidivism for black offenders (45%) than white offenders (23%).
  • Amazon’s employment practice - About six years ago, Amazon caught that their hiring process was biased against women. This was because the data used was based on the number of resumes submitted over the past ten years, and most of the applicants were men. Thus, the model was trained to favor males over females.

Making non-biased algorithms is hard—like for some, making non-biased decisions is also a challenge. Of course, to create non-biased algorithms, the data used has to be bias-free and the individuals creating these models need to be free of bias as well – to ensure discrimination of any sort does not leak into the work.

Some helpful hints to safeguard against bias in your work:

  • Represent “what should be” and not “what is.”
  • Implement a governance or check system across your organization.
  • Evaluate content fairly. Learning from the instances above, we should strive to ensure that metrics are consistent when comparing different social groups, whether gender, ethnicity, or age.

Making sure our data represents everyone equally and in a way that does not discriminate against a group of people will likely be one of the more significant challenges of the AI era… and quite possibly our workforce.