Companies Need to Mitigate A.I. Bias
By: Ashley C
Tags: Machine Learning, Artificial Intelligence, Bias, Ethics, Software

Hiring Bias At Amazon

Amazon implemented an A.I. recruiting process in 2014. The project was based on reviewing the applicant’s resumes and rating their applicants through A.I. powered algorithms to create a more efficient hiring system.

Unfortunately, in 2015 Amazon figured out that their A.I. system was rating candidates with a bias, showing bias against female applicants. How did this even happen? This old training set was a huge contributor to the bias, because the dataset was 10 years old, and at the time there was a huge male dominance across the tech industry and men were forming 60% of Amazon’s employees. As a result of the bias in the data, the A.I. was taught to prefer male candidates over females. The model would penalize resumes that included the word “women’s,” and fortunately as soon as Amazon discovered this bias in the A.I., they stopped utilizing the algorithm.

Racial Bias in Healthcare A.I.

A recent study in 2020 by ScienceMag.org, proved that a common healthcare risk-prediction algorithm developed by Optum demonstrated racial bias. The unfortunate part is that this algorithm had been used on around 200 million U.S. citizens (Analysis took place in September 2020).

The purpose of this algorithm was to determine which patients would benefit from additional medical care, but after careful analysis of the model, researchers found that it significantly ignored the needs of black patients. Why did this even happen? The creators of the algorithm used previous patient’s healthcare purchases as a guideline for medical needs. This flawed the model because historical data shows that income and race are highly correlated metrics, and making assumptions based on one variable creates a huge bias which outputs very inaccurate results.

Cosmetic ads were being advertised to women on FaceBook based on the stereotype that females value physical appearance as high importance.

Biased Advertisements

Human bias can additionally be apparent in tech platforms, and a particular situation of this is seen in advertisements.

Many social media platforms and websites track data to later train models, leading to more biased machine learning models.

Quite recently in 2019, Facebook allowed for advertisers A.I. software to deliberately target audiences on the basis of gender, race, and religion. Women were being targeted in job advertisements related to secretarial or educational work, in contrast to other job advertisements such as a taxi driver role, mostly being shown to men, in particular men who were minorities.

Obviously, people were outraged and critiqued FaceBook’s questionable decision, and due to the backlash, recently Facebook altered their policy to no longer allow employers to target factors such as age, gender, or race within their advertisements.

Chatbot Show’s The Consequences of Negative Human Influence

March in 2016, Microsoft's Twitter chatbot, Tay, revealed how an A.I. can be trained to be very biased. The purpose of Tay was to engage people in dialogue through tweets and direct messages, and Tay played the persona of a teenage girl. The goal was to release Tay to the internet, let the bot discover patterns of language through communication, which would be used in her future conversations. Eventually, the creators hoped for Tay to sound like someone on the internet.

In the beginning, Tay was harmlessly engaging with her followers, using comedic and harmless jokes. Within a few hours, this whole demeanour changed when Tay started tweeting extremely offensive things including: “I f@#%&*# hate feminists and they should all die and burn in hell”.

16 hours later, Tay had tweeted over 95,000 times and the majority of those tweets were abusive and highly offensive. As Tay went viral, the outrage was expressed and Microsoft had no choice but to deactivate the account.

What was thought to be a harmless and fun experiment went wrong and many different ways. Is the internet really that racist, misogynistic, and anti-Semitic?

After investigation, it was revealed that 4chan, a troll platform, shared a link to Tay’s account and encouraged users to flood the bot with racist, misogynistic, and anti-semitic tweets.

The experiment of Tay, may not have been successful in fulfilling Microsoft’s goals, but the failed attempt goes to show how easily A.I. can be trained to exhibit bias during decision making.