Towards Value-Based Machine Learning
By: Ron Bodkin
Tags: Machine Learning, Ethics, Values, Technology Trends, Ai Responsibility

Machine Learning (ML) has become integral in many aspects of modern life, as digital experiences proliferate and we increasingly rely on automated algorithms for discovery, curation and guiding our choices in areas as diverse as entertainment content (e.g., Medium and Tik Tok), communication (Slack and gmail), navigation (Google Maps) and shopping (Amazon and Stitch Fix).

ML is often viewed as a value-neutral technology and objective: not aligned with or dependent on values. But the reality is that ML is a tool and like any tool, it is used based on values and the consequences it creates impact our values.

I have been responsible for applying ML to real world problems since 2007 and have found that repeatedly, use of ML leads to unintended consequences. Much like an evil genie, ML models will often grant your exact wish (optimize what you specify) but not what you really intended. Ten years ago when I was VP of Engineering at Quantcast, we would often be frustrated to see ML models we created that didn’t work properly. They would exploit subtle errors in our data or problem set up and we had to work hard to understand what made them work so we could fix our data and fix our objectives (or loss functions) to achieve the results we intended.

More recently, there have been important examples of this phenomenon of unintended consequences of ML in areas like biased ML models. Examples include Amazon’s hiring algorithms that were biased against hiring female engineers and Alphabet Jigsaw’s algorithms for toxic content that were biased against targeted identity groups. More generally, recommendation systems show a bias towards trashy, inflammatory, click-bait content (e.g., the impact of recent Facebook algorithmic changes). There are also value challenges from explicitly harming customer interests, e.g., Amazon changing ML-driven search results in favor of Amazon’s profitability over what customers want.

To date most schemes to address these problems have been based on the premise that you can optimize for a value-neutral objective function (like maximizing revenue, or maximizing time spent on an app) while building in various guard rails. Common techniques used include filtering out problematic cases, preparing data to avoid bias, model explanation tools, and tracking secondary metrics (e.g., tracking for long-term cohort engagement and not short-term response).

I believe this approach is fundamentally insufficient — value-neutral objectives are amoral by definition. Instead, I believe we must do more — we must encode values in the objectives we measure and work on ways to explicitly produce good outcomes with ML in addition to other ethical AI practices.

An Example of the Importance of Values

Consider content recommendation systems. Tristan Harris co-founder ofthe Center for Humane Technology has a nice way of thinking about this problem. Consider a spectrum of content ranging from regenerative, i.e., reflective and thoughtful content on the left (e.g., articles in the BBC or Wall Street Journal) to extractive, i.e., inflammatory and extreme content on the right (e.g., Infowars or Daily Kos).

A Content Spectrum

A few years ago, systems like this were often trained to optimize clicks. This resulted in a lot of trashy “click-bait” content that wasn’t engaging for users (or regenerative!) The next evolution was to optimize for aggregate engagement time. This has resulted in more sustained user engagement but has led inevitably to more inflammatory and extreme content (like conspiracy theories, political extremism and criminal activity). In that spectrum of content, the system is tilted to the right — user engagement flows towards the more sensationalist and alarming material.

Content Spectrum: Tilted Towards Attention Grabbing Content

It has also increased digital addiction and in a meaningful way, it has downgraded the users of systems and society more broadly. The use of human review, policing of terms of service and secondary systems to block and/or not recommend and/or not serve ads for inaccurate or other bad content has helped. However, bad actors continuously find ways to probe the boundary and create the worst, most addictive content, that will be promoted. You can think of these as barriers against the worst content while still keeping the same harmful flow.

A Content Spectrum with Barriers for the Worst Content

Conversely, imagine a system that explicitly optimized for high quality engagement. This should include ideas like matching resources spent by users to their intentions like “I’d like to enjoy no more than five hours a week of entertaining content,” “I don’t want to be drawn into watching cat videos until 2 am” and “I’d like to spend at most $50 per month on video entertainment.” It should also include impact on society as a whole, e.g., by informing people, provoking meaningful engagement, encouraging deep personal interactions, and allowing political discourse to evolve. Of course, these are hard concepts to represent as a simple mathematical function — and there is a great diversity of perspective of what high quality engagement means.

Amplifying Regenerative Content Instead

How to Proceed?

This shouldn’t be viewed as an obstacle to progress but a grand challenge that the technology community should embrace. How can we better create objective functions that optimize our values? How can we better anticipate and mitigate unintended consequences? How can we create better transparency and visibility into the objectives our systems are encoding? I believe it will become a critical engineering discipline — blending technical acumen with broader insight into policy goals and ethics. Likewise, François Chollet (the creator of Keras) views having the right objectives for ML as important, predicting that “loss function engineer is probably going to be a job title in the future.”

There is a lot of opportunity to close the gap between long-term research in AI value alignment (e.g., Inverse Reinforcement Learning) and practical goals in systems today. But the most important step is to take ownership of the ethics of AI systems and incorporate values into their design, especially their objectives!