Fast.ai Practical Data Ethics lesson 5.1 notes-The problem with metrics
By: Risto Hinno
Tags: Goodharts Law, Social Media, Fastai, Ethics, Measuring

Lesson 5.1 materials are here. These are my notes and thus not complete and may contain mistakes.

Goodhart’s and Campbell’s Laws

Using target to optimize algorithms might degrade how good the target is:

Goodhart’s Law : “When a measure becomes a target, it ceases to be a good measure.” (source)

: “When a measure becomes a target, it ceases to be a good measure.” (source) Campbell’s Law: ”The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” (source)

Overemphasizing a metric leads to:

Manipulation

Gaming

Myopic focus on short-term goals

Unexpected negative consequences (especially it we don’t regard longer perspective and goals)

Illustrative example is from lesson 2 (notes), where top predictors for stroke had some funny features. Prior stroke and previous cardiovascular disease should really have predictive power for stroke. What does then for example prior accidental injury show?

Top predictors of future stroke in terms of coefficient magnitude. Source

Problem with this study is that we haven’t really measured “stroke”. We’ve measured if person has symptoms, goes to a doctor, gets tested, AND receive a diagnosis of stroke.

Targets are given a lot of attention. System are optimizing themselves so that targets would be achieved. Even if that means that other parts of performance which doesn’t get measured suffer. One study (What’s Measured Is What Matters: Targets and Gaming in the English Public Health Care System) found that targets around ER waiting times led to:

Cancelling scheduled operations to draft extra staff to ER.

Requiring patients to wait in queues of ambulances.

Turning stretchers into “beds” by putting them in hallways.

Big discrepancies in #s reported by hospitals vs. by patients.

Following graph shows how putting pressure on those performing below the target level to do better, but also providing a perverse incentive for those doing better than the target to allow their performance to deteriorate to the standard and more generally to crowd performance towards the target (source).

Crowding towards the target. Source

These are probably not outcomes that support giving high-quality healthcare. But as these aspects are not measured they don’t receive much attention.

How data is created

Measuring targets assumes that you have good data in your hands. Otherwise no matter how well target is chosen it has distortions in it.

One example from US illustrates how using automatic grading of essays got out of hand. For example sentences that had nothing to do with another received high marks (source). It is questionable how much automatic grading could be used in this case, because great essays might have some unique quality which is not discovered by algorithm. Measuring creativity is very subjective and thus not easy task for machines.

Another example is from TikTok where Chipotle #boorito campaign got 3.9 billion views. That was surprising as TikTok had been downloaded by that time only 145 million times in the US. Highly unlikely that these users created that amount of views (source). That shows what counts as a view might be a bit of mystery (or algorithm that does the counting might be flawed).

Zach King’s Chipotle TikTok video. Source

Example from Facebook which exaggerated video watch time by as much as 900% (source). This mistake led some organizations to lay off reporters and editors by the score, making room for videographers and producers. To later understand that they had done (a least partial) mistake.

Failure modes of Goodhart’s Law

Modes based on Goodhart’s Law where metric used for optimizing a system is used in a extent that further optimizing is pointless or creates harm (source):

Regressional Goodhart: Difference between proxy and goal is important.

Difference between proxy and goal is important. Extremal Goodhart: Relationship between proxy and goal is different than taken to an extreme.

Relationship between proxy and goal is different than taken to an extreme. Causal Goodhart: Non-casual relationship between proxy and goal, intervening on one may fail to impact the other.

Non-casual relationship between proxy and goal, intervening on one may fail to impact the other. Adversarial Goodhart: Adversaries try attempt to game the chosen proxy.

Issues with online environments

Addictive environment. Source

Online environments might have been created for good, but they have some serious issues (source, source):