Reflections
On the meaningful understanding of thelogic of automated decision-making
Eric Horvitz
Berkeley Center for Law & Technology
Privacy Law Forum: Silicon Valley
March 24, 2017

horvitz@microsoft.com
http://erichorvitz.com

General Data Protection Regulation(EU) 2016/679, 25 May 2018
Article 15 (1)(h):"existence of automated decision-making…and…meaningful information about the logic involved
Many questions about interpreting Article.

Multiple aspects of AI decision-making pipeline.

Data  Predictions  Decisions
Training cases
DatasetML algorithm

Predictive model

Predictions
Decision modelDecisions

Data  Predictions  Decisions
Training cases
DatasetML algorithm

Predictive model

Predictions
Decision model

DecisionsLogic of processing
Personal
Data

Training
Data

Algorithm

Predictions

Decisionsmeaningful information about the logic involved

UseLogic of processing
Personal
Data

Training
Data
What, who, when?
Accuracy?

Algorithm

Predictions

Procedure used?

Validation?
Accuracy?

What, when?

Decisions
Decision policy?
Values, cost/benefit?

UseFully automated?
Decision support?

Localization & Personalization
Personal
Data

Training
DomainTraining
Data

Algorithm

PredictionsValidation?
Accuracy?

DecisionsUseLocalization & Personalization

March 2017

Machine learning “contact lens” for children

A. Howard, C. Zhang, E. Horvitz (2010). Addressing Bias in Machine
Learning Algorithms: A Pilot Study on Emotion Recognition for Intelligent
Systems. IEEE Workshop on Advanced Robotics and its Social Impacts.

Localization & Personalization

March 2017

Machine learning “contact lens” for children

A. Howard, C. Zhang, E. Horvitz (2010). Addressing Bias in Machine
Learning Algorithms: A Pilot Study on Emotion Recognition for Intelligent
Systems. IEEE Workshop on Advanced Robotics and its Social Impacts.

Moving into High-Stakes Arenas

Localization & Personalization
Readmissions Manager

M. Bayati, M. Braverman, M. Gillam, K.M. Mack, G. Ruiz, M.S. Smith, E.
Horvitz (2014). Data-Driven Decisions for Reducing Readmissions for Heart
Failure: General Methodology and Case Study. PLOS One Medicine.

Site A

Site B

Site-specific evidenceSite-specific pts, prevalencies

Site C

Site covariate dependencies

Site A

Site B

Site-specific evidenceSite-specific pts, prevalencies

Site C

Site covariate dependencies

Hospital A
Community hospital: 180 beds, 10,000 admissions/year.
Hospital B
Acute care teaching hospital: 250 beds, 15, 000 inpatients/year.

Hospital C
Major teaching & research hospital: 900 beds, 40,000 inpatient/year.

Same cityChallenges of Localization & Personalization
Personal
Data

Transfer
Local
Training
Learning
Training
DomainTraining
Data

Algorithm

Predictions

Decisions

Use

Validation?
Accuracy?

J. Wiens, J. Guttag, and E. Horvitz (2014). A Study in Transfer Learning:
Leveraging Data from Multiple Hospitals to Enhance Hospital-Specific
Predictions, Journal of the American Medical Informatics Association.

On Decisions & Values
Personal
Data

Training
DomainTraining
Data

Algorithm

Predictions
Validation?
Accuracy?

DecisionsUse

On Decisions & Values
Personal
Data

Training
DomainTraining
Data

Algorithm

Predictions
Validation?
Accuracy?

DecisionsUse

On Decisions & Values
Personal
Data

Training
DomainTraining
Data

Algorithm

Predictions
Validation?
Accuracy?

DecisionsUse

On Decisions & Values
Personal
Data

Training
DomainTraining
Data

Algorithm

Predictions
Validation?
Accuracy?

Decisions

UseNarrowing Focus
Personal
Data

Training
Data

Algorithm

Predictions

DecisionsInterpretability & explainabilitymeaningful information about the logic involved

UseMeaningful Information about ML Procedure?

M. Zeiler, R. Fergus Visualizing and Understanding Convolutional Networks, Arxiv (2013)

Interpretability & Explanation of Machine Learning
Rich & open-area of research
One approach: Enable end users to understandcontribution of individual features
What influence does changing observations x have ifother values are not changed?

Interpretability-Power Tradeoff
Neural networks
Promising space
Logistic regression

Interpretability & Explanation

Insights—and Errors

AI Journal 1996

Inspection & Troubleshooting

Cooper, et al. 1996

Inspection & Troubleshooting(!)

Having our Cake and…

Caruana, et al. 2015

Directions• Research on understandability & explanation• Best practices, norms, standards on comprehension• What aspects of AI pipelines?
• How much detail is “meaningful” understanding?

• For whom? when? What uses & contexts?