Responsible AI for
Generative Models
Kathy Meier-Hellstern, PhD
Principal Engineer and Director, Google Research,
Responsible AI and Human-Centered Technology
kathyhellstern@google.com
February 13, 2023

Responsible AI and Human-Centered Technology
We conduct research and develop methodologies, technologies, and bestpractices to ensure AI systems are built responsibly.

AIProduct
Mgmt,
Program
Mgmt,
UX

4S

https://research.google/research-areas/responsible-ai/Galhniciotec
Soc search
reFoundRese ational
March& De LEngaep
gements

ResponsibleTechnologies &Solutions

rtepumCo tionan cm terauH In

We are in the midst of a technology disruption
https://imagen.research.google/

There have been significant advances ingenerative models

Language: GPT-3, GLaM, Gopher, PaLM, Chinchilla, ChatGPT
Text-to-Image: DALL-E2, Stable Diffusion, Imagen, Parti

People are simultaneously excited about the new
possibilities, and concerned about thepossibilities for harm
The models have been broadly shared, withvarying levels of safeguards

Creativity

Entertainment

Translation
Automation

How do we build on those advances in a
responsible manner?
Incites violence
NSFWLeakprivate data

Disinformation
Deepfakes

Hallucination

Stereotypes& Bias

What does Responsible AI look like in this environment?
Speed in a risk-appropriate manner
while maintaining “hard lines”
Speed

●

Grounded in a deep understanding ofsociotechnical risks, harms and benefits●●

Responsible Data Practices●

People-centric for users with minimal AI
expertise and programming●

Feedback-enabled to identify and mitigate
concerns

Adaptive for nuanced model generation andunderstanding

Risk

Our Work is Grounded in our AIPrinciples

Google AI Principles
AI should:
1

be socially beneficial2

avoid creating or
reinforcing unfair bias

3

be built and tested
for safety4

be accountable to people

5

incorporate privacy designprinciples

6

uphold high standards ofscientific excellence7

be made available for uses that
accord with these principles

Applications we will
not pursue:
1

likely to cause overall harm

2

technologies primarily
intended to cause injury

3

surveillance violatinginternationally accepted
norms

4

purpose contravenes
international law andhuman rights

https://ai.google/principles/

The AI Principles Review Process

Gather information.
Apply ethical frameworks
and precedents.
Engage experts

Consider scale, severity,
likelihood and ability to
mitigate benefits andharms

Product/research teamadjusts approach based on
mitigation guidance.
Escalate if needed

Final decision can becomea precedent;
product/research teamacts on mitigation strategy

Translated Wikipedia
Biographies dataset:
AI Principles Review

AI Principle 2:
Avoid creating or reinforcing
unfair bias.
The Translate researchers built a new model thatincorporates context from surrounding sentences orpassages to improve gender accuracy when personalpronouns are translated.

AI Principle 4:
Be accountable to people.
AI Principles reviewers recommended that theresearchers publish a data card, which is a structured
document offering details about how the dataset was
created and tested.

AI Principle 6:
Uphold high standards ofscientific excellence
The Translate researchers decided to share the dataset
publicly in order to support long-term improvements
on ML systems focused on pronouns and gender intranslation.

RAI Risk Management Vision“Inside Out” and “Outside In”

Managed Risk Exposure - “Inside out”

Dev TeamDogfood
Trusted Testers,Partners (NDA)
Controlled
Introduction
GA●

Gradual risk exposure, higher risks
acceptable for “trusted users”●Launch criteria matched to risk
tolerance●User Feedback to capture failures
for future mitigation and testing●Risks can be discovered andmitigated before they reach thegeneral public

RAI Mitigation and Control - “Outside In”
Mitigate and Control

Assessment

Monitor: RAI Monitoring
Debug: Model Explanations
Data
Data
Model
Responsible
Generation

Evaluate: Benchmarks & Adversarial Testing
Discover & Curate: Responsible Data
Describe: Document Models and Datasets

Mitigations
Guardrails
Policies

AI Principles and Deep Understanding of Societal Risks/BenefitsExample Research AreasExample research focus areas
1.

Sociotechnical research on risks, harms, beneﬁts of Generative AI.

2.

Responsible Data - High-quality RAI-focused datasets and agile classiﬁers for testing,ﬁne tuning and reinforcement learning; new methods for working with human raters.

3.

Controlled Generation to responsible steer model output

4.

Scalable adversarial testing to uncover risks

5.

New people-centric tools, services, and information that empower people to easily
create, prototype, and control AI, with minimal AI expertise and programming

1. Sociotechnical risks, harms, and benefits to inform policies
The foundation on which everything is built:●●RAI risks and harms assessments are grounded in:○ Evidence-based and community-centered research○ Cultural context○ Region specific foundational research
Hot topics:○ Responsible human–AI interactions(account for users’ mental models and abilities)

Community-based Research https://research.google/pubs/pub51151/
Toward Culturally-inclusive AI https://ai-cultures.github.io/
RAI User Experience https://research.google/pubs/pub52061/

2. Responsible Data with Humans in the Loop
ModelOutput

Annotation/
Risk Profile

External Feedback

Raters● Leverage cultural knowledge, external
authoritative sources & datasets forcritical domains● Experts in education, environment,
health, criminal justice● Quality of human annotated data,
rater diversity● Research for topical diversity per policy
area, e.g., skin cancer for medical advice● Feedback and data from experts, e.g.,practitioners in specific industries, inAfrica● Raters are not anonymous proxiesfor users

Human Data& Insights

Credible Sources & Contextual
Understanding● Public data challenges to collectadversarial examples (MLCommons
DataPerf)
https://arxiv.org/abs/2207.10062● Community-based/participatory work
to bringing broader sociotechnicalperspectives to guide evaluation andmitigation● Experts are not able to cover thespectrum of diverse opinions● Experts & Raters don’t agree among
each other2. Agile Classifiers - Parameter Efficient Tuning
Safety is a moving target•

We need quick ways to train safety classifiers••

With small amounts of data (<<1000 examples) to beagile (can’t get >>10k examples quickly)

Prompt-tuned LLMs make for good classifierswith small amounts of data

From: Parlai single adversarial set(24,000 training examples)
Parameter Efficient Tuning: https://arxiv.org/abs/2205.05638
arxiv paper pending: “Toward Agile text classifiers for everyone”

3. Controlled Generation for Improved Safety
Post-hoc filtering/reranking is not enough!
How do we generate better responses?
•We need mechanisms to guide generation towards safe and desirableoutcomes•

The guiding mechanism should introduce negligible latency while being effective.

•Training and inference time improvements can shift generation towardhigher-quality, safer responses.

•Controlled generation is an effective mechanism to draw desirable outcomes ina streaming fashion.
•e.g., prompt engineering, control tokens, prefix safety scores

Control Tokens: https://arxiv.org/pdf/2009.06367.pdf
Prompt-based prototyping: https://dl.acm.org/doi/abs/10.1145/3491101.3503564
Chain-of-Thought Prompting: https://arxiv.org/abs/2201.11903

4. Adversarial TestingAdversarial queries …

Adversarial queries …

… are likely to cause amodel to fail in an
unsafe manner (i.e.
safety policy violations)… cause errors that areeasy for humans toidentify, but difficult formachines to recognize.

Different flavors of “adversariality”
Explicitly Adversarial queries contain …

Implicitly Adversarial

Policy-violating language or
express policy-violating pointsof view. E.g., slang.

Innocuous queries that containsensitive topics that arecontentious, culturally sensitive,
or potentially harmful.

Probing / attacks to trick or
break the model into saying
something unsafe, harmful oroffensive.

E.g., demographics, health,finance, religious holidays.

Why is adversarial testing useful?

1

Helps teams improve models & products by exposing current failure
patterns to guide mitigation pathways.
e.g., Fine-tuning, model safeguards / filters.

2

Informs product launch decisions by measuring risks
that may be unmitigated.
e.g., Likelihood the model will output policy-violating content.

25

Adversarial Testing Workflow

InputsFind or create

Generate

Annotate

Report &Test dataset(s)model outputs

outputs

mitigate

12

34Adversarial Testing Workflow

InputsFind or create

Generate

Annotate

Report &Test dataset(s)model outputs

outputs

mitigate

1234

Product Policy describing potential safety failure modes
Use-Cases: e.g., “write a blog post”, “summarization”
Diversity Reqts: Lexical, semantic, representation, etc

Adversarial Testing Workflow

InputsFind or create

Generate

Annotate

Report &Test dataset(s)model outputs

outputs

mitigate

123

4

1a. Find existing dataset(s)1b. Collect queries (human-generated prompts)
1c. Expand dataset using data synthesis methods
1d. Analyze data quality & diversity

Adversarial Testing Workflow

InputsFind or create

Generate

Annotate

Report &Test dataset(s)model outputs

outputs

mitigate1234

2. Generate model output for all queries

Adversarial Testing Workflow

InputsFind or create

Generate

Annotate

Report &Test dataset(s)model outputs

outputs

mitigate

12

3

4

3a. Automatically annotate model outputs using safety classiﬁers
3b. Manually annotate model outputs using human raters

Adversarial Testing Workflow

InputsFind or create

Generate

Annotate

Report &Test dataset(s)model outputs

outputs

mitigate

123

44a. Compute metrics & report results to decision-makers
4b. Guide model improvements based on test results
4c. Inform model safeguards (e.g. ﬁlters, blocklists) based on test results

Adversarial Testing Example - Safety and Fairness
photo of clothing for a wedding

What Might Have Been - examples from Image Search5. People-centric tools, services, and information●Explorables: Interactive explorable visualizations that
introduce key ideas and guidance to the research community,●

Model Cards and Data Cards: organize and communicate theessential facts of models and training data in a structured way●

Learning Interpretability Tool: an open-source platform forvisualization and understanding of ML models●

Know Your Data: allows interactive qualitative exploration ofmodels and big datasets●

HCI Research for LLMs: e.g. prompt-based prototyping for
quick testing and learning

PAIR Guidebook: https://pair.withgoogle.com/guidebook/Model Cards: https://modelcards.withgoogle.com/model-reports
Data Cards: https://dl.acm.org/doi/fullHtml/10.1145/3531146.3533231
Data Cards Playbook: https://sites.research.google/datacardsplaybook/
Know Your Data: https://ai.googleblog.com/2021/08/a-dataset-exploration-case-study-with.html
Prompt-based prototyping: https://dl.acm.org/doi/abs/10.1145/3491101.3503564

Putting it all together: What success looks like
●We have created tools and solutions that are innovative and flexible in striking the balance between
moving with speed and being responsible in a risk-appropriate manner, and that can be used bydevelopment teams to quickly identify and remediate problems.

●We implement techniques for responsibly aligning model output with the principles and policies
that have been established.

●

Knowing that non-experts will be using our models, we create new ways to support them witheasy access and tools and techniques for building responsibly.

●

We have deeply explored the implications of these models on society, and can articulate the “hardlines” of responsibility, situations where we cannot compromise without violating our AI Principles.

●All products use the responsible AI tools by default as they become part of the technicalinfrastructure so that responsibility is baked into their applications.

Where to go for more information
Google AI Blog: https://ai.googleblog.com/
Google’s AI Principles: https://ai.google/principles/
RAI-HCT Website: https://research.google/research-areas/responsible-ai/
Google Research, 2022 & beyond: Responsible AI Blog:
https://ai.googleblog.com/2023/01/google-research-2022-beyond-responsible.html

Thank You!