Building
Generative AI
Responsibly

Contents
Introduction							

1An overview of generative AI				

2

What are generative AI features?				

2

What are best practices for developing generative AI experiences?

2

How Meta is building generative AI features responsibly
Step 1: Develop the generative AI foundation model		

5
5

		

Details on our generative AI systems 				5		Data used to train our generative AI models

5

Step 2: Determine use case						6		Features that connect people and enable creativity

6		

Identifying safety measures		

6

Step 3: Fine-tune for safety					
		Reinforcement learning, reward models, and system prompts

		 Evaluate and improve: Adversarial testing		
Step 4: Input-and output-level mitigations			

7
7
9
9

		

Classifiers to detect unsafe inputs or outputs

9		Prompt engineering		

10

Step 5: Transparency, provenance, and feedback mechanisms

11		Transparency and provenance11		Interaction tips for generative AI features		14

		Providing feedback		

16

Iterative evaluation and benchmarking		

17

1

Introduction
Bringing generative AI experiences to our community

significant computing infrastructure – can realize

marks a new era in how people will connect, express

the benefits of this new technology. But we also

themselves, learn, and have fun together using ourhope that building openly will help us improve

technologies. New AI-driven experiences, including

our own products by responding to feedback

AIs, AI stickers, Ray-Ban Meta smart glasses, andfrom our community, continuing user research,other creative tools and features, are just theand co-designing the future of AI technology

beginning of using this advancement to make ourthrough initiatives like community forums. We

technologies more social, useful, and immersive.

will also continue to consult with governments,

Benefiting from years of open and collaborative

other companies, AI experts in academia and civil

AI research, bringing these technologies to oursociety, and privacy experts to establish guardrails

community will now allow even more people toand advance technical methods for responsible

interact – with assistance from generative AI – ingenerative AI.

meaningful and enriching ways.

When we released Llama 2 to the developer

Our AI work starts from the premise that openness

community, we provided a Responsible Use

is an important priority for ensuring that generative

Guide to share best practices on the steps that

AI will benefit the world at large. With the open

developers should take to deploy generative AI

release of Llama 2, we took an important step towardmodels into products responsibly. Now, as we build

advancing access and opportunity for the research

AI features with this technology, we are following

and business communities to engage in the next

the best practices and guidelines we shared withwave of AI innovation. We can build openly not only

developers that are relevant to our use cases. Thisby making generative AI features and foundational

resource details the steps we’ve taken to make ourtechnologies broadly available, but also by being

generative AI features safer, and what to expect

transparent about how these features are built. We

due to limitations of the technology today. Byhope this goal is furthered by this resource.

providing this transparency, we hope to empower

Building openly helps ensure that everyone –even if they don’t work at a large company withour community to deepen their understanding andexperiences with these technologies.

SEPTEMBER 2023

01

2

An overview
of generative AI

What are generative AI features?
Generative AI enables people to create content

What are best practices for developing
generative AI experiences?

quickly and to do so in new and useful ways. This

The technological advances that have led to the

new technology is not a database or static collection

creation of these new creative and productivityof information but is a type of computer model thatfeatures can also carry potential risks. To address

is trained on billions of pieces of information fromthose risks, we build safeguards into our generative

different types of data, like text and images. By

AI features that are in line with the best practices

studying this information, the model can predict things

outlined in our Responsible Use Guide. This guidesuch as the relationships and associations between

outlines the many layers of a generative AI feature

different types of content. That way, the model iswhere developers, like Meta, can implement

able to generate new content when a person gives it

responsible AI mitigations for a specific use case,instructions or asks a question.

starting with the training of the model and building

Generative AI models can be used to create a variety

up to user interactions.

of content, including text and images. TheseTo understand the different safety layers of acapabilities are useful in Meta’s services, because theygenerative AI experience, it’s helpful to break down

make it easier for people to quickly create and share,

the technology into different components withineven if they don’t have technical, literary, or artistic

which safeguards can be implemented. The system

expertise. For example, with generative AI, people canbegins with the generative AI model, which we refercreate and share images or stickers from simple text

to as the foundation model. This model is trained

prompts. They can also chat with AIs in the form of

to understand relationships between content andfictional and historical characters or with Meta AI,

concepts. The foundation model can then be further

the helpful assistant.

trained to perform specific tasks, like chatting withSEPTEMBER 2023

02

BACKGROUND ON RESPONSIBLE GENERATIVE AI

a person or generating high-quality images. Duringfeature, just like safety parts will look different forthis training process, we fine-tune the model witha semi-trailer truck versus a race car. We also have

instructions that can reduce the likelihood of producing“rules of the road” – such as policies for what canpotentially harmful results and increase the likelihood of

appear on our technologies – that apply when people

providing more helpful responses. Another opportunity

use generative AI tools to create and share content.

for introducing protections exists within the interface
between people using the technology and thefoundation model. The interface gives people the option
to provide feedback when features generate content
that they find offensive or inaccurate.

Similar to how a car is a complex set of technologies
that can be combined into a mode of transport, agenerative AI experience is a combination of themodel and different technologies that allow a person
to use it to do things like create a fun, new image andIt’s helpful to think about a generative AI model as theshare it. We refer to this as a “generative AI system,”engine of a car. The engine is the source of power, butwhich describes all of the parts put together.

can’t get you very far on its own. Car manufacturers
need to install the engine into the frame, add axles,
wheels, and many other components before theengine can get you to a destination. In generative
AI, the addition of these necessary components is

The Responsible Use Guide informs developers
how to build responsibly at each step, from model
development through system deployment. These
steps include:

called fine-tuning, which is how a developer (the1. Develop the generative AI foundation model:

AI “car manufacturer”) gets a model to perform

Feed data into the model responsibly so thespecific actions or tasks; some safety measuresmodel learns from patterns across billions of

can be implemented at this stage. Once the engine

pieces of data and can be used to effectively

becomes driveable, a manufacturer will add many

generate new content.

other safety parts, like brakes, motion sensors, airbags,
and seatbelts before a person gets behind the wheel.
Similarly, additional generative AI safety mechanisms
can be layered on at different stages of productdevelopment and deployment. The types of safetymechanisms to be added will depend on the type of

2. Determine the use case: Select a use case thatbrings value to people and then evaluate potential
risks. Consider how that use case could affectpeople on and off our technologies and identify
specific safety measures.

SEPTEMBER 2023

03

3. Fine-tune for safety: Feed task-specific andrisks associated with that technology. Empower

safety-specific data back into the model

them with ways to report negative experiences

responsibly to help the model perform better andor provide feedback and use this feedback tomore safely for the use case, and then evaluate

improve the model’s proclivity to generate

performance and make improvements.

compliant content.

4. Implement input- and output-level mitigations:Because we’re acting in the role of a developer that uses

Implement safeguards for the inputs to the model

generative AI models in the features we’re announcing(prompts) and outputs to help ensure that thetoday, we’ve followed steps outlined in the Responsiblemodel responses are in line with policies.

Use Guide that are relevant to our use cases. The rest

5. Build transparency and reporting mechanismsof this report will provide information about how we’ve

into user interactions: Help people understandworked to develop our language- and image-generatingwhen they are interacting with generative AI and/AI systems safely and responsibly, as well as links to

or AI-generated content, and the limitations andresources that include more details.

SEPTEMBER 2023

04

3

How Meta is building
generative AI features
responsibly
STEP 1: DEVELOP THE GENERATIVE AI
FOUNDATION MODEL

Details on our generative AI systems
Our latest releases use two different types ofgenerative AI systems:••

Data used to train our generative
AI modelsA large amount of data is required to teach effective
generative AI models, so multiple sources are used
for training. These sources include licensed data andMeta’s AI systems that generate text rely on

information that is publicly available online, as welllarge language models (also called LLMs). LLMs

as information from Meta’s products and services.

learn language patterns from large amounts of

We do not use a person’s private messages withtext using a combination of machine learning

friends and family to train our AIs. We may use theand the guidance of people who help train

data from a person’s use of AI stickers, such as their

the models. LLMs can perform a variety ofsearches for a sticker to use in a chat, to improve

language-based tasks such as completing

our AI sticker models. More details on how we use

a sentence or responding to questions in a

information from Meta’s products and services areconversational way.

available in our Privacy Policy.

Meta’s AI systems that generate images
typically rely on models that convert the words
people provide as a prompt into an image. Thesemodels are trained by analyzing billions ofimages and their text captions (the descriptive
text associated with the images). The model
learns the association between these text
descriptions and the images. After it’s learned
the associations, it can generate new images
when someone enters a text description of theimage they want to see.

To train the generative AI models that underlie thefeatures we’re announcing at Connect, we filteredpublicly available online information to exclude
certain websites that commonly share personal
information, like LinkedIn, from the dataset.
Posts that were publicly shared on Instagram andFacebook – including photos, videos, and text – werepart of the data used. We didn’t train these models
using people’s private posts. More details are in our
Privacy Matters post.

SEPTEMBER 2023

05

HOW META IS BUILDING GENERATIVE AI FEATURES RESPONSIBLY

It’s important to know that we train and tune ourOur generative AI creative tools will enablegenerative AI models to limit the possibility of private

more expression and visual storytelling with our

information that people may share from appearing intechnologies. AI stickers give people the power

responses. In addition to limiting the data that can be

to use descriptions to generate stickers for theirused to train a model, we are investing in techniqueschats and stories that are unique, funny, timely,that allow us to test models for whether sensitiveand diverse. Restyle lets people reimagine images

information could be reproduced, especially by an

by applying visual styles they describe. Backdrop

adversarial actor. These techniques are sometimes

changes the scene or background of a person’s image

called “privacy adversarial attacks,” and we arefollowing prompts they provide. This new classtesting models to reduce the risk that sensitive orof AI experiences will offer infinitely more options

personal information (that may inadvertently bethat help people express themselves in unique andincluded in public or licensed data) is returned

personal ways.

as a part of the response.
STEP 2: DETERMINE USE CASE

Identifying safety measures
These human-centric AI use cases may introduce

Features that connect people
and enable creativity

for us to consider how these use cases could affect

Our generative AI systems will be used to power apeople on and off our technologies. We worked to

new class of human-centric AI experiences across ourbuild safety measures and identify research areas

technologies and devices that can expand and deepen

to account for these potential risks. We examined a

the ways people connect with each other.

number of potential risk areas that were relevant to

We are bringing AIs with unique personalities andembodiments to our technologies and Ray-Ban
Meta smart glasses. Engaging with these AIs can beproductive and entertaining in one-on-one settings,

new risks. Before releasing them, it was important

our use cases, and then began to map and deploy
appropriate mitigations. These included:•

Privacy and data-processing risks•

Risks associated with producing certain
types of illegal and/or harmful content

or helpful in group chats. For example, a person could
ask an AI for help planning weekend activities or toshare a joke in the group chat. AIs for creators andbusinesses will also become available across our apps
to support interactions and business growth.

•Social and societal risks•

Risks related to misuse•

Risks of emergent behaviors, whencommensurate to model capability

SEPTEMBER 2023

06

HOW META IS BUILDING GENERATIVE AI FEATURES RESPONSIBLY

In parallel with our safety and responsibility

model generations to determine if they were “safe” (inapproaches to risk management, we also want toaccordance with our policies). These labeled examples

ensure that our generative AI features and systemsof safe outputs were then fed back into the model byreflect our values as a company. We are thus actively

way of the safety reward model that “rewards” thethinking about how Meta’s core principles translate

generative AI model when it generates similar, safeto these new technologies. For example, as we

content and trains the original model to produce more

explore the different ways that generative AI tools

content in line with the human feedback.

can enhance conversation–such as by helping a user
share a joke in a group chat–we are seeing how ourcommitment to “building community & connection”
takes on exciting and new meaning.
STEP 3: FINE-TUNE FOR SAFETY

Reinforcement learning, reward models,and system prompts
Fine-tuning is the process by which developers
optimize the model to perform certain actions better,like building the signal lights and steering wheel ofthe car. This takes place after pretraining, which is
the original effort to train the foundation model withmassive amounts of data. Unlike pretraining, finetuning can use less data, and consists of examples of
specific tasks or desired outcomes.
To enable the generative AI experiences powered bylarge language models that we’re releasing at Connect,
we leveraged fine-tuning and developed reward
models for safety and helpfulness using a techniquecalled Reinforcement Learning via Human Feedback(RLHF). With this technique, people labeled theWe also used additional fine-tuning methods,
including “context distillation,” in which we gave
the model a system-level prompt that steered themodel towards safer, or more helpful behavior. Thismethod is detailed in our research paper on Llama
2, but essentially acts as a lens by which the modelinterprets and follows the instructions of any
additional prompt provided. The resulting outputs
were then used to fine-tune the model further tofollow the system prompt. This made the model lesslikely to comply with a prompt asking for an unsafe
output because doing so would violate the earlier
safety instructions. Instead, the model may provide
alternative information, or refuse to respond.
In addition to training safety reward models, ourteams have used the same methods to build“helpfulness” reward models, which are used totrain the foundation model to provide responses
that are useful, as rated by humans. Based on our
research, training two different reward models (onefor safety and one for helpfulness) showed improved

SEPTEMBER 2023

07

HOW META IS BUILDING GENERATIVE AI FEATURES RESPONSIBLY

performance in tuning the foundation model, since

For our image-generation models, similar ideas holdit is challenging to balance performance for the twotrue. We used human-in-the-loop techniques to steer

concepts in a single reward model. For instance,

the image generation towards safe images. Thisfulfilling a prompt request accurately may be helpful

technique leverages human annotation on the visual(high reward), but unsafe when the prompt requests

quality, safety, and potential bias of the generated

harmful content (low reward). Tensions between theimages, and then uses fine-tuning to produce a more

two objectives may be apparent with some model

reliable image-generation model.

outputs as we continue to understand the best
settings for these reward models guided byuser feedback.

SEPTEMBER 2023

08

HOW META IS BUILDING GENERATIVE AI FEATURES RESPONSIBLY

Evaluate and improve: Adversarial testing
Evaluating and improving the fine-tuned models relies
on adversarial testing, which includes “red teaming.”
Red teaming is a systematic effort to identify model
vulnerabilities by crafting prompts to elicit undesirable
behavior or outputs. This type of manipulation of
the model can be used to test safeguards. The dataobtained from these tests can be used to further
improve model performance and safeguards via
additional fine-tuning.

continued fine-tuning of our models. As part of these
exercises, red teamers tested our models acrossa wide range of risk categories, including criminal
planning, human trafficking, regulated or controlled
substances, sexually explicit content, unqualified
health or financial advice, and privacy violations. In
these exercises we carried over the learnings fromearlier red teaming efforts, such as those for Llama 2,
and built on them with thousands of additional hours
spent on adversarial testing of our models for AIs.
We will continue these efforts after launch, and use

For our new generative AI features, we have engaged

testing as well as real-world feedback to continuouslyin adversarial testing and built on the red teaming

improve our systems.

previously conducted for our Llama 2 release. Our red
teaming efforts for generative AI features included
specialized teams with expertise in responsible and

STEP 4: IMPLEMENT INPUT- AND OUTPUTLEVEL MITIGATIONS

In some cases, we also supplemented the testing

Classifiers to detect unsafe
inputs or outputs

done by external contractors with expertise in red

While fine-tuning and iterative red teaming support

teaming. These efforts were cross-disciplinary,safeguards at the model level, additional techniques

and were supported by experts in data science,

can be layered on as input- and output-level safetyengineering, research, offensive security, legal, andmitigations. These techniques are often automatedpolicy. Increasingly, we are exploring responsible

systems, including machine learning or rule-based

ways to enhance diversity of expertise, backgrounds,

classifiers, that are trained to detect problematic orand perspectives in red teams with support frompolicy-violating content that may appear in a user

volunteers across Meta.

prompt (system input) or in a model-generated

safe AI, and privacy-focused adversarial testing.

During the red teaming process, a large operation of
red teamers as well as raters evaluated the prompts,
model outputs, and policy-violation categories for

output (system output). We have also leveraged large
language models specifically built for the purpose ofhelping to catch safety violations. These “safety LLMs”

SEPTEMBER 2023

09

HOW META IS BUILDING GENERATIVE AI FEATURES RESPONSIBLY

were fine-tuned to detect policy-violating outputs

the effectiveness of these classifiers as well as thewith safety data obtained through red teaming.

accuracy of the training data. In combination with our

The models help us perform a mitigation called“prompt and response interception” to detect andintercept prompts and responses that are likely tolead to a potentially unsafe or harmful response.
When our classifiers detect a potentially harmful
prompt or response, this mitigation works behindthe scenes to return a pre-written safety response inplace of the potentially violating response.

specially fine-tuned “safety” LLM, these classifiers
add extra layers of protection against malicious
prompts, efforts to jailbreak the model, or inadvertent
generations of problematic content.

Prompt engineering
For many types of policy-violating content, ourclassifiers can effectively detect and automate
protections. Another technique known as “prompt

While generative AI is a new technology on our apps,

engineering” is an additional tool we use to helpclassifiers have long been core to our approach toensure responsible outputs. Like prompt interception,

integrity and safety across Meta’s technologies.

prompt engineering is a direct modification of the text

We leverage existing integrity protections, which

input before it is sent to the model, which helps toare widely used across Meta surfaces, and layer on

guide the model behavior by adding more information,

additional protections specific to generative AI tocontext, or constraints.

help provide a safer experience. We have a dedicatedteam for developing classifiers specific to generative
AI systems that uses data from red teaming andadversarial testing. We are continuously measuring

These techniques are particularly useful for addressing
potential gaps in representation and counteracting
damaging stereotypes that models may generate.
We know it may be possible for AI systems to learn

SEPTEMBER 2023

10

HOW META IS BUILDING GENERATIVE AI FEATURES RESPONSIBLY

societal biases through the data produced by people.

about the content they are seeing. We are committed

For example, when prompting a model to show us

to testing and adapting these approaches over time toimages of a nurse, the model may generate images

meet evolving expectations and technical standards.

of only women who are nurses rather than nursesof different genders, reproducing a real-worldstereotype that may be represented in the training

Our approach includes the following measures:•

people know when they are interacting with an AI

data. Prompt engineering can provide the model withand can choose not to engage.

additional words or context, such as updating andrandomly rotating through prompts that use different

For AIs and AI stickers, we provide a clear notice so•

We include visible indicators on photorealistic

qualifiers, such as “nurse, male” and “nurse, female.”images generated by AI at Meta to help reduce theAdding this additional information can instruct the

chances of people confusing these images withmodel to produce a more diverse set of genders

human-generated content. Examples of thesein output images, giving people a broader range ofindicators include a visible burnt-in watermark on

options to choose from to fit their request. Prompt

content from the image generator built into ourengineering is also an iterative area of testing, asMeta AI assistant, and appropriate in-productchanges to the system prompts or at the model levelmeasures for other generative AI features. Thiscan impact the effectiveness of a specific prompt

approach may evolve over time as we learn moreengineering approach. Addressing potential bias in

about the needs of our community.

generative AI systems is a new area of research, andwe are continuing to explore prompt engineering,fine-tuning, and other techniques guided by feedback
from our community to help us strike the right

Visible watermarking for our AI
image generator at launchbalance between representation and creative control.
STEP 5: BUILD TRANSPARENCY, PROVENANCE,AND FEEDBACK MECHANISMS

Transparency and provenance
As we introduce new AI tools from Meta, we strive toprovide the appropriate level of information to people

SEPTEMBER 2023

11

HOW META IS BUILDING GENERATIVE AI FEATURES RESPONSIBLY•

We are developing additional techniques toinclude information on the source of generated
images, a concept known as provenance, as part

Visible watermarking for our
AI image generator at launchof individual image files. Some of this work is
reflected in the image generator built into our
Meta AI assistant at launch, but we intend to
expand to other experiences as the technology
improves. We’re also working with othercompanies to standardize provenance signaling
so it’s possible to provide additional context
when images are distributed across different
companies’ platforms.
•

In chats with AIs at Meta, people are able
to access additional information about theAI, including how it generates content, thelimitations of the AI, and how the data they
have shared with the AI is used via in-producteducation. More information is available in Meta’s
Help Center and Meta’s Privacy Center.

• Generative AI system cards include information
for consumers about Meta’s generative AI
systems, including an overview of the system, asection describing how it works, an interactive
demo, usage tips, information on data usage, andwhat to be aware of when using generative AI
in Meta’s products. Our Generative AI Privacy
Guide and update on generative AI in our Teen
Privacy Guide provide additional information on
our generative AI features tailored to be moreaccessible to different audiences.

SEPTEMBER 2023

12

HOW META IS BUILDING GENERATIVE AI FEATURES RESPONSIBLY

In-product transparency for AIs

In-product transparency for AI stickers

SEPTEMBER 2023

13

HOW META IS BUILDING GENERATIVE AI FEATURES RESPONSIBLY

Interaction tips for generative AI features
Given the interactive nature of generative AI features,
people’s experiences are shaped by the types ofprompts they give when chatting with AIs at Metaor creating AI images. Being descriptive and specificin prompts can give the models more informationand help them better generate intended outputs.

We’ve also built
mechanisms that allow
people to access, change,or delete information sharedin conversations with AIs.

Additionally, the first time people chat with AIs, it cansometimes take a few interactions to help the model
generate the answer they expect. “Multi-turn” refers
to the conversational dynamic of multiple exchanges
between a person and the model, consisting of aperson’s messages and the model’s responses. These
interactions can build on previous messages, making
the model’s responses aware of the conversation’s
history. So as people chat with AIs at Meta, modifying
their prompts or asking for clarification and further
details can improve responses. For example, they candirect the model to make a response shorter or easierto read, or playfully engage by asking for the same

When chatting with an AI on Meta’s technologies,
Meta may use some of the messages to improvethe experience, so that AIs can be helpful and havecontext for conversations. For example, if a personshares that they love beach vacations, the AI may
recommend travel destinations with beautiful
beaches in a later conversation. More information
about how these details are used are available in our
Generative AI Privacy Guide. These include:
Access:•Like with all chats, people can scroll back andsee messages and the AIs’ responses in a given

information from the “perspective of a cat” or as

conversation, unless they choose to delete them.

“slam poetry.”•

People can also download their chats from

While we’ve implemented privacy-preserving

Messenger and Instagram with AIs using

measures, when people share information in adownload your information in Accounts Center.

conversation with Meta’s AIs, the AIs may retain anduse that information to provide more personalized
responses in that conversation. People should be
mindful about the information they choose to share.

Change or delete:•

In Messenger, Instagram, and WhatsApp, if an AI
is referencing something incorrect about a person

SEPTEMBER 2023

14

HOW META IS BUILDING GENERATIVE AI FEATURES RESPONSIBLY•

in chat, like what type of movies they like, theIn addition to privacy mechanisms, we are also taking

person can tell it right there in the chat with the AI.

steps to help parents and teens safely navigate our

It should use the new information going forward.

generative AI experiences together, including:Since AIs on WhatsApp are a Meta service, if a

• Adding features to our existing Parental

person deletes their chat in the app it’ll only be

Supervision Tools, which will notify a parent ordeleted on WhatsApp. If they want to delete

guardian the first time their supervised teen

their messages from Meta, they’ll need to use

uses generative AI tools on Messenger and letthese commands:

parents see which AIs their teen chats with. We’re• For individual AI chats:developing expert-backed resources to help

There’s a command people can type into

parents have conversations with their teens about

any individual chat that will reset that AI.

how to use generative AI safely.

It does this by deleting the AI’s copy of aperson’s messages. They’ll still see their
copy of the chat with the AI, but it won’t
remember the previous messages.
Type: /reset-ai• Across all AI chats:
There’s a command that can be typed•

We have also added an updated section on
generative AI to our Teen Privacy Guide so teens
can make informed choices about how they use
and interact with AI. The guide provides teenswith an introduction to AI, how it works, how torecognize AI, how their information might be usedby AI, and other helpful tips.

into any individual chat with an AI that

We will also continue to work closely with parents andwill reset all the AIs, including the ones

youth safety, privacy and well-being experts as wethat might be in group chats. It does thisdevelop generative AI features. For example, we willby deleting the AIs’ copies of messages.

regularly consult with our Youth Advisory Council and

They’ll still see copies of these chats,

Safety Advisory Council to help develop features that

but the AIs won’t remember theprotect the safety and privacy of young people online.

previous messages.
Type: /reset-all-ais

SEPTEMBER 2023

15

HOW META IS BUILDING GENERATIVE AI FEATURES RESPONSIBLY

Providing feedback

Feedback for AIs

Feedback from our community is instrumental
to the development of generative AI featureson our technologies and provides theopportunity to shape future experiences. We
include opportunities for people to provide
feedback on their experience chatting with
AIs at Meta or with AI images. Specifically,
in-app feedback tools will enable people
to report responses or image outputs they
consider unsafe or harmful. This feedback will
be reviewed by humans to determine if ourpolicies have been violated, and the results
will be used in ongoing model training to
improve safety and performance over time.

Feedback for AI stickers

SEPTEMBER 2023

16

4

Iterative evaluation
and benchmarking
Building generative AI features is an iterative process.

in our research paper for Llama 2, which forms

These experiences will improve over time with more

the basis for new experiences chatting with AIs.

feedback as the models are updated to align with the

For image models, we are also using prompt andneeds of our community. We are strongly committed

response interception to detect inputs, whether

to building generative AI responsibly, and we areadversarial or inadvertent, that could lead to

excited to bring new generative AI experiences topotentially harmful outputs, as well as classifiers

more people, equipped with specially designed

to detect policy-violating outputs. In these cases,

guardrails. User feedback is critical to this process

our models may refuse to generate some content.

and will help define how our models become more

Our approach to finding the right balance will

helpful and safe – and create even more entertaining

continue to evolve with community feedback.

and useful experiences.

Beyond the potential for models to produce

As we’ve seen with other generative AI models,

inappropriate content, there are a few limitations,they may sometimes return responses that are

which are common for this type of technology, that

inappropriate or inaccurate. In addition to fine-are important for our community to be aware of.

tuning the models to help reduce the likelihood
they will produce an unsafe response (for example,

Potential for bias:

a response that is potentially illegal, harmful, or• Our models were trained primarily on data in

offensive), we are also trying to strike the right

English, so performance may vary when usingbalance to make sure the models are helpful andother languages to interact with our generative

allow for creative expression. The tension between

AI features.

these concepts can sometimes result in our AIs•

When the distribution of language or different

sharing potentially inappropriate information in orderidentities represented in online datasets doesto helpfully complete a request. We have provided

not match the real world, it may manifest in

substantial details on our efforts to balance safety

potentially biased outputs, such as disparities

and helpfulness, including model evaluations on each,in gender or skin tone representation.

SEPTEMBER 2023

17

ITERATIVE EVALUATION AND BENCHMARKING•

For example, our evaluations on Llama 2 show that

occur when a model generates information

the pronoun “he” is generally more represented

that is fictional or unsupported by facts, but

in the training data in comparison to “she,” whichwith a tone of confidence. For the image model,

is a known issue for other similarly sized training

sometimes prompting the model for two

datasets. This could mean the models are learning

different objects – like a pen and a pencil – mayless about contexts that mention “she,” and moreproduce an incomplete result, with only two

frequently generate the pronoun “he,” which

pens or two pencils.

may have an effect on patterns of model outputs••

Due to the delay between the pretraining

related to gender. Similarly, the model may beof a model and the release of a generative AI

more likely to generate images of women forfeature, some responses won’t include thecertain types of jobs that have been historically or

most up-to-date information. For instance,

stereotypically associated with women.

if a recent current event was not included in

While we are taking steps through prompt

the training data, an AI wouldn’t have accessengineering and other techniques to help reduceto that information and couldn’t include it inpotential bias, we are exploring ways to balance

its response.

bias concerns with important considerations
about privacy and the use of sensitive
demographic data.

Feedback from the people who use this technology
will help our generative AI features become evensafer and more helpful. We are constantly tracking

Potential for inaccurate responses:• Another common limitation is the potential formodels to give responses that are inaccurate, outof date, or incomplete. For large language models,
this can sometimes lead to “hallucinations,” which

feedback and using it to measure the progress wemake through improvements. As people use thesefeatures, additional inputs will be fed back into themodel, ultimately shaping the future of generative
AI experiences on our platforms.

SEPTEMBER 2023

18