Gartner Research

Best Practices for the
Responsible Use of Natural
Language Technologies
By Gabriele Rigon, Anthony Mullen, Avivah Litan,
Svetlana Sicular, Wilco van Ginkel
26 April 2023

Best Practices for the Responsible Use of Natural
Language Technologies
Published 26 April 2023 - ID G00764808 - 15 min read
By Analyst(s): Gabriele Rigon, Anthony Mullen, Avivah Litan, Svetlana Sicular, Wilco vanGinkelInitiatives: Artiﬁcial Intelligence; Build Trust and Mature D&A Culture

Using NLTs responsibly is challenging due to the constant evolutionof the responsible AI discipline and of NLTs. Applications andsoftware engineering leaders should take the actions covered inthis research to mitigate ethical, liability and social risks arisingfrom the application of NLTs.

Overview
Key Findings■

The responsible AI framework is becoming more important and gets better
understood by vendors, buyers, society and legislators, as the requirements of thegeneral public and authorities for the responsible use of natural language
technologies (NLTs) are becoming more demanding.

■As NLT-enabled solutions are evolving rapidly and further hyped by generative AI,
leaders and employees feel urged to leverage such technologies to gain competitive
advantage or improve efﬁciency, often neglecting the security and privacy risks they
imply.

■

Due to its unstructured and casual adoption, responsible AI tooling for “biasmitigation” or “explainability enhancement” creates a false sense of security inorganizations.

■

The absence of a broadly accepted industry standard and commitment for
responsible AI and poorly deﬁned operationalization of high-level guidelines makeethical principles look good on paper, but ineffective in reality.

Gartner, Inc. | G00764808

Page 1 of 13

Recommendations■

Shield your organization from liability and social risks, such as data rights, humanrights and labor issues, by deﬁning ethics and governance guidelines for the use of
NLTs that anticipate regulations and developing a plan of actions to operationalize
such guidelines.

■

Mitigate security and privacy risks by accounting for the complete end-to-end ﬂow of
the applications, and by adopting privacy-enhancing and data scrambling
techniques as well as privacy-preserving practices for sharing data.

■

Mitigate risks entailed by bias and opacity of NLT models by adopting bias detectionand explainable AI (XAI) tools, and fairness and transparency practices andapproaches, throughout the life cycle of the NLT-enabled initiative.

■

Operationalize accountability and governance by deﬁning roles and responsibilities
and ensuring policies account for the needs of any internal or external stakeholder.

Introduction
The rapid evolution of NLTs and the sudden hype about generative AI tools (such as
ChatGPT and Bard), foundation models and large language models (LLMs) are creating
new use cases and opportunities for organizations to improve business outcomes (seeInnovation Insight for Artiﬁcial Intelligence Foundation Models and Innovation Insight for
Generative AI). But organizations can be exposed to well-known and new risks. The
excitement around the potential of AI technologies and the increasing availability ofrelated services may tempt organizations to lower their guard and move fast, neglecting
potential ethical and responsibility issues.
To shield their organizations from potential data rights, human rights and labor issues,
applications and software engineering leaders should understand NLTs’ limitations, theethical boundaries of their use and the risks they may pose during their life cycles.
This research is intended to provide guidelines to ensure NLT-enabled initiatives aredesigned, implemented and maintained in an ethical way, and to mitigate theaforementioned and other risks. In particular, applications and software engineering
leaders should consider the principles listed in Table 1. The corresponding tooling andpractices are illustrated in detail in the next section.

Gartner, Inc. | G00764808

Page 2 of 13

Table 1: Best Practices for the Responsible Use of NLTs
Human
Centricity and
Lawfulness■

Proportionality■

Compliance■

Misuseprevention■

Environmental
sustainability

Security andPrivacy

Fairness andTransparency

Accountability
andGovernance■

Overallapplication
security■Biasdetection andmonitoring■

Roles andResponsibilities— HITL■

Robustness
againstadversarial
attacks■

Content
moderation
techniques■

Documentation
enforcement■

End users’
feedback andacceptance■

Recurrent
audits■■■■Encryption
techniques
Data
scramblingprocedures
Privacyenhancingtechniques■

Confidential
information■

Syntheticdata

Responsible
data labelingandannotation■

Composite AI■

Know yourlanguagemodels■XAI testing■

XAI
monitoring

Source: Gartner

Gartner, Inc. | G00764808

Page 3 of 13

Analysis
Applying responsible AI principles to NLTs may increase time to market and costs.
Organizations may lose competitiveness in the race when trying to narrow down thescope of the use cases. On the other hand, not applying responsible AI principles is evenworse in the medium to long term. Many organizations are still unaware of AI’s
unintended consequences, they focus on mere regulatory compliance and tooling for bias
mitigation, explainability and privacy protection, and they neglect a disciplined AI ethics
and governance approach. The proliferation of “responsible AI” techniques of various
kinds (products, open-source software frameworks, individual tools) may generate
confusion and, when adopted without a structured approach, lull organizations into a
false sense of security.
At the same time, the responsible AI framework is becoming more prominent and getsbetter understood by vendors, buyers, society and legislators. Tech giants like Google,
Microsoft, IBM and Amazon have been investing in responsible AI programs and teams
for years now. On the other hand, frameworks like the recent Partnership on AI’s (PAI)
Responsible Practices for Synthetic Media which is supported by companies such as
Adobe, OpenAI, Bumble and Synthesia, reveal how industry standards are consolidating.
However, such standards, intended as a broadly accepted code of conduct for responsible
use of NLTs or AI, are still lacking. At the same time, the requirements of the general public
and the authorities for the responsible use of NLTs are becoming more demanding.
The following addresses the application of responsible AI principles to NLT-enabledinitiatives.

Human Centricity and Lawfulness
Proportionality
Decide whether the use of the NLT is proportional, justiﬁed by the accomplishment ofspeciﬁc legitimate interests of the business and accepted by users. This means, forexample, to avoid proﬁling customers or employees leveraging NLT-based speech or text
analytics tools without explicitly informing them or “just in case.” If the use of thetechnology cannot be justiﬁed along these lines, then the other responsible AI principles
become irrelevant.
Compliance■

Involve legal and compliance teams in the discussion in the early stages of theinitiative, when use cases are identiﬁed and initiatives are ideated.

Gartner, Inc. | G00764808

Page 4 of 13

■

Identify compliance requirements, related to internal AI governance policies, as wellas established and emerging regulatory frameworks, such as the General Data
Protection Regulation (GDPR) and AI-speciﬁc regulations in the European Union, theAlgorithmic Accountability Act of 2022 in the U.S. and Bill C-27 in Canada.
Compliance does not derive from the technologies per se, but depends on theirapplication.

■

Protect organizations’ confidential information and personal identifiable
information (PII), and mitigate copyright violation risks by introducing best practices
for human supervision and review in the workﬂows that include generative AI
techniques and products, such as the research preview version of OpenAI’s ChatGPT.

■

Once speciﬁc requirements are identiﬁed, shortlist the NLT vendors that holdcertiﬁcations to any relevant regional or industry-speciﬁc standards, such as the
GDPR, the ISO/IEC 27001, the Health Insurance Portability and Accountability Act(HIPAA) and the Federal Risk and Authorization Management Program (FedRAMP).

Misuse Prevention
Some NLTs pose speciﬁcal issues with respect to their misuse. Speech synthesis, forexample, can be used to replicate voices of unaware users for a number of malicious
purposes. The Microsoft research team developed VALL-E, a language model for speechsynthesis. It claims “if the model is generalized to unseen speakers in the real world, itshould include a protocol to ensure that the speaker approves the use of their voice and asynthesized speech detection model.”Environmental Sustainability
Carbon emissions reporting for large language models in particular is currently a topic ofdiscussion in many research communities. 1 It is unclear how the exact carbon footprintof machine learning (ML) models in general can be assessed, but, as explained in Quick
Answer: How Do I Make AI Environmentally Sustainable?, techniques exist that help create
and run models at the lowest carbon footprint, also improving business outcomes for thebusiness.

Security and Privacy
For Securing the End-to-End Flow Where NLT Is Used
Overall Application Security

Gartner, Inc. | G00764808

Page 5 of 13

Watch out for vulnerabilities in every component of the application that may expose it tointrusions and data breaches. This relates not only to the NLT-enabled modules, but also
to the implementation of custom front ends, such as chatbot widgets, which may offer
attackers opportunities to steal personal data in user authentication or online payment
journeys. Deciding whether on-premises vs. cloud deployment should be preferred is often
key to the overall security of the application, and the choice may be strictly conditioned byorganizations’ policies.
Robustness Against Adversarial Attacks
Test the robustness of the model against adversarial attacks, which are distortions or
perturbations of inputs intentionally designed to trick ML systems. Tools like TextAttack or
OpenAttack help developers improve the robustness of natural language processing (NLP)
models against this kind of attack. LLMs such as GPT-3 are not immune to them either,and attempts have been made to benchmark their robustness against a variety ofadversarial perturbations.
Encryption Techniques
Safeguard interactions with the NLT-enabled applications and protect personal data that
may be shared by users, by adopting encryption techniques. For example, end-to-end
encryption in enterprise-grade conversational AI applications ensures nobody other than
the sender and the receiver of the messages (the user and the chatbot) can read theconversation.
For Privacy Protection
Data Scrambling Procedures
Ensure the training corpus undergoes speciﬁc data scrambling procedures, as it may
contain PII and other sensitive data. Storing such information poses security andregulatory compliance risks, and personal data may bear patterns that will be learned andwill bias the output of the system. This is particularly crucial for recruitment activities, forexample CV screening or classiﬁcation tasks.
Protect personal data that users share while interacting with the NLT-enabled application
by applying data scrambling techniques and logging such interactions. This is particularly
relevant for speech transcription and analytics in contact center use cases, thetranscription of conversations between users and chatbots, and documents shared withneural machine translation (NMT) engines.
Gartner, Inc. | G00764808

Page 6 of 13

Synthetic Data
Although the use of synthetic data for NLP is at its early stages, such data is designed to
be intrinsically less prone to contain PII and bias, and it can represent a safer training
dataset for language models.
Privacy-Enhancing Techniques
Further mitigate privacy risks by including privacy enhancement techniques andframeworks, such as TensorFlow Privacy library or Syft, when building the language
model. For example, differential privacy allows sharing information about a dataset bydescribing the aggregated patterns of groups within the corpus, while withholding
information about individuals. Federated learning is another technique intended to ensure
conﬁdential data is not shared when training ML models. The Gartner research Three
Critical Use Cases for Privacy-Enhancing Computation Techniques recommends“treat[ing] privacy risk in AI model training by applying differential privacy and/or synthetic
data to protect identiﬁable data, and federated machine learning (ML) to enhance privacy
across training stages.”
Confidential Information
Do not use conﬁdential information to train the language models, unless that is done on
purpose, and be aware of information leak risks when submitting prompts to third-party
models. Avoid sharing conﬁdential information with unprotected applications andtransmit it only to authorized parties on a need-to-know basis. For example, conversations
with OpenAI’s ChatGPT application and, in particular, the information included in theprompts are to be used to train the model and to be reviewed by trainers. The privacy
protection measures taken by OpenAI in relation to ChatGPT started to be scrutinized bysome governments. 2

Fairness and Transparency
For Mitigating Bias Risks of NLT Applications
Bias Detection and Monitoring

Gartner, Inc. | G00764808

Page 7 of 13

■Training data. Reduce bias in training data by adopting speciﬁc bias detection
techniques, such as Google’s What-If, IBM’s AI Fairness 360, FairML and AllenNLP
Fairness module. The aim is to avoid features like gender, ethnicity and user’s
location, which are over- or under-represented. Such bias is ampliﬁed by thelanguage models and reﬂected in the output of the system. Gender bias is a wellknown problem, for instance, in machine translation. 3■

Models. Check and mitigate undesired bias that may hide in the language models.
Models amplify bias unintentionally left in training data, but may also generate
unfair inferences that may exploit spurious correlations in the data to produce
results where causal relationships are wrong. For example, a text classiﬁcation
model trained on online contents, such as movie reviews, can identify “spurious
correlations” between the name of certain actors and directors and apositive/negative sentiment, even though the names by themselves do not convey
any sentiment by themselves. 4■

Continuous monitoring. Monitor bias in the deployed application by deﬁning
dedicated metrics and thresholds, and activating notiﬁcations whenever anomalies
are detected in the inference.

Content Moderation Techniques
Filter out contents that are offensive, obscene, illegal or harmful in the training data byleveraging content moderation techniques. Such techniques are normally used to identify
inappropriate user-generated content in public communities on the internet. However,content moderation API endpoints should be leveraged to screen training data as well, if
such data derives from online forums and social network discussions. Among others,
Microsoft and Cohere offer content moderation services for text. Content moderation canalso be applied to deployed applications. For example, OpenAI provides a moderation
endpoint which is free to use to classify inputs and outputs of their own APIs.
Responsible Data Labeling and Annotation
Employ ethical data labeling and annotation techniques and practices. Favor teamdiversity to ensure labels are designed and assigned minimizing any form of bias or
discrimination and create equitable work conditions for the people involved. The processitself could be partially automated, so as to reduce the workers’ effort in such a delicate
and demanding task.
For Enhancing Transparency and Explainability
Gartner, Inc. | G00764808

Page 8 of 13

Composite AI
Improve the explainability of models by leveraging composite AI approaches, where MLdriven models are combined with rule- and ontology-based reasoning, or other symbolic,more transparent AI approaches. Language-speciﬁc rules, knowledge graphs andontologies are interpretable by humans, so they enhance the transparency of the models’
inferences. This holds true, for example, in the case of chatbots’ natural language
understanding (NLU).
Know Your Language Models
Gather’s insight about the training data used to create language models, including how alanguage model was built, is also critical to make the NLT-enabled inference more
understandable and intuitive for human beings.
Explainable AI (XAI) Testing
Adopt dedicated XAI tools and frameworks, such as open-source ones like IBM’s AI
Explainability 360 library, Microsoft’s InterpretML toolkit, Local Interpretable ModelAgnostic Explanations (LIME), SHapley Additive exPlanations (SHAP), or licensed
products like Amazon SageMaker Clarify. We recommend assessing the effectiveness ofsuch tools when applied to each speciﬁc language model. In principle, they may be
employed through the whole life cycle of the NLT-enabled initiative. When ﬁne-tuning themodels, they allow for a better understanding of the features and weights that are learned
and show whether predictions align with human intuition (intrinsic explainability).
Explainability is particularly relevant in the context of speech and text analytics
techniques applied to sensitive use cases, such as fraud detection and compliancemonitoring in the ﬁnancial domain.
XAI Monitoring
Monitor explainability in the deployed application by deﬁning dedicated metrics andthresholds, and activating notiﬁcations whenever anomalies are detected in the inference.

Large language models pose speciﬁc issues in terms of explainability because of theirsize and the overall complexity of the inference process. This applies also toconversational/generative AI applications grounded on foundation models, such as
OpenAI’s ChatGPT, ChatGPT-powered Bing and Google’s Bard.

Gartner, Inc. | G00764808

Page 9 of 13

Accountability and Governance
Roles and Responsibilities — HITL
Deﬁne roles and responsibilities, that is, who will be accountable for the performance of
the NLT-enabled system and its outputs. While developers and their leadership should betrained on AI ethics and governance, organizations should assess whether humans could,
should or must be part of the NLT workﬂow and supervise its processing case by case.
This is particularly relevant in the case of generative AI use cases, where contents are
generated automatically by the system (a ChatGPT-like application, also an NMT engine).
Depending on the level of risk and exposure of the output, human post-editing and fact
checking may be critical (human-in-the-loop, HITL).
Documentation Enforcement
Enforce project documentation, so as to keep track of decisions made about data
sourcing and labeling, as well as model deﬁnition, expectations and risks related tosecurity, compliance and reputation. Documentation will improve the overall explainabilityof the system. Frameworks that apply closely to dataset documentation and model
reporting are, for example, data sheets for datasets and model cards for model reporting.
End Users’ Feedback and Acceptance
Ensure the needs of stakeholders not involved in the development of the initiative (endusers such as employees and customers) are considered when deﬁning governanceguidelines. Customer and employee satisfaction and acceptance of the technology may
impact the adoption, its ROI and the very same feasibility of the NLT-enabled initiative.
Recurrent Audits
Set a calendar of recurrent audits of the initiative, focusing on key compliance aspects.
Disclaimer: The organizations or the tools proﬁled in this research are provided forillustrative purposes only, and do not constitute an exhaustive list of examples in this ﬁeld
nor an endorsement by Gartner of the referenced organizations or tools.

Gartner, Inc. | G00764808

Page 10 of 13

Evidence
This research is grounded on interactions (inquiries and brieﬁngs) with vendors andclients about responsible AI approaches adopted in NLTs. It is also based on previous
research conducted by Garner on responsible AI framework and public information on the
topic (including news articles, specialized blog posts, academic papers and company
policies).
1

Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model,

arxiv.org.
2

The Italian Data Protection Authority started to scrutinize OpenAI ChatGPT’s
compliance with the GDPR requirements in March 2023. This led to the temporary blockof the application in Italy and, sequentially, the creation of a task force by the European

Data Protection Board to share information on similar enforcement actions undertaken by
European Data Protection Authorities in April 2023. This ban was the ﬁrst one proclaimed
by a European country with the goal of protecting users’ privacy. Bans enforced by othercountries in early 2023 were justiﬁed on a different basis. China’s ban, for example,
occurred in February 2023 and seemed more related to the Chinese government’s concernthat content generated by ChatGPT may spread misinformation.
3

Gender Bias in Machine Translation, Association for Computational Linguistics.

4

Identifying Spurious Correlations for Robust Text Classiﬁcation (PDF), Association for

Computational Linguistics.

Gartner, Inc. | G00764808

Page 11 of 13

Acronym Key and Glossary Terms
FedRAMP

Federal Risk and Authorization Management Program

GDPR

General Data Protection Regulation

HIPAA

Health Insurance Portability and Accountability Act

HITL

human-in-the-loop

LIME

Local Interpretable Model-Agnostic Explanations

LLM

large language modelMLmachine learning

NLP

natural language processing

NLT

natural language technology

NLU

natural language understanding

NMT

neural machine translation

PII

personal identiﬁable information

SHAP

SHapley Additive exPlanations

XAI

explainable AI

Recommended by the Authors
Some documents may not be available as part of your current Gartner
subscription.
A Comprehensive Guide to Responsible AI
AI Ethics: Use 5 Common Principles as Your Starting Point
Design and Implement Human-in-the-Loop Interfaces for Control, Performance andTransparency of AI
How to Responsibly Use ChatGPT (and Other LLM Applications) in Your Business
Interactions
Three Critical Use Cases for Privacy-Enhancing Computation Techniques
Innovation Insight for Composite AI

Gartner, Inc. | G00764808

Page 12 of 13

© 2023 Gartner, Inc. and/or its afﬁliates. All rights reserved. Gartner is a registered trademark of
Gartner, Inc. and its afﬁliates. This publication may not be reproduced or distributed in any form
without Gartner's prior written permission. It consists of the opinions of Gartner's researchorganization, which should not be construed as statements of fact. While the information contained inthis publication has been obtained from sources believed to be reliable, Gartner disclaims all warranties
as to the accuracy, completeness or adequacy of such information. Although Gartner research may
address legal and ﬁnancial issues, Gartner does not provide legal or investment advice and its researchshould not be construed or used as such. Your access and use of this publication are governed by
Gartner's Usage Policy. Gartner prides itself on its reputation for independence and objectivity. Its
research is produced independently by its research organization without input or inﬂuence from any
third party. For further information, see "Guiding Principles on Independence and Objectivity." Gartner
research may not be used as input into or for the training or development of generative artiﬁcial
intelligence, machine learning, algorithms, software, or related technologies.

Gartner, Inc. | G00764808

Page 13 of 13

Table 1: Best Practices for the Responsible Use of NLTs
Human Centricity and
Lawfulness

Security and Privacy

Fairness and Transparency

Accountability andGovernance■ Proportionality■ Overall application security■ Bias detection and monitoring■ Roles and Responsibilities — HITL

■ Compliance■ Robustness against adversarial
attacks■ Content moderation techniques■ Documentation enforcement■ Encryption techniques■ Responsible data labeling andannotation■ End users’ feedback andacceptance■ Data scrambling procedures■ Composite AI■ Recurrent audits■ Privacy enhancing techniques■ Know your language models■ Confidential information■ XAI testing■ Synthetic data

■ XAI monitoring■ Misuse prevention■ Environmental sustainability

Source: Gartner

Gartner, Inc. | G00764808

Page 1A of 1A

Connect With Us
Get actionable, objective insight to deliver on your mission-criticalpriorities. Our expert guidance and tools enable faster, smarter
decisions and stronger performance. Contact us to become a client:
U.S.: 1 855 811 7593
International: +44 (0) 3330 607 044
Become a Client

Stay connected to the latest insights
Attend a Gartner conference
View Webinars© 2023 Gartner, Inc. All rights reserved. CTMKT_2323718