Package: azure-ai-contentsafety
Azure AI Content Safety client library for PythonAzure AI Content Safetydetects harmful user-generated and AI-generated content in applications and services. Content Safety includes text and image APIs that allow you to detect material that is harmful:Text Analysis API: Scans text for sexual content, violence, hate, and self-harm with multi-severity levels.Image Analysis API: Scans images for sexual content, violence, hate, and self-harm with multi-severity levels.Text Blocklist Management APIs: The default AI classifiers are sufficient for most content safety needs; however, you might need to screen for terms that are specific to your use case. You can create blocklists of terms to use with the Text API.DocumentationVarious documentation is available to help you get startedAPI reference documentationProduct documentationGetting startedPrerequisitesPython 3.7 or later is required to use this package.You need anAzure subscriptionto use this package.AnAzure AI Content Safetyresource, if no existing resource, you couldcreate a new one.Install the packagepipinstallazure-ai-contentsafetyAuthenticate the clientGet the endpointYou can find the endpoint for your Azure AI Content Safety service resource using theAzure PortalorAzure CLI:# Get the endpoint for the Azure AI Content Safety service resourceazcognitiveservicesaccountshow--name"resource-name"--resource-group"resource-group-name"--query"properties.endpoint"Create a ContentSafetyClient/BlocklistClient with API keyTo use an API key as thecredentialparameter.Step 1: Get the API key.
The API key can be found in theAzure Portalor by running the followingAzure CLIcommand:azcognitiveservicesaccountkeyslist--name"<resource-name>"--resource-group"<resource-group-name>"Step 2: Pass the key as a string into an instance ofAzureKeyCredential.fromazure.core.credentialsimportAzureKeyCredentialfromazure.ai.contentsafetyimportContentSafetyClient,BlocklistClientendpoint="https://<my-custom-subdomain>.cognitiveservices.azure.com/"credential=AzureKeyCredential("<api_key>")content_safety_client=ContentSafetyClient(endpoint,credential)blocklist_client=BlocklistClient(endpoint,credential)Create a ContentSafetyClient/BlocklistClient with Microsoft Entra ID token credentialStep 1: Enable Microsoft Entra ID for your resource.
Please refer to this documentAuthenticate with Microsoft Entra IDfor the steps to enable Microsoft Entra ID for your resource.The main steps are:Create resource with a custom subdomain.Create Service Principal and assign Cognitive Services User role to it.Step 2: Set the values of the client ID, tenant ID, and client secret of the Microsoft Entra application as environment variables:AZURE_CLIENT_ID,AZURE_TENANT_ID,AZURE_CLIENT_SECRET.DefaultAzureCredential will use the values from these environment variables.fromazure.identityimportDefaultAzureCredentialfromazure.ai.contentsafetyimportContentSafetyClient,BlocklistClientendpoint="https://<my-custom-subdomain>.cognitiveservices.azure.com/"credential=DefaultAzureCredential()content_safety_client=ContentSafetyClient(endpoint,credential)blocklist_client=BlocklistClient(endpoint,credential)Key conceptsAvailable featuresThere are different types of analysis available from this service. The following table describes the currently available APIs.FeatureDescriptionText Analysis APIScans text for sexual content, violence, hate, and self-harm with multi-severity levels.Image Analysis APIScans images for sexual content, violence, hate, and self-harm with multi-severity levels.Text Blocklist Management APIsThe default AI classifiers are sufficient for most content safety needs. However, you might need to screen for terms that are specific to your use case. You can create blocklists of terms to use with the Text API.Harm categoriesContent Safety recognizes four distinct categories of objectionable content.CategoryDescriptionHateHate and fairness-related harms refer to any content that attacks or uses pejorative or discriminatory language with reference to a person or identity group based on certain differentiating attributes of these groups including but not limited to race, ethnicity, nationality, gender identity and expression, sexual orientation, religion, immigration status, ability status, personal appearance, and body size.SexualSexual describes language related to anatomical organs and genitals, romantic relationships, acts portrayed in erotic or affectionate terms, pregnancy, physical sexual acts, including those portrayed as an assault or a forced sexual violent act against one's will, prostitution, pornography, and abuse.ViolenceViolence describes language related to physical actions intended to hurt, injure, damage, or kill someone or something; describes weapons, guns and related entities, such as manufacturers, associations, legislation, and so on.Self-harmSelf-harm describes language related to physical actions intended to purposely hurt, injure, damage one's body or kill oneself.Classification can be multi-labeled. For example, when a text sample goes through the text moderation model, it could be classified as both Sexual content and Violence.Severity levelsEvery harm category the service applies also comes with a severity level rating. The severity level is meant to indicate the severity of the consequences of showing the flagged content.Text: The current version of the text model supports the full 0-7 severity scale. By default, the response will output 4 values: 0, 2, 4, and 6. Each two adjacent levels are mapped to a single level. Users could use "outputType" in request and set it as "EightSeverityLevels" to get 8 values in output: 0,1,2,3,4,5,6,7. You can refertext content severity levels definitionsfor details.[0,1] -> 0[2,3] -> 2[4,5] -> 4[6,7] -> 6Image: The current version of the image model supports the trimmed version of the full 0-7 severity scale. The classifier only returns severities 0, 2, 4, and 6; each two adjacent levels are mapped to a single level. You can referimage content severity levels definitionsfor details.[0,1] -> 0[2,3] -> 2[4,5] -> 4[6,7] -> 6Text blocklist managementFollowing operations are supported to manage your text blocklist:Create or modify a blocklistList all blocklistsGet a blocklist by blocklistNameAdd blocklistItems to a blocklistRemove blocklistItems from a blocklistList all blocklistItems in a blocklist by blocklistNameGet a blocklistItem in a blocklist by blocklistItemId and blocklistNameDelete a blocklist and all of its blocklistItemsYou can set the blocklists you want to use when analyze text, then you can get blocklist match result from returned response.ExamplesThe following section provides several code snippets covering some of the most common Content Safety service tasks, including:Analyze textAnalyze imageManage text blocklistPlease refer tosample datafor the data used here. For more samples, please refer tosamples.Analyze textAnalyze text without blocklistsimportosfromazure.ai.contentsafetyimportContentSafetyClientfromazure.ai.contentsafety.modelsimportTextCategoryfromazure.core.credentialsimportAzureKeyCredentialfromazure.core.exceptionsimportHttpResponseErrorfromazure.ai.contentsafety.modelsimportAnalyzeTextOptionskey=os.environ["CONTENT_SAFETY_KEY"]endpoint=os.environ["CONTENT_SAFETY_ENDPOINT"]# Create a Content Safety clientclient=ContentSafetyClient(endpoint,AzureKeyCredential(key))# Construct a requestrequest=AnalyzeTextOptions(text="You are an idiot")# Analyze texttry:response=client.analyze_text(request)exceptHttpResponseErrorase:print("Analyze text failed.")ife.error:print(f"Error code:{e.error.code}")print(f"Error message:{e.error.message}")raiseprint(e)raisehate_result=next(itemforiteminresponse.categories_analysisifitem.category==TextCategory.HATE)self_harm_result=next(itemforiteminresponse.categories_analysisifitem.category==TextCategory.SELF_HARM)sexual_result=next(itemforiteminresponse.categories_analysisifitem.category==TextCategory.SEXUAL)violence_result=next(itemforiteminresponse.categories_analysisifitem.category==TextCategory.VIOLENCE)ifhate_result:print(f"Hate severity:{hate_result.severity}")ifself_harm_result:print(f"SelfHarm severity:{self_harm_result.severity}")ifsexual_result:print(f"Sexual severity:{sexual_result.severity}")ifviolence_result:print(f"Violence severity:{violence_result.severity}")Analyze text with blocklistsimportosfromazure.ai.contentsafetyimportContentSafetyClientfromazure.core.credentialsimportAzureKeyCredentialfromazure.ai.contentsafety.modelsimportAnalyzeTextOptionsfromazure.core.exceptionsimportHttpResponseErrorkey=os.environ["CONTENT_SAFETY_KEY"]endpoint=os.environ["CONTENT_SAFETY_ENDPOINT"]# Create a Content Safety clientclient=ContentSafetyClient(endpoint,AzureKeyCredential(key))blocklist_name="TestBlocklist"input_text="I h*te you and I want to k*ll you."try:# After you edit your blocklist, it usually takes effect in 5 minutes, please wait some time before analyzing with blocklist after editing.analysis_result=client.analyze_text(AnalyzeTextOptions(text=input_text,blocklist_names=[blocklist_name],halt_on_blocklist_hit=False))ifanalysis_resultandanalysis_result.blocklists_match:print("\nBlocklist match results: ")formatch_resultinanalysis_result.blocklists_match:print(f"BlocklistName:{match_result.blocklist_name}, BlockItemId:{match_result.blocklist_item_id}, "f"BlockItemText:{match_result.blocklist_item_text}")exceptHttpResponseErrorase:print("\nAnalyze text failed: ")ife.error:print(f"Error code:{e.error.code}")print(f"Error message:{e.error.message}")raiseprint(e)raiseAnalyze imageimportosfromazure.ai.contentsafetyimportContentSafetyClientfromazure.ai.contentsafety.modelsimportImageCategoryfromazure.core.credentialsimportAzureKeyCredentialfromazure.core.exceptionsimportHttpResponseErrorfromazure.ai.contentsafety.modelsimportAnalyzeImageOptions,ImageDatakey=os.environ["CONTENT_SAFETY_KEY"]endpoint=os.environ["CONTENT_SAFETY_ENDPOINT"]image_path=os.path.abspath(os.path.join(os.path.abspath(__file__),"..","./sample_data/image.jpg"))# Create a Content Safety clientclient=ContentSafetyClient(endpoint,AzureKeyCredential(key))# Build requestwithopen(image_path,"rb")asfile:request=AnalyzeImageOptions(image=ImageData(content=file.read()))# Analyze imagetry:response=client.analyze_image(request)exceptHttpResponseErrorase:print("Analyze image failed.")ife.error:print(f"Error code:{e.error.code}")print(f"Error message:{e.error.message}")raiseprint(e)raisehate_result=next(itemforiteminresponse.categories_analysisifitem.category==ImageCategory.HATE)self_harm_result=next(itemforiteminresponse.categories_analysisifitem.category==ImageCategory.SELF_HARM)sexual_result=next(itemforiteminresponse.categories_analysisifitem.category==ImageCategory.SEXUAL)violence_result=next(itemforiteminresponse.categories_analysisifitem.category==ImageCategory.VIOLENCE)ifhate_result:print(f"Hate severity:{hate_result.severity}")ifself_harm_result:print(f"SelfHarm severity:{self_harm_result.severity}")ifsexual_result:print(f"Sexual severity:{sexual_result.severity}")ifviolence_result:print(f"Violence severity:{violence_result.severity}")Manage text blocklistCreate or update text blocklistimportosfromazure.ai.contentsafetyimportBlocklistClientfromazure.ai.contentsafety.modelsimportTextBlocklistfromazure.core.credentialsimportAzureKeyCredentialfromazure.core.exceptionsimportHttpResponseErrorkey=os.environ["CONTENT_SAFETY_KEY"]endpoint=os.environ["CONTENT_SAFETY_ENDPOINT"]# Create a Blocklist clientclient=BlocklistClient(endpoint,AzureKeyCredential(key))blocklist_name="TestBlocklist"blocklist_description="Test blocklist management."try:blocklist=client.create_or_update_text_blocklist(blocklist_name=blocklist_name,options=TextBlocklist(blocklist_name=blocklist_name,description=blocklist_description),)ifblocklist:print("\nBlocklist created or updated: ")print(f"Name:{blocklist.blocklist_name}, Description:{blocklist.description}")exceptHttpResponseErrorase:print("\nCreate or update text blocklist failed: ")ife.error:print(f"Error code:{e.error.code}")print(f"Error message:{e.error.message}")raiseprint(e)raiseList text blocklistsimportosfromazure.ai.contentsafetyimportBlocklistClientfromazure.core.credentialsimportAzureKeyCredentialfromazure.core.exceptionsimportHttpResponseErrorkey=os.environ["CONTENT_SAFETY_KEY"]endpoint=os.environ["CONTENT_SAFETY_ENDPOINT"]# Create a Blocklist clientclient=BlocklistClient(endpoint,AzureKeyCredential(key))try:blocklists=client.list_text_blocklists()ifblocklists:print("\nList blocklists: ")forblocklistinblocklists:print(f"Name:{blocklist.blocklist_name}, Description:{blocklist.description}")exceptHttpResponseErrorase:print("\nList text blocklists failed: ")ife.error:print(f"Error code:{e.error.code}")print(f"Error message:{e.error.message}")raiseprint(e)raiseGet text blocklistimportosfromazure.ai.contentsafetyimportBlocklistClientfromazure.core.credentialsimportAzureKeyCredentialfromazure.core.exceptionsimportHttpResponseErrorkey=os.environ["CONTENT_SAFETY_KEY"]endpoint=os.environ["CONTENT_SAFETY_ENDPOINT"]# Create a Blocklist clientclient=BlocklistClient(endpoint,AzureKeyCredential(key))blocklist_name="TestBlocklist"try:blocklist=client.get_text_blocklist(blocklist_name=blocklist_name)ifblocklist:print("\nGet blocklist: ")print(f"Name:{blocklist.blocklist_name}, Description:{blocklist.description}")exceptHttpResponseErrorase:print("\nGet text blocklist failed: ")ife.error:print(f"Error code:{e.error.code}")print(f"Error message:{e.error.message}")raiseprint(e)raiseDelete text blocklistimportosfromazure.ai.contentsafetyimportBlocklistClientfromazure.core.credentialsimportAzureKeyCredentialfromazure.core.exceptionsimportHttpResponseErrorkey=os.environ["CONTENT_SAFETY_KEY"]endpoint=os.environ["CONTENT_SAFETY_ENDPOINT"]# Create a Blocklist clientclient=BlocklistClient(endpoint,AzureKeyCredential(key))blocklist_name="TestBlocklist"try:client.delete_text_blocklist(blocklist_name=blocklist_name)print(f"\nDeleted blocklist:{blocklist_name}")exceptHttpResponseErrorase:print("\nDelete blocklist failed:")ife.error:print(f"Error code:{e.error.code}")print(f"Error message:{e.error.message}")raiseprint(e)raiseAdd blockItemsimportosfromazure.ai.contentsafetyimportBlocklistClientfromazure.ai.contentsafety.modelsimportAddOrUpdateTextBlocklistItemsOptions,TextBlocklistItemfromazure.core.credentialsimportAzureKeyCredentialfromazure.core.exceptionsimportHttpResponseErrorkey=os.environ["CONTENT_SAFETY_KEY"]endpoint=os.environ["CONTENT_SAFETY_ENDPOINT"]# Create a Blocklist clientclient=BlocklistClient(endpoint,AzureKeyCredential(key))blocklist_name="TestBlocklist"block_item_text_1="k*ll"block_item_text_2="h*te"block_items=[TextBlocklistItem(text=block_item_text_1),TextBlocklistItem(text=block_item_text_2)]try:result=client.add_or_update_blocklist_items(blocklist_name=blocklist_name,options=AddOrUpdateTextBlocklistItemsOptions(blocklist_items=block_items))forblock_iteminresult.blocklist_items:print(f"BlockItemId:{block_item.blocklist_item_id}, Text:{block_item.text}, Description:{block_item.description}")exceptHttpResponseErrorase:print("\nAdd block items failed: ")ife.error:print(f"Error code:{e.error.code}")print(f"Error message:{e.error.message}")raiseprint(e)raiseList blockItemsimportosfromazure.ai.contentsafetyimportBlocklistClientfromazure.core.credentialsimportAzureKeyCredentialfromazure.core.exceptionsimportHttpResponseErrorkey=os.environ["CONTENT_SAFETY_KEY"]endpoint=os.environ["CONTENT_SAFETY_ENDPOINT"]# Create a Blocklist clientclient=BlocklistClient(endpoint,AzureKeyCredential(key))blocklist_name="TestBlocklist"try:block_items=client.list_text_blocklist_items(blocklist_name=blocklist_name)ifblock_items:print("\nList block items: ")forblock_iteminblock_items:print(f"BlockItemId:{block_item.blocklist_item_id}, Text:{block_item.text}, "f"Description:{block_item.description}")exceptHttpResponseErrorase:print("\nList block items failed: ")ife.error:print(f"Error code:{e.error.code}")print(f"Error message:{e.error.message}")raiseprint(e)raiseGet blockItemimportosfromazure.ai.contentsafetyimportBlocklistClientfromazure.core.credentialsimportAzureKeyCredentialfromazure.ai.contentsafety.modelsimportTextBlocklistItem,AddOrUpdateTextBlocklistItemsOptionsfromazure.core.exceptionsimportHttpResponseErrorkey=os.environ["CONTENT_SAFETY_KEY"]endpoint=os.environ["CONTENT_SAFETY_ENDPOINT"]# Create a Blocklist clientclient=BlocklistClient(endpoint,AzureKeyCredential(key))blocklist_name="TestBlocklist"block_item_text_1="k*ll"try:# Add a blockItemadd_result=client.add_or_update_blocklist_items(blocklist_name=blocklist_name,options=AddOrUpdateTextBlocklistItemsOptions(blocklist_items=[TextBlocklistItem(text=block_item_text_1)]),)ifnotadd_resultornotadd_result.blocklist_itemsorlen(add_result.blocklist_items)<=0:raiseRuntimeError("BlockItem not created.")block_item_id=add_result.blocklist_items[0].blocklist_item_id# Get this blockItem by blockItemIdblock_item=client.get_text_blocklist_item(blocklist_name=blocklist_name,blocklist_item_id=block_item_id)print("\nGet blockitem: ")print(f"BlockItemId:{block_item.blocklist_item_id}, Text:{block_item.text}, Description:{block_item.description}")exceptHttpResponseErrorase:print("\nGet block item failed: ")ife.error:print(f"Error code:{e.error.code}")print(f"Error message:{e.error.message}")raiseprint(e)raiseRemove blockItemsimportosfromazure.ai.contentsafetyimportBlocklistClientfromazure.core.credentialsimportAzureKeyCredentialfromazure.ai.contentsafety.modelsimport(TextBlocklistItem,AddOrUpdateTextBlocklistItemsOptions,RemoveTextBlocklistItemsOptions,)fromazure.core.exceptionsimportHttpResponseErrorkey=os.environ["CONTENT_SAFETY_KEY"]endpoint=os.environ["CONTENT_SAFETY_ENDPOINT"]# Create a Blocklist clientclient=BlocklistClient(endpoint,AzureKeyCredential(key))blocklist_name="TestBlocklist"block_item_text_1="k*ll"try:# Add a blockItemadd_result=client.add_or_update_blocklist_items(blocklist_name=blocklist_name,options=AddOrUpdateTextBlocklistItemsOptions(blocklist_items=[TextBlocklistItem(text=block_item_text_1)]),)ifnotadd_resultornotadd_result.blocklist_itemsorlen(add_result.blocklist_items)<=0:raiseRuntimeError("BlockItem not created.")block_item_id=add_result.blocklist_items[0].blocklist_item_id# Remove this blockItem by blockItemIdclient.remove_blocklist_items(blocklist_name=blocklist_name,options=RemoveTextBlocklistItemsOptions(blocklist_item_ids=[block_item_id]))print(f"\nRemoved blockItem:{add_result.blocklist_items[0].blocklist_item_id}")exceptHttpResponseErrorase:print("\nRemove block item failed: ")ife.error:print(f"Error code:{e.error.code}")print(f"Error message:{e.error.message}")raiseprint(e)raiseTroubleshootingGeneralAzure AI Content Safety client library will raise exceptions defined inAzure Core. Error codes are defined as below:Error CodePossible reasonsSuggestionsInvalidRequestBodyOne or more fields in the request body do not match the API definition.1. Check the API version you specified in the API call.2. Check the corresponding API definition for the API version you selected.InvalidResourceNameThe resource name you specified in the URL does not meet the requirements, like the blocklist name, blocklist term ID, etc.1. Check the API version you specified in the API call.2. Check whether the given name has invalid characters according to the API definition.ResourceNotFoundThe resource you specified in the URL may not exist, like the blocklist name.1. Check the API version you specified in the API call.2. Double check the existence of the resource specified in the URL.InternalErrorSome unexpected situations on the server side have been triggered.1. You may want to retry a few times after a small period and see it the issue happens again.2. Contact Azure Support if this issue persists.ServerBusyThe server side cannot process the request temporarily.1. You may want to retry a few times after a small period and see it the issue happens again.2.Contact Azure Support if this issue persists.TooManyRequestsThe current RPS has exceeded the quota for your current SKU.1. Check the pricing table to understand the RPS quota.2.Contact Azure Support if you need more QPS.LoggingThis library uses the standardlogginglibrary for logging.Basic information about HTTP sessions (URLs, headers, etc.) is logged atINFOlevel.DetailedDEBUGlevel logging, including request/response bodies andunredactedheaders, can be enabled on the client or per-operation with thelogging_enablekeyword argument.See full SDK logging documentation with exampleshere.Optional ConfigurationOptional keyword arguments can be passed in at the client and per-operation level. The azure-corereference documentationdescribes available configurations for retries, logging, transport protocols, and more.Next stepsAdditional documentationFor more extensive documentation on Azure Content Safety, see theAzure AI Content Safetyon docs.microsoft.com.ContributingThis project welcomes contributions and suggestions. Most contributions require
you to agree to a Contributor License Agreement (CLA) declaring that you have
the right to, and actually do, grant us the rights to use your contribution.
For details, visithttps://cla.microsoft.com.When you submit a pull request, a CLA-bot will automatically determine whether
you need to provide a CLA and decorate the PR appropriately (e.g., label,
comment). Simply follow the instructions provided by the bot. You will only
need to do this once across all repos using our CLA.This project has adopted theMicrosoft Open Source Code of Conduct. For more information,
see the Code of Conduct FAQ or contactopencode@microsoft.comwith any
additional questions or comments.