Package: assemblyai
AssemblyAI's Python SDKBuild with AI models that can transcribe and understand audioWith a single API call, get access to AI models built on the latest AI breakthroughs to transcribe and understand audio and speech data securely at large scale.OverviewAssemblyAI's Python SDKOverviewDocumentationQuick StartInstallationExamplesCore ExamplesLeMUR ExamplesAudio Intelligence ExamplesReal-Time ExamplesPlaygroundsAdvancedHow the SDK handles Default ConfigurationsDefining DefaultsOverriding DefaultsSynchronous vs AsynchronousPolling IntervalsRetrieving Existing TranscriptsRetrieving a Single TranscriptRetrieving Multiple Transcripts as a GroupRetrieving Transcripts AsynchronouslyDocumentationVisit ourAssemblyAI API Documentationto get an overview of our models!Quick StartInstallationpipinstall-UassemblyaiExamplesBefore starting, you need to set the API key. If you don't have one yet,sign up for one!importassemblyaiasaai# set the API keyaai.settings.api_key=f"{ASSEMBLYAI_API_KEY}"Core ExamplesTranscribe a Local Audio Fileimportassemblyaiasaaitranscriber=aai.Transcriber()transcript=transcriber.transcribe("./my-local-audio-file.wav")print(transcript.text)Transcribe an URLimportassemblyaiasaaitranscriber=aai.Transcriber()transcript=transcriber.transcribe("https://example.org/audio.mp3")print(transcript.text)Export Subtitles of an Audio Fileimportassemblyaiasaaitranscriber=aai.Transcriber()transcript=transcriber.transcribe("https://example.org/audio.mp3")# in SRT formatprint(transcript.export_subtitles_srt())# in VTT formatprint(transcript.export_subtitles_vtt())List all Sentences and Paragraphsimportassemblyaiasaaitranscriber=aai.Transcriber()transcript=transcriber.transcribe("https://example.org/audio.mp3")sentences=transcript.get_sentences()forsentenceinsentences:print(sentence.text)paragraphs=transcript.get_paragraphs()forparagraphinparagraphs:print(paragraph.text)Search for Words in a Transcriptimportassemblyaiasaaitranscriber=aai.Transcriber()transcript=transcriber.transcribe("https://example.org/audio.mp3")matches=transcript.word_search(["price","product"])formatchinmatches:print(f"Found '{match.text}'{match.count}times in the transcript")Add Custom Spellings on a Transcriptimportassemblyaiasaaiconfig=aai.TranscriptionConfig()config.set_custom_spelling({"Kubernetes":["k8s"],"SQL":["Sequel"],})transcriber=aai.Transcriber()transcript=transcriber.transcribe("https://example.org/audio.mp3",config)print(transcript.text)LeMUR ExamplesUse LeMUR to Summarize Multiple Transcriptsimportassemblyaiasaaitranscriber=aai.Transcriber()transcript_group=transcriber.transcribe_group(["https://example.org/customer1.mp3","https://example.org/customer2.mp3",],)result=transcript_group.lemur.summarize(context="Customers asking for cars",answer_format="TLDR")print(result.response)Use LeMUR to Ask Questions on a Single Transcriptimportassemblyaiasaaitranscriber=aai.Transcriber()transcript=transcriber.transcribe("https://example.org/customer.mp3")# ask some questionsquestions=[aai.LemurQuestion(question="What car was the customer interested in?"),aai.LemurQuestion(question="What price range is the customer looking for?"),]result=transcript.lemur.question(questions)forqinresult.response:print(f"Question:{q.question}")print(f"Answer:{q.answer}")Use LeMUR to Ask for Action Items from a Single Transcriptimportassemblyaiasaaitranscriber=aai.Transcriber()transcript=transcriber.transcribe("https://example.org/customer.mp3")result=transcript.lemur.action_items(context="Customers asking for help with resolving their problem",answer_format="Three bullet points",)print(result.response)Use LeMUR to Ask Anything with a Custom Promptimportassemblyaiasaaitranscriber=aai.Transcriber()transcript=transcriber.transcribe("https://example.org/customer.mp3")result=transcript.lemur.task("You are a helpful coach. Provide an analysis of the transcript ""and offer areas to improve with exact quotes. Include no preamble. ""Start with an overall summary then get into the examples with feedback.",)print(result.response)Use LeMUR to with Input Textimportassemblyaiasaaitranscriber=aai.Transcriber()config=aai.TranscriptionConfig(speaker_labels=True,)transcript=transcriber.transcribe("https://example.org/customer.mp3",config=config)# Example converting speaker label utterances into LeMUR input texttext=""foruttintranscript.utterances:text+=f"Speaker{utt.speaker}:\n{utt.text}\n"result=aai.Lemur().task("You are a helpful coach. Provide an analysis of the transcript ""and offer areas to improve with exact quotes. Include no preamble. ""Start with an overall summary then get into the examples with feedback.",input_text=text)print(result.response)Delete data previously sent to LeMURimportassemblyaiasaai# Create a transcript and a corresponding LeMUR request that may contain senstive information.transcriber=aai.Transcriber()transcript_group=transcriber.transcribe_group(["https://example.org/customer1.mp3",],)result=transcript_group.lemur.summarize(context="Customers providing sensitive, personally identifiable information",answer_format="TLDR")# Get the request ID from the LeMUR responserequest_id=result.request_id# Now we can delete the data about this requestdeletion_result=aai.Lemur.purge_request_data(request_id)print(deletion_result)Audio Intelligence ExamplesPII Redact a Transcriptimportassemblyaiasaaiconfig=aai.TranscriptionConfig()config.set_redact_pii(# What should be redactedpolicies=[aai.PIIRedactionPolicy.credit_card_number,aai.PIIRedactionPolicy.email_address,aai.PIIRedactionPolicy.location,aai.PIIRedactionPolicy.person_name,aai.PIIRedactionPolicy.phone_number,],# How it should be redactedsubstitution=aai.PIISubstitutionPolicy.hash,)transcriber=aai.Transcriber()transcript=transcriber.transcribe("https://example.org/audio.mp3",config)To request a copy of the original audio file with the redacted information "beeped" out, setredact_pii_audio=Truein the config.
Once theTranscriptobject is returned, you can access the URL of the redacted audio file withget_redacted_audio_url, or save the redacted audio directly to disk withsave_redacted_audio.importassemblyaiasaaitranscript=aai.Transcriber().transcribe("https://example.org/audio.mp3",config=aai.TranscriptionConfig(redact_pii=True,redact_pii_policies=[aai.PIIRedactionPolicy.person_name],redact_pii_audio=True))redacted_audio_url=transcript.get_redacted_audio_url()transcript.save_redacted_audio("redacted_audio.mp3")Read more about PII redaction here.Summarize the content of a transcript over timeimportassemblyaiasaaitranscriber=aai.Transcriber()transcript=transcriber.transcribe("https://example.org/audio.mp3",config=aai.TranscriptionConfig(auto_chapters=True))forchapterintranscript.chapters:print(f"Summary:{chapter.summary}")# A one paragraph summary of the content spoken during this timeframeprint(f"Start:{chapter.start}, End:{chapter.end}")# Timestamps (in milliseconds) of the chapterprint(f"Healine:{chapter.headline}")# A single sentence summary of the content spoken during this timeframeprint(f"Gist:{chapter.gist}")# An ultra-short summary, just a few words, of the content spoken during this timeframeRead more about auto chapters here.Summarize the content of a transcriptimportassemblyaiasaaitranscriber=aai.Transcriber()transcript=transcriber.transcribe("https://example.org/audio.mp3",config=aai.TranscriptionConfig(summarization=True))print(transcript.summary)By default, the summarization model will beinformativeand the summarization type will bebullets.Read more about summarization models and types here.To change the model and/or type, pass additional parameters to theTranscriptionConfig:config=aai.TranscriptionConfig(summarization=True,summary_model=aai.SummarizationModel.catchy,summary_type=aai.SummarizationType.headline)Detect Sensitive Content in a Transcriptimportassemblyaiasaaitranscriber=aai.Transcriber()transcript=transcriber.transcribe("https://example.org/audio.mp3",config=aai.TranscriptionConfig(content_safety=True))# Get the parts of the transcript which were flagged as sensitiveforresultintranscript.content_safety.results:print(result.text)# sensitive text snippetprint(result.timestamp.start)print(result.timestamp.end)forlabelinresult.labels:print(label.label)# content safety categoryprint(label.confidence)# model's confidence that the text is in this categoryprint(label.severity)# severity of the text in relation to the category# Get the confidence of the most common labels in relation to the entire audio fileforlabel,confidenceintranscript.content_safety.summary.items():print(f"{confidence*100}% confident that the audio contains{label}")# Get the overall severity of the most common labels in relation to the entire audio fileforlabel,severity_confidenceintranscript.content_safety.severity_score_summary.items():print(f"{severity_confidence.low*100}% confident that the audio contains low-severity{label}")print(f"{severity_confidence.medium*100}% confident that the audio contains mid-severity{label}")print(f"{severity_confidence.high*100}% confident that the audio contains high-severity{label}")Read more about the content safety categories.By default, the content safety model will only include labels with a confidence greater than 0.5 (50%). To change this, passcontent_safety_confidence(as an integer percentage between 25 and 100, inclusive) to theTranscriptionConfig:config=aai.TranscriptionConfig(content_safety=True,content_safety_confidence=80,# only include labels with a confidence greater than 80%)Analyze the Sentiment of Sentences in a Transcriptimportassemblyaiasaaitranscriber=aai.Transcriber()transcript=transcriber.transcribe("https://example.org/audio.mp3",config=aai.TranscriptionConfig(sentiment_analysis=True))forsentiment_resultintranscript.sentiment_analysis:print(sentiment_result.text)print(sentiment_result.sentiment)# POSITIVE, NEUTRAL, or NEGATIVEprint(sentiment_result.confidence)print(f"Timestamp:{sentiment_result.start}-{sentiment_result.end}")Ifspeaker_labelsis also enabled, then each sentiment analysis result will also include aspeakerfield.# ...config=aai.TranscriptionConfig(sentiment_analysis=True,speaker_labels=True)# ...forsentiment_resultintranscript.sentiment_analysis:print(sentiment_result.speaker)Read more about sentiment analysis here.Identify Entities in a Transcriptimportassemblyaiasaaitranscriber=aai.Transcriber()transcript=transcriber.transcribe("https://example.org/audio.mp3",config=aai.TranscriptionConfig(entity_detection=True))forentityintranscript.entities:print(entity.text)# i.e. "Dan Gilbert"print(entity.entity_type)# i.e. EntityType.personprint(f"Timestamp:{entity.start}-{entity.end}")Read more about entity detection here.Detect Topics in a Transcript (IAB Classification)importassemblyaiasaaitranscriber=aai.Transcriber()transcript=transcriber.transcribe("https://example.org/audio.mp3",config=aai.TranscriptionConfig(iab_categories=True))# Get the parts of the transcript that were tagged with topicsforresultintranscript.iab_categories.results:print(result.text)print(f"Timestamp:{result.timestamp.start}-{result.timestamp.end}")forlabelinresult.labels:print(label.label)# topicprint(label.relevance)# how relevant the label is for the portion of text# Get a summary of all topics in the transcriptforlabel,relevanceintranscript.iab_categories.summary.items():print(f"Audio is{relevance*100}% relevant to{label}")Read more about IAB classification here.Identify Important Words and Phrases in a Transcriptimportassemblyaiasaaitranscriber=aai.Transcriber()transcript=transcriber.transcribe("https://example.org/audio.mp3",config=aai.TranscriptionConfig(auto_highlights=True))forresultintranscript.auto_highlights.results:print(result.text)# the important phraseprint(result.rank)# relevancy of the phraseprint(result.count)# number of instances of the phrasefortimestampinresult.timestamps:print(f"Timestamp:{timestamp.start}-{timestamp.end}")Read more about auto highlights here.Real-Time ExamplesRead more about our Real-Time service.Stream your Microphone in Real-Timeimportassemblyaiasaaidefon_open(session_opened:aai.RealtimeSessionOpened):"This function is called when the connection has been established."print("Session ID:",session_opened.session_id)defon_data(transcript:aai.RealtimeTranscript):"This function is called when a new transcript has been received."ifnottranscript.text:returnifisinstance(transcript,aai.RealtimeFinalTranscript):print(transcript.text,end="\r\n")else:print(transcript.text,end="\r")defon_error(error:aai.RealtimeError):"This function is called when an error occurs."print("An error occured:",error)defon_close():"This function is called when the connection has been closed."print("Closing Session")# Create the Real-Time transcribertranscriber=aai.RealtimeTranscriber(on_data=on_data,on_error=on_error,sample_rate=44_100,on_open=on_open,# optionalon_close=on_close,# optional)# Start the connectiontranscriber.connect()# Open a microphone streammicrophone_stream=aai.extras.MicrophoneStream()# Press CTRL+C to aborttranscriber.stream(microphone_stream)transcriber.close()Transcribe a Local Audio File in Real-Timeimportassemblyaiasaaidefon_data(transcript:aai.RealtimeTranscript):"This function is called when a new transcript has been received."ifnottranscript.text:returnifisinstance(transcript,aai.RealtimeFinalTranscript):print(transcript.text,end="\r\n")else:print(transcript.text,end="\r")defon_error(error:aai.RealtimeError):"This function is called when the connection has been closed."print("An error occured:",error)# Create the Real-Time transcribertranscriber=aai.RealtimeTranscriber(on_data=on_data,on_error=on_error,sample_rate=44_100,)# Start the connectiontranscriber.connect()# Only WAV/PCM16 single channel supported for nowfile_stream=aai.extras.stream_file(filepath="audio.wav",sample_rate=44_100,)transcriber.stream(file_stream)transcriber.close()PlaygroundsVisit one of our Playgrounds:LeMUR PlaygroundTranscription PlaygroundAdvancedHow the SDK handles Default ConfigurationsDefining DefaultsWhen noTranscriptionConfigis being passed to theTranscriberor its methods, it will use a default instance of aTranscriptionConfig.If you would like to re-use the sameTranscriptionConfigfor all your transcriptions,
you can set it on theTranscriberdirectly:config=aai.TranscriptionConfig(punctuate=False,format_text=False)transcriber=aai.Transcriber(config=config)# will use the same config for all `.transcribe*(...)` operationstranscriber.transcribe("https://example.org/audio.wav")Overriding DefaultsYou can override the default configuration later via the.configproperty of theTranscriber:transcriber=aai.Transcriber()# override the `Transcriber`'s config with a new configtranscriber.config=aai.TranscriptionConfig(punctuate=False,format_text=False)In case you want to override theTranscriber's configuration for a specific operation with a different one, you can do so via theconfigparameter of a.transcribe*(...)method:config=aai.TranscriptionConfig(punctuate=False,format_text=False)# set a default configurationtranscriber=aai.Transcriber(config=config)transcriber.transcribe("https://example.com/audio.mp3",# overrides the above configuration on the `Transcriber` with the followingconfig=aai.TranscriptionConfig(dual_channel=True,disfluencies=True))Synchronous vs AsynchronousCurrently, the SDK provides two ways to transcribe audio files.The synchronous approach halts the application's flow until the transcription has been completed.The asynchronous approach allows the application to continue running while the transcription is being processed. The caller receives aconcurrent.futures.Futureobject which can be used to check the status of the transcription at a later time.You can identify those two approaches by the_asyncsuffix in theTranscriber's method name (e.g.transcribevstranscribe_async).Polling IntervalsBy default we poll theTranscript's status each3s. In case you would like to adjust that interval:importassemblyaiasaaiaai.settings.polling_interval=1.0Retrieving Existing TranscriptsRetrieving a Single TranscriptIf you previously created a transcript, you can use its ID to retrieve it later.importassemblyaiasaaitranscript=aai.Transcript.get_by_id("<TRANSCRIPT_ID>")print(transcript.id)print(transcript.text)Retrieving Multiple Transcripts as a GroupYou can also retrieve multiple existing transcripts and combine them into a singleTranscriptGroupobject. This allows you to perform operations on the transcript group as a single unit, such as querying the combined transcripts with LeMUR.importassemblyaiasaaitranscript_group=aai.TranscriptGroup.get_by_ids(["<TRANSCRIPT_ID_1>","<TRANSCRIPT_ID_2>"])summary=transcript_group.lemur.summarize(context="Customers asking for cars",answer_format="TLDR")print(summary)Retrieving Transcripts AsynchronouslyBothTranscript.get_by_idandTranscriptGroup.get_by_idshave asynchronous counterparts,Transcript.get_by_id_asyncandTranscriptGroup.get_by_ids_async, respectively. These functions immediately return aFutureobject, rather than blocking until the transcript(s) are retrieved.See the above section onSynchronous vs Asynchronousfor more information.