Package: batchalign
TalkBank | Batchalign2Welcome!Batchalign2is a Python suite of language sample analysis (LSA) software from the TalkBank project. It is used to interact with conversation audio files and their transcripts, and provides a whole host of analyses within this space.The TalkBank Project, of which Batchalign is a part, is supported by NIH grant HD082736.Quick StartThe following instructions is a quick start to install Batchalign.For most users, we recommend youvisit this detailed guidefor more detailed instructions.The remaining instructions on this page provides a very rough overview of the primary functionality ofbatchalign, and assumes familiarity with Python and the terminal.Get PythonWe support Python versions 3.9, 3.10, and 3.11.We do not support Python 3.12(no PyTorch support)To install Python, follow the instructions...for macOSInstall Brew:visit this linkInstall Python: executebrew install python@3.11for WindowsInstall Python 3.11:via this linkIf later commands reportpip module not found,this page may helpyour distribution's instructions for LinuxInstall and Update the PackageYou can get Batchalign from PyPi, and you can update the package in the same way:macOS/Linux:pip3 install -U batchalignWindows:py -m pip3 install -U batchalignRock and RollThere are two main ways of interacting with Batchalign. Batchalign can be used as a program to batch-process CHAT (hence the name), or a Python LSA library.to get started with the Batchalign program,tap hereto get started on the Batchalign Library (assumes familiarity with Python),tap hereQuick Start: Command LineBasic UsageOnce installed, you can invoke the Batchalign CLI program via thebatchaligncommand.It is used in the following basic way:batchalign [verb] [input_dir] [output_dir]Whereverbincludes:transcribe- placing only an audio of video file (.mp3/.mp4/.wav) in the input directory, perform ASR on the audio, diarizes utterances, identifies some basic conversational features like retracing and filled pauses, and generate word-level alignments. You must supply a language code flag:--lang=[three letter ISO language code]for the ASR system to know what language the transcript is in. You can choose the flags--revto use Rev.AI, a commercial ASR service, or--whisper, to use a local copy of OpenAI Whisper.align- placing both an audio of video file (.mp3/.mp4/.wav) and anutterance-alignedCHAT file in the input directory, generate word-level alignmentsmorphotag- placing a CHAT file in the input directory, uses Stanford NLP Stanza to generate morphological and dependency analyses. You must supply a language code flag:--lang=[three letter ISO language code]for the alignment system to know what language the transcript is in.You can get a CHAT transcript to experiment withat the TalkBank website, under any of the "Banks" that are available. You can also generate and parse a CHAT transcript viathe Python program.Sample CommandsFor input files (CHAT and audio foralign, CHAT only formorphotag, and audio only fortranscribe), located in~/ba_inputdumping the output to~/ba_output, one could write:ASR + Segmentationbatchalign transcribe --lang=eng ~/ba_input ~/ba_outputmorphosyntactic analysisbatchalign morphotag ~/ba_input ~/ba_outputforced alignmentbatchalign align --lang=eng ~/ba_input ~/ba_outputFollow instructions frombatchalign --helpandbatchalign [verb] --helpto learn more about other options.VerbosityPlacing one or multiple-vbehind the wordbatchalign(i.e. behind the[verb]will not work) increases the verbosity of Batchalign. The default mode and one-vwill use the normal Batchalign interface, whereas Batchalign with more than 1-vwill switch to the text-based "logging" interface.For instance, here is the instruction for running Batchalign to perform forced-alignment:batchalign align input outputWith one-v, you can get stack trace information about any files that crashes:batchalign -v align input outputand with two-vv, we will ditch the loading bar user interface and instead switch to a logging-based interface that has more information about what Batchalign is doing under the hood:batchalign -vv align input outputQuick Start: PythonLet's begin!importbatchalignasbaDocumentTheDocumentis the most basic object in Bachalign. All processing pipelines expectDocumentas input, and will spit outDocumentas output.doc=ba.Document.new("Hello, this is a transcript! I have two utterances.",media_path="audio.mp3",lang="eng")# navigating the documentfirst_utterance=doc[0]first_form=doc[0][0]the_comma=doc[0][1]assertthe_comma.text==','assertthe_comma.type==ba.TokenType.PUNCT# taking a transcriptsentences=doc.transcript(include_tiers=False,strip=True)Notably, if you have a Document that you haven't transcribed yet, you still can make a Document!doc=ba.Document.new(media_path="audio.mp3",lang="eng")PipelinesQuick PipelineSay you wanted to perform ASR, and then tag morphology of the resulting output.nlp=ba.BatchalignPipeline.new("asr,morphosyntax",lang="eng",num_speakers=2)doc=ba.Document.new(media_path="audio.mp3",lang="eng")doc=nlp(doc)# this is equivalent to nlp("audio.mp3"), we will make the initial doc for youfirst_word_pos=doc[0][0].morphologyfirst_word_time=doc[0][0].timefirst_utterance_time=doc[0].alignmentThe quick API (right now) has support for the following tasks, which you can pass in a comma-separated list in the first argument:asr: ASR!morphosyntax: PoS and dependency analysisfa: Forced Alignment (require utterance-level timings already)We will support many, many, many more tasks soon with this API. For now, to gain access to the whole suite of tools, use the second pipeline API discussed below.Manual PipelineBatchalign ships with a plurality of engines which preform the actual processing. For instance, to recreate the demo we had above using the Engines API, we would write# ASRwhisper=ba.WhisperEngine(lang="eng")# retracing and disfluency analysisretrace=ba.NgramRetraceEngine()disfluency=ba.DisfluencyReplacementEngine()# morphosyntaxmorphosyntax=ba.StanzaEngine()# create a pipelinenlp=ba.BatchalignPipeline(whisper,retrace,disfluency,morphosyntax)# and run it!doc=nlp("audio.mp3")Here's a listof available engines.FormatsWe currently support reading and writing two transcript formats: TalkBank CHAT, and Praat TextGrid.CHATHere's how to read and write a CHAT file to parse a TalkBank transcript!# readingchat=ba.CHATFile(path="chat.cha")doc=chat.doc# writingchat=ba.CHATFile(doc=doc)chat.write("chat.cha")We will automatically detect audio files located within the same directory as the CHAT file, and associate it with the Batchalign Document.TextGridImportantly, there are two ways a TextGrid could be written: we can either place eachutterancein an individualIntervalTier, or eachwordin its ownIntervalTier; we leave that decision up to you. To learn more about TextGrid,visit this page.# reading; recall we can either interpret each IntervalTier as a word or utterancetg_utterance=ba.TextGridFile("utterance",path="tg_ut.TextGrid",lang="eng")tg_word=ba.TextGridFile("word",path="tg_w.TextGrid",lang="eng")doc1=tg_utterance.docdoc2=tg_word.doc# writingtg_utterance=ba.TextGridFile("utterance",doc=doc1)tg_word=ba.TextGridFile("word",doc=doc2)tg_utterance.write("tg_ut.TextGrid")tg_word.write("tg_w.TextGrid")Questions?If you have any questions or concerns, please reach out! If something isn't working right,open an issue on GitHub; if you need support, please feel free to emailhoujun@cmu.eduandmacw@cmu.edu.