Package: autobi-py
autobi_pyA Python wrapper and interfacing library aroundAuToBI. AuToBI is effectively the only publically available system for the automatic generation of prosody transcriptions, created by Andrew Rosenberg for his PhD in 2012.AuToBI_py provides a simple Python interface for AuToBI's feature generation systems, as well as the parts of AuToBI that are available from the command line.UsageTo get started, the JVM needs to be running, as the Java code can otherwise not be called. To start the JVM, use theAutobiJVMHandlerclass like so:fromautobi.coreimportAutobiJVMHandlerwithAutobiJVMHandler()asjvm:do_stuff(jvm)AutobiJVMHandlerreturns a handle to the JVM (typeautobi.core.AutobiJVM) that you can pass into functions from this library to let them access it. The constructor can take one argument, a string name, that is used to distinguish separate views of the JVM as follows:fromautobi.coreimportAutobiJVMHandlerwithAutobiJVMHandler("test")asjvm1:withAutobiJVMHandler("test")asjvm2:assertjvm1isjvm2withAutobiJVMHandler("second_test1")asjvm1:withAutobiJVMHandler("second_test2")asjvm2:assertjvm1isnotjvm2# The default argument is "main"withAutobiJVMHandler()asjvm1:withAutobiJVMHandler("main")asjvm2:assertjvm1isjvm2If you want to hard shutdown the JVM and invalidate allAutobiJVMinstances for some reason, runautobi.core._shutdown().Running AuToBI like from the commandlineTo run AuToBI like you would from the commandline, use theRunDefaultclass:fromautobiimportRunDefaultfromautobi.coreimportAutobiJVMHandlerwithAutobiJVMHandler()asjvm:RunDefault(jvm).run("-input_file=test.TextGrid -wav_file=test.wav ...")# One argument has been added to argument list in the adapter# "-adapter_test", which prints "TEST SUCCESS: The adapter is running"RunDefault(jvm).run("-adapter_test")To make creating arguments for running like this easier, there is theArgumentBuilderclass:frompathlibimportPathfromautobiimportRunDefault,ArgumentBuilderfromautobi.coreimportAutobiJVMHandlerwithAutobiJVMHandler("test")asjvm:builder=ArgumentBuilder(jvm)builder.with_output_file(Path()/"out.arff")builder.with_input_wav(Path()/"test.wav")builder.with_input_TextGrid(Path()/"test.TextGrid")builder.with_charset("hello")args=builder.to_args_string()RunDefault(jvm).run(args)Extracting featuresThe most important part of the adapter is the ability to use AuToBI's excellent feature extraction algorithms on their own. The majority of Rosenberg's PhD was dedicated to feature extraction, and a large majority of the code is, too.Creating a FeatureSet to extract.The primary interface for this is theautobi.FeatureSetclass, which is simply a wrapper around a list of strings, where each string represents a valid feature.To generate one of these feature sets, use theFeaturenamesBuilderclass:fromautobiimportFeaturenamesBuilderfromautobi.coreimportAutobiJVMHandler# With individual features:withAutobiJVMHandler()asjvm:builder=FeaturenamesBuilder(jvm)builder.with_feature(r"cog[subregionC[delta[prodC[znormC[log[f0]],rnormC[I],0.1]],subregion[200ms]]]")builder.with_feature(r"mean[subregionC[znormC[f0],subregion[200ms]]]")featureset=builder.build()# With default feature sets from AuToBi:withAutobiJVMHandler()asjvm:builder=FeaturenamesBuilder(jvm)builder.with_default_features("PhraseAccentClassificationFeatureSet")featureset=builder.build()# Of course, these can be combinded:withAutobiJVMHandler()asjvm:builder=FeaturenamesBuilder(jvm)builder.with_default_features("PhraseAccentClassificationFeatureSet")builder.with_feature(r"cog[subregionC[delta[prodC[znormC[log[f0]],rnormC[I],0.1]],subregion[200ms]]]")builder.with_feature(r"mean[subregionC[znormC[f0],subregion[200ms]]]")featureset=builder.build()What individual features are valid is not documented anywhere for AuToBI. The default feature sets, however, are simply the names of theJava classes that inherit from FeatureSetin the original AuToBI, which the adapter gets out with java reflections and runs to get the features they generate out.Extracting a FeatureSetTo actually extract any features from input data, the API gets a little messier. The input expects a.wavfile and a praat.TextGridfile, bothfrom disk, meaning the functions take file paths as arguments. AuToBI should also accept different input formats, but I have not tested those.The necessity of the.wavfile is probably expected. However, the.TextGridfile might be surprising. The reason this is needed is because ToBI is a transcription format that is defined as something to addon top ofa normal, textual, transcription, and AuToBI only adds this layer. To get a base transcription in TextGrid format from raw audio, you can, for example, check out another project I wrote for my BA thesis:transcribe_allign_textgrid.The format AuToBI expects the TextGrids to be in is a grid with a single interval tier, called "words", that contains the force-alligned transcription. It cannot deal with empty intervals, but those can be replaced with dashes instead.The actual data extraction is done with theDatasetBuilder, which takes an argument list from the ArgumentBuilder:fromautobi.coreimportAutobiJVMHandlerfromautobiimportArgumentBuilder,DatasetBuilderwithAutobiJVMHandler()asjvm:argument_builder=ArgumentBuilder(jvm)argument_builder.with_input_wav(input_wav)argument_builder.with_input_TextGrid(input_grid)args=argument_builder.to_args_string()data=DatasetBuilder(jvm,args)ThisDatasetBuildershould then be told what features to extract. This can, of course, be done by giving it an instance ofFeatureSet, although it also has the same interface asFeaturenamesBuilderto add more to it (The advantage of feature sets being you can pass them around your in code if you want to create multiple data sets)fromautobi.coreimportAutobiJVMHandlerfromautobiimportArgumentBuilder,DatasetBuilder,FeaturenamesBuilderwithAutobiJVMHandler()asjvm:builder=FeaturenamesBuilder(jvm)builder.with_default_features("PhraseAccentClassificationFeatureSet")featureset=builder.build()argument_builder=ArgumentBuilder(jvm)argument_builder.with_input_wav(input_wav)argument_builder.with_input_TextGrid(input_grid)args=argument_builder.to_args_string()databuilder=DatasetBuilder(jvm,args)databuilder.with_feature_set(featureset)databuilder.with_feature(r"mean[subregionC[znormC[f0],subregion[200ms]]]")databuilder.with_default_features("PitchAccentClassificationFeatureSet")Finally, the data can be extracted. To do this, the output can either be written do csv, arff, or liblinear formats, or exported to a pandas dataframe directly, which simply writes to a csv, and then reads from that csv.# Please note you can only call one: Calling write invalidates the builder object.databuilder.write_csv(output_csv)databuilder.write_arff(output_arff)databuilder.write_liblinear(output_liblinear)One common pattern you might find yourself using is:withTemporaryDirectory()astemp_dir:tempfile=Path(temp_dir)/"out.csv"databuilder.write_csv(tempfile)df=pd.read_csv(tempfile,delimiter=",",na_values=["?"]).astype(float)which is added as a buildin functiondatasetbuilder.build_pandas()ExamplesA file that uses autobi_py to extract features might look a little like this, which is a simplified excerpt from the code I wrote for the actual experiment of my BA thesis (which can be foundhere)importfunctoolsfrompathlibimportPathfromdataclassesimportdataclassimportnumpyasnpimportpandasaspdfromautobi.coreimportAutobiJVM,AutobiJVMHandlerfromautobiimportArgumentBuilder,DatasetBuilder,FeaturenamesBuilder,FeatureSet@dataclassclassDataSet:accent_detection:pd.DataFrameaccent_classification:pd.DataFramedefget_autobi_features(grid_file:Path,wav_file:Path,jvm:AutobiJVM,featureset:FeatureSet)->pd.DataFrame:args=(ArgumentBuilder(jvm).with_input_wav(input_wav).with_input_TextGrid(input_grid).to_args_string())returnDatasetBuilder(jvm,args).with_feature_set(featureset).build_pandas()defcreate_features(df:DataSet,get_accent_detection_features:Callable[[pd.DataFrame],pd.DataFrame],get_accent_classification_features:Callable[[pd.DataFrame],pd.DataFrame],)->DataSet:df.accent_detection=(df.accent_detection.groupby("grid_path",group_keys=False).apply(get_accent_detection_features).drop(["wav_path","grid_path","start","end","word"],inplace=False,axis=1).reset_index(drop=True))df.accent_classification=(df.accent_classification.groupby("grid_path",group_keys=False).apply(get_accent_classification_features).drop(["wav_path","grid_path","start","end","word"],inplace=False,axis=1).reset_index(drop=True))returndfdeffeature_creation_autobi(df:DataSet)->DataSet:withAutobiJVMHandler("main")asjvm:accent_detection_features=(FeaturenamesBuilder(jvm).with_default_features("PitchAccentDetectionFeatureSet").build())accent_classification_features=(FeaturenamesBuilder(jvm).with_default_features("PitchAccentClassificationFeatureSet").build())defget_features(features:FeatureSet,df:pd.DataFrame)->pd.DataFrame:autobi_features=get_autobi_features(df["grid_path"].iloc[0],df["wav_path"].iloc[0],jvm,features,)returnpd.concat([df,autobi_features],axis=1,join="inner")returncreate_features(df,get_accent_detection_features=functools.partial(get_features,accent_detection_features),get_accent_classification_features=functools.partial(get_features,accent_classification_features),)Code that transforms the output fromtranscribe_allign_textgridto the format that AuToBI expects, making use of thePraatIOlibrary, might look something like this:frompraatio.data_classes.interval_tierimportIntervalTierfrompraatio.data_classes.textgridimportTextGridfrompraatio.utilities.constantsimportIntervaldefreplace_empty(item:Interval)->Interval:ifnotitem.label:returnitem._replace(label="-")returnitemdefreformat_textgrid(grid:TextGrid)->TextGrid:words=[replace_empty(word)forwordingrid.tierDict["words"].entryList]new_grid=tg.Textgrid(minTimestamp=grid.minTimestamp,maxTimestamp=grid.maxTimestamp)new_grid.addTier(IntervalTier("words",words,minT=grid.minTimestamp,maxT=grid.maxTimestamp))returnnew_gridNotes on weirdness of styleAn attentive reader might have noticed this adapter overuses the Builder pattern and generally has a very weird Python API. This is because of the limitations that come from the Java-Python bridge. The bridge is only sending strings (sometimes serialized as Json) to and from Java. Because of this, the builder pattern is great, as it sends single strings to the JVM at a time, with thebuildmethod returning only strings to be used for other parts of the API. Moreover, having to call the functions on objects means that those objects can keep a reference to the JVM view themselves.Similarly, the overuse of the ArgumentBuilder is because AuToBI was originally designed as an application, not a library. Internal Java functions all want an instance of AuToBIArguments, which I construct like this.A more complete Python project would probably add another layer of abstraction on top of the present library to provide a more Pythonic API.