Package: lidbox
lidboxSpoken language identification (LId) out of the box using TensorFlow.Parallel feature extraction usingtf.data.Dataset, with STFT computations on the GPU using thetf.signalpackage.Only metadata (e.g. utt2path, utt2label) is fully loaded into memory, rest is done in linear passes  over the dataset with thetf.data.Datasetiterator.Spectrograms, source audio, and utterance ids can be written into TensorBoard summaries.Model training withtf.keras, some model examples are availablehere.Average detection cost (C_avg) implemented as atf.keras.metrics.Metric.You can also trylidboxfor speaker recognition, since no assumptions will be made of the signal labels. E.g. use utt2speaker as utt2label and see what happens.Hereis a full example notebook showing whatlidboxcan do.Why would I want to use this?You need a simple, deep learning based speech classification pipeline.
For example: waveform -> VAD filter -> augment audio data -> serialize all data to a single binary file -> extract log-scale Mel-spectra or MFCC -> use DNN/CNN/LSTM/GRU/attention (etc.) to classify by signal labelsYou have thousands of hours of speech dataYou have a TensorFlow/Keras model that you train on the GPU and want thetf.data.Datasetextraction pipeline to also be on the GPUYou want an end-to-end pipeline that uses TensorFlow 2 as much as possibleWhy would I not want to use this?You are happy doing everything withKaldior some other toolkitsYou don't want to debug by reading the source code when something goes wrongYou don't want to install TensorFlow 2 and configure its dependencies (CUDA etc.)You want to train phoneme recognizers or use CTCInstallingYou need to have Python 3 installed.With the examplegit clone --depth 1 https://github.com/matiaslindgren/lidbox.git
python3 -m pip install ./lidboxCheck that the command line entry point is working:lidbox -hIf not, make sure thesetuptoolsentry point scripts (e.g. directory$HOME/.local/bin) are on your path.Without the examplepython3 -m pip install lidboxTensorFlowTensorFlow 2 is not included in the package requirements because you might want to do custom configuration to get the GPU working etc.If you don't want to customize anything and instead prefer something that just works for now, the following should be enough:python3 -m pip install tensorflowIf everything is working, seethisfor a simple example to get started.Editable installIf you plan on making changes to the code, it is easier to installlidboxas a Python package in setuptools develop mode:git clone --depth 1 https://github.com/matiaslindgren/lidbox.git
python3 -m pip install --editable ./lidboxThen, if you make changes to the code, there's no need to reinstall the package since the changes are reflected immediately.
Just be careful not to make changes whenlidboxis running, because TensorFlow will use itsautographpackage to convert some of the Python functions to TF graphs, which might fail if the code changes suddenly.X-vector embeddingsOne benefit of deep learning classifiers is that you can first train them on large amounts of data and then use them as feature extractors to produce low-dimensional, fixed-length language vectors from speech.
See e.g. thex-vectorapproach by Snyder et al.Below is a visualization of test set language embeddings for 4 languages in 2-dimensional space.
Each data point represents 2 seconds of speech in one of the 4 languages.PLDA + Naive Bayes classifierThere is a simplelanguage embedding classifier backendavailable.
To use it, you need to first installPLDA:python3 -m pip install plda@https://github.com/RaviSoji/plda/archive/184d6e39b01363b72080f2752819496cd029f1bd.zip