Package: hearpreprocess
hear-preprocessDataset preprocessing code for the HEAR Benchmark and for all the tasks used during
the 2021 HEAR NeurIPS challenge. To find out more about HEAR please visithttps://hearbenchmark.com.Unless you need to pre-process HEAR benchmark tasks yourself or want to contribute a task,
you won't need this repo. Usehear-eval-kitto
evaluate your embedding models on these tasks.Pre-processed datasets (at 48000Hz) for all HEAR Benchmark tasks are available onzenodo. Other sampling rates
(16000, 22050, 32000, 44100), are available for download (requester pays) from Google Storagegs://hear2021-archive/tasks/This preprocessing is slow and disk-intensive but safe and careful.Cloud UsageSeehear-eval's
README.spottyfor information on how to use spotty.Installationpip3 install hearpreprocessTested with Python 3.7 and 3.8. Python 3.9 is not officially supported
because pip3 installs are very finicky, but it might work.DevelopmentClone repo:git clone https://github.com/hearbenchmark/hear-preprocess
cd hear-preprocessInstall in development mode:pip3 install -e ".[dev]"Make sure you have pre-commit hooks installed:pre-commit installRunning tests:python3 -m pytestPreprocessingYou probably don't need to do this unless you can't use theavailable pre-processed
datasetsand need to preprocess
the data yourself..If you want to run preprocessing yourself:You will needffmpeg>=4.2installed (possibly from conda-forge).You will needsoxrsupport, which might require package
libsox-fmt-ffmpeg orinstalling from
source.These Luigi pipelines are used to preprocess the evaluation tasks
into a common format for downstream evaluation.To run the preprocessing pipeline for all available tasks, with all
available modes for each task:python3 -m hearpreprocess.runner all --mode allYou can instead just call a specific single taskpython3 -m hearpreprocess.runner task1 --mode allor specific multiple tasks:python3 -m hearpreprocess.runner task1 task2 --mode allTasksList of available tasks used in HEAR 2021:Task NameModesdcase2016_task2fullnsynth_pitch5h, 50hspeech_commands5h, fullbeehive_states_fold05h, fullbeehive_states_fold15h, fullbeijing_operafullesc50fullfsd50kfullgunshot_triangulationfulllibricountfullmaestro5hmridangam_strokefullmridangam_tonicfulltfds_crema_dfulltfds_gtzanfulltfds_gtzan_music_speechfullvocal_imitationfullvox_lingua_top10fullPipelinesEach pipeline will download and preprocess each dataset according
to the following DAG:DownloadCorpusExtractArchiveExtractMetadata: Create splits over the entire corpus and find
the label metadata for them.SubcorpusSplit (subsample each split) => MonoWavSplit => TrimPadSplit => SubcorpusData (symlinks)SubcorpusData => {SubcorpusMetadata, ResampleSubcorpus}SubcorpusMetadata => MetadataVocabularyFinalCombine => TarCorpus => FinalizeCorpusIn terms of sampling:We create a 60/20/20 split if train/valid/test does not exist.We cap each split at 3/1/1/ hours of audio, defined asIf further small sampling happens, that chooses a particular
number of audio samples per task.These commands will download and preprocess the entire dataset. An
intermediary directory defined by the optionluigi-dir(default_workdir) will be created, and then a final directory defined by
the optiontasks-dir(defaulttasks) will contain the completed
dataset.Options:Options:
  --num-workers INTEGER  Number of CPU workers to use when running. If not
                         provided all CPUs are used.
  --sample-rate INTEGER  Perform resampling only to this sample rate. By
                         default we resample to 16000, 22050, 44100, 48000.
  --tmp-dir TEXT         Temporary directory to save all the intermediate
                         tasks (will not be deleted afterwords). (default:
                         _workdir/)
  --tasks-dir TEXT       Directory to save the final task output (default:
                         tasks/)
  --tar-dir TEXT         Directory to save the tar'ed output (default: .)
  --mode TEXT            default, all, or small mode for each task.
  --help                 Show this message and exit.To check the stats of an audio directory:python3 -m hearpreprocess.audio_dir_stats {input folder} {output json file}Stats include: audio_count, audio_samplerate_count, mean meadian
and certain (10, 25, 75, 90) percentile durations.  This is helpful
in getting a quick glance of the audio files in a folder and helps
in decideing the preprocessing configurations.The pipeline will also generate some stats of the original and
preprocessed data sets, e.g.:speech_commands-v0.0.2/01-ExtractArchive/test_stats.json
speech_commands-v0.0.2/01-ExtractArchive/train_stats.json
speech_commands-v0.0.2/03-ExtractMetadata/labelcount_test.json
speech_commands-v0.0.2/03-ExtractMetadata/labelcount_train.json
speech_commands-v0.0.2/03-ExtractMetadata/labelcount_valid.jsonFaster preprocessing, for developmentThe small flag runs the preprocessing pipeline on a small version
of each dataset stored atDownsampled HEAR Open
Tasks. This
is used for development and continuous integration tests for the
pipeline.These small versions of the data can be generated
deterministically with the following command:python3 -m hearpreprocess.sampler <taskname>NOTE:--mode smallis used to run the task on a
small version of the dataset for development.Breaking change for hear-evalIf the open tasks have changed enough to break the downstream CI,
(for example in the heareval repo), thePreprocessed Downsampled HEAR Open
Tasksshould be updated. An example of an obvious breaking changes can be modification of the task configuration.The version should be bumped up inhearpreprocess/__init__.pyand the pipeline should
be run for the open tasks with--mode smallflagThereafter, the following command can be used to copy the tarred files produced by running the pipeline for the open tasks to the repo( Please clone the repo )git clone git@github.com:hearbenchmark/hear2021-open-tasks-downsampled.git
cp hear-LATEST-speech_commands-v0.0.2-small-44100.tar.gz ./hear2021-open-tasks-downsampled/preprocessed/
cp hear-LATEST-nsynth_pitch-v2.2.3-small-44100.tar.gz ./hear2021-open-tasks-downsampled/preprocessed/
cp hear-LATEST-dcase2016_task2-hear2021-small-44100.tar.gz ./hear2021-open-tasks-downsampled/preprocessed/
cp hear-2021.0.6-speech_commands-v0.0.2-small-44100.tar.gz ./hear2021-open-tasks-downsampled/preprocessed/
cp hear-2021.0.6-nsynth_pitch-v2.2.3-small-44100.tar.gz ./hear2021-open-tasks-downsampled/preprocessed/
cp hear-2021.0.6-dcase2016_task2-hear2021-small-44100.tar.gz ./hear2021-open-tasks-downsampled/preprocessed/