Package: ultrastar-pitch
ultrastar_pitchThis python application automates the pitch detection for ultrastar deluxe projects.usageThe software can be used with user interface or as command line application. The windowsbinaryautomatically starts as a graphical user application.For cli execution, just run the command in your project folder. Additional flags are listed below. If the usdx file is named "notes.txt", no arguments are needed. Otherwise it has to be explicitly stated:ultrastar-pitch name.txtIf everything went well a new file "notes_new.txt" should appear. In case a different output name is desired, it can be changed with the "-o" flag:ultrastar-pitch -o name_new.txtspleeterThespleeter projectuses deep learning to separate the vocal- and instrumental part of a song. In some cases (mainly acoustic songs), ultrastar-pitch performs better with the isolated vocal data. In other cases, its accuracy drops due to the introduced artifacts / information loss.In order to use spleeter with ultrastar-pitch, a couple of steps need to be performed:spleeter produces multiple outputs. The vocals.wav file needs to be placed in the same directory as the notes.txt file.in the notes.txt file the "#MP3" tag needs to be changed, that it refers to the vocal file (e.g. "#MP3:vocals.wav").after running ultrastar-pitch, the "#MP3" tag needs to be reverted backinstallationIf you are using thebinary, everything should run out of the box.In case of an error, try to install Microsoft Visual C++ Redistributable x64 (2010+2015).If you want to use the python application do the following:windowsInstall the newest version ofPython 3Install the newest build offfmpegOpen CMD/Powershell and type:pip install ultrastar-pitchlinux (debian)Open a terminal and type:sudo apt-get install python3 python3-pip ffmpegpip install ultrastar-pitchflagsCommand line options for nono graphical execution:flagdescription-hshow this help message and exit-ospecify output file name-genable graphical user interface-mdisable stochastic postprocessing-ashow prediction accuracy (debug flag)-lset logging level (debug flag)developer informationbuild instructions (windows only)The software can be compiled into a single standalone binary. To achieve this, an additional package has to be installed.pip install pyinstallerTo include ffmpeg into the binary, it needs to be placed as specified by the setup.spec file. The default would be "ffmpeg\bin\ffmpeg.exe" within the project root directory.The building process is fairly easy. Just execute the following command within the cmd/powershell:pyinstaller setup.specimplementationThe software takes a timed usdx project file and the corresponding audio file. The song is converted into a mono wav and gets split into the predefined audio segments. These chunks are divided into blocks to be transformed into features. The output is then fed into a neuronal network to determine the pitches. Statistical postprocessing is used to determine pseudo key of the song. The predicted pitches are reevaluated to match the pseudo key.The deep learning model was trained on a large karaoke database. Details for building your own model can be found in the dev/ folder.accuracyThe precision of this method changes greatly with the analyzed audio. For example a ballad with slow background music and a strong female voice can get an accuracy of over 90%, while a rock song with loud background music and a rough male voice can drop below 30%.apiThe software consists of various modules:moduledescriptionProjectParserparse note.txt project file for singable audio segmentsAudioPreprocessortransform audio segments into features for pitch detectionPitchClassifierpredict pitches from featuresStochasticPostprocessorincrease prediction accuracy by applying stochasticsDetectionPipelineexecute the models above in one pipelineEach modules can be used in your own project. Just import them like this:from ultrastar_pitch import modulechangelogversionchangelog1.0.1fixed encoding issues and enforce python version1.0.0minimalistic gui, new preprocessing algo, new model, api restructuring0.82use dynamic stride to accelerate processing, removed model initializer, minor fixes0.81fixed exception for silent audio input0.80switched to onnx framework for model load and inference0.73added support for ansi encoded note.txt files0.72optimized performance with micro optimizations0.71switched from median to highest likelihood pitch evaluation0.70use statistical distribution to improve the prediction0.64switched from scipy.io to wave library to load audio0.6ximproved model accuracy and prediction speed0.60switching from average to median pitch evaluation, changed license0.50implemented PCA, bumped tensorflow to version 2 and improved model accuracy and speed0.41fixed behavior for some edge cases0.40complete restructuring and application works as command line application0.34bug fixes0.33using absolute paths instead of relative ones0.32added model to PyPi repo -> is now used by default0.31substitute license field with classifier and updated installer script0.30added deep learning support and improved source code readability0.21got pyinstaller running -> binary doesn't need separate ffmpeg anymore0.20replaced pydub by subprocess and scipy wavfile read -> faster processing0.10first running implementationlegacy downloadsOld BinariesOld Python Wheelstodowrite unit testsmore sophisticated statistical postprocessingevaluate approaches for partial automate timing detectionsiamese neural network for multi octave detectionevaluate accuracy with higher sampleratesintegrate preprocessing into modelevaluate fcnn networks with variable input lengthintegrate resampling into modelevaluate accuracy with sequence models (rnns, grus, lstms)more logging