Package: descript-audio-codec
Descript Audio Codec (.dac): High-Fidelity Audio Compression with Improved RVQGANThis repository contains training and inference scripts
for the Descript Audio Codec (.dac), a high fidelity general
neural audio codec, introduced in the paper titledHigh-Fidelity Audio Compression with Improved RVQGAN.arXiv Paper: High-Fidelity Audio Compression with Improved RVQGAN📈Demo Site⚙Model Weights👉 With Descript Audio Codec, you can compress44.1 KHz audiointo discrete codes at alow 8 kbps bitrate.🤌 That's approximately90x compressionwhile maintaining exceptional fidelity and minimizing artifacts.💪 Our universal model works on all domains (speech, environment, music, etc.), making it widely applicable to generative modeling of all audio.👌 It can be used as a drop-in replacement for EnCodec for all audio language modeling applications (such as AudioLMs, MusicLMs, MusicGen, etc.)UsageInstallationpip install descript-audio-codecORpip install git+https://github.com/descriptinc/descript-audio-codecWeightsWeights are released as part of this repo under MIT license.
We release weights for models that can natively support 16 kHz, 24kHz, and 44.1kHz sampling rates.
Weights are automatically downloaded when you first runencodeordecodecommand. You can cache them using one of the following commandspython3-mdacdownload# downloads the default 44kHz variantpython3-mdacdownload--model_type44khz# downloads the 44kHz variantpython3-mdacdownload--model_type24khz# downloads the 24kHz variantpython3-mdacdownload--model_type16khz# downloads the 16kHz variantWe provide a Dockerfile that installs all required dependencies for encoding and decoding. The build process caches the default model weights inside the image. This allows the image to be used without an internet connection.Please refer to instructions below.Compress audiopython3 -m dac encode /path/to/input --output /path/to/output/codesThis command will create.dacfiles with the same name as the input files.
It will also preserve the directory structure relative to input root and
re-create it in the output directory. Please usepython -m dac encode --helpfor more options.Reconstruct audio from compressed codespython3 -m dac decode /path/to/output/codes --output /path/to/reconstructed_inputThis command will create.wavfiles with the same name as the input files.
It will also preserve the directory structure relative to input root and
re-create it in the output directory. Please usepython -m dac decode --helpfor more options.Programmatic UsageimportdacfromaudiotoolsimportAudioSignal# Download a modelmodel_path=dac.utils.download(model_type="44khz")model=dac.DAC.load(model_path)model.to('cuda')# Load audio signal filesignal=AudioSignal('input.wav')# Encode audio signal as one long file# (may run out of GPU memory on long files)signal.to(model.device)x=model.preprocess(signal.audio_data,signal.sample_rate)z,codes,latents,_,_=model.encode(x)# Decode audio signaly=model.decode(z)# Alternatively, use the `compress` and `decompress` functions# to compress long files.signal=signal.cpu()x=model.compress(signal)# Save and load to and from diskx.save("compressed.dac")x=dac.DACFile.load("compressed.dac")# Decompress it back to an AudioSignaly=model.decompress(x)# Write to filey.write('output.wav')Docker imageWe provide a dockerfile to build a docker image with all the necessary
dependencies.Building the image.docker build -t dac .Using the image.Usage on CPU:docker run dac <command>Usage on GPU:docker run --gpus=all dac <command><command>can be one of the compression and reconstruction commands listed
above. For example, if you want to run compression,docker run --gpus=all dac python3 -m dac encode ...TrainingThe baseline model configuration can be trained using the following commands.Pre-requisitesPlease install the correct dependenciespip install -e ".[dev]"Environment setupWe have provided a Dockerfile and docker compose setup that makes running experiments easy.To build the docker image do:docker compose buildThen, to launch a container, do:docker compose run -p 8888:8888 -p 6006:6006 devThe port arguments (-p) are optional, but useful if you want to launch a Jupyter and Tensorboard instances within the container. The
default password for Jupyter ispassword, and the current directory
is mounted to/u/home/src, which also becomes the working directory.Then, run your training command.Single GPU trainingexport CUDA_VISIBLE_DEVICES=0
python scripts/train.py --args.load conf/ablations/baseline.yml --save_path runs/baseline/Multi GPU trainingexport CUDA_VISIBLE_DEVICES=0,1
torchrun --nproc_per_node gpu scripts/train.py --args.load conf/ablations/baseline.yml --save_path runs/baseline/TestingWe provide two test scripts to test CLI + training functionality. Please
make sure that the trainig pre-requisites are satisfied before launching these
tests. To launch these tests please runpython -m pytest testsResults