Package: encodec
EnCodec: High Fidelity Neural Audio CompressionThis is the code for the EnCodec neural codec presented in theHigh Fidelity Neural Audio Compression[abs].
paper. We provide our two multi-bandwidth models:A causal model operating at 24 kHz on monophonic audio trained on a variety of audio data.A non-causal model operationg at 48 kHz on stereophonic audio trained on music-only data.The 24 kHz model can compress to 1.5, 3, 6, 12 or 24 kbps, while the 48 kHz model
support 3, 6, 12 and 24 kbps. We also provide a pre-trained language model for each
of the models, that can further compress the representation by up to 40% without
any further loss of quality.For reference, we also provide the code for our novel MS-STFT discriminator.SamplesSamples including baselines are provided onour sample page.
You can also have a quick demo of what we achieve for 48 kHz music with EnCodec, along with
entropy coding, by clicking the thumbnail (original tracks provided byLucille CrewandVoyageur I).What's up?Seethe changelogfor details on releases.InstallationEnCodec requires Python 3.8, and a reasonably recent version of PyTorch (1.11.0 ideally).
To install EnCodec, you can run from this repository:pipinstall-Uencodec# stable releasepipinstall-Ugit+https://git@github.com/facebookresearch/encodec#egg=encodec# bleeding edge# of if you cloned the repo locallypipinstall.UsageYou can then use the EnCodec command, either aspython3-mencodec[...]# orencodec[...]If you want to directly use the compression API, checkoutencodec.compressandencodec.model. See hereafter for instructions on how to extract the discrete
representation.Model storageThe models will be automatically downloaded on first use using Torch Hub.
For more information on where those models are stored, or how to customize
the storage location,checkout their documentation.Compressionencodec[-bTARGET_BANDWIDTH][-f][--hq][--lm]INPUT_FILE[OUTPUT_FILE]Given any audio file supported by torchaudio on your platform, compresses
it with EnCodec to the target bandwidth (default is 6 kbps, can be either 1.5, 3, 6, 12 or 24).
OUTPUT_FILE must end in.ecdc. If not provided it will be the same asINPUT_FILE,
replacing the extension with.ecdc.
In order to use the model operating at 48 kHz on stereophonic audio, use the--hqflag.
The-fflag is used to force overwrite an existing output file.
Use the--lmflag to use the pretrained language model with entropy coding (expect it to
be much slower).If the sample rate or number of channels of the input doesn't match that of the model,
the command will automatically resample / reduce channels as needed.Decompressionencodec[-f][-r]ENCODEC_FILE[OUTPUT_WAV_FILE]Given a.ecdcfile previously generated, this will decode it to the given output wav file.
If not provided, the output will default to the input with the.wavextension.
Use the-ffile to force overwrite the output file (be carefull if compress then decompress,
not to overwrite your original file !). Use the-rflag if you experience clipping, this will
rescale the output file to avoid it.Compression + Decompressionencodec[-r][-bTARGET_BANDWIDTH][-f][--hq][--lm]INPUT_FILEOUTPUT_WAV_FILEWhenOUTPUT_WAV_FILEhas the.wavextension (as opposed to.ecdc), theencodeccommand will instead compress and immediately decompress without storing the intermediate.ecdcfile.Extracting discrete representationsThe EnCodec model can also be used to extract discrete representations from the audio waveform.fromencodecimportEncodecModelfromencodec.utilsimportconvert_audioimporttorchaudioimporttorch# Instantiate a pretrained EnCodec modelmodel=EncodecModel.encodec_model_24khz()model.set_target_bandwidth(6.0)# Load and pre-process the audio waveformwav,sr=torchaudio.load("<PATH_TO_AUDIO_FILE>")wav=wav.unsqueeze(0)wav=convert_audio(wav,sr,model.sample_rate,model.channels)# Extract discrete codes from EnCodecencoded_frames=model.encode(wav)codes=torch.cat([encoded[0]forencodedinencoded_frames],dim=-1)# [B, n_q, T]Note that the 48 kHz model processes the audio by chunks of 1 seconds, with an overlap of 1%,
and renormalizes the audio to have unit scale. For this model, the output ofmodel.encode(wav)would a list (for each frame of 1 second) of a tuple(codes, scale)withscalea scalar tensor.Installation for developmentThis will install the dependencies and aencodecin developer mode (changes to the files
will directly reflect), along with the dependencies to run unit tests.pip install -e '.[dev]'TestYou can run the unit tests withmake testsCitationIf you use this code or results in your paper, please cite our work as:@article{defossez2022highfi,
  title={High Fidelity Neural Audio Compression},
  author={Défossez, Alexandre and Copet, Jade and Synnaeve, Gabriel and Adi, Yossi},
  journal={arXiv preprint arXiv:2210.13438},
  year={2022}
}LicenseThis repository is released under the CC-BY-NC 4.0. license as found in theLICENSEfile.