Spaces:
Runtime error
Runtime error
mattricesound
commited on
Commit
•
1ef06f0
1
Parent(s):
051ea71
Update README
Browse files
README.md
CHANGED
@@ -1,7 +1,7 @@
|
|
1 |
# General Purpose Audio Effect Removal
|
2 |
Removing multiple audio effects from multiple sources using compositional audio effect removal and source separation and speech enhancement models.
|
3 |
|
4 |
-
This repo contains the code for the paper [General Purpose Audio Effect Removal](https://arxiv.org/abs/2110.00484). (Todo: Link broken, Add video, Add img, citation)
|
5 |
|
6 |
|
7 |
# Setup
|
@@ -13,18 +13,18 @@ pip install -e . ./umx
|
|
13 |
```
|
14 |
# Usage
|
15 |
This repo can be used for many different tasks. Here are some examples.
|
16 |
-
## Run RemFX Detect on a single file
|
17 |
First, need to download the checkpoints from [zenodo](https://zenodo.org/record/8179396)
|
18 |
```
|
19 |
./download_checkpoints.sh
|
20 |
./remfx_detect.sh wet.wav -o dry.wav
|
21 |
```
|
22 |
-
## Download the [General Purpose Audio Effect Removal evaluation datasets](https://zenodo.org/record/8183649/)
|
23 |
```
|
24 |
./download_eval_datasets.sh
|
25 |
```
|
26 |
|
27 |
-
## Download the starter datasets
|
28 |
```
|
29 |
python scripts/download.py vocalset guitarset dsd100 idmt-smt-drums
|
30 |
```
|
@@ -35,7 +35,7 @@ Then set the dataset root :
|
|
35 |
export DATASET_ROOT={path/to/datasets}
|
36 |
```
|
37 |
|
38 |
-
## Training
|
39 |
Before training, it is important that you have downloaded the starter datasets (see above) and set DATASET_ROOT.
|
40 |
This project uses the [pytorch-lightning](https://www.pytorchlightning.ai/index.html) framework and [hydra](https://hydra.cc/) for configuration management. All experiments are defined in `cfg/exp/`. To train with an existing experiment run
|
41 |
```
|
@@ -69,7 +69,7 @@ If you have generated the dataset separately (see Generate datasets used in the
|
|
69 |
|
70 |
Also note that the training assumes you have a GPU. To train on CPU, set `accelerator=null` in the config or command-line.
|
71 |
|
72 |
-
## Evaluate models on the General Purpose Audio Effect Removal evaluation datasets (Table 4 from the paper)
|
73 |
First download the General Purpose Audio Effect Removal evaluation datasets (see above).
|
74 |
To use the pretrained RemFX model, download the checkpoints
|
75 |
```
|
@@ -105,7 +105,7 @@ Then run the evaluation script.
|
|
105 |
The script assumes that RemFX_eval_datasets is in the top-level directory.
|
106 |
Metrics and hyperparams will be logged in `./lightning_logs/{timestamp}`
|
107 |
|
108 |
-
## Generate other datasets
|
109 |
The datasets used in the experiments are customly generated from the starter datasets. In short, for each training/val/testing example, we select a random 5.5s segment from one of the starter datasets and apply a random number of effects to it. The number of effects applied is controlled by the `num_kept_effects` and `num_removed_effects` parameters. The effects applied are controlled by the `effects_to_keep` and `effects_to_remove` parameters.
|
110 |
|
111 |
Before generating datasets, it is important that you have downloaded the starter datasets (see above) and set DATASET_ROOT.
|
@@ -155,7 +155,7 @@ Some relevant dataset/training parameters descriptions
|
|
155 |
- `delay`
|
156 |
|
157 |
# DO WE NEED THIS?
|
158 |
-
## Evaluate RemFXwith a custom directory
|
159 |
Assumes directory is structured as
|
160 |
- root
|
161 |
- clean
|
|
|
1 |
# General Purpose Audio Effect Removal
|
2 |
Removing multiple audio effects from multiple sources using compositional audio effect removal and source separation and speech enhancement models.
|
3 |
|
4 |
+
This repo contains the code for the paper [General Purpose Audio Effect Removal](https://arxiv.org/abs/2110.00484). (Todo: Link broken, Add video, Add img, citation, licence)
|
5 |
|
6 |
|
7 |
# Setup
|
|
|
13 |
```
|
14 |
# Usage
|
15 |
This repo can be used for many different tasks. Here are some examples.
|
16 |
+
## Run RemFX Detect on a single file
|
17 |
First, need to download the checkpoints from [zenodo](https://zenodo.org/record/8179396)
|
18 |
```
|
19 |
./download_checkpoints.sh
|
20 |
./remfx_detect.sh wet.wav -o dry.wav
|
21 |
```
|
22 |
+
## Download the [General Purpose Audio Effect Removal evaluation datasets](https://zenodo.org/record/8183649/)
|
23 |
```
|
24 |
./download_eval_datasets.sh
|
25 |
```
|
26 |
|
27 |
+
## Download the starter datasets
|
28 |
```
|
29 |
python scripts/download.py vocalset guitarset dsd100 idmt-smt-drums
|
30 |
```
|
|
|
35 |
export DATASET_ROOT={path/to/datasets}
|
36 |
```
|
37 |
|
38 |
+
## Training
|
39 |
Before training, it is important that you have downloaded the starter datasets (see above) and set DATASET_ROOT.
|
40 |
This project uses the [pytorch-lightning](https://www.pytorchlightning.ai/index.html) framework and [hydra](https://hydra.cc/) for configuration management. All experiments are defined in `cfg/exp/`. To train with an existing experiment run
|
41 |
```
|
|
|
69 |
|
70 |
Also note that the training assumes you have a GPU. To train on CPU, set `accelerator=null` in the config or command-line.
|
71 |
|
72 |
+
## Evaluate models on the General Purpose Audio Effect Removal evaluation datasets (Table 4 from the paper)
|
73 |
First download the General Purpose Audio Effect Removal evaluation datasets (see above).
|
74 |
To use the pretrained RemFX model, download the checkpoints
|
75 |
```
|
|
|
105 |
The script assumes that RemFX_eval_datasets is in the top-level directory.
|
106 |
Metrics and hyperparams will be logged in `./lightning_logs/{timestamp}`
|
107 |
|
108 |
+
## Generate other datasets
|
109 |
The datasets used in the experiments are customly generated from the starter datasets. In short, for each training/val/testing example, we select a random 5.5s segment from one of the starter datasets and apply a random number of effects to it. The number of effects applied is controlled by the `num_kept_effects` and `num_removed_effects` parameters. The effects applied are controlled by the `effects_to_keep` and `effects_to_remove` parameters.
|
110 |
|
111 |
Before generating datasets, it is important that you have downloaded the starter datasets (see above) and set DATASET_ROOT.
|
|
|
155 |
- `delay`
|
156 |
|
157 |
# DO WE NEED THIS?
|
158 |
+
## Evaluate RemFXwith a custom directory
|
159 |
Assumes directory is structured as
|
160 |
- root
|
161 |
- clean
|