Update README.md
Browse files
README.md
CHANGED
@@ -8,17 +8,21 @@ inference: false
|
|
8 |
**Music2Vec** is accepted as 2-page abstract in Late Breaking Demos (LBD) at the ISMIR 2022.
|
9 |
It is a completely unsupervised model trained on 1000 hour music audios.
|
10 |
We release the **crop5s** version as music2vec-v1.
|
11 |
-
Our model is SOTA-comparable on multiple MIR tasks even under probing settings, while keeping fine-tunable on a single 2080Ti.
|
12 |
Larger models trained with more data are on the way~
|
13 |
|
|
|
|
|
|
|
14 |
![Model Architecture](music2vec.png)
|
15 |
|
|
|
|
|
|
|
16 |
![Performance Comparison](music2vec_performance.png)
|
17 |
|
18 |
# Model Usage
|
19 |
|
20 |
-
## Huggingface Loading
|
21 |
-
|
22 |
```python
|
23 |
from transformers import Wav2Vec2Processor, Data2VecAudioModel
|
24 |
import torch
|
|
|
8 |
**Music2Vec** is accepted as 2-page abstract in Late Breaking Demos (LBD) at the ISMIR 2022.
|
9 |
It is a completely unsupervised model trained on 1000 hour music audios.
|
10 |
We release the **crop5s** version as music2vec-v1.
|
11 |
+
Our base model is SOTA-comparable on multiple MIR tasks even under probing settings, while keeping fine-tunable on a single 2080Ti.
|
12 |
Larger models trained with more data are on the way~
|
13 |
|
14 |
+
# Model Architecture
|
15 |
+
Music2Vec Framework. During pre-training, the student model aims to
|
16 |
+
reconstruct the masked music audio by taking the contextualized representations provided by the teacher model as prediction targets.
|
17 |
![Model Architecture](music2vec.png)
|
18 |
|
19 |
+
# Performance Comparison
|
20 |
+
With 95M parameters and relatively small training data (1k hr), our base Music2Vec representation achieves comparable performance to the SOTA Jukebox-5B representation.
|
21 |
+
Note that our base model size is 2% of Jukebox-5B.
|
22 |
![Performance Comparison](music2vec_performance.png)
|
23 |
|
24 |
# Model Usage
|
25 |
|
|
|
|
|
26 |
```python
|
27 |
from transformers import Wav2Vec2Processor, Data2VecAudioModel
|
28 |
import torch
|