speech-test commited on
Commit
f80d68b
1 Parent(s): f225ffc

Update info

Browse files
Files changed (1) hide show
  1. README.md +87 -0
README.md ADDED
@@ -0,0 +1,87 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ datasets:
4
+ - superb
5
+ tags:
6
+ - speech
7
+ - audio
8
+ - wav2vec2
9
+ license: apache-2.0
10
+ ---
11
+
12
+ # Wav2Vec2-Base for Intent Classification
13
+
14
+ ## Model description
15
+
16
+ This is a ported version of [S3PRL's Wav2Vec2 for the SUPERB Intent Classification task](https://github.com/s3prl/s3prl/tree/master/s3prl/downstream/fluent_commands).
17
+
18
+ The base model is [wav2vec2-base](https://huggingface.co/facebook/wav2vec2-base), which is pretrained on 16kHz
19
+ sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz.
20
+
21
+ For more information refer to [SUPERB: Speech processing Universal PERformance Benchmark](https://arxiv.org/abs/2105.01051)
22
+
23
+ ## Task and dataset description
24
+
25
+ Intent Classification (IC) classifies utterances into predefined classes to determine the intent of
26
+ speakers. SUPERB uses the
27
+ [Fluent Speech Commands](https://fluent.ai/fluent-speech-commands-a-dataset-for-spoken-language-understanding-research/)
28
+ dataset, where each utterance is tagged with three intent labels: **action**, **object**, and **location**.
29
+
30
+ For the original model's training and evaluation instructions refer to the
31
+ [S3PRL downstream task README](https://github.com/s3prl/s3prl/tree/master/s3prl/downstream#ic-intent-classification---fluent-speech-commands).
32
+
33
+
34
+ ## Usage examples
35
+
36
+ You can use the model directly like so:
37
+ ```python
38
+ import torch
39
+ import librosa
40
+ from datasets import load_dataset
41
+ from transformers import Wav2Vec2ForSequenceClassification, Wav2Vec2FeatureExtractor
42
+
43
+ def map_to_array(example):
44
+ speech, _ = librosa.load(example["file"], sr=16000, mono=True)
45
+ example["speech"] = speech
46
+ return example
47
+
48
+ # load a demo dataset and read audio files
49
+ dataset = load_dataset("anton-l/superb_demo", "ic", split="test")
50
+ dataset = dataset.map(map_to_array)
51
+
52
+ model = Wav2Vec2ForSequenceClassification.from_pretrained("superb/wav2vec2-base-superb-ic")
53
+ feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained("superb/wav2vec2-base-superb-ic")
54
+
55
+ # compute attention masks and normalize the waveform if needed
56
+ inputs = feature_extractor(dataset[:4]["speech"], sampling_rate=16000, padding=True, return_tensors="pt")
57
+
58
+ logits = model(**inputs).logits
59
+
60
+ action_ids = torch.argmax(logits[:, :6], dim=-1).tolist()
61
+ action_labels = [model.config.id2label[_id] for _id in action_ids]
62
+
63
+ object_ids = torch.argmax(logits[:, 6:20], dim=-1).tolist()
64
+ object_labels = [model.config.id2label[_id + 6] for _id in object_ids]
65
+
66
+ location_ids = torch.argmax(logits[:, 20:24], dim=-1).tolist()
67
+ location_labels = [model.config.id2label[_id + 20] for _id in location_ids]
68
+ ```
69
+
70
+ ## Eval results
71
+
72
+ The evaluation metric is accuracy.
73
+
74
+ | | **s3prl** | **transformers** |
75
+ |--------|-----------|------------------|
76
+ |**test**| `0.9235` | `N/A` |
77
+
78
+ ### BibTeX entry and citation info
79
+
80
+ ```bibtex
81
+ @article{yang2021superb,
82
+ title={SUPERB: Speech processing Universal PERformance Benchmark},
83
+ author={Yang, Shu-wen and Chi, Po-Han and Chuang, Yung-Sung and Lai, Cheng-I Jeff and Lakhotia, Kushal and Lin, Yist Y and Liu, Andy T and Shi, Jiatong and Chang, Xuankai and Lin, Guan-Ting and others},
84
+ journal={arXiv preprint arXiv:2105.01051},
85
+ year={2021}
86
+ }
87
+ ```