language: en
datasets:
- superb
tags:
- speech
- audio
- hubert
- s3prl
license: apache-2.0
Hubert-Base for Intent Classification
Model description
This is a ported version of S3PRL's Hubert for the SUPERB Intent Classification task.
The base model is hubert-base-ls960. It is pretrained on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz.
For more information refer to SUPERB: Speech processing Universal PERformance Benchmark
Task description
Intent Classification (IC) classifies utterances into predefined classes to determine the intent of speakers. SUPERB uses the Fluent Speech Commands dataset, where each utterance is tagged with three intent labels: action, object, and location.
For the original model's training and evaluation instructions refer to the S3PRL downstream task README.
Usage examples
You can use the model directly like so:
import torch
import numpy as np
from datasets import load_dataset
from transformers import HubertForSequenceClassification, Wav2Vec2FeatureExtractor
# TODO: replace with the official superb dataset
superb_ks = load_dataset("anton-l/superb_dummy", "ic", split="test")
model = HubertForSequenceClassification.from_pretrained("superb/hubert-base-superb-ic")
feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained("superb/hubert-base-superb-ic")
audio = np.array(superb_ks[0]["speech"])
# compute attention masks and normalize the waveform if needed
inputs = feature_extractor(audio, sampling_rate=16_000, return_tensors="pt")
logits = model(**inputs).logits[0]
action_ids = torch.argmax(logits[:, :6], dim=-1).tolist()
action_labels = [model.config.id2label[_id] for _id in action_ids]
object_ids = torch.argmax(logits[:, 6:20], dim=-1).tolist()
object_labels = [model.config.id2label[_id + 6] for _id in object_ids]
location_ids = torch.argmax(logits[:, 20:24], dim=-1).tolist()
location_labels = [model.config.id2label[_id + 20] for _id in location_ids]
Eval results
The evaluation metric is accuracy.
s3prl |
transformers |
|
|---|---|---|
test |
TBA | TBA |
BibTeX entry and citation info
@article{yang2021superb,
title={SUPERB: Speech processing Universal PERformance Benchmark},
author={Yang, Shu-wen and Chi, Po-Han and Chuang, Yung-Sung and Lai, Cheng-I Jeff and Lakhotia, Kushal and Lin, Yist Y and Liu, Andy T and Shi, Jiatong and Chang, Xuankai and Lin, Guan-Ting and others},
journal={arXiv preprint arXiv:2105.01051},
year={2021}
}