Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,47 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
## Model Card: Wav2vec_Classroom_FT
|
| 2 |
+
|
| 3 |
+
### Model Overview
|
| 4 |
+
**Model Name:**Wav2vec_Classroom_FT
|
| 5 |
+
**Version:** 1.0
|
| 6 |
+
**Developed By:** Ahmed Adel Attia (University of Maryland and Stanford University)
|
| 7 |
+
**Date:** 2025
|
| 8 |
+
|
| 9 |
+
**Description:**
|
| 10 |
+
NCTE-Baseline-ASR is an automatic speech recognition (ASR) model trained for classroom speech transcription using direct fine-tuning on a small set of human-verified gold-standard transcriptions. Unlike **NCTE-WSP-ASR**, this model does not leverage weak transcriptions for intermediate training and is solely trained on high-quality annotations.
|
| 11 |
+
|
| 12 |
+
This model is adapted from **[Wav2vec-Classroom](https://huggingface.co/aadel4/Wav2vec_Classroom)**, which was trained using continued pretraining (CPT) on large-scale unlabeled classroom speech data. The adaptation involves direct fine-tuning on a limited transcribed dataset.
|
| 13 |
+
|
| 14 |
+
**Use Case:**
|
| 15 |
+
- Speech-to-text transcription for classroom environments.
|
| 16 |
+
- ASR applications requiring high precision with limited data.
|
| 17 |
+
- Benchmarking ASR performance without weakly supervised pretraining.
|
| 18 |
+
|
| 19 |
+
### Model Details
|
| 20 |
+
**Architecture:** Wav2vec2.0-based model fine-tuned with Fairseq
|
| 21 |
+
|
| 22 |
+
**Training Data:**
|
| 23 |
+
- **NCTE-Gold:** 13 hours of manually transcribed classroom recordings.
|
| 24 |
+
|
| 25 |
+
**Training Strategy:**
|
| 26 |
+
1. **Direct Fine-tuning:** The model is fine-tuned directly on NCTE-Gold without any pretraining on weak transcripts.
|
| 27 |
+
2. **Evaluation:** The model is tested on classroom ASR tasks to compare its performance with WSP-based models.
|
| 28 |
+
|
| 29 |
+
### Evaluation Results
|
| 30 |
+
**Word Error Rate (WER) comparison on NCTE and MPT test sets:**
|
| 31 |
+
|
| 32 |
+
| Training Data | NCTE WER | MPT WER |
|
| 33 |
+
|--------------|----------|---------|
|
| 34 |
+
| **Baseline (TEDLIUM-trained ASR)** | 55.82 / 50.56 | 55.11 / 50.50 |
|
| 35 |
+
| **NCTE-Gold only (NCTE-Baseline-ASR)** | 21.12 / 16.47 | 31.52 / 27.93 |
|
| 36 |
+
| **NCTE-WSP-ASR (NCTE-Weak → NCTE-Gold)** | **16.54 / 13.51** | **25.07 / 23.70** |
|
| 37 |
+
|
| 38 |
+
### Limitations
|
| 39 |
+
- The model is trained on a small dataset (13 hours), which limits its ability to generalize beyond classroom speech.
|
| 40 |
+
- Performance is lower than **NCTE-WSP-ASR**, which benefits from weak transcripts for pretraining.
|
| 41 |
+
- Background noise, overlapping speech, and speaker variations may still impact transcription quality.
|
| 42 |
+
|
| 43 |
+
### Usage Request
|
| 44 |
+
If you use the NCTE-Baseline-ASR model in your research, please acknowledge this work and refer to the original paper submitted to Interspeech 2025.
|
| 45 |
+
|
| 46 |
+
For inquiries or collaborations, please contact the authors of the original paper.
|
| 47 |
+
|