EIDORA/Wav2Vec2_Base_Speech

Wav2Vec2 Base — Speech Embeddings

Wav2Vec2 Base learns representations from speech audio. EIDORA produces one normalized 768-dimensional embedding for each selected short speech excerpt.

Training Data and Checkpoint Provenance

• Architecture: Wav2Vec2 Base self-supervised speech encoder.

• Original checkpoint: facebook/wav2vec2-base.

• Pinned revision: 0b5b8e868dd84f03fd87d01f9c4ff0f080fecfe8.

• Training data: Pretrained speech representations from the original facebook/wav2vec2-base checkpoint.

• Training method: Self-supervised speech pretraining; the linked checkpoint information here does not identify an exhaustive corpus list.

• Source: Wav2Vec2 Base (original author attribution retained).

Model Overview

• Model: Wav2Vec2 Base — Speech Embeddings
• EIDORA identifier: eidora/audio-wav2vec2-base-medium
• Modality: Audio
• Embedding dimensions: 768
• Embedding normalization: L2-normalized
• Similarity measure: Cosine similarity
• Package schema: Version 3
• Model version: 1.0.0

EIDORA Use and Supported Modalities

Supported modality: Audio/speech only.

Researchers can use this package to:

• Explore similarity among speech recordings.
• Cluster speech excerpts using learned representations.
• Compare speech-oriented audio embeddings across a collection.

Input and Preprocessing

EIDORA prepares speech as a mono waveform:

1. Decode audio and use a 16,000 Hz sampling rate.
2. Take the configured beginning excerpt up to 32,000 samples (two seconds).
3. Pad shorter waveforms as needed and supply a valid_samples count so padding is not treated as recorded speech.
4. Pass waveform as float32 shape [1, 32000] and valid_samples as int64 shape [1].

EIDORA performs audio preparation outside the ONNX graph. The model processes one recording at a time.

Output Embeddings

The ONNX output is float32 with shape [1, 768]. Each vector has 768 dimensions, is L2-normalized, and is intended for cosine similarity comparisons.

EIDORA selects a short waveform excerpt, tracks the valid sample count to exclude padding, mean-pools time-dependent encoder features and L2-normalizes the resulting vector.

Runtime and Compatibility

The package uses model.onnx and schema-v3 runtime declarations for audio preparation and embedding output.

It declares CPUExecutionProvider. Actual execution availability depends on the installed ONNX Runtime and EIDORA Desktop environment. config.yaml is authoritative for graph inputs, preprocessing, provider and embedding dimensions.

Limitations

• The model is not a speech-to-text, speaker identification, emotion recognition or music-classification system.
• EIDORA uses a maximum two-second segment at 16 kHz; longer recordings are excerpted.
• The ONNX encoder requires at least 400 valid samples, which is a technical minimum, not a recommended duration.

Results should be evaluated against the intended research data and questions.

Research Context and References

Main paper

**wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations **

Paper: https://arxiv.org/abs/2006.11477

The publication describes the upstream learning method and original model. The embeddings returned by EIDORA additionally depend on the specific preprocessing and output adaptation documented here.

Primary citation (BibTeX)

@inproceedings{baevski2020wav2vec,
  title={wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations},
  author={Baevski, Alexei and Zhou, Yuhao and Mohamed, Abdelrahman and Auli, Michael},
  booktitle={Advances in Neural Information Processing Systems},
  year={2020}
}

License and Attribution

The upstream facebook/wav2vec2-base checkpoint is listed under Apache License 2.0. See the included LICENSE. EIDORA attributes the original speech model to Meta AI and the wav2vec 2.0 researchers.

See the LICENSE file included in this repository.

Package Information

• Model repository: EIDORA/Wav2Vec2_Base_Speech
• Package version: 1.0.0
• Package schema: 3
• EIDORA configuration: config.yaml
• ONNX graph(s): model.onnx
• Supported modality: Audio
• License identifier: apache-2.0
• Declared provider: CPUExecutionProvider
• Required supplementary files: LICENSE

The configuration defines the authoritative model-facing runtime contract. Other packaged assets, where present, do not introduce additional inference modalities.