Support for Kaldi formatted Audio files, especially "segments"
Author: picheny-nyu
Pull request: no

# 🚀 Feature request

Kaldi is a popular open source speech recognition system that manages audio in a simple but unique format. It would be great to have some addition to the datasets object that would take a Kaldi directory and turn this into a dataset. Barring this, it would be very helpful to have a modification of the datasets audio class that would permit processing of parts of long audio files rather than have to create a new set of audio files from the larger set. 

## Motivation

Would make it much easier to have cross system portability and allow HuggingFace to easily support a much larger collection of audio data. 



Comments:
Thanks for your interest, @picheny-nyu!
Indeed, this could be quite useful. The logic can either be implemented in `huggingface/datasets` for each dataset in Kaldi format individually, or we could add an example script to `transformers/examples` that converts Kaldi datasets. Feel free to open a PR if you'd like to contribute those :) 

Also pinging @patrickvonplaten since he has a bit more experience with Kaldi 

Hey @picheny-nyu,

I very much agree that we should have better support to allow datasets of long audio files to be chunked and I agree that this should happen in `datasets`.

Regarding interoperability with Kaldi, I think it's a nice idea, but I'm a bit worried about whether this is very easy to do or not especially given that Kaldi is written in C++. 
Kaldi is also not very actively maintained anymore I think as the original authors seem to focus more and more on Python-based libraries for ASR:
- https://github.com/lhotse-speech/lhotse
- https://github.com/k2-fsa/icefall 



Also cc @lhoestq 

Thanks for the response. I was primarily thinking of kaldi input formats for waveforms and transcriptions, not kaldi functionality in general, though that would also be nice. They have some very nice features for organizing and processing input data that looks like it would not be too hard to reproduce inside huggingface processing. We could ask the kaldi authors if they plan to continue this in their new python version too. 

Let's see what @lhoestq thinks here

Hi ! Yes feel free to open an issue in https://github.com/huggingface/datasets to discuss the loading of Kaldi structured datasets. We can imagine having something like
```python
from datasets import load_dataset

my_kaldi_dataset = load_dataset("kaldi_folder", data_dir="path/to/my/kaldi/data")
```

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

If someone else needs this, here is how I tackled the problem:
```python
import datasets
import kaldiio  # https://github.com/nttcslab-sp/kaldiio

_DATA_DIR = "path/to/kaldi_dir"
_FILEPATHS = {
    "train": {
        "feats": "dump/train/feats.scp",
        "texts": "data/train/text",
    },
    "valid": {
        "feats": "dump/valid/feats.scp",
        "texts": "data/valid/text",
    },
    "test": {
        "feats": "dump/test/feats.scp",
        "texts": "data/test/text",
    },
}

class DatasetWithPrecomputedFeatures(datasets.GeneratorBasedBuilder):

    def _split_generators(self, _):

        return [
            datasets.SplitGenerator(
                name=datasets.Split.TRAIN,
                gen_kwargs={"split": "train"},
            ),
            datasets.SplitGenerator(
                name=datasets.Split.VALIDATION,
                gen_kwargs={"split": "valid"},

            ),
            datasets.SplitGenerator(
                name=datasets.Split.TEST,
                gen_kwargs={"split": "test"},

            ),
        ]

    def _generate_examples(self, split):

        with open(os.path.join(_DATA_DIR, _FILEPATHS[split]["texts"])) as texts:
            texts = dict(map(lambda s: s.strip().split(maxsplit=1), f))

        featfile = os.path.join(_DATA_DIR, _FILEPATHS[split]["feats"])
        feats_generator = kaldiio.load_scp(featfile)
        
        for key, (uttid, transcript) in enumerate(texts.items()):
            feats = feats_generator[uttid]
            if feats.ndim > 2:
                if feats.shape[0] != 1:
                    raise ValueError(f"Too many dimensions for {uttid}: {feats.shape}")
                feats = feats.squeeze(0)

            yield key, {"feats": feats, "text": transcript, "id": uttid}
```


@qmeeus this looks great! Would you like to open a pull requests in `datasets` to add this dataset download file under `datasets/datasets/kaldi_folder`? 

cc @polinaeterna @lhoestq