Adding utilities to chunk large audio files and read directly from microphone
Author: Narsil
Pull request: yes

------------
This PR and was slowly merged bits by bits (with sometimes major changes) into transformers.
Keeping it open for VAD chunking which is in this PR and not yet available in transormers
------------

- chunk_files, requires scipy only, relatively straightforward.
- vad_files, requires webrtcvad, will chunk both large chunks and if
  voice is unactivated (good potential if lots of silence, but might
miss large portions of the audio).
- Both require ffmpeg too, maybe move to `av` (initial implementation
  was 4x slower and more complex though.)
- ffmpeg_microphone will stream audio from the machine's microphone.
  ~~no streaming (meaning temporary results while the whole frame is being
  processed) yet, but should be relatively easy to do and probably
  pipeline agnostic.~~ streaming support (but a bit manual)

All those functions have many knobs to turn which can affect the end
result quite drastically, so there are no "sane" defaults (afaik).
For now those are explicitely separate from the core of pipeline meaning
they are likely to change, and simply meant as helper functions to keep
simple APIs even on more challenging data, and make demos easy to do.
Another benefit is that we can expose all those knobs without exploding
the pipeline's complexity (they are not exposed yet).

Current defaults yield both for chunk_files and vad_files 47WER on AMI
with `facebook/wav2vec2-base-960h` which is on par with expectations.


Wer script:

```python
from jiwer import wer                                                                
                                                                                                                          
from datasets import load_dataset                                                    
from transformers import pipeline                                                    
from transformers.pipelines.audio_utils import chunk_files, vad_files                
import tqdm                                                                          
import numpy as np                                                                   
import re                                                                            
                                                                                     
                                                                                     
def evaluate():                                                                      
    dataset = load_dataset("ami", "headset-single", split="validation")              
                                                                                     
    pipe = pipeline("automatic-speech-recognition", device=0)                        
    sampling_rate = pipe.feature_extractor.sampling_rate                             
                                                                                     
    non_letters = re.compile(r"[^a-z'\s]+")                                                                
    multi_space = re.compile(r"\s+")                                                 
                                                                                     
    vad_wers = []                                                                    
    chunk_wers = []                                                                  
    max_chunk_duration_s = 20                                                        
    for item in tqdm.tqdm(dataset):                                                  
        words = item["words"]                                                        
        filename = item["file"]                                                      
        target_text = " ".join(words).lower()                                        
        target_text = non_letters.sub("", target_text)                               
        target_text = multi_space.sub(" ", target_text)                              
                                                                                     
        pred_text = ""                                                               
        for item in tqdm.tqdm(pipe(chunk_files([filename], sampling_rate, max_chunk_duration_s))):   
            pred_text += " " + item["text"]                                       
        pred_text = pred_text.lower()                                             
        chunk_wers.append(wer(target_text, pred_text))                            
                                                                                  
        pred_text = ""                                                            
        for item in tqdm.tqdm(pipe(vad_files([filename], sampling_rate, max_chunk_duration_s))):   
            pred_text += " " + item["text"]                                       
        pred_text = pred_text.lower()                                             
        vad_wers.append(wer(target_text, pred_text))                              
                                                                                  
    return np.mean(chunk_wers), np.mean(vad_wers)                                 
                                                                                  
                                                                                  
if __name__ == "__main__":                                                        
    score = evaluate()                                                            
    print("WER: ", score)  
```

Microphone streaming:

```python
import datetime                                                                      
import sys                                                                                                                
from transformers import pipeline                                                    
from transformers.pipelines.audio_utils import ffmpeg_microphone                     
                                                                                     
nlp = pipeline("automatic-speech-recognition", device=0)                             
sampling_rate = nlp.feature_extractor.sampling_rate                                  
                                                                                     
                                                                                     
start = datetime.datetime.now()                                                      
                                                                                     
max_chunk_duration_s = 5                                                             
stream_chunk_ms = 50                                                                 
N = max_chunk_duration_s * 1000 / stream_chunk_ms                                    
for i, item in enumerate(                                                            
    nlp(                                                                             
        ffmpeg_microphone(                                                                                 
            sampling_rate=sampling_rate,                                             
            format_for_conversion="f32le",                                           
            max_chunk_duration_s=max_chunk_duration_s,                               
            stream_chunk_ms=stream_chunk_ms,                                         
        ),                                                                           
        batch_size=1,                                                                
        num_workers=1,                                                               
    )                                                                                
):                                                                                   
    sys.stdout.write("\033[K")                                                       
    print(item["text"], end="\r")                                                    
    if i % N == N - 1:                                                               
        print("") 
```

Edit: Separating this from ChunkPipeline which in the end is totally
unrelated work (linked to decision to keep these as helpers instead
of within the pipeline).


# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)

@LysandreJik
@anton-l


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Reopening since I am using stuff from this PR for testing (`ffmpeg_microphone` namely)

Awesome that you found a way to stream audio!!!

If possible I'd be happy to first get https://github.com/huggingface/transformers/pull/14896#pullrequestreview-841262000 merged to enable offline decoding of very large files and once that's done it would be great to tackle the online streaming case. Could we maybe put the VAD utilities in a new seperate PR since it is a bit unrelated in my opinion to "online streaming" as explained here: https://github.com/huggingface/transformers/pull/14896#discussion_r776267284 ? 



This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

unstale

Looking now!

Sorry @patrickvonplaten I induced you in error.

This is the main PR, from which we decided I would stem sub, smaller PRs:

This one is the smaller (which I need to update too apparently) https://github.com/huggingface/transformers/pull/15046

I merely rebased this one so it wouldn't be too stale..

No worries! Will take a look at the new one tomorrow first thing then :-)

Sorry is this PR still relevant?

Well, it's still contains the `vad` chunking. 

It's more of a safekeep PR, I'll mark it as draft since we shouldn't merge it.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.