[WIP] Adding LID (Language Identification) Head for M-CTC-T Model
Author: cwkeam
Pull request: yes

# What does this PR do?

Adding LID head to the M-CTC-T Model.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@patrickvonplaten 


Comments:
Developing this as a `SequenceClassification` head. This is correct right? @patrickvonplaten 

TODO:
- [ ] Export LID head weights from flashlight model
- [ ] make new `cwkeam/m-ctct-t-large-lid` checkpoint with above exported weights
- [ ] write tests

This looks good to me @lorenlugosch and @anton-l  what do you think? 

Looks good---I'll give you the mapping from logit index to language shortly.

```
0: ab
1: ar
2: as
3: br
4: ca
5: cnh
6: cs
7: cv
8: cy
9: de
10: dv
11: el
12: en
13: eo
14: es
15: et
16: eu
17: fa
18: fi
19: fr
20: fy-NL
21: ga-IE
22: hi
23: hsb
24: hu
25: ia
26: id
27: it
28: ja
29: ka
30: kab
31: ky
32: lg
33: lt
34: lv
35: mn
36: mt
37: nl
38: or
39: pa-IN
40: pl
41: pt
42: rm-sursilv
43: rm-vallader
44: ro
45: ru
46: rw
47: sah
48: sl
49: sv-SE
50: ta
51: th
52: tr
53: tt
54: uk
55: vi
56: vot
57: zh-CN
58: zh-HK
59: zh-TW
```

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_17671). All of your documentation changes will be reflected on that endpoint.

@lorenlugosch if that was the intention for the original implementation then yes, for sure! We can also turn the model into `MCTCForAudioFrameClassification` (similar to `Wav2Vec2ForAudioFrameClassification`) to make it clearer that the outputs are frame-level.

@anton-l That sounds like the perfect solution! Will get going with it.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@anton-l do you want to help finalize the PR here or should I do it?

@patrickvonplaten @anton-l 

Firstly, I apologize for being irresponsible with going through with something I started. Life really got in the way here but this is so close to being done.

I've put up the weights as well from my last run at this: https://huggingface.co/cwkeam/m-ctc-t-large-lid



@cwkeam no worries, and thank you for the weights! 
Do you need a hand with any part that's left (tests, modelcard, etc.)? I'd be happy to help or take over the PR if you don't have time! :hugs: 

Actually, I realized I actually had finished the full working implementation. I think because of little nits I didn't fully notify you guys but I think it's worth a review:

```python
from transformers import MCTCTProcessor, MCTCTForAudioFrameClassification
from datasets import load_dataset
import torch

dataset = load_dataset("hf-internal-testing/librispeech_asr_demo", "clean", split="validation")
dataset = dataset.sort("id")
sampling_rate = dataset.features["audio"].sampling_rate

processor = MCTCTProcessor.from_pretrained("cwkeam/m-ctc-t-large-lid")
model = MCTCTForAudioFrameClassification.from_pretrained("cwkeam/m-ctc-t-large-lid")

# audio file is decoded on the fly
inputs = processor(dataset[0]["audio"]["array"], sampling_rate=sampling_rate, return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits
predicted_ids = torch.argmax(logits, dim=-1)
predicted_ids
>>> tensor([[12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12,
         12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12,
         12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12,
         12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12,
         12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12,
         12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12,
         12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12,
         12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12,
         12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12,
         12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12,
         12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12, 12]])
```


Where, as you can see from @lorenlugosch 's comment above, 12 is English, which we expect from librispeech demo.



@cwkeam perfect! Then I'll go over the PR this week and prepare it for merging :) 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@anton-l and I won't have any time soon to look into this. Should we let it stale for now?
Or @cwkeam, @lorenlugosch would you be interested in taking this PR over? 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.