Handle python 3.11 for huggingface["video"] and dependencies
Author: CalOmnie
Pull request: no
Labels: name: Feature request

### Feature request

Currently, the only thing preventing the installation of:
- huggingface[dev]
- huggingface[all]
- huggingface[video]

On python 3.11 is the `av==9.2.0` requirement. That requirement cites issue with `av==10.0.0` as reason for pinning this version, however the latest version is now `14.0.1`, it would be nice to revisit that pin, especially now that it forces users on python 3.10.

### Motivation

It would allow contributors and users of huggingface[video] to use python 3.11.

### Your contribution

I've looked into the issue with av==9.2.0, the original failure is Cython generating bad C:
```
      av/logging.pyx:351:28: Cannot assign type 'void (void *, int, const char *, va_list) except * nogil' to 'av_log_callback' (alias of 'void (*)(void *, int, const char *, va_list) noexcept nogil'). Exception values are incompatible. Suggest adding 'noexcept' to the type of 'log_callback'.
```

This can be circumvented by forcing an earlier version of Cython:
```
$ echo "cython<3.0" >> c.txt
$ PIP_CONSTRAINT=c.txt pip install av==9.2.0
```
But that leads to this error:
`src/av/stream.c:2999:29: error: ‘struct AVStream’ has no member named ‘codec’`
Which, looking into it, means that ffmpeg is too recent for this av code, that's where I give up because I realised that asking users to downgrade ffmpeg was probably not the way to go.

I'm attempting to run the tests on python 3.10 w/ av==9.2.0 vs python 3.11 w/ av==14.0.1, but `make test` takes absolute ages even on 32 cores and 64Gb RAM and it just seems to fail randomly so clearly I am under equiped for this undertaking. Any tips to have a better times with running tests? Is there a way to only run those using `av` besides `grep "import av"`?

Comments:
Hi @CalOmnie, you can call individual tests with `pytest tests/.../test_file.py -k 'test_name'`. Once you identify a failing/passing test, you can just check that specific test, rather than running everything with `make test`, which will definitely take hours!

Hey @Rocketknight1 , I was aware of that option, but my issue was "how do I test every files that import `av`?" I can get a list of modules importing it with `grep -RIn "import av"`, but then I have to repeat that for every result to capture the modules importing the modules importing `av` etc. Is there a better approach?

Hmmn, I don't think we have a clean way to do that. However, for framework incompatibility issues like this, you probably don't need to run **every** test that uses `av`, right? I assumed that the Cython incompatibility would occur in most cases when `av` is used, so you don't need complete coverage to tell if it's been resolved or not

Alright, so if I understand correctly I can just select an arbitrary subset of functions that use `av` and only run the tests for those?

Yes, I think so! Once you have a proposed solution, we can a full set of slow tests on it later

So, I've looked into it and it seems that... pyav is just not tested anywhere. Also, the pin was made due to "issues with the audio stream" but... pyav is never used to extract audio.
It's used there:
- `transformer.image_utils` # No test, only extract video
- `transformers.pipelines.video_classification` # Test doesn't use pyav, only extract video
- `transformers.models.git.convert_git_to_pytorch` # Only extract video

I've run the following script I found in the documentation to check if python 3.10 + av==9.2.0 created different results than python 3.11 + av==14.0.1 and the results were the same:
```
import av
import numpy as np

from transformers import AutoImageProcessor, VideoMAEModel
from huggingface_hub import hf_hub_download

from transformers.image_utils import read_video_pyav


def sample_frame_indices(clip_len, frame_sample_rate, seg_len):
    """
    Sample a given number of frame indices from the video.

    Args:
        clip_len (`int`): Total number of frames to sample.
        frame_sample_rate (`int`): Sample every n-th frame.
        seg_len (`int`): Maximum allowed index of sample's last frame.

    Returns:
        indices (`List[int]`): List of sampled frame indices
    """
    converted_len = int(clip_len * frame_sample_rate)
    end_idx = np.random.randint(converted_len, seg_len)
    start_idx = end_idx - converted_len
    indices = np.linspace(start_idx, end_idx, num=clip_len)
    indices = np.clip(indices, start_idx, end_idx - 1).astype(np.int64)
    return indices

file_path = hf_hub_download(
        repo_id="nielsr/video-demo", filename="eating_spaghetti.mp4", repo_type="dataset"
)
video = read_video_pyav(file_path, 16)
image_processor = AutoImageProcessor.from_pretrained("MCG-NJU/videomae-base")
model = VideoMAEModel.from_pretrained("MCG-NJU/videomae-base")
inputs = image_processor(list(video), return_tensors="pt")

outputs = model(**inputs)
print(outputs.last_hidden_state)
import torch
torch.save(outputs.last_hidden_state, "311.pt")
```

I don't really know how to proceed from there. Please advise.