Inconsistent Documentation for `⁠dataset_index` Requirement Across ViTPose Models
Author: harpreetsahota204
Pull request: no
Labels: name: bug

### System Info

## Description

There's confusion regarding the `dataset_index` parameter requirement across the ViTPose model family. The documentation only mentions this requirement for **some** of the models, initially when the model was released it was only for the `usyd-community/vitpose-plus-base` checkpoint.

But now other checkpoints also fail without it. I'm concerned about:

1. Whether ALL models in the family require this parameter, even if some don't explicitly fail without it
2. Whether adding this parameter to models that don't explicitly require it could affect inference results

## Current Behavior
When following the examples in the model cards, these checkpoints fail with:
```
ValueError: dataset_index must be provided when using multiple experts (num_experts=6). Please provide dataset_index to the forward pass.
```

Affected checkpoints:
- `usyd-community/vitpose-plus-small`
- `usyd-community/vitpose-plus-large`
- `usyd-community/vitpose-plus-huge`

## Questions
1. Should the `dataset_index` parameter be added to ALL ViTPose+ models for consistency?
2. Will adding `inputs["dataset_index"] = torch.tensor([0], device=device)` to models that don't explicitly require it affect inference results?
3. Is there a recommended approach for handling this parameter in applications that need to work with multiple ViTPose+ variants?

## Context
I've implemented a plugin using these models, and it's failing on certain checkpoints. I want to ensure any fix I implement won't compromise results for other checkpoints that currently work without this parameter.

## Suggested Documentation Update
If all models require this parameter, please update all model cards to include:
```python
inputs["dataset_index"] = torch.tensor([0], device=device)
```

If only certain models require it, please clearly indicate which ones do and which ones don't.

### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

## Steps to Reproduce

1. Load any of the affected models:
- `usyd-community/vitpose-plus-small`
- `usyd-community/vitpose-plus-large`
- `usyd-community/vitpose-plus-huge`

2. Process an image following the model card example and run inference

3. Observe the error:
   ```
   ValueError: dataset_index must be provided when using multiple experts (num_experts=6). Please provide dataset_index to the forward pass.
   ```



### Expected behavior

1. All model cards should document whether the `dataset_index` parameter is required for that specific checkpoint.

2. Documentation should be consistent across all models in the ViTPose+ family that share the same requirements.

3. Ideally, the model cards for all affected checkpoints should include the necessary code snippet:
   ```python
   inputs["dataset_index"] = torch.tensor([0], device=device)
   ```

4. Clear guidance on whether this parameter affects inference results and if it should be applied universally across all ViTPose models for consistency.



Comments:
cc @nielsrogge for ViTPose

Pinging @qubvel here. 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hey @harpreetsahota204, sorry, missed the issue! All vitpose '-plus' models require passing dataset index 👍 
cc @stevhliu re docs refactoring

Thanks for the heads up @qubvel, I'll make a note of it in #37089 👍 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

@stevhliu was this resolved?

Not yet, #37089 was closed so I'll open a new PR and include clearer guidance for it :)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Still to be done @stevhliu?

Still to be done??