feat: allow output_hidden_states and output_attensions to record outputs of specific layers
Author: cloudhan
Pull request: yes

# What does this PR do?

This PR enable model forward to record optional outputs at specified layers. This will be particularly useful for large model with long context when explorering the aesthetics of the attention maps to design sparse attention. Without it, with moderate size model (say 7B), it can easily OOM with only 1k level context.

```py
outputs = model.forward(input_ids, output_hidden_states=10, output_attentions=[10])
```
now it only keeps `outputs.attentions[10]`, outputs of other layers are set to `None` to save memory.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker @Cyrilvallez
- vision models: @yonigozlan @molbap
- audio models: @eustlb @ebezzam @vasqu
- multimodal models: @zucchini-nlp
- graph models: @clefourrier

Library:

- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- continuous batching: @remi-or @ArthurZucker @McPatate
- pipelines: @Rocketknight1
- tokenizers: @ArthurZucker and @itazap
- trainer: @SunMarc
- attention: @vasqu @ArthurZucker @CyrilVallez
- model loading (from pretrained, etc): @CyrilVallez
- distributed: @3outeille @ArthurZucker
- CIs: @ydshieh

Integrations:

- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization: @SunMarc @MekkCyber
- kernels: @MekkCyber @drbh
- peft: @BenjaminBossan @githubnemo

Devices/Backends:

- AMD ROCm: @ivarflakstad
- Intel XPU: @IlyasMoutawwakil
- Ascend NPU: @ivarflakstad 

Documentation: @stevhliu

Research projects are not maintained and should be taken as is.

 -->


Comments:
Hmmn - this adds some code complexity that may complicate maintenance for a relatively niche feature. Probably the easiest way to extract these without huge memory is just to make a function that extracts the layers you want and then compile it? Torch should be able to correctly drop the unwanted tensors and avoid OOM in that scenario.

Might be useful for certain use-cases as in https://github.com/huggingface/transformers/issues/33698 ig. Depends on how much memory is being wasted on VLMs/diffusers, it's a common pattern for them to output hidden states  

I guess, but I think we should try to avoid adding features that need updates across all modeling code unless they're really important. Plus I think the `compile()` solution should work fine!

Yeah, we defi don't want to update all models. Anyway, just wanted to link related issues to have a full picture :)

@Rocketknight1 this is actually a common pattern for this feature.
https://github.com/huggingface/transformers/blob/b40b7a767d569b067a115ce115fd270ac323cd36/src/transformers/models/qwen3/modeling_qwen3.py#L358-L361
and then it will be handled by the `generic.py`

Unless you want to support all models, you don't need to update all modeling. Only some legacy models are using per modeling pattern
https://github.com/huggingface/transformers/blob/b40b7a767d569b067a115ce115fd270ac323cd36/src/transformers/models/gpt_neo/modeling_gpt_neo.py#L449-L452

Good point, I forgot we'd factored that out into `_can_record_outputs` for a lot of models. If you want to attempt a PR to update the generic code there to allow specific layer selection, I guess we could consider it! No guarantees, though, I'd need to talk with the rest of the team about whether the extra complexity is worth it

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=43213&sha=25a8fa