Add RWKV7 model support
Author: chinoll
Pull request: no
Labels: name: Feature request

### Feature request

### Model description

I would like to propose adding RWKV7 support to Transformers.

RWKV7 is the latest generation of the RWKV recurrent language model family. It is a causal language model with recurrent state updates, but it is not compatible with the existing `rwkv` implementation in Transformers, which targets earlier RWKV variants. RWKV7 has different token-mixing/state-update formulas, different checkpoint structure, and different configuration conventions, so I think it should be added as a separate model family under `src/transformers/models/rwkv7` rather than extending the existing `rwkv` model with another version flag.

The proposed Transformers integration would include:

- `Rwkv7Config`
- `Rwkv7Model`
- `Rwkv7ForCausalLM`
- `Rwkv7Tokenizer`
- Auto class registration
- checkpoint conversion from upstream RWKV-LM `.pth` checkpoints to Transformers `safetensors`
- tokenizer conversion to standard Transformers assets, including `vocab.json`, `tokenizer.json`, `tokenizer_config.json`, `merges.txt`, and `chat_template.jinja`
- tests for model loading, tokenizer round-trip, generation, and numerical parity against the upstream RWKV-LM implementation

I found the open RWKV6 PR https://github.com/huggingface/transformers/pull/34918 while checking for overlapping work. This RWKV7 proposal is not meant to duplicate that PR: RWKV7 uses a different architecture/checkpoint format from the currently supported RWKV implementation and from the RWKV6 work. I am opening this issue first to coordinate before submitting a PR.

Local validation has been performed against the upstream RWKV-LM implementation:

- converted RWKV7 checkpoints can be loaded through the Transformers `AutoModelForCausalLM` path
- greedy generation was compared against RWKV-LM
- logits and top-k token ordering were compared against RWKV-LM
- tokenizer encode/decode behavior was checked after converting the upstream vocabulary to Transformers tokenizer files
- the initial implementation prioritizes a plain PyTorch path for numerical correctness; optimized kernels can be discussed separately if maintainers prefer

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

Reference implementation:

https://github.com/BlinkDL/RWKV-LM

Official RWKV7 model repository:

https://huggingface.co/BlinkDL/rwkv7-g1

Example upstream checkpoints:

- https://huggingface.co/BlinkDL/rwkv7-g1/resolve/main/rwkv7-g1d-0.1b-20260129-ctx8192.pth
- https://huggingface.co/BlinkDL/rwkv7-g1/resolve/main/rwkv7-g1g-1.5b-20260526-ctx8192.pth

Example converted Transformers-format checkpoint:

https://huggingface.co/chinoll/rwkv7-g1d-0.1b

Related existing Transformers RWKV work:

- Existing RWKV implementation: https://github.com/huggingface/transformers/tree/main/src/transformers/models/rwkv
- Open RWKV6 PR: https://github.com/huggingface/transformers/pull/34918

### Motivation

RWKV7 checkpoints are already available on the Hugging Face Hub, but they currently require custom code to load and run. Native Transformers support would make these models easier to use with the standard `AutoModelForCausalLM`, tokenizer, generation, conversion, and downstream fine-tuning workflows.

### Your contribution

I can contribute the initial PyTorch implementation, tokenizer support, checkpoint conversion script, converted test checkpoint, documentation, and tests. I have also run local checks against the upstream RWKV-LM implementation and can include those results in the PR.

Comments: