Add RWKV7 model support
Author: chinoll
Pull request: no
Labels: name: New model

### Model description

I would like to propose adding RWKV7 support to Transformers.

RWKV7 is the latest generation of the RWKV recurrent language model family. It is a causal language model with recurrent state updates, but it is not compatible with the existing `rwkv` implementation in Transformers, which targets earlier RWKV variants. RWKV7 has different token-mixing/state-update formulas, different checkpoint structure, and different configuration conventions, so I think it should be added as a separate model family under `src/transformers/models/rwkv7` rather than extending the existing `rwkv` model with another version flag.

The proposed Transformers integration would include:

- `Rwkv7Config`
- `Rwkv7Model`
- `Rwkv7ForCausalLM`
- `Rwkv7Tokenizer`
- Auto class registration
- checkpoint conversion from upstream RWKV-LM `.pth` checkpoints to Transformers `safetensors`
- tokenizer conversion to standard Transformers assets, including `vocab.json`, `tokenizer.json`, `tokenizer_config.json`, `merges.txt`, and `chat_template.jinja`
- tests for model loading, tokenizer round-trip, generation, and numerical parity against the upstream RWKV-LM implementation

I found the open RWKV6 PR https://github.com/huggingface/transformers/pull/34918 while checking for overlapping work. This RWKV7 proposal is not meant to duplicate that PR: RWKV7 uses a different architecture/checkpoint format from the currently supported RWKV implementation and from the RWKV6 work. I am opening this issue first to coordinate before submitting a PR.

Local validation has been performed against the upstream RWKV-LM implementation:

- converted RWKV7 checkpoints can be loaded through the Transformers `AutoModelForCausalLM` path
- greedy generation was compared against RWKV-LM
- logits and top-k token ordering were compared against RWKV-LM
- tokenizer encode/decode behavior was checked after converting the upstream vocabulary to Transformers tokenizer files
- the initial implementation prioritizes a plain PyTorch path for numerical correctness; optimized kernels can be discussed separately if maintainers prefer

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

Reference implementation:

https://github.com/BlinkDL/RWKV-LM

Official RWKV7 model repository:

https://huggingface.co/BlinkDL/rwkv7-g1

Example upstream checkpoints:

- https://huggingface.co/BlinkDL/rwkv7-g1/resolve/main/rwkv7-g1d-0.1b-20260129-ctx8192.pth
- https://huggingface.co/BlinkDL/rwkv7-g1/resolve/main/rwkv7-g1g-1.5b-20260526-ctx8192.pth

Example converted Transformers-format checkpoint:

https://huggingface.co/chinoll/rwkv7-g1d-0.1b

Related existing Transformers RWKV work:

- Existing RWKV implementation: https://github.com/huggingface/transformers/tree/main/src/transformers/models/rwkv
- Open RWKV6 PR: https://github.com/huggingface/transformers/pull/34918

### Open source status

- [x] The model implementation is available
- [x] The model weights are available

### Provide useful links for the implementation

_No response_

Comments:
 Hi @ArthurZucker  @Cyrilvallez , I have implemented RWKV7 support locally as a separate `rwkv7` model family, including model/config/tokenizer, checkpoint conversion, docs, and tests.

  I have also validated it against the upstream RWKV-LM implementation with greedy generation, logits comparison, top-k token ordering, and tokenizer round-trip checks.

  Since RWKV7 differs from the existing RWKV/RWKV6 implementations, would it be OK for me to open a PR for review?

Hey @chinoll, looking at the released hub checkpoints for this model, it does not seem to really follow Transformers format/conventions (it does not even offer `safetensors` checkpoints), so I'm really not sure it's a good candidate for a new model integration.
Sorry about this