---
license: apache-2.0
datasets:
- stingning/ultrachat
- kaist-ai/CoT-Collection
- mesolitica/google-translate-commitpackft
- Wanfq/Explore_Instruct_Rewriting_32k
- Wanfq/Explore_Instruct_Rewriting_10k
- Wanfq/Explore_Instruct_Brainstorming_16k
- xiyuez/red-dot-design-award-product-description
---

# RWKV v4 7B world model 
finetuned with ultrachat , COT and some novel instructions data, commitpackft and so on

use full ultrachat and cot data, about 3B tokens

if you wanna do Role play, use [this model](https://huggingface.co/xiaol/RWKV-4-world-one-state-ultrachat-COT-65k/blob/main/RWKV-world-novel-one-state-ultrachat-cot-tuned-Role-play-65k.pth)


# Contributor
[@JL-er](https://huggingface.co/JL-er) 
[@Remixa](https://huggingface.co/Remixa)


# Design of experiment
this model lose multi-turn chat ability,cause from using whole ultrachat datasets.

so i continue tuned multi-turn datasets with 2 aspects

1.[role play](https://huggingface.co/xiaol/RWKV-4-world-one-state-ultrachat-COT-65k/blob/main/RWKV-world-novel-one-state-ultrachat-cot-tuned-Role-play-65k.pth)

2.[novel multiturn instruction](https://huggingface.co/xiaol/RWKV-4-world-one-state-ultrachat-COT-65k/blob/main/rwkv-world-one-novel-cot-ultrachat-novel-instructions.pth)

# Training details
[wandb.ai](https://wandb.ai/one-/one-rwkv-64k)

# CAses

![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/_1dJo549ldgX6q0JUwC6c.jpeg)

![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/7969wbHaJpBq2n6xvfC7C.jpeg)

# Usage
adjust tempp and topp on different scenario.
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/cGDF6b4-x_9rcwMdl1KPp.png)

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/_QduMpcGkCoC00DTlXQop.png)

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/9CqrfEpcJcxtLoX5ffEFo.png)

# COT and lookback

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/hUxTVgjLBMcFqxQX9HoxL.png)
[this model](https://huggingface.co/xiaol/RWKV-4-world-one-state-novel-tuned-65k) can do above task with 100% acc.

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/mXyfoD1_jlNqBElQfn_Fk.png)

## role play model

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/2ns794U56M1592w6Uy1dk.png)

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/6CQp2kh56FFa-TmNUrld3.png)

## novel

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/wPXTYWFWfi-mPmoPbmfWg.png)

![image/png](https://cdn-uploads.huggingface.co/production/uploads/6176b32847ee6431f632981e/ZgRK8N6jnxR3HawvFiUve.png)
# demo site(temporary)
[online showcase](https://rwkv.ai-creator.net/risu)