Instructions to use myxy/recursive-compressor-2-7b-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use myxy/recursive-compressor-2-7b-instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="myxy/recursive-compressor-2-7b-instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import RecursiveCompressorLM
model = RecursiveCompressorLM.from_pretrained("myxy/recursive-compressor-2-7b-instruct", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use myxy/recursive-compressor-2-7b-instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "myxy/recursive-compressor-2-7b-instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "myxy/recursive-compressor-2-7b-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/myxy/recursive-compressor-2-7b-instruct

SGLang

How to use myxy/recursive-compressor-2-7b-instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "myxy/recursive-compressor-2-7b-instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "myxy/recursive-compressor-2-7b-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "myxy/recursive-compressor-2-7b-instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "myxy/recursive-compressor-2-7b-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use myxy/recursive-compressor-2-7b-instruct with Docker Model Runner:
```
docker model run hf.co/myxy/recursive-compressor-2-7b-instruct
```

RecursiveCompressorLM (Instruct)

English | 日本語

日本語

モデル概要

RecursiveCompressor アーキテクチャによる日英バイリンガル対話モデルです。事前学習版（pretrain）の重みから対話データセットでファインチューニングしました。

アーキテクチャ: RecursiveCompressorLM (PreTrainedModel)
パラメータ数: TODO（例: 約2.7B）
ベースモデル: myxy/recursive-compressor-2-7b
コンテキスト長: 2048（学習時）
トークナイザ: elyza/ELYZA-japanese-Llama-2-7b-fast
dtype: bfloat16

対話フォーマット

本モデルは以下のテンプレートで学習されています:

<s>[QUERY]質問1[ANSWER]回答1<s>[QUERY]質問2[ANSWER]回答2<s>...

各ターンが [QUERY]質問[ANSWER]回答 の形で、ターン間は BOS (<s>) で区切られます。 [QUERY] [ANSWER] は特殊トークンではなくプレーンテキストのマーカーです。

推論例

prompt = "[QUERY]日本の首都はどこですか？[ANSWER]"
# → モデルが回答を生成

訓練データ

事前学習に加え、以下の対話データセットで追加学習しました:

データセット	言語	ライセンス・由来
`shi3z/ja_conv_wikipedia_llama2pro8b_30k`	日本語	Llama2-Pro 8B 生成（Llama 2 ToS継承）
`shi3z/ja_conv_wikipedia_orion14B_100K`	日本語	Orion-14B 生成
`HuggingFaceH4/ultrachat_200k`	英語	MIT（GPT系で生成された合成対話）

訓練設定

ベースモデルからの warm-start
オプティマイザ: Muon (隠れ層2D重み) + AdamW (embedding/head/bias/norm)
学習率: 5e-5
並列方式: パイプライン並列 (PyTorch PipelineStage + Schedule1F1B, 6 GPUs)
バッチ: マイクロバッチ6 × バッチサイズ6
混合精度: fp32マスター重み + bfloat16 autocast

用途

想定用途: 日本語/英語の質問応答、対話、文章生成
非想定用途: 医療・法律・金融などの高リスク判断、安全性が重要な用途、事実確認を要する用途

使い方

本モデルは独自アーキテクチャ RecursiveCompressorLM を使用するため、以下のリポジトリをクローンしてその中のクラス定義を読み込む必要があります:

リポジトリ: https://github.com/myxyy/RecursiveCompressorHF

git clone https://github.com/myxyy/RecursiveCompressorHF.git -b v1.0
cd RecursiveCompressorHF
uv sync

HuggingFaceの generate() メソッドに対応しています:

import torch
from transformers import AutoTokenizer, TextStreamer
from recursive_compressor_lm import RecursiveCompressorLM

model = RecursiveCompressorLM.from_pretrained(
    "myxy/recursive-compressor-2-7b-instruct",
    torch_dtype=torch.bfloat16,
).to("cuda").eval()
tokenizer = AutoTokenizer.from_pretrained("elyza/ELYZA-japanese-Llama-2-7b-fast")

prompt = "[QUERY]猫の足は何本ですか？[ANSWER]"
input_ids = tokenizer.encode(prompt, return_tensors="pt").to("cuda")

output_ids = model.generate(
    input_ids,
    max_new_tokens=256,
    do_sample=True,
    temperature=0.8,
    top_p=0.9,
    streamer=TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True),
)
#print(tokenizer.decode(output_ids[0], skip_special_tokens=True))

リポジトリ内の predict.py / predict_stream.py も同等の機能を提供します（インタラクティブREPL等）。

注: beam searchは未対応です。do_sample=Falseで貪欲生成、do_sample=Trueで確率サンプリングが使えます。

制限・バイアス

ベースモデルのバイアスを継承
対話データの大半が他のLLMによる合成データのため、それらモデルのスタイル・誤りを反映しうる
事実関係の誤り（hallucination）が頻発する可能性
安全性フィルタは未実装。有害な出力を生成する可能性がある
マルチターン対話で長い文脈を保持する能力は限定的

ライセンス

Llama 2 Community License に従います。

ライセンス全文: https://llama.meta.com/llama2/license/
「Built with Meta Llama 2」と明示する必要があります
MAU 7億超の事業者は別途ライセンス取得が必要

トークナイザがLlama 2派生であること、および対話学習データの一部 (shi3z) がLlama2-Proで生成されていることから、本モデルはこのライセンスに従います。

引用

TODO

English

Model Description

A bilingual (Japanese / English) conversational model built on the RecursiveCompressor architecture. Fine-tuned from the pretrain variant on dialogue datasets.

Architecture: RecursiveCompressorLM (extends PreTrainedModel)
Parameters: TODO (e.g., ~2.7B)
Base model: MODEL_REPO_ID/pretrain
Training context length: 2048
Tokenizer: elyza/ELYZA-japanese-Llama-2-7b-fast
dtype: bfloat16

Conversation Format

The model is trained with the following template:

<s>[QUERY]question1[ANSWER]answer1<s>[QUERY]question2[ANSWER]answer2<s>...

Each turn is [QUERY]question[ANSWER]answer, separated by BOS (<s>). [QUERY] and [ANSWER] are plain text markers, not special tokens.

Inference Example

prompt = "[QUERY]What is the capital of Japan?[ANSWER]"
# → model generates the answer

Training Data

In addition to the pretrain corpus, fine-tuned on these dialogue datasets:

Dataset	Language	License / Origin
`shi3z/ja_conv_wikipedia_llama2pro8b_30k`	Japanese	Generated by Llama2-Pro 8B (inherits Llama 2 ToS)
`shi3z/ja_conv_wikipedia_orion14B_100K`	Japanese	Generated by Orion-14B
`HuggingFaceH4/ultrachat_200k`	English	MIT (synthetic dialogues generated by GPT models)

Training Setup

Warm-started from the pretrain model
Optimizers: Muon (2D hidden weights) + AdamW (embedding/head/bias/norm)
Learning rate: 5e-5
Parallelism: pipeline parallel (PyTorch PipelineStage + Schedule1F1B, 6 GPUs)
Batch: 6 microbatches × batch size 6
Mixed precision: fp32 master weights + bfloat16 autocast

Usage

This model uses the custom RecursiveCompressorLM architecture, so you need to clone the repository to import the class definitions:

Repository: https://github.com/myxyy/RecursiveCompressorHF

git clone https://github.com/myxyy/RecursiveCompressorHF.git -b v1.0
cd RecursiveCompressorHF
uv sync

The model supports HuggingFace's generate() method:

import torch
from transformers import AutoTokenizer, TextStreamer
from recursive_compressor_lm import RecursiveCompressorLM

model = RecursiveCompressorLM.from_pretrained(
    "myxy/recursive-compressor-2-7b-instruct",
    torch_dtype=torch.bfloat16,
).to("cuda").eval()
tokenizer = AutoTokenizer.from_pretrained("elyza/ELYZA-japanese-Llama-2-7b-fast")

prompt = "[QUERY]How many legs does a cat have?[ANSWER]"
input_ids = tokenizer.encode(prompt, return_tensors="pt").to("cuda")

output_ids = model.generate(
    input_ids,
    max_new_tokens=256,
    do_sample=True,
    temperature=0.8,
    top_p=0.9,
    streamer=TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True),
)
#print(tokenizer.decode(output_ids[0], skip_special_tokens=True))

See predict.py / predict_stream.py in the repository for an interactive REPL.

Note: beam search is not supported (use do_sample=False for greedy or do_sample=True for sampling).

Intended Use

Intended: Japanese / English question answering, dialogue, text generation
Not intended: High-stakes decisions in medical / legal / financial domains; safety-critical use; fact verification

Limitations & Bias

Inherits biases from the base model
Most of the dialogue training data is synthetic data generated by other LLMs, so this model may reflect the style and errors of those models
May frequently produce factually incorrect statements (hallucination)
No safety filtering; may produce harmful outputs
Limited ability to maintain long conversational context across many turns

License

Llama 2 Community License.

Full text: https://llama.meta.com/llama2/license/
"Built with Meta Llama 2" attribution required
Organizations with > 700M MAU must seek a separate license

The tokenizer is derived from Llama 2, and part of the instruction tuning data (shi3z) was generated by Llama2-Pro, so this model inherits the license.

Citation

TODO

Acknowledgments

Meta AI for Llama 2
ELYZA for the Japanese-extended tokenizer
shi3z, HuggingFaceH4 for dialogue datasets

Downloads last month: 3

Safetensors

Model size

3B params

Tensor type

BF16