Instructions to use myxy/recursive-compressor-2-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use myxy/recursive-compressor-2-7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="myxy/recursive-compressor-2-7b")

# Load model directly
from transformers import RecursiveCompressorLM
model = RecursiveCompressorLM.from_pretrained("myxy/recursive-compressor-2-7b", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use myxy/recursive-compressor-2-7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "myxy/recursive-compressor-2-7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "myxy/recursive-compressor-2-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/myxy/recursive-compressor-2-7b

SGLang

How to use myxy/recursive-compressor-2-7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "myxy/recursive-compressor-2-7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "myxy/recursive-compressor-2-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "myxy/recursive-compressor-2-7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "myxy/recursive-compressor-2-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use myxy/recursive-compressor-2-7b with Docker Model Runner:
```
docker model run hf.co/myxy/recursive-compressor-2-7b
```

RecursiveCompressorLM (Pretrain)

English | 日本語

日本語

モデル概要

RecursiveCompressor アーキテクチャによる日英バイリンガル言語モデルの事前学習版です。独自の再帰的圧縮機構によりトークン間の長距離依存をサブリニアな計算量で扱います。

アーキテクチャ: RecursiveCompressorLM (PreTrainedModel)
パラメータ数: TODO（例: 約2.7B）
コンテキスト長: 2048（学習時）／推論時はより長いコンテキストも理論上対応
トークナイザ: elyza/ELYZA-japanese-Llama-2-7b-fast
dtype: bfloat16

モデル構造

パラメータ	値
d_model	2048
num_heads	16
d_ff	6144
chunk_size	4
compress_size	1
num_layers	16

訓練データ

事前学習に以下の文書データセットを使用しました（対話データは含みません）:

データセット	言語	ライセンス
`wikimedia/wikipedia` (20231101.ja)	日本語	CC BY-SA 4.0 / GFDL
`wikimedia/wikipedia` (20231101.en)	英語	CC BY-SA 4.0 / GFDL
`hotchpotch/cc100-ja-documents`	日本語	Common Crawl Terms of Use
`JeanKaddour/minipile`	英語	MIT (The Pileの派生)

訓練設定

オプティマイザ: Muon (隠れ層2D重み) + AdamW (embedding/head/bias/norm)
学習率: 5e-5
並列方式: パイプライン並列 (PyTorch PipelineStage + Schedule1F1B, 6 GPUs)
バッチ: マイクロバッチ6 × バッチサイズ6
混合精度: fp32マスター重み + bfloat16 autocast
ハードウェア: NVIDIA RTX 3090 × 6

用途

想定用途: 日本語/英語の文章生成、続きの生成、追加ファインチューニングのベース
非想定用途: 質問応答・対話（→ instruct版を参照）、医療/法律/金融などの高リスク判断

使い方

本モデルは独自アーキテクチャ RecursiveCompressorLM を使用するため、以下のリポジトリをクローンしてその中のクラス定義を読み込む必要があります:

リポジトリ: https://github.com/myxyy/RecursiveCompressorHF

git clone https://github.com/myxyy/RecursiveCompressorHF.git -b v1.0
cd RecursiveCompressorHF
uv sync

HuggingFaceの generate() メソッドに対応しています:

import torch
from transformers import AutoTokenizer, TextStreamer
from recursive_compressor_lm import RecursiveCompressorLM

model = RecursiveCompressorLM.from_pretrained(
    "myxy/recursive-compressor-2-7b",
    torch_dtype=torch.bfloat16,
).to("cuda").eval()
tokenizer = AutoTokenizer.from_pretrained("elyza/ELYZA-japanese-Llama-2-7b-fast")

prompt = "吾輩は猫である。"
input_ids = tokenizer.encode(prompt, return_tensors="pt").to("cuda")

output_ids = model.generate(
    input_ids,
    max_new_tokens=256,
    do_sample=True,
    temperature=0.8,
    top_p=0.9,
    streamer=TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True),
)
#print(tokenizer.decode(output_ids[0], skip_special_tokens=True))

リポジトリ内の predict.py / predict_stream.py も同等の機能を提供します（インタラクティブREPL等）。

注: beam searchは未対応です（do_sample=Falseの貪欲生成、またはdo_sample=Trueの確率サンプリングを使用してください）。

制限・バイアス

訓練データのバイアス（Wikipediaやウェブ文書の偏り）を反映する可能性
事実関係の誤り（hallucination）が起こりうる
数値計算・論理推論は不得手
安全性フィルタは未実装

ライセンス

Llama 2 Community License に従います。

ライセンス全文: https://llama.meta.com/llama2/license/
「Built with Meta Llama 2」と明示する必要があります
MAU 7億超の事業者は別途ライセンス取得が必要

トークナイザが Llama 2 派生であるため、本モデルもこのライセンスに従います。

引用

TODO

English

Model Description

A bilingual (Japanese / English) pretrained causal language model using the RecursiveCompressor architecture, which handles long-range token dependencies in sublinear computational complexity via recursive compression.

Architecture: RecursiveCompressorLM (extends PreTrainedModel)
Parameters: TODO (e.g., ~2.7B)
Training context length: 2048 (longer contexts theoretically supported at inference)
Tokenizer: elyza/ELYZA-japanese-Llama-2-7b-fast
dtype: bfloat16

Architecture

Parameter	Value
d_model	2048
num_heads	16
d_ff	6144
chunk_size	4
compress_size	1
num_layers	16

Training Data

Pretrained on document datasets only (no dialogue data):

Dataset	Language	License
`wikimedia/wikipedia` (20231101.ja)	Japanese	CC BY-SA 4.0 / GFDL
`wikimedia/wikipedia` (20231101.en)	English	CC BY-SA 4.0 / GFDL
`hotchpotch/cc100-ja-documents`	Japanese	Common Crawl Terms of Use
`JeanKaddour/minipile`	English	MIT (subset of The Pile)

Training Setup

Optimizers: Muon (2D hidden weights) + AdamW (embedding/head/bias/norm)
Learning rate: 5e-5
Parallelism: pipeline parallel (PyTorch PipelineStage + Schedule1F1B, 6 GPUs)
Batch: 6 microbatches × batch size 6
Mixed precision: fp32 master weights + bfloat16 autocast
Hardware: NVIDIA RTX 3090 × 6

Usage

This model uses the custom RecursiveCompressorLM architecture, so you need to clone the repository to import the class definitions:

Repository: https://github.com/myxyy/RecursiveCompressorHF

git clone https://github.com/myxyy/RecursiveCompressorHF.git -b v1.0
cd RecursiveCompressorHF
uv sync

The model supports HuggingFace's generate() method:

import torch
from transformers import AutoTokenizer, TextStreamer
from recursive_compressor_lm import RecursiveCompressorLM

model = RecursiveCompressorLM.from_pretrained(
    "myxy/recursive-compressor-2-7b",
    torch_dtype=torch.bfloat16,
).to("cuda").eval()
tokenizer = AutoTokenizer.from_pretrained("elyza/ELYZA-japanese-Llama-2-7b-fast")

prompt = "Once upon a time"
input_ids = tokenizer.encode(prompt, return_tensors="pt").to("cuda")

output_ids = model.generate(
    input_ids,
    max_new_tokens=256,
    do_sample=True,
    temperature=0.8,
    top_p=0.9,
    streamer=TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True),
)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))

See predict.py / predict_stream.py in the repository for an interactive REPL.

Note: beam search is not supported (use do_sample=False for greedy or do_sample=True for sampling).

Intended Use

Intended: Text completion in Japanese / English; base for downstream fine-tuning
Not intended: Question answering / dialogue (see the instruct variant); high-stakes decisions in medical, legal, or financial domains

Limitations & Bias

May reflect biases present in Wikipedia and web text
May produce factually incorrect statements (hallucination)
Limited capability in arithmetic and logical reasoning
No safety filtering

License

Llama 2 Community License.

Full text: https://llama.meta.com/llama2/license/
"Built with Meta Llama 2" attribution required
Organizations with > 700M MAU must seek a separate license

The tokenizer is derived from Llama 2, hence this model inherits the license.

Citation

TODO

Acknowledgments

Meta AI for Llama 2
ELYZA for the Japanese-extended tokenizer
Wikimedia Foundation, hotchpotch, JeanKaddour for training data

Downloads last month: 6

Safetensors

Model size

3B params

Tensor type

BF16