Alpha-R1: Alpha Screening with LLM Reasoning via Reinforcement Learning

Alpha-R1

English | 中文

Alpha-R1 是一个面向量化 Alpha 筛选的推理增强型 LLM:基于 Qwen3-8B,通过 GRPO 强化学习(verl)以市场反馈奖励训练。它阅读 Alpha101 因子的语义化描述——每个因子如何起作用、何时有效、何时失效——并针对当前市场环境筛选出最值得激活的因子组合。

模型概览 (Model Overview)

Alpha-R1 framework overview

项目 内容
Base model Qwen/Qwen3-8B
训练方法 GRPO(verl),市场反馈奖励
输入 决策上下文 prompt:拼接的因子语义描述 α_des
输出 <alpha_list> 中列出的选中因子
候选因子池 82 个 Alpha101 因子(论文筛选后)
推荐解码 temperature=0(greedy),top_p=0.7,max_new_tokens=4096

快速开始 (Quick Start)

transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "FinStep/Alpha-R1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", device_map="auto")

prompt = "<decision context: concatenated factor descriptions>"  # see the GitHub repo for the prompt builder
inputs = tokenizer.apply_chat_template(
    [{"role": "user", "content": prompt}],
    add_generation_prompt=True, return_tensors="pt",
).to(model.device)

# paper setting: temperature=0 (greedy), top_p=0.7
out = model.generate(inputs, max_new_tokens=4096, do_sample=False)
print(tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

vLLM

from vllm import LLM, SamplingParams

llm = LLM(model="FinStep/Alpha-R1")
params = SamplingParams(temperature=0.0, top_p=0.7, max_tokens=4096)
outputs = llm.chat([[{"role": "user", "content": prompt}]], params)

完整的端到端管线(因子描述生成 → Alpha-R1 推理 → 输出解析 → qlib 策略回测)见 GitHub 仓库

输出契约 (Output Contract)

模型在 <alpha_list>...</alpha_list> 中输出选中的因子 id,例如:

<alpha_list>alpha001, alpha021, alpha053</alpha_list>

GitHub 仓库的 src/alpha_r1/parsing/ 提供了配套的校验与解析脚本。

表现 (Performance)

12 个月样本外测试(2025-01-01 ~ 2025-12-31,论文 Table 1):

Backtest NAV comparison on S&P 500 (left) and CSI 300 (right)

类型 方法 S&P 500 CSI 300
AR (%) SR MDD (%) AR (%) SR MDD (%)
Non-LLMBuy & Hold19.340.8018.7522.161.3110.49
PCA7.980.2717.302.930.1714.46
XGBoost3.490.0318.458.990.5016.26
LightGBM-5.42-0.4320.9318.441.0514.92
A2C10.820.4017.7022.961.2014.86
PPO7.680.2514.9714.960.8112.95
DDPG2.53-0.0215.041.970.1216.54
TD35.540.1416.588.660.5210.26
SAC37.601.4415.189.770.5611.68
LLMGemini 2.5 Pro14.230.5517.0116.290.9014.01
Claude 3.7 Sonnet10.920.4018.8810.130.5714.49
DeepSeek‑R121.940.9314.3614.660.8114.60
Qwen3‑8B12.850.4719.5215.440.7914.38
Alpha‑R1 (Ours)47.871.6216.9140.572.236.58

域外泛化(无需重训,论文 Table 2):Russell 2000 上 80.54% AR(SR 2.46),CSI 1000 上 73.52% AR(SR 2.80)。AR = 年化收益,SR = 超额夏普比率,MDD = 最大回撤。

训练 (Training)

基于 Qwen3-8B,使用 verl 进行 GRPO 训练,奖励为市场反馈奖励(R_final = R_adjusted - P_structural,论文 §3.4)。训练配置与参考奖励实现见 GitHub 仓库的 training/ 目录。

局限性 (Limitations)

  • 本模型面向学术研究场景,输出不构成任何投资建议。
  • 因子筛选依赖上游的描述生成与回测管线(见 GitHub 仓库),模型本身不直接产出可交易信号。

引用 (Citation)

@article{jiang2025alphar1,
  title={Alpha-R1: Alpha Screening with LLM Reasoning via Reinforcement Learning},
  author={Jiang, Zuoyou and Zhao, Li and Sun, Rui and Sun, Ruohan and Li, Zhongjian and Li, Jing and Jiang, Daxin and Bai, Zuo and Hua, Cheng},
  journal={arXiv preprint arXiv:2512.23515},
  year={2025}
}

License

本项目基于 MIT License 发布。

Downloads last month
-
Safetensors
Model size
8B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for FinStep/Alpha-R1

Finetuned
Qwen/Qwen3-8B
Finetuned
(2072)
this model

Paper for FinStep/Alpha-R1