SWARM-Lab/CoCo-Qwen3-4B-LoRA

CoCo: Qwen3-4B LoRA adapter

The LLM component of TSFM Meets LLM: Context as Covariates (CoCo), shared by SWARM-Lab. This repository contains the adapter from the paper's submitted supplementary package, not the base model weights.

CoCo uses a text-conditioned LLM to generate numerical forecast rollouts. Their median forecast and per-step variance provide forecast and confidence covariates to an adapted Chronos2 model.

Companion adapter: SWARM-Lab/CoCo-Chronos2-LoRA.

Checkpoint

• Base model: Qwen/Qwen3-4B-Instruct-2507.
• PEFT LoRA rank: 32; alpha: 64; dropout: 0.
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj.
• Adapter file: 264,300,328 bytes.

Loading the component

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3-4B-Instruct-2507"
tokenizer = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(
    base_id, torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(base, "SWARM-Lab/CoCo-Qwen3-4B-LoRA")
model.eval()

This loads the LLM component only. Full CoCo forecasting additionally needs the paper's prompt, parsing, history-only normalization, rollout aggregation, covariate construction, and adapted Chronos2. A unified public CoCo inference package is being prepared. This loading example has not yet been GPU-tested as part of the release.

The submitted inference environment uses vLLM 0.11.2 and PEFT 0.18.1. The LLM can also be served with vLLM's LoRA support. Paper inference uses five rollouts and a text window of 16 timestamps.

Training and intended use

The paper describes GRPO adaptation using a frozen Chronos2 reward model and a mixture of synthetic text/time-series examples and FNSPID examples. Intended use is research on text-aided time-series forecasting. Results depend on the informativeness and reliability of the supplied text.

Base-model license

The base Qwen checkpoint is distributed under Apache-2.0 by its publisher.