daidv1112/FoundationalTS-phase2-harsleep-balanced

FoundationalTS — Phase-2 (HAR + Sleep, balanced)

Joint Phase-2 downstream checkpoint of OpenTSLMSPFixPos2D, a multimodal time-series LLM. Warm-started from a Phase-1 joint-pretrained checkpoint and fine-tuned on the HAR and Sleep chain-of-thought (CoT) downstream tasks with balanced 1:1 sampling between the two tasks.

Model

• Backbone LLM: google/gemma-3-270m + LoRA (r=4, α=8)
• Encoder/decoder: MLP (--enc_dec_type mlp, hidden 64 / ff 256)
• Patch size: 32 · Positional encoding: sinusoidal 2-D (sincos) · no frequency branch
• Training: Phase-2 downstream (joint_phase2_downstream), warm-started from Phase-1 (extended-balanced), 20 epochs, PHASE2_COT_ONLY=1, SLEEP_UPSAMPLE=9 (→ HAR:Sleep ≈ 1:1)
• Best val loss: 0.636 (epoch 20)

Contents

• best_model.pt — full checkpoint (encoder / projector / decoder / decoder_projector / mse_projector state, LoRA adapters, resized LLM embeddings, plus optimizer and scheduler state so training can be resumed).

Reference results (Context-is-Key benchmark, mean RCRPS ↓)

Setting	RCRPS
context ON	0.380
context OFF	0.411

(For comparison the Phase-1 forecaster scores 0.300 no-context; this Phase-2 checkpoint is CoT-tuned, which trades some forecast quality for text/CoT ability.)

Loading

Use the FoundationalTS code (OpenTSLMSPFixPos2D). Enable LoRA before loading the state, then load encoder/decoder/projector states and LoRA adapters. See evaluation/cot/eval_cot_phase2.py and hf_eval/ in the repo: https://github.com/Davido111200/FoundationalTS