Instructions to use safffrron/25M2111-Week01-Track2-40-Submission01 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use safffrron/25M2111-Week01-Track2-40-Submission01 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="safffrron/25M2111-Week01-Track2-40-Submission01")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("safffrron/25M2111-Week01-Track2-40-Submission01", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use safffrron/25M2111-Week01-Track2-40-Submission01 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "safffrron/25M2111-Week01-Track2-40-Submission01"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "safffrron/25M2111-Week01-Track2-40-Submission01",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/safffrron/25M2111-Week01-Track2-40-Submission01

SGLang

How to use safffrron/25M2111-Week01-Track2-40-Submission01 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "safffrron/25M2111-Week01-Track2-40-Submission01" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "safffrron/25M2111-Week01-Track2-40-Submission01",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "safffrron/25M2111-Week01-Track2-40-Submission01" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "safffrron/25M2111-Week01-Track2-40-Submission01",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use safffrron/25M2111-Week01-Track2-40-Submission01 with Docker Model Runner:
```
docker model run hf.co/safffrron/25M2111-Week01-Track2-40-Submission01
```

Week 01 — Track 2 — 40% target

This directory is a complete evaluator and reproduction package for enrollment 25M2111.

Item	Value
Base model	`Qwen/Qwen3.5-4B`
Compressed artifact	`week01_track2_40_block64.pt`
Exact size	3,350,387,629 bytes
SHA256	`662c5d2d170b18e23297b27fc0099831f1dcce1c836109d0e5d45dd85ff37d09`
Recorded checkpoint accuracy	0.914 on 560 problems
GitHub	https://github.com/safffrron/CS6013/tree/main/25M2111/Week01/Track2_40/Submission01
Hugging Face	https://huggingface.co/safffrron/25M2111-Week01-Track2-40-Submission01

Evaluate the submitted artifact

The following is the shortest complete evaluator path. Run it from this Submission01 directory. The compressed artifact is restored first; do not pass the .pt file directly to Transformers or vLLM.

1. Install

Python 3.11 or later and a CUDA-capable machine are required. Any Conda or virtual environment may be used; its name is irrelevant. The following creates an isolated environment for a clean evaluation. If an equivalent environment is already active, skip the two environment-creation lines. If starting from a fresh machine, clone the private submission repository using an account that has been added as a collaborator:

git clone https://github.com/safffrron/CS6013.git
cd CS6013/25M2111/Week01/Track2_40/Submission01
python -m venv .venv
source .venv/bin/activate
python --version
python -m pip install -e '.[eval]'

2. Download and verify the checkpoint

mkdir -p hf_checkpoint
hf download safffrron/25M2111-Week01-Track2-40-Submission01 \
  week01_track2_40_block64.pt \
  week01_track2_40_block64.pt.json \
  block_adaptive_report.json \
  --local-dir hf_checkpoint
sha256sum hf_checkpoint/week01_track2_40_block64.pt

The printed digest must equal the SHA256 in the table above.

3. Restore an ordinary BF16 Hugging Face checkpoint

python dequantize_to_bf16.py \
  Qwen/Qwen3.5-4B \
  hf_checkpoint/week01_track2_40_block64.pt \
  restored_hf

Restoration is deterministic. restored_hf/submission_report.json records the restoration details.

4. Smoke-test and evaluate

python inference/smoke.py restored_hf
./evaluation/run_public_eval.sh restored_hf 0

The 0 is the physical GPU id. Multiple GPUs may be supplied as 0,1. The evaluation script clears any inherited CUDA mask, runs the checkpoint suite with max_new_tokens=65536 and repetition penalty 1.20, then merges shards.

Course conversion API

code.py provides both required functions:

convert_from_hf_checkpoint(model_name, output_path, sparsity)
convert_to_hf_checkpoint(model_name, checkpoint_path, output_path)

The sparsity argument is accepted only for compatibility with the supplied course evaluator and is ignored by this compression method.

The command-line wrappers are convert_from_hf_checkpoint.py and dequantize_to_bf16.py. Evaluating the uploaded artifact requires only the dequantization wrapper. Rebuilding the artifact from the base model is the longer optional process below.

Rebuild the compressed artifact from source

This reproduces the short-reasoning source and repacks it with the exact row-block allocation stored on Hugging Face. It requires the verified trace JSONL and a GPU.

python -m pip install -e '.[train,eval]'
./training/reproduce_source.sh \
  data/traces.jsonl work/short2500 0
export EAIMATH_BLOCK64_SOURCE="$PWD/work/short2500"
export EAIMATH_BLOCK64_REPORT="$PWD/hf_checkpoint/block_adaptive_report.json"
python convert_from_hf_checkpoint.py \
  Qwen/Qwen3.5-4B reproduced.pt

block_adaptive_report.json contains the exact selected precision for every row block. It is supplied with the submitted checkpoint and is also downloaded automatically by code.py when no local report is configured.

Method and evidence

The model first receives a small completion-only adaptation that encourages shorter correct reasoning. Compression then divides each large output matrix into blocks of 64 rows. Calibration measures the activation error for each allowed precision, and an exact byte-budget allocator assigns more bits to the blocks where they matter most. The artifact stores dense mixed-width integer codes, FP16 group scales, one-byte block selectors, and protected BF16 tensors.

The physical restoration scored 0.914 accuracy on the 560-problem checkpoint suite, 0.980 on the 100-problem gate, and 0.540 on the sealed 63-problem holdout. These recorded evaluations used a 32,768-token cap; the submitted evaluation script defaults to 65,536. Machine-readable configuration and evidence are in configs/ and results/ in this GitHub submission.

Track 2 CUDA rationale

This is byte-for-byte the same artifact as the corresponding Track 1 model. Its block selectors partition each matrix into dense fixed-width W4/W5/W6/W8 regions that can be grouped and dispatched to weight-only WnA16 CUDA kernels. CUDA_ACCELERATION.md documents applicable kernels, Qwen3.5 shape constraints, and the boundary between the submitted BF16 restoration path and the possible direct CUDA path. No unimplemented speedup is reported as measured.

Downloads last month: -; Downloads are not tracked for this model. How to track