Instructions to use sandeep123/ML-LaySum-Qwen3-1.7B-SFT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use sandeep123/ML-LaySum-Qwen3-1.7B-SFT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="sandeep123/ML-LaySum-Qwen3-1.7B-SFT")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("sandeep123/ML-LaySum-Qwen3-1.7B-SFT")
model = AutoModelForCausalLM.from_pretrained("sandeep123/ML-LaySum-Qwen3-1.7B-SFT", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use sandeep123/ML-LaySum-Qwen3-1.7B-SFT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sandeep123/ML-LaySum-Qwen3-1.7B-SFT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sandeep123/ML-LaySum-Qwen3-1.7B-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/sandeep123/ML-LaySum-Qwen3-1.7B-SFT

SGLang

How to use sandeep123/ML-LaySum-Qwen3-1.7B-SFT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "sandeep123/ML-LaySum-Qwen3-1.7B-SFT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sandeep123/ML-LaySum-Qwen3-1.7B-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "sandeep123/ML-LaySum-Qwen3-1.7B-SFT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sandeep123/ML-LaySum-Qwen3-1.7B-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use sandeep123/ML-LaySum-Qwen3-1.7B-SFT with Docker Model Runner:
```
docker model run hf.co/sandeep123/ML-LaySum-Qwen3-1.7B-SFT
```

ML-LaySum Qwen3-1.7B SFT

A full-parameter supervised fine-tuning of Qwen3-1.7B for English lay summarization of machine-learning research. It takes the abstract, introduction, and conclusion of a paper and generates a summary for a non-specialist reader. Generation uses Qwen3's non-thinking mode.

Training

Setting	Value
Base model	Qwen/Qwen3-1.7B (chat checkpoint)
Base revision	70d244cc86ccca08cf5af4e1e306ecf908b1ad5e
Dataset	ML-LaySum; ICML 2025 and 2026 papers and lay summaries
Dataset revision	a983411ae5100d8805f2852e10ebd43647001586
Training / validation / held-out test	6,911 / 879 / 852
Source	Abstract + introduction + conclusion, where available
Target	Reference lay summary
Objective	Cross-entropy on summary tokens and the turn-ending token; prompt tokens masked
Training	Full-parameter SFT, 3 epochs, 1,296 optimizer steps
Learning rate	2e-5; cosine schedule; 3% warmup
Optimizer	AdamW; weight decay 0
Effective batch size	16 (8 GPUs × 1 example × 2 accumulation steps)
Precision	FP32 weights; BF16 mixed-precision computation
Hardware	8 AMD MI210 GPUs
Context	Maximum configured length 32,768 tokens; no truncation or packing
Seed	42
Selection	Last checkpoint, step 1,296; not selected by best validation loss

Training resumed at step 1,000 after a storage-quota interruption, retaining optimizer, scheduler and random states. Test data were not used for training or checkpoint selection. Missing conclusion sections remain missing; no synthetic conclusions were inserted.

Usage

Install a Qwen3-compatible Transformers version (training used 4.57.6), PyTorch and Accelerate.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "sandeep123/ML-LaySum-Qwen3-1.7B-SFT"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto"
)
paper_text = "Abstract: ...\n\nIntroduction: ...\n\nConclusion: ..."
instruction = (
    "Write a lay summary of the following machine learning research for a "
    "non-specialist reader. Explain the problem, main approach, and findings in "
    "clear language, faithfully using only the supplied paper text. Return only "
    "the summary.\n\nPaper text:\n"
)
text = tokenizer.apply_chat_template(
    [{"role": "user", "content": instruction + paper_text}],
    tokenize=False, add_generation_prompt=True, enable_thinking=False,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.inference_mode():
    output = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True))

The decoding settings above are an example, not a validated optimal configuration. Uploaded weights are FP32 (approximately 6.9 GB); loading in BF16 reduces weight memory to approximately 3.4 GB, excluding activations and the generation cache.

Evaluation and limitations

No held-out generation benchmark or human evaluation has yet been completed. This is an SFT baseline, with no claim of state-of-the-art performance. Outputs can omit qualifications, misstate findings or introduce unsupported explanations. Check summaries against the paper before publication. Validation cross-entropy is a training diagnostic, not a measure of lay-reader comprehension. See training_summary.json for validation losses.

Files and license

This repository contains inference weights, tokenizer files and sanitized training metadata. Optimizer states, account credentials, raw training records and private cluster paths are not uploaded. Model weights are released under Apache 2.0, following the base model license. Dataset and source-paper terms are separate; see the dataset card. Dataset access may require permission independently of this public model.

Downloads last month: 156

Safetensors

Model size

2B params

Tensor type

F32

Model tree for sandeep123/ML-LaySum-Qwen3-1.7B-SFT

Base model

Qwen/Qwen3-1.7B-Base

Finetuned

Qwen/Qwen3-1.7B

Finetuned

(1214)

this model