Instructions to use centenohugo/SFTChatBot-comp4221 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use centenohugo/SFTChatBot-comp4221 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="centenohugo/SFTChatBot-comp4221")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("centenohugo/SFTChatBot-comp4221")
model = AutoModelForCausalLM.from_pretrained("centenohugo/SFTChatBot-comp4221", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use centenohugo/SFTChatBot-comp4221 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "centenohugo/SFTChatBot-comp4221"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "centenohugo/SFTChatBot-comp4221",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/centenohugo/SFTChatBot-comp4221

SGLang

How to use centenohugo/SFTChatBot-comp4221 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "centenohugo/SFTChatBot-comp4221" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "centenohugo/SFTChatBot-comp4221",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "centenohugo/SFTChatBot-comp4221" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "centenohugo/SFTChatBot-comp4221",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use centenohugo/SFTChatBot-comp4221 with Docker Model Runner:
```
docker model run hf.co/centenohugo/SFTChatBot-comp4221
```

My first trained model -> SmolLM2-135M fine-tuned for grounded FAQ answers

The model reads a supplied fictional campus fact and answers in exactly two lines:

Answer: <answer>
Source: <fact ID>

Training

Single run of the provided trainer, python train.py --steps 100 --output checkpoint. Full settings in run_config.json, per-update loss in loss.csv.

Setting	Value
Base model	`HuggingFaceTB/SmolLM2-135M`
Hardware	NVIDIA GeForce RTX 4090 D
Updates	100 (8 one-conversation micro-batches each)
Training conversations	300
Sequence length	256
Optimizer	AdamW, learning rate 2e-5, gradient clipping 1.0
Loss	Conversation-balanced label smoothing, ε = 0.1, final assistant response only
Seed	4221

Training loss: 3.204 (update 1) → 1.456 (update 100).

Evaluation

20 held-out prompt framings, greedy decoding, 96-new-token cap.

Model	Format pass	Grounded-format pass
Base `SmolLM2-135M`	0 / 20	0 / 20
This checkpoint	20 / 20	20 / 20

Usage

The prompt format matches encode_messages in the assignment code: each message is Role:\n<content>, non-assistant messages end with \n, and the prompt ends with Assistant:\n.

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "<centenohugo>/SFTChatBot-comp4221"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)

prompt = (
    "System:\nUse the supplied fictional campus fact. Reply exactly two lines: Answer: "
    "followed by the answer, then Source: followed by its fact ID. Do not add any other text.\n"
    "User:\nFact [F001]: What is the booking limit for study room S01? 2 hours per booking.\n"
    "Question: What is the booking limit for study room S01?\n"
    "Assistant:\n"
)
ids = tokenizer(prompt, return_tensors="pt", add_special_tokens=False)
out = model.generate(**ids, max_new_tokens=96, do_sample=False,
                     eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(out[0, ids["input_ids"].shape[1]:], skip_special_tokens=True))

Downloads last month: 453

Safetensors

Model size

0.1B params

Tensor type

F32

Model tree for centenohugo/SFTChatBot-comp4221

Base model

HuggingFaceTB/SmolLM2-135M

Finetuned

(992)

this model