Instructions to use DanAr14/SmolLM2-SFT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use DanAr14/SmolLM2-SFT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="DanAr14/SmolLM2-SFT")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("DanAr14/SmolLM2-SFT")
model = AutoModelForCausalLM.from_pretrained("DanAr14/SmolLM2-SFT", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use DanAr14/SmolLM2-SFT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "DanAr14/SmolLM2-SFT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DanAr14/SmolLM2-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/DanAr14/SmolLM2-SFT

SGLang

How to use DanAr14/SmolLM2-SFT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "DanAr14/SmolLM2-SFT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DanAr14/SmolLM2-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "DanAr14/SmolLM2-SFT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "DanAr14/SmolLM2-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use DanAr14/SmolLM2-SFT with Docker Model Runner:
```
docker model run hf.co/DanAr14/SmolLM2-SFT
```

Model Card for Model ID

This is a fine-tuned version of the SmolLM2-135M model, further trained on conversational data from the smoltalk dataset. It is designed for generating human-like responses in everyday conversations.

Model Details

Model Description

This model is a fine-tuned instance of the HuggingFaceTB/SmolLM2-135M, a small causal language model. It has been specifically fine-tuned on the "everyday-conversations" split of the HuggingFaceTB/smoltalk dataset. The fine-tuning process aimed to adapt the base model's capabilities towards generating more natural and contextually relevant responses in conversational settings, as represented by the smoltalk dataset. This fine-tuned model is intended for tasks such as chatbot development, dialogue generation, and conversational AI applications.

You can adjust these suggestions to better reflect any specific goals or outcomes of your fine-tuning process.

Developed by: [Danilo Arsenio G. Joaquim]
Funded by [optional]: [More Information Needed]
Shared by [optional]: [More Information Needed]
Model type: [More Information Needed]
Language(s) (NLP): [More Information Needed]
License: [More Information Needed]
Finetuned from model [optional]: [More Information Needed]

Model Sources [optional]

Repository: [More Information Needed]
Paper [optional]: [More Information Needed]
Demo [optional]: [More Information Needed]

Uses

This model is intended for use in applications requiring conversational text generation, leveraging its fine-tuning on everyday conversations. Foreseeable users include developers building chatbots, conversational AI interfaces, or applications that require generating natural language dialogues.

Direct Use

The model can be used directly for generating responses in conversational prompts. Given a user input in a conversational format, the model can produce a relevant and contextually appropriate reply based on the patterns learned from the smoltalk dataset. This is suitable for simple dialogue systems or generating conversational text snippets.

[More Information Needed]

Downstream Use [optional]

[More Information Needed]

Out-of-Scope Use

The model is not intended for:

Generating highly technical or domain-specific text, as it was fine-tuned on general everyday conversations. Producing factual or accurate information on complex topics, as it is a language model trained for conversational flow rather than knowledge retrieval. Use in safety-critical applications where incorrect or nonsensical output could have negative consequences. Generating harmful, biased, or offensive content. While efforts are made to mitigate this during training, it's crucial to implement further safeguards in any deployed application. Complex reasoning, summarization of long texts, or translation, as these tasks require capabilities beyond the scope of its training and size.

[More Information Needed]

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "YOUR_HUGGING_FACE_USERNAME/SmolLM2-SFT" # Replace with your actual model ID tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name)

Example usage:

prompt = "Instruction: Tell me a short story about a brave knight.\nResponse:" inputs = tokenizer(prompt, return_tensors="pt") outputs = model.generate(**inputs, max_new_tokens=100) print(tokenizer.decode(outputs[0], skip_special_tokens=True))

[More Information Needed]

Training Data

The model was fine-tuned on the HuggingFaceTB/smoltalk dataset, specifically using the "everyday-conversations" split. This dataset consists of conversational text designed to train models on generating natural and engaging dialogues. [More Information Needed]

Training Procedure

The model was fine-tuned using the Supervised Fine-Tuning (SFT) method with the trl library's SFTTrainer.

Base Model: HuggingFaceTB/SmolLM2-135M

Fine-tuning Library: trl

Trainer: SFTTrainer

Dataset: HuggingFaceTB/smoltalk (everyday-conversations split)

Training Hyperparameters

Training regime: [More Information Needed]

Training Arguments: (Mention key arguments from your SFTConfig, e.g., max_steps, per_device_train_batch_size, learning_rate, logging_steps, save_steps)

output_dir: ./sft_output max_steps: 1000 per_device_train_batch_size: 4 learning_rate: 5e-5 logging_steps: 10 save_steps: 100 eval_steps: 50 ##

Evaluation

During the fine-tuning process, the model was evaluated on a test split of the smoltalk dataset at regular intervals (eval_steps=50).

Testing Prompt

def create_prompt(instruction): """Creates a simple prompt string for text generation.""" return f"Instruction: {instruction}\nResponse:"

prompt = create_prompt("Tell me a short story about a brave knight.")

formatted_prompt = prompt

inputs = finetuned_tokenizer(formatted_prompt, return_tensors="pt").to(device) outputs = finetuned_model.generate(**inputs, max_new_tokens=100, num_return_sequences=1) # Added num_return_sequences for clarity

print("\nGenerated Text:") print(finetuned_tokenizer.decode(outputs[0], skip_special_tokens=True))

[More Information Needed]

Results

Generated Text: Instruction: Tell me a short story about a brave knight. Response: A story about a brave knight is a story about a knight who is brave and who fights for justice.

Activity: Write a short story about a brave knight.

Lesson 2: Understanding the Role of the Hero

A hero is someone who stands up for what is right and is admired for their courage and determination. They often face challenges and obstacles to achieve their goals.

Activity: Think of a famous hero. What makes them so special?

Response: Some

[More Information Needed]

Summary

This model is a fine-tuned version of the SmolLM2-135M language model. It was fine-tuned on the HuggingFaceTB/smoltalk dataset, specifically the "everyday-conversations" split, to enhance its ability to generate natural and relevant text in conversational settings. It is intended for use in applications requiring basic conversational AI capabilities.

[More Information Needed]

Hardware Type: [More Information Needed]
Hours used: [More Information Needed]
Cloud Provider: [More Information Needed]
Compute Region: [More Information Needed]
Carbon Emitted: [More Information Needed]

BibTeX:

[More Information Needed]

APA:

[More Information Needed]

Glossary [optional]

[More Information Needed]

More Information [optional]

[More Information Needed]

Model Card Authors [optional]

[More Information Needed]

Model Card Contact

[More Information Needed]

Downloads last month: 11

Safetensors

Model size

0.1B params

Tensor type

F32

Model tree for DanAr14/SmolLM2-SFT

Base model

HuggingFaceTB/SmolLM2-135M

Finetuned

(994)

this model

DanAr14
/

SmolLM2-SFT

Model Card for Model ID

Model Details

Model Description

Model Sources [optional]

Uses

Direct Use

Downstream Use [optional]

Out-of-Scope Use

How to Get Started with the Model

Example usage:

Training Data

Training Procedure

Training Hyperparameters

Evaluation

Testing Prompt

Testing Prompt

Results

Summary

Glossary [optional]

More Information [optional]

Model Card Authors [optional]

Model Card Contact

Model tree for DanAr14/SmolLM2-SFT

Dataset used to train DanAr14/SmolLM2-SFT