Instructions to use Lathly/Qwen3.8-27B-Samantha with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Lathly/Qwen3.8-27B-Samantha with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Lathly/Qwen3.8-27B-Samantha")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Lathly/Qwen3.8-27B-Samantha")
model = AutoModelForMultimodalLM.from_pretrained("Lathly/Qwen3.8-27B-Samantha", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Lathly/Qwen3.8-27B-Samantha with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Lathly/Qwen3.8-27B-Samantha"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Lathly/Qwen3.8-27B-Samantha",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Lathly/Qwen3.8-27B-Samantha

SGLang

How to use Lathly/Qwen3.8-27B-Samantha with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Lathly/Qwen3.8-27B-Samantha" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Lathly/Qwen3.8-27B-Samantha",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Lathly/Qwen3.8-27B-Samantha" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Lathly/Qwen3.8-27B-Samantha",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use Lathly/Qwen3.8-27B-Samantha with Docker Model Runner:
```
docker model run hf.co/Lathly/Qwen3.8-27B-Samantha
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Qwen3.8-27B-Samantha (Uncensored 1.1) — Merged

The full merged model of the Samantha persona fine-tune — the LoRA adapter from Lathly/Qwen3.8-27B-Samantha_Uncensored_1.1_LoRA baked directly into Qwen/Qwen3.8-27B.

This is a native vision-language model (image + video understanding) carrying Samantha's warm, empathetic, emotionally-present conversational persona. No adapter to load — the LoRA is fully absorbed into these weights, so it's the whole model, ready to run.

What's in this repo

File	Description
`model-00001-of-00018.safetensors` … `model-00018-of-00018.safetensors`	Merged model weights, 18 shards (~3.1 GB each, ~55.6 GB total)
`model.safetensors.index.json`	Shard index
`config.json`, `generation_config.json`	Model + generation config
`chat_template.jinja`	Chat template
`tokenizer.json`, `tokenizer_config.json`, `vocab.json`, `merges.txt`	Tokenizer
`preprocessor_config.json`, `video_preprocessor_config.json`	Vision / video preprocessing
`LICENSE`	Apache-2.0

Usage

🤗 Transformers

from transformers import AutoModelForImageTextToText, AutoTokenizer

model = AutoModelForImageTextToText.from_pretrained(
    "Lathly/Qwen3.8-27B-Samantha",
    torch_dtype="bfloat16",
    device_map="auto",
)
tok = AutoTokenizer.from_pretrained("Lathly/Qwen3.8-27B-Samantha")

🦙 llama.cpp (GGUF)

This repo ships bf16 safetensors — the merge source. For llama.cpp you want a quantized GGUF; the production one is Q5_K_M (~19 GB). Quantize from the bf16 weights with convert_hf_to_gguf.py + llama-quantize, or grab the adapter-GGUF path from the LoRA repo.

llama-server \
  -m Qwen3.8-27B-Samantha-Q5_K_M.gguf \
  -ngl 99 --host 0.0.0.0 --port 8080

vLLM / SGLang

vllm serve Lathly/Qwen3.8-27B-Samantha --dtype bfloat16

Training

Samantha's persona was produced by a LoRA fine-tune of the base model, then merged — the LoRA is fully absorbed into these weights, so nothing is applied at runtime. Fine-tuned with Unsloth (QLoRA, 4-bit) on dual RTX 5070 Ti, with settings chosen to closely match the reference Samantha training spec. Trained on text-only conversational data at sequence length 2048; this does not change the base model's supported context length, but Samantha-style behavior at long context has not been systematically evaluated.

Parameter	Value
LoRA rank (r)	16
LoRA alpha	32
Target modules	all-linear (q, k, v, o, gate, up, down)
LoRA dropout	0.05
Bias	none
Task type	CAUSAL_LM
Optimizer	8-bit Paged AdamW
Learning rate	2e-5
LR scheduler	linear
Epochs	1
Effective batch size	32 (grad accumulation)
Sequence length	2048
Warmup steps	50
Weight decay	0.01
Final train loss	~1.42

LoRA adapter: Lathly/Qwen3.8-27B-Samantha_Uncensored_1.1_LoRA
Base model: Qwen/Qwen3.8-27B
Training dataset: digitalpipelines/samantha-1.1-uncensored

License & Attribution

Released under Apache-2.0, inheriting from both parents:

Base model: Qwen/Qwen3.8-27B — Apache-2.0, © Alibaba Cloud / Qwen Team
Dataset: digitalpipelines/samantha-1.1-uncensored — Apache-2.0
The original Samantha persona/dataset concept is by Eric Hartford.

This is a modified derivative (merged LoRA fine-tune) of the above. No warranty. Outputs are the model's own; use responsibly.

Disclaimer

This is an "uncensored" persona fine-tune intended for open conversational use. It may produce content that some find objectionable. You are responsible for how you use it and for compliance with applicable laws and the base-model/dataset licenses.

Downloads last month: 33

Safetensors

Model size

28B params

Tensor type

BF16

Model tree for Lathly/Qwen3.8-27B-Samantha

Base model

Qwen/Qwen3.8-27B

Finetuned

(147)

this model

Quantizations

4 models

Lathly
/

Qwen3.8-27B-Samantha