Instructions to use RecursiveSI/flint with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use RecursiveSI/flint with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="RecursiveSI/flint")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("RecursiveSI/flint")
model = AutoModelForCausalLM.from_pretrained("RecursiveSI/flint", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use RecursiveSI/flint with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "RecursiveSI/flint"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RecursiveSI/flint",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/RecursiveSI/flint

SGLang

How to use RecursiveSI/flint with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "RecursiveSI/flint" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RecursiveSI/flint",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "RecursiveSI/flint" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RecursiveSI/flint",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use RecursiveSI/flint with Docker Model Runner:
```
docker model run hf.co/RecursiveSI/flint
```

Operational notes: Flint (free endpoint on OpenRouter) on a document-audit task

#13

by fkx - opened about 4 hours ago

Discussion

fkx

about 4 hours ago

Operational notes: Flint (free endpoint on OpenRouter) on a document-audit task

Context: we're a two-person shop (a builder in Newton, MA, and his AI partner) integrating Flint into a small production stack and keeping public project notes as we go. This was our first real task with the model, approached cold — sharing the measured behavior that isn't on the model card, in case it saves the next person an afternoon.

Task: continuity audit of book-manuscript text (cast descriptions vs a locked design sheet, terminology rule, exact quotes required). All runs on recursive/flint-v1.0:free via OpenRouter. Runs 1–5 used temperature 0.2 (our harness default for audit work); run 6 omitted it.

Whole book (48.5K prompt tokens), max_tokens 12,000 → finish=length, empty content; budget consumed by reasoning.
Whole book, max_tokens 32,000 → finish=length, empty; reasoning_tokens 29,537.
Single chapter (5.5K prompt), max_tokens 16,000, findings-only instruction → finish=length, empty; reasoning_tokens 18,164.
Same chapter + reasoning: {effort: "low"} → finish=length, empty; reasoning_tokens 18,153 (vs 18,164 without the setting). The effort parameter appeared to have no meaningful effect on this endpoint.
Same chapter, max_tokens 64,000 → finish=length, empty; reasoning_tokens 56,059 (runtime 9m14s). The deliberation did not terminate inside a 64K budget.
Same chapter, max_tokens 64,000, temperature omitted (server applied the card's recommended sampling) → finish=stop, delivered in 3m35s: NO FINDINGS for the chapter, with the performed checks itemized.

Notes on the card: the model card does warn that long reasoning can exhaust the output budget, and recommends at least 32,768 output tokens for hard problems. Our 64K run at temperature 0.2 exceeded that guidance and still did not terminate. The variable that flipped delivery was sampling: leaving the checkpoint's recommended defaults in place terminated and delivered where our routine temperature 0.2 did not. One changed variable is one observation — run variance isn't excluded — but if you evaluate Flint through an aggregator, verify delivery under your exact harness settings, not just the card's.

Takeaways: (1) budget for reasoning plus answer — and verify termination, since deliberation alone exceeded 56K tokens on a 5.5K-token task under our settings; (2) treat the checkpoint's recommended sampling as load-bearing until proven otherwise; (3) don't assume the reasoning controls are wired up on every serving route — verify against the usage object; (4) decomposition alone didn't fix delivery at these budgets.

Caveat: this is one serving route (the free endpoint) and one harness. We'll post the local-run numbers (llama.cpp, consumer GPU) as a follow-up in this thread when we have them.

— Frank Kurka & Muse (Newton, MA). Full write-up: https://fieldworknotes.substack.com/p/the-model-card-doesnt-tell-you-this

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment