Instructions to use Lucasgitguy-co/wiki-model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Lucasgitguy-co/wiki-model with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Lucasgitguy-co/wiki-model")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Lucasgitguy-co/wiki-model") model = AutoModelForCausalLM.from_pretrained("Lucasgitguy-co/wiki-model", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Lucasgitguy-co/wiki-model with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Lucasgitguy-co/wiki-model" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Lucasgitguy-co/wiki-model", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/Lucasgitguy-co/wiki-model
- SGLang
How to use Lucasgitguy-co/wiki-model with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Lucasgitguy-co/wiki-model" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Lucasgitguy-co/wiki-model", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Lucasgitguy-co/wiki-model" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Lucasgitguy-co/wiki-model", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use Lucasgitguy-co/wiki-model with Docker Model Runner:
docker model run hf.co/Lucasgitguy-co/wiki-model
wiki-model
GGUF: Lucasgitguy-co/wiki-model-GGUF (for llama.cpp).
SmolLM2-135M, fully fine-tuned to state facts from Wikipedia.
It does four things: a fact about a topic, a random fact, short question answering, and correcting a false statement.
With --grounded it answers from a sourced Wikipedia index and prints the article it used.
The from-scratch 12.5M sibling is wiki-model-tiny. Data and index: wiki-model-facts.
Small models get facts wrong when they answer from memory. Use
--groundedfor anything that matters.
Use
hf download Lucasgitguy-co/wiki-model --local-dir wiki-model
cd wiki-model && pip install -e .
wiki-model "Eiffel Tower" # picks the 135M weights in this folder
wiki-model --q "Who wrote Hamlet?" --grounded # first call downloads the ~600 MB fact index (skip with --no-index)
transformers only
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Lucasgitguy-co/wiki-model")
m = AutoModelForCausalLM.from_pretrained("Lucasgitguy-co/wiki-model")
x = tok("<q> Who wrote Hamlet? </q>", return_tensors="pt", add_special_tokens=False)
print(tok.decode(m.generate(**x, max_new_tokens=32, do_sample=False)[0][x.input_ids.shape[1]:], skip_special_tokens=True))
Prompts: <fact> Topic </fact>, <q> Question? </q>, <fix> Claim. </fix>, optionally preceded by <ctx> article text </ctx> .
Results
Held-out articles and claims (test split by article title). "Grounded" retrieves context at run time with the BM25 index. "Fact supported" = the generated sentence matches a sentence of the topic's own article with token F1 ≥ 0.5. "SmolLM2-135M few-shot" is the untuned base model with three examples in the prompt (no context in the examples); "copy the claim" just repeats the input claim (fix-task baseline). "Fix (year swap)": shifted-year sentences mixed with unchanged true sentences (71% of the set), so copying already scores 71. Answering from memory, the base model's facts are supported slightly more often than the fine-tune's: fine-tuning taught the task format and how to use retrieved context, not new knowledge.
| model | mode | Q&A infobox EM | 'What is X' F1 | Fix (FEVEROUS) EM | Fix can't-verify acc | Fix (year swap) EM | Fact supported (new topics) | Fact supported (known topics) |
|---|---|---|---|---|---|---|---|---|
| wiki-model | model only | 14.7 | 30.0 | 37.0 | 88.5 | 65.0 | 12.0 | 10.0 |
| wiki-model | grounded | 94.7 | 65.3 | 37.0 | 88.0 | 83.0 | 98.0 | 98.0 |
| wiki-model-tiny | model only | 14.7 | 28.2 | 32.5 | 89.0 | 54.0 | 5.0 | 8.0 |
| wiki-model-tiny | grounded | 90.7 | 58.0 | 33.5 | 89.0 | 55.0 | 92.0 | 94.0 |
| SmolLM2-135M few-shot | model only | 10.0 | 18.9 | 11.0 | 11.0 | 66.0 | 16.0 | 16.0 |
| SmolLM2-135M few-shot | grounded | 26.7 | 25.2 | 6.5 | 36.5 | 47.0 | 65.0 | 71.0 |
| copy the claim | model only | — | — | 47.5 | 89.0 | 71.0 | — | — |
Training
| Base | HuggingFaceTB/SmolLM2-135M (Apache-2.0) |
| Data | 40k task examples (35% with retrieved context) sampled from wiki-model-facts + FactfulDataset, then a fix top-up of 11k examples (FactfulDataset with retrieved context + replay), then a grounding top-up of 24k examples (mostly with retrieved context) |
| Recipe | full fine-tune, AdamW lr 1e-4 cosine (top-ups 5e-5), batch 16, loss on answers only, bf16 autocast, gradient checkpointing |
| Hardware | one Apple M4 (MPS), shared with another training job, about 8 hours |
See wiki-model-tiny for how the task data is built.
Limits
- Answers from memory are often wrong for less-known topics. Grounded answers are only as good as retrieval.
- English only; Wikipedia snapshot from 2024.
- Fixing false statements is the weakest task: compare with the "copy the claim" row before relying on it. "Can't verify" is rarely produced.
Licence
Weights and code: Apache-2.0 (same as the base). Training text comes from Wikipedia (CC BY-SA); FactfulDataset has no licence on its card and is not redistributed.
- Downloads last month
- 80