Librarian

The language model behind the Librarian at sadas.at/chat. She runs in the visitor's browser and recites passages of Daniel Rodríguez's writing from her own weights.

Base model Qwen/Qwen2.5-0.5B-Instruct, its 24 layers as published
Weights 4 bits, ONNX
Vocabulary 103,120 of the base model's 151,665 tokens; other scripts than Latin are read as bytes
Memory 82 passages in 6 parts, written into thin matrices beside the base weights
Training none by gradient: every edit is a closed-form fit
First download 260 MB
Each part of memory 7.1 to 7.8 MB, fetched when a question calls for it
Reply up to 160 tokens
Runs on WebGPU, or WebAssembly on the CPU; about 2 GB of free memory
Runtime Transformers.js with ONNX Runtime Web
Languages English, some Spanish
Licence Apache-2.0

Files

  • onnx/model_q4.onnx and onnx/model_q4.onnx_data.gz, in 3 parts: the graph and the base weights. The graph takes one edit as inputs, and with none it is the base model.
  • memory/1.bin.gz to memory/8.bin.gz: the edits. One decides whether a message calls for a passage, one is her voice, and the others recite the passages.
  • chooser.json: a table of words that picks the passage. It stores none.
  • tokenizer.json.gz, tokenizer_config.json, config.json, generation_config.json.
  • LICENSE and NOTICE: the licence, and what was changed from the base model.

A file of a megabyte or more is gzipped. The parts of the weights are joined before they are unpacked.

Limits

She was made for one page, which brings her instructions, the rule that reads the table, and the sentences she says when she does not know. She is experimental and can be wrong: she may pick the wrong passage or misremember the right one. She is not Daniel and does not speak for him.

Downloads last month
1
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for sadasat/librarian

Quantized
(396)
this model