Instructions to use sadasat/librarian with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers.js
How to use sadasat/librarian with Transformers.js:
// npm i @huggingface/transformers import { pipeline } from '@huggingface/transformers'; // Allocate pipeline const pipe = await pipeline('text-generation', 'sadasat/librarian');
Librarian
The language model behind the Librarian at sadas.at/chat. She runs in the visitor's browser and recites passages of Daniel Rodríguez's writing from her own weights.
| Base model | Qwen/Qwen2.5-0.5B-Instruct, its 24 layers as published |
| Weights | 4 bits, ONNX |
| Vocabulary | 103,120 of the base model's 151,665 tokens; other scripts than Latin are read as bytes |
| Memory | 82 passages in 6 parts, written into thin matrices beside the base weights |
| Training | none by gradient: every edit is a closed-form fit |
| First download | 260 MB |
| Each part of memory | 7.1 to 7.8 MB, fetched when a question calls for it |
| Reply | up to 160 tokens |
| Runs on | WebGPU, or WebAssembly on the CPU; about 2 GB of free memory |
| Runtime | Transformers.js with ONNX Runtime Web |
| Languages | English, some Spanish |
| Licence | Apache-2.0 |
Files
onnx/model_q4.onnxandonnx/model_q4.onnx_data.gz, in 3 parts: the graph and the base weights. The graph takes one edit as inputs, and with none it is the base model.memory/1.bin.gztomemory/8.bin.gz: the edits. One decides whether a message calls for a passage, one is her voice, and the others recite the passages.chooser.json: a table of words that picks the passage. It stores none.tokenizer.json.gz,tokenizer_config.json,config.json,generation_config.json.LICENSEandNOTICE: the licence, and what was changed from the base model.
A file of a megabyte or more is gzipped. The parts of the weights are joined before they are unpacked.
Limits
She was made for one page, which brings her instructions, the rule that reads the table, and the sentences she says when she does not know. She is experimental and can be wrong: she may pick the wrong passage or misremember the right one. She is not Daniel and does not speak for him.
- Downloads last month
- 1