ZaraAI
Collection
Open-weights models trained on the Standard Works of The Church of Jesus Christ of Latter-day Saints, for religious research. • 3 items • Updated
ZaraEmbed
A verse embedder for the Standard Works of The Church of Jesus Christ of Latter-day Saints that finds the right verse first far more often than OpenAI on Book of Mormon, D&C and Pearl of Great Price chapters it has never seen. Ask a question in plain modern English, with or without the names in the verse, and ZaraEmbed returns the verse that answers it. It is ZaraBERTa fine-tuned for retrieval, so it reads Nephi, Zarahemla and LORD as the scripture-native words ZaraBERTa learned, not as fragments.
Trained by Brigham Young University-Idaho on public-domain text. Chapter headings, footnotes and Church study aids were not used.
ZaraAI family
Part of ZaraAI, BYU-Idaho's scripture-native model collection:
| Model | Role |
|---|---|
| BYU-Idaho/ZaraBERTa | Cased, scripture-native language model (the base for both below) |
| BYU-Idaho/ZaraEmbed | Verse embedder: question to verse retrieval (this model) |
| BYU-Idaho/ZaraRerank | Cross-encoder reranker for ZaraEmbed's top 100 |
Purpose
These models are released for research on scripture and on domain-specific language models. They are also released as aids for personal scripture study. They are offered in the hope that they help readers find and understand the Book of Mormon, and come unto Christ.
What is new
1. It finds verses from chapters it never trained on
The held-out tests use 289 verses from 37 Book of Mormon, D&C and Pearl of Great Price chapters that no training pair touched. A separate language model, not one used for the training questions, wrote one question per verse in two styles, and the same 289 verses appear in both tests:
- no-names: a teenager's search that avoids the verse's names and distinctive words.
- story: a reader naming the speaker, audience or event.
Right verse first (top-1), both models embedding bare verse text:
| Test | ZaraEmbed | OpenAI text-embedding-3-large | Significance |
|---|---|---|---|
| no-names (289) | 191 (66%) | 126 (44%) | McNemar p = 2e-11; top-10 271 vs 232 |
| story (289) | 175 (61%) | 143 (49%) | p = 5e-4; top-10 249 vs 222 |
On the Book of Mormon questions alone (149 of the 289), ZaraEmbed leads 103 vs 58 on no-names and 98 vs 81 on story.
For an open-weights reference, bge-large-en-v1.5 places 63 no-names and 78 story questions first.
2. On the human-written benchmark
The 152-question benchmark holds human-written study questions across all five volumes. Here OpenAI leads overall, and the Restoration-scripture questions are a tie:
| Benchmark | ZaraEmbed | OpenAI text-embedding-3-large | Significance |
|---|---|---|---|
| Book of Mormon, D&C, Pearl of Great Price (63) | 36 | 38 | not significant |
| Book of Mormon only (40) | 18 | 20 | not significant |
| All five volumes (152) | 92 | 108 | OpenAI ahead, p = 0.014 |
ZaraEmbed's strength is the Restoration scripture it was built for. Paired with ZaraRerank, the pipeline moves ahead of OpenAI on the benchmark's Restoration questions too (47 vs 38 for OpenAI on bare verses, 41 with chapter context); see that card.
3. Trained on both bare verses and verses with chapter context
Half of the documents seen in training were shown as "chapter summary + verse", so the model accepts either form. The summaries are retrieval-oriented descriptions generated for this project, not Church chapter headings. All numbers on this card use bare verses.
Usage
Mean-pool last_hidden_state over real tokens, then L2-normalise. Prefix every text with a single space
so verse-initial words tokenize like mid-sentence words. Questions use up to 160 tokens; verses up to 256.
import torch
import torch.nn.functional as F
from transformers import AutoTokenizer, AutoModel
tok = AutoTokenizer.from_pretrained("BYU-Idaho/ZaraEmbed")
model = AutoModel.from_pretrained("BYU-Idaho/ZaraEmbed", add_pooling_layer=False).eval()
def embed(texts, max_length):
batch = tok([" " + t for t in texts], padding=True, truncation=True,
max_length=max_length, return_tensors="pt")
with torch.no_grad():
hidden = model(**batch).last_hidden_state
mask = batch["attention_mask"].unsqueeze(-1).to(hidden.dtype)
pooled = (hidden * mask).sum(1) / mask.sum(1)
return F.normalize(pooled, dim=-1)
question = embed(["Why does helping other people count as serving God?"], max_length=160)
verses = embed([
"And behold, I tell you these things that ye may learn wisdom; that ye may learn that when ye are in the service of your fellow beings ye are only in the service of your God.", # Mosiah 2:17
"And Cainan lived seventy years, and begat Mahalaleel:", # Genesis 5:12
], max_length=256)
print((question @ verses.T).tolist())
# [[0.7151, -0.0116]] -> Mosiah 2:17 scores far above the unrelated verse (observed on CPU)
For search, embed all verses once, then take the top 100 by cosine similarity and pass them to ZaraRerank.
Model details
- Architecture: RobertaModel, 24 layers, hidden 1024, cased, vocabulary 52,176 (ZaraBERTa's), 1024-d embeddings.
- Base: BYU-Idaho/ZaraBERTa.
- Objective: symmetric InfoNCE with in-batch negatives, temperature 0.05.
- Training: 3 epochs, batch 1,024 through gradient caching, lr 3e-5, bf16, about 5.6 hours on one NVIDIA GB10. This checkpoint is epoch 3, from a single training seed.
- 270,875 training pairs:
- 233,856 question-verse pairs, synthetic questions generated by large language models: 8 plain-language questions per Restoration verse; 8 story-frame questions per Restoration verse, written from the chapter summary and neighbouring verses; earlier synthetic questions; and 1,594 name-disambiguation questions.
- Verse-summary, neighbouring-verse and parallel-passage pairs.
- Hard negatives mined by an earlier ZaraEmbed version, with NV-Retriever positive-aware filtering (negatives scoring above 95% of the gold verse are dropped), plus a same-chapter hard negative for each story-frame question.
pooling.jsonrecords the pooling settings and the per-epoch training history.
Intended uses
- Semantic search over the Standard Works from modern-English questions.
- First-stage retrieval for ZaraRerank or for a retrieval-augmented assistant.
- Verse-to-verse similarity and finding related passages.
Limitations and responsible use
- These models are study aids, not doctrinal authorities, and they are not an official publication of The Church of Jesus Christ of Latter-day Saints.
- Retrieval scores measure textual relevance, not interpretation; read every verse in its context.
- Results come from a single training seed, and the held-out questions were written by a language model.
Provenance and license
Base model roberta-large by Meta AI, MIT license, through ZaraBERTa; that notice applies to any
redistribution of these weights. Training text: public-domain editions of the Old Testament, New
Testament, Book of Mormon, Doctrine and Covenants and Pearl of Great Price. Training questions and chapter
summaries were generated by language models over that text. Chapter headings, footnotes and Church study
aids were not used. Trained and evaluated by Brigham Young University-Idaho, 2026. Released as open weights
under the MIT license.
Citation
@misc{zaraembed2026,
title = {ZaraEmbed: a scripture-native verse embedder for the Standard Works},
author = {Vallejo, Ron and Brigham Young University-Idaho},
year = {2026},
url = {https://huggingface.co/BYU-Idaho/ZaraEmbed}
}
- Downloads last month
- -