Llama-LLEMEtizer-8b-2606

Model Description

pelcra/Llama-LLEMEtizer-8b-2606 is a fully fine-tuned model based on meta-llama/Meta-Llama-3.1-8B, designed for Early Modern English lemmatisation and part-of-speech tagging.

Given an Early Modern English sentence, the model returns a JSON array containing one object per word. Each object includes:

• the original word or punctuation mark,
• the assigned part-of-speech tag,
• the normalised lemma,
• whether a space follows the token in the original input.

The model is intended for historical linguistics, corpus annotation, digital humanities, and Early Modern English text-processing workflows.

Model Details

• Model type: Causal language model / instruction-following text generation model
• Base model: meta-llama/Meta-Llama-3.1-8B
• Architecture: Llama 3.1
• Parameters: 8B
• Fine-tuning method: Full fine-tuning
• Training examples: ~18,000 sentence instruction samples (~277,000 words)
• Task: Early Modern English lemmatisation and POS tagging
• Output format: JSON array
• Format: Safetensors, sharded
• Number of shards: 4
• Language: English, with a focus on Early Modern English
• License: Llama 3.1 Community License

Intended Use

This model is intended for:

• lemmatising Early Modern English sentences,
• assigning simplified POS tags to Early Modern English tokens,
• producing structured token-level JSON annotations,
• supporting historical linguistics and digital humanities research,
• assisting corpus annotation and normalisation workflows.

It is not intended for:

• general-purpose chat,
• high-stakes automated decision-making,
• legal, medical, financial, or safety-critical use cases,
• use as a substitute for expert philological or linguistic judgement.

Input and Output Format

The model expects a sentence as input and returns a JSON array with one object per token.

Each output object has the following structure:

{
  "word": "<original token>",
  "POS": "<part-of-speech tag>",
  "lemma": "<normalised lemma>",
  "space_after": true
}

The space_after field indicates whether a space followed the token in the original text.

Example

Input

That all your concerns may have blessing and success is earnestly praied by, my honord Lord, Your Lordships Most humble and affectionat ser=vt=, Jo.

Output

[
  {"word": "That", "POS": "CONJ*", "lemma": "that", "space_after": true},
  {"word": "all", "POS": "Q*", "lemma": "all", "space_after": true},
  {"word": "your", "POS": "PRO$", "lemma": "your", "space_after": true},
  {"word": "concerns", "POS": "N*", "lemma": "concern", "space_after": true},
  {"word": "may", "POS": "MD*", "lemma": "may", "space_after": true},
  {"word": "have", "POS": "H*", "lemma": "have", "space_after": true},
  {"word": "blessing", "POS": "N*", "lemma": "blessing", "space_after": true},
  {"word": "and", "POS": "CONJ*", "lemma": "and", "space_after": true},
  {"word": "success", "POS": "N*", "lemma": "success", "space_after": true},
  {"word": "is", "POS": "BE*", "lemma": "be", "space_after": true},
  {"word": "earnestly", "POS": "ADV*", "lemma": "earnestly", "space_after": true},
  {"word": "praied", "POS": "V*", "lemma": "pray", "space_after": true},
  {"word": "by", "POS": "P", "lemma": "by", "space_after": false},
  {"word": ",", "POS": ".", "lemma": ",", "space_after": true},
  {"word": "my", "POS": "PRO$", "lemma": "my", "space_after": true},
  {"word": "honord", "POS": "V*", "lemma": "honour", "space_after": true},
  {"word": "Lord", "POS": "N*", "lemma": "lord", "space_after": false},
  {"word": ",", "POS": ".", "lemma": ",", "space_after": true},
  {"word": "Your", "POS": "PRO$", "lemma": "your", "space_after": true},
  {"word": "Lordships", "POS": "N*", "lemma": "lordship", "space_after": true},
  {"word": "Most", "POS": "Q*", "lemma": "most", "space_after": true},
  {"word": "humble", "POS": "ADJ*", "lemma": "humble", "space_after": true},
  {"word": "and", "POS": "CONJ*", "lemma": "and", "space_after": true},
  {"word": "affectionat", "POS": "ADJ*", "lemma": "affectionate", "space_after": true},
  {"word": "ser=vt=", "POS": "N*", "lemma": "servant", "space_after": false},
  {"word": ",", "POS": ".", "lemma": ",", "space_after": true},
  {"word": "Jo.", "POS": "NPR*", "lemma": "john", "space_after": false}
]

How to Use

import re

from transformers import AutoTokenizer, AutoModelForCausalLM


SYSTEM_MESSAGE = """You are a historical linguistics professor specialising in Early Modern English. Your task is to tokenise, assign parts of speech, and lemmatise every word in a given sentence with precision and scholarly accuracy."""

TASK_DESCRIPTION = """Definition:
A lemma is the base morphological form of a word — the canonical form listed at the head of a dictionary entry. Lemmas must conform to British spelling conventions as defined by the Oxford English Dictionary.

---

Instructions:

1. Contextual Analysis (Internal)
Before lemmatising each word, carefully consider it in its full sentential context. Reflect on:
- How the word functions grammatically in the sentence.
- Its semantic role and meaning.
- Any morphological or orthographic features that may influence the lemma.
- The historical spelling conventions of Early Modern English.

This analysis is internal only. Do not output any explanation or reasoning.

---

2. Part of Speech Assignment
Assign each word exactly one of the following tags:

| Tag      | Description                                   |
|----------|-----------------------------------------------|
| N*       | Noun                                          |
| NPR*     | Proper noun                                   |
| PRO*     | Personal pronoun                              |
| PRO$     | Possessive pronoun                            |
| W*       | Wh-word                                       |
| D*       | Article or demonstrative pronoun              |
| ADJ*     | Adjective                                     |
| ADV*     | Adverb                                        |
| Q*       | Quantifier                                    |
| NUM*     | Numeral (digits and number words)             |
| V*       | Lexical verb                                  |
| BE*      | Verb "to be"                                  |
| H*       | Verb "to have"                                |
| DO*      | Verb "to do"                                  |
| MD*      | Modal verb                                    |
| TO       | Infinitival "to"                              |
| NEG*     | Negative particle                             |
| RP       | Verbal particle                               |
| P        | Preposition                                   |
| CONJ*    | Conjunction                                   |
| GEN      | Genitive marker                               |
| INTJ     | Interjection                                  |
| FW       | Foreign word                                  |
| +        | Compound or merged phrase                     |
| .        | Punctuation mark                              |
| UNKNOWN  | Unknown                                       |

---

3. Lemmatisation Rules

General Rules:
- Verbs: reduce to the bare infinitive (without "to").
- Nouns: reduce to the singular, non-possessive form.
- Adjectives: reduce to the positive (base) degree.
- Adverbs: reduce to the base adverb form.
- The lemma must always share the same morphological root as the original word.
- The lemma must always match the part of speech of the original word.
- Punctuation marks: use the punctuation character itself as the lemma.

Important corollaries:
- Nouns derived from another POS must remain nouns (e.g., "blessing" -> "blessing", not "bless").
- Adverbs must remain adverbs (e.g., "modernly" -> "modernly").
- Gerunds and present/past participles must be lemmatised as verbs (e.g., "remembring" -> "remember").
- Do not substitute a suppletive root: "better" -> "better" (not "good"); "worse" -> "worse" (not "bad"); "further" -> "further" (not "far"); "people" -> "people" (not "person"); "these" -> "these" (not "this"); "those" -> "those" (not "that"); "themselves" -> "themselves" (not "themself").
- "any" used pronominally or as a modifier is tagged Q*; "such" used as a modifier is tagged D*.

Exceptions:
- All forms of "be" (is, was, were, art, wast, etc.) -> "be".
- Modal verbs must be converted to their present-tense base form: "could" -> "can"; "would" -> "will"; "should" -> "shall"; "might" -> "may".
- Foreign words: output only <foreign-word> as the lemma.

---

4. Spelling and Proper Nouns
- Return all lemmas in lowercase, including proper nouns.
- Normalise historical and archaic spellings to their modern British equivalents (e.g., "hardnesse" -> "hardness"; "lyuynge" -> "live"; "y=e=" -> "the").
- Standardise proper nouns to their modern recognised British forms where identifiable (e.g., "Thamys" -> "thames"). If not identifiable, preserve the given form in lowercase.
- Strip possessive inflections from proper nouns (e.g., "Thames's" -> "thames"; "Amelias" -> "amelia").

---

5. Output Format
Process every word in the input sentence in order, including punctuation marks. Note that & when functioning as a conjunction must be tagged CONJ*. Return a JSON array, one object per word:

[
  {
    "word": "<original word as it appears>",
    "POS": "<assigned POS tag>",
    "lemma": "<lemmatised base form>",
    "space_after": <true if a space follows this word in the original text, false otherwise>
  },
  ...
]

Return the JSON array wrapped in a ```json``` code block. Do not include any explanation or preamble.

---

Sentence to lemmatise:
<SENTENCE>"""


def normalise_sentence(sentence: str) -> str:
    """Strip space before punctuation to match real pasted-text format."""
    return re.sub(r' ([,\.;:!\?\)\]\'"])', r'\1', sentence)


model_id = "pelcra/Llama-LLEMEtizer-8b-2606"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype="auto",
)

sentence = (
    "That all your concerns may have blessing and success is earnestly "
    "praied by, my honord Lord, Your Lordships Most humble and affectionat "
    "ser=vt=, Jo."
)

messages = [
    {"role": "system", "content": SYSTEM_MESSAGE},
    {
        "role": "user",
        "content": TASK_DESCRIPTION.replace(
            "<SENTENCE>",
            normalise_sentence(sentence),
        ),
    },
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=1024,
    temperature=0.1,
    do_sample=False,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Part-of-Speech Tagset

The model uses the following POS tags:

Tag	Description
N*	Noun
NPR*	Proper noun
PRO*	Personal pronoun
PRO$	Possessive pronoun
W*	Wh-word
D*	Article or demonstrative pronoun
ADJ*	Adjective
ADV*	Adverb
Q*	Quantifier
NUM*	Numeral, including digits and number words
V*	Lexical verb
BE*	Verb “to be”
H*	Verb “to have”
DO*	Verb “to do”
MD*	Modal verb
TO	Infinitival “to”
NEG*	Negative particle
RP	Verbal particle
P	Preposition
CONJ*	Conjunction
GEN	Genitive marker
INTJ	Interjection
FW	Foreign word
+	Compound or merged phrase
.	Punctuation mark
UNKNOWN	Unknown

Lemmatisation Principles

The model was trained to follow task-specific lemmatisation rules, including:

• verbs are reduced to the bare infinitive,
• nouns are reduced to the singular non-possessive form,
• adjectives are reduced to the positive degree,
• adverbs are reduced to the base adverb form,
• punctuation marks use the punctuation character itself as the lemma,
• lemmas are lowercased, including proper nouns,
• historical and archaic spellings are normalised to modern British forms where possible,
• the lemma must preserve the same morphological root and part of speech as the original token,
• forms of “be” are lemmatised as be,
• modal verbs are normalised to their present-tense base form where applicable,
• foreign words use <foreign-word> as the lemma.

Training Data

The source texts come from the Penn-Helsinki Parsed Corpus of Early Modern English (PPCEME).

The training data was produced through a multi-stage annotation workflow:

1. PPCEME text samples were lemmatised using carefully curated prompts designed specifically for Early Modern English lemmatisation.
2. Multiple prompts were used for different linguistic situations and edge cases.
3. Additional cases were handled programmatically where appropriate.
4. Every row was manually analysed and corrected by at least two people. There was a group of trained annotators with extensive Early Modern English lemmatisation knowledge. On top of that, an expert supervisor validated every decision.
5. A subset of ~18,000 sentence level instruction samples was selected for full fine-tuning. This resulted in ~277,000 words for the training phase.

The resulting dataset is task-specific and focused on Early Modern English lemmatisation, POS tagging, and whitespace-aware tokenisation.

Training Procedure

• Fine-tuning method: Full fine-tuning
• Base model: meta-llama/Meta-Llama-3.1-8B
• Training samples: ~18,000
• Training objective: Instruction-following generation of structured JSON annotations

Evaluation

The model was evaluated on three held-out datasets.

Two external evaluation datasets were constructed from texts drawn from the Early English Books Online corpus and the Corpus of Early English Correspondence. These were designed to test performance outside the training corpus:

1. a continuous-text dataset of approximately 2,000 words, imitating a longer passage from early English books;
2. a fragmented dataset consisting of selected words and sentences drawn from different places.

An additional internal held-out test set was created from the PPCEME corpus and excluded from training.

Evaluation dataset	Description	Accuracy	Samples
External continuous-text dataset	Continuous passage-style data from Early English Books Online / Corpus of Early English Correspondence	97.05%	2,000
External fragmented dataset	Selected words and sentences from different places in Early English Books Online / Corpus of Early English Correspondence	95.33%	1,178
Internal PPCEME held-out test set	PPCEME samples held out from training	95.43%	920

Accuracy is reported over the evaluated samples. The external datasets are intended to give a broader estimate of model performance on Early Modern English material that falls outside the PPCEME training distribution. The results show that the model generalises well beyond its training data, and performance is in fact stronger on continuous, naturally occurring texts than on the difficult examples deliberately selected to test edge cases.

Limitations

This model may:

• produce malformed JSON, especially on very long or unusual inputs,
• misanalyse highly ambiguous Early Modern English forms,
• fail on spellings or abbreviations not represented in the training distribution,
• incorrectly normalise rare proper nouns,
• produce plausible but incorrect lemmas,
• require human review for scholarly publication-quality annotation.

The model is intended as an annotation aid and should not be treated as an infallible linguistic authority.

Bias, Risks, and Safety

The model is specialised for historical linguistic annotation and is not designed as a general-purpose assistant. Its outputs may reflect the composition, editorial conventions, and annotation choices present in the source corpus and manually corrected training data.

Users should evaluate the model carefully before using it in research pipelines, published corpora, or downstream quantitative linguistic analysis.

Files

This repository contains:

chat_template.jinja
config.json
generation_config.json
model.safetensors.index.json
model-00001-of-00004.safetensors
model-00002-of-00004.safetensors
model-00003-of-00004.safetensors
model-00004-of-00004.safetensors
special_tokens_map.json
tokenizer.json
tokenizer_config.json
README.md

These files are everything needed to run the model locally on one's own hardware.

License

This model is based on meta-llama/Meta-Llama-3.1-8B and is released under the Llama 3.1 Community License.

Use of this model is subject to the Llama 3.1 Community License and the Llama 3.1 Acceptable Use Policy. Please review the original Llama 3.1 license terms before using, modifying, or redistributing this model.

Required notice:

Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.

Citation

If you use this model, please cite:

[soon]

Please also cite the Penn-Helsinki Parsed Corpus of Early Modern English (PPCEME), Early English Books Online, and the Corpus of Early English Correspondence where appropriate.