LexisLM-500m-Instruct

LexisLM-500m-Instruct is a Greek-focused instruction model fine-tuned from the 502-million-parameter LexisLM-500M base checkpoint. It is a decoder-only model exported in BF16 and compatible with the Transformers Llama architecture.

Model details

Property	Value
Layers	16
Hidden size	1,536
Attention heads	16
Context length	4,096 tokens
Vocabulary	32,004 tokens
Tokenizer	Greek byte-level BPE with chat tokens
Fine-tuning checkpoint	runs/greek_500m_22bn_ctx4k_sft_mixed/best.pt, step 3,802
Fine-tuning data	Greek EuroAlpaca, Aya Greek SFT, and Greek chat SFT mixtures

The chat template prepends <eos>, formats each message as <|role|> followed by a newline and content, and ends each turn with <|endofturn|>. Generation stops at <eos> or <|endofturn|>.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "ComputationalLinguisticsPlatform/LexisLM-500m-Instruct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
torch.set_num_threads(4)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32)
torch.manual_seed(42)
messages = [{"role": "user", "content": "Η πρωτεύουσα της Ελλάδας είναι"}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_tensors="pt", return_dict=True,
)
inputs = {key: value.to(model.device) for key, value in inputs.items()
          if key in ("input_ids", "attention_mask")}
output = model.generate(
    **inputs, max_new_tokens=64, do_sample=True, temperature=0.1, top_p=0.9,
    eos_token_id=[tokenizer.eos_token_id,
                  tokenizer.convert_tokens_to_ids("<|endofturn|>")],
    pad_token_id=tokenizer.pad_token_id,
)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Output from this example (CPU FP32, seed 42):

Η πρωτεύουσα της Ελλάδας είναι η Αθήνα.

Evaluation

The selected SFT checkpoint was evaluated with this project's Greek answer-likelihood protocol. ARC used 25 demonstrations, MMLU used 5, and TruthfulQA used 0. These scores are specific to that protocol and are not instruction-following evaluations.

Benchmark	Metric	Score
ARC-Easy	Character-normalized accuracy	41.58%
ARC-Challenge	Character-normalized accuracy	25.26%
MMLU	Accuracy	26.04%
TruthfulQA	MC1 accuracy	23.13%
TruthfulQA	MC2 score	41.35%

Limitations

The model may produce incorrect, biased, or unsafe text. Greek is the primary target language. The benchmarks above do not establish reliability for a specific application.

Provenance and license

Pretraining used Greek Wikipedia and FineWeb2, which is released under ODC-By 1.0. Supervised fine-tuning used the three data mixtures listed above, including EuroAlpaca. EuroAlpaca is licensed under CC BY-NC 4.0, so this instruction-tuned model is released for noncommercial use under CC BY-NC 4.0. The export was produced with scripts/export_hf.py from the selected training checkpoint.