hannah-khallaf/e2r-strategy-multilingual-e5-large-bce

Multilingual Easy-to-Read Strategy Classifier: Multilingual-E5 Large

This model identifies the simplification strategies realised between a standard-language sentence and its Easy-to-Read rewrite.

It is a multilingual sentence-pair multi-label classifier covering Arabic, Catalan, English, French, Italian and Spanish.

Predicted strategies

The model predicts any combination of the following six strategies:

1. Synonymy
2. Modulation
3. Compression
4. Explanation
5. Syntactic Change
6. Omission

The label order is fixed and must not be changed.

Model architecture

The model uses:

• intfloat/multilingual-e5-large as the multilingual encoder;
• paired input consisting of the standard sentence and its Easy-to-Read rewrite;
• the prefix query: on both input sequences;
• first-token/CLS pooling;
• dropout of 0.1;
• one linear classification layer with six independent outputs;
• sigmoid probabilities for multi-label prediction;
• a global decision threshold of 0.23;
• a maximum input length of 512 tokens.

The classification architecture is:

Multilingual-E5 Large
→ first-token/CLS representation
→ dropout
→ linear layer with six outputs
→ sigmoid probabilities
→ threshold at 0.23

The model was trained using binary cross-entropy without positive-label weighting.

Released checkpoint

This repository contains seed 87, selected using development Macro-F1.

Result	Value
Selected seed	87
Best epoch	4
Development Macro-F1	69.68
Three-seed test Macro-F1, mean ± SD	61.35 ± 1.09
Global decision threshold	0.23

The development result refers to the released seed. The test result is the mean and standard deviation across seeds 13, 42 and 87.

Installation

pip install torch transformers

Important preprocessing

The prefix query: must be added to both sequences:

query: <standard sentence>
query: <Easy-to-Read rewrite>

The standard sentence must be supplied first and the Easy-to-Read rewrite second.

Basic usage

This repository contains the exact custom classification architecture, so load it with trust_remote_code=True.

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "hannah-khallaf/e2r-strategy-multilingual-e5-large-bce"

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True,
)

model = AutoModelForSequenceClassification.from_pretrained(
    model_id,
    trust_remote_code=True,
)

standard_sentence = (
    "The committee postponed the implementation of the measure."
)

easy_to_read_rewrite = (
    "The committee decided to use the measure later."
)

inputs = tokenizer(
    "query: " + standard_sentence,
    "query: " + easy_to_read_rewrite,
    return_tensors="pt",
    truncation=True,
    max_length=512,
)

model.eval()

with torch.inference_mode():
    logits = model(**inputs).logits
    probabilities = torch.sigmoid(logits)[0]

labels = [
    model.config.id2label[index]
    for index in range(model.config.num_labels)
]

thresholds = model.config.e2r_classifier["thresholds"]

scores = {
    label: float(probability)
    for label, probability in zip(labels, probabilities)
}

predicted_labels = [
    label
    for label in labels
    if scores[label] >= float(thresholds[label])
]

print("Predicted strategies:", predicted_labels)
print("Scores:", scores)

Reusable prediction function

from __future__ import annotations

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer


MODEL_ID = "hannah-khallaf/e2r-strategy-multilingual-e5-large-bce"

tokenizer = AutoTokenizer.from_pretrained(
    MODEL_ID,
    trust_remote_code=True,
)

model = AutoModelForSequenceClassification.from_pretrained(
    MODEL_ID,
    trust_remote_code=True,
)

model.eval()


def classify_rewrite(
    standard_sentence: str,
    easy_to_read_rewrite: str,
) -> dict:
    inputs = tokenizer(
        "query: " + standard_sentence,
        "query: " + easy_to_read_rewrite,
        return_tensors="pt",
        truncation=True,
        max_length=512,
    )

    with torch.inference_mode():
        logits = model(**inputs).logits
        probabilities = torch.sigmoid(logits)[0]

    labels = [
        model.config.id2label[index]
        for index in range(model.config.num_labels)
    ]

    thresholds = model.config.e2r_classifier["thresholds"]

    scores = {
        label: float(probability)
        for label, probability in zip(labels, probabilities)
    }

    predicted_labels = [
        label
        for label in labels
        if scores[label] >= float(thresholds[label])
    ]

    return {
        "predicted_labels": predicted_labels,
        "scores": scores,
        "thresholds": thresholds,
    }

Input requirements

The model was trained using the following exact input contract:

inputs = tokenizer(
    "query: " + standard_sentence,
    "query: " + easy_to_read_rewrite,
    return_tensors="pt",
    truncation=True,
    max_length=512,
)

Do not omit the prefixes or reverse the sentence order.

Output format

The model returns six logits in the following exact order:

Synonymy
Modulation
Compression
Explanation
Syntactic Change
Omission

Apply torch.sigmoid() to obtain probabilities. A strategy is predicted when its probability is at least 0.23.

The thresholds are also available from:

model.config.e2r_classifier["thresholds"]

Intended use

The model is intended for:

• multilingual Easy-to-Read research;
• analysis of rewriting and simplification strategies;
• automatic annotation support;
• corpus exploration;
• comparison of standard and simplified sentence pairs.

Its predictions should be reviewed by a qualified annotator when used in research or data creation.

Out-of-scope use

The model should not be used as evidence that a text is accessible to a particular person or reader group.

It should not be used to make medical, legal, educational or administrative decisions about individuals.

Limitations

The model assigns labels at sentence level. It does not:

• identify the exact words or spans responsible for a strategy;
• evaluate whether the rewrite preserves the original meaning;
• measure the overall quality of the rewrite;
• certify compliance with Easy-to-Read guidelines;
• establish accessibility for a particular reader.

Performance differences between languages may reflect differences in corpus, domain, simplification practice, annotation procedure and label harmonisation. They should not be interpreted as inherent differences between languages.

Training languages

• Arabic
• Catalan
• English
• French
• Italian
• Spanish

Reproducibility files

The repository includes:

• the selected model weights in SafeTensors format;
• the exact custom Transformers model class;
• the fixed label order;
• the selected global threshold;
• the preprocessing contract;
• the resolved training configuration;
• the selected seed's development summary;
• the selected seed's test summary.

The exported Transformers model was checked against the original PairClassifier using the same prefixed sentence-pair input. The original and exported implementations produced identical logits, probabilities and thresholded predictions.

Citation

Please cite the accompanying paper when using this model. The complete bibliographic entry will be added after publication.