🦦 otter-cross-mmbert

Otter is a multilingual, open-type named entity recognizer. You give it a piece of text and a list of entity types in plain language -- ["person", "band", "chemical compound"] -- and it returns the character spans of the entities of those types. There is no fixed label set and no fine-tuning step: the types are part of the input.

Cross-encoder. The entity types are prepended to the text as a [LABEL] <type> ... [SEP] prefix, so a single encoder sees labels and text together and they attend to each other. More accurate than the bi-encoder, at the cost of re-encoding the text for every label set.

• Encoder: jhu-clsp/mmBERT-base (vocabulary extended with a [LABEL] token)
• Max sequence length: 1024 tokens
• Max span length: 30 tokens

Usage

from transformers import AutoModel

model = AutoModel.from_pretrained("whoisjones/otter-cross-mmbert", trust_remote_code=True)
model.eval()

entities = model.predict(
    "Angela Merkel besuchte gestern das Brandenburger Tor in Berlin.",
    labels=['person', 'organization', 'location'],
)

for entity in entities:
    print(f"{entity['text']!r:25} {entity['label']:15} {entity['score']:.2f}")

'Angela Merkel'           person          0.99
'Brandenburger Tor'       location        0.82
'Berlin'                  location        0.90

Each entity is a dict with text, label, start, end (character offsets into the input string) and score. Pass a list of strings to run on a batch; you then get one list of entities per input, in the same order:

model = model.to("cuda")

texts = ["Angela Merkel besuchte das Brandenburger Tor.", "Sony was founded in Tokyo."]
results = model.predict(texts, labels=["person", "organization", "location"], batch_size=16)

Threshold

predict keeps spans scoring above threshold, which defaults to config.prediction_threshold (0.5 for this checkpoint, chosen by calibrating macro-F1 across the evaluation suite). Lower it for higher recall, raise it for higher precision:

entities = model.predict(text, labels=labels, threshold=0.1)

Because the label set is part of the input, the useful threshold shifts with how many types you ask for and how specific they are. If you have a few hundred annotated sentences from your own domain, re-calibrating on those is worth more than any default.

Writing good label names

The label is read as natural language, so it carries meaning. "politician" and "person" select different spans, and a phrase like "chemical compound" works as well as a single word. Prefer the wording you would use to describe the type to a person.

Prompt format

The model is trained on inputs of the form

[LABEL] person [LABEL] organization [SEP] John Doe works at OpenAI.

model.build_prompt(labels) returns that prefix if you want to build inputs yourself. Note that the prefix counts against max_seq_length, so very long label sets leave less room for the text.

Fine-tuning

collate_fn.py in this repository holds the training and evaluation collators. See the GitHub repository for the full training pipeline, the evaluation suite, and the data preparation scripts.

Model family

Model	Architecture	Encoder
whoisjones/otter-bi-mmbert	bi-encoder	mmBERT-base
whoisjones/otter-cross-mmbert	cross-encoder	mmBERT-base
whoisjones/otter-bi-rembert	bi-encoder	RemBERT
whoisjones/otter-cross-rembert	cross-encoder	RemBERT

The cross-encoders are the stronger models; the bi-encoders are cheaper when one label set is applied across a large corpus.

License

Apache 2.0.