knowledgator/gliformer-large-v1

GLiFormer Large v1

One encoder for PDF layout processing, entity recognition, classification, relation extraction, structured records, and text embeddings.

knowledgator/gliformer-large-v1 is the 575.6M-parameter large release of GLiFormer. It accepts task labels and extraction schemas at inference time, with task heads sharing a DeBERTa encoder. The layout-aware architecture supports text and document-layout inputs; the usage examples and quality results below focus on text tasks.

Usage

pip install gliformer -U

Or install the GLiFormer framework in a Python 3.10+ environment from the source:

git clone https://github.com/Knowledgator/GLiFormer.git
cd GLiFormer
pip install -e .

Optional CUDA attention kernels are available with pip install -e ".[flash]". CPU inference uses eager attention. The following examples reuse model:

import torch
from gliformer import GLiFormer

model = GLiFormer.from_pretrained(
    "knowledgator/gliformer-large-v1",
    load_tokenizer=True,
)
model = model.to("cuda" if torch.cuda.is_available() else "cpu").eval()

For a local copy, replace the model ID with the checkpoint directory.

Named entity recognition

Specify the entity types at inference time:

text = "Alice works at Acme in London."
entities = model.predict_entities(
    text,
    ["person", "organization", "location"],
    threshold=0.5,
)
for entity in entities:
    print(entity["text"], entity["label"], entity["score"])

Each entity includes text, label, start, end, and score. Offsets are character positions with an exclusive end. Pass a list of texts and batch_size=8 for batched extraction.

Text classification

predictions = model.classify(
    "The new search feature is fast and easy to use.",
    ["positive", "negative", "neutral"],
    threshold=0.5,
)
print(predictions)  # Label dictionaries containing class_name and score.

Named groups are also supported, for example {"sentiment": ["positive", "negative"], "topic": ["product", "support"]}.

Joint relation extraction

Supply entity and relation labels together to use this checkpoint's joint relation head:

results = model.inference(
    "Alice works at Acme.",
    joint_relations={
        "employment": {
            "entities": ["person", "organization"],
            "relations": ["works_at"],
        }
    },
    threshold=0.5,
)
for relation in results["joint_relex"][0]:
    print(relation["head"]["text"], relation["relation"], relation["tail"]["text"])

inference returns a dictionary of task outputs, each containing one result per input text. The separate predict_relations convenience method requires an open relation head; use joint_relations for this model.

Structured extraction

Extract records directly into a Python dictionary:

records = model.structure(
    "Alice works at Acme.",
    {"employee": ["name", "company"]},
)
print(records)
# {'employee': [{'name': 'Alice', 'company': 'Acme'}]}

Nested Pydantic schemas support multilevel records:

from pydantic import BaseModel

class Employee(BaseModel):
    name: str
    role: str

class Department(BaseModel):
    name: str
    employees: list[Employee]

class Company(BaseModel):
    name: str
    departments: list[Department]

records = model.structure(
    "At Acme, Engineering includes Alice, a software engineer, and Bob, "
    "a designer. Sales includes Carol, an account manager.",
    {"company": Company},
    validate_output=True,
)
print(records)

The decoder assembles source-grounded fields and parent–child relationships into nested records. Predictions depend on the schema, input, and thresholds; Pydantic validation checks the output schema, not factual correctness.

Multiple tasks in one call

results = model.inference(
    "Alice joined Acme as a software engineer.",
    entities=["person", "organization"],
    classes=["business", "sports", "technology"],
    structures={"employee": ["name", "company"]},
)
print(results["ner"][0])
print(results["classification"][0])
print(results["structuring"][0])

Text embeddings

import torch.nn.functional as F

embeddings = model.embed_text([
    "A scientist works in a laboratory.",
    "A researcher conducts an experiment.",
])
print(embeddings.shape)  # torch.Size([2, 1024])
print(F.cosine_similarity(embeddings[0:1], embeddings[1:2]).item())

Reported evaluation results

All scores below are percentages. These are the existing GLiFormer-large measurements reported in the project's evaluation notes and manuscript; they were not rerun for this model card. The historical reports do not record checkpoint hashes, so they cannot establish an exact binary match to this release.

Task	Metric	Score
NER, 26 datasets / 131,156 examples	Mean dataset strict entity F1	50.91
CrossNER, 5 domains / 2,505 examples	Mean domain strict entity F1	64.35
Classification, 13 datasets / 79,828 examples	Mean dataset macro-F1	75.03
Multilevel structuring, 500 examples	Order-free, boundary-tolerant JSON F1	91.10

Dataset means weight datasets equally. NER requires both the entity span and type to match. Classification macro-F1 averages class F1 scores within each dataset. Structuring compares flattened JSON value paths after aligning records without requiring their original order and allowing the evaluator's limited boundary repairs; it is not exact JSON match. The manuscript corrects the structuring evaluation size to 500; historical notes contain obsolete bucket counts totaling 300.

Named entity recognition

Dataset	Examples	Precision	Recall	F1
ACE 2004	812	51.67	29.00	37.15
ACE 2005	1,060	44.88	22.42	29.90
AnatEM	3,830	26.18	31.74	28.70
bc2gm	5,000	45.46	51.53	48.30
bc4chemd	26,364	40.74	67.58	50.84
bc5cdr	4,797	61.92	72.05	66.60
Broad Tweet Corpus	2,000	55.55	70.99	62.33
CoNLL 2003	3,453	58.96	72.82	65.16
CrossNER_AI	431	51.69	51.83	51.76
CrossNER_literature	416	63.11	60.10	61.57
CrossNER_music	465	67.48	66.17	66.82
CrossNER_politics	650	70.46	72.52	71.48
CrossNER_science	543	71.63	68.69	70.13
FabNER	2,064	27.17	18.47	21.99
FindVehicle	20,777	42.11	51.21	46.21
GENIA_NER	1,854	48.42	57.23	52.46
HarveyNER	1,303	9.95	22.54	13.81
mit-movie	2,442	63.10	52.98	57.60
mit-restaurant	1,520	38.88	30.25	34.03
MultiNERD	10,000	54.96	91.91	68.79
ncbi	940	47.63	64.05	54.63
Ontonotes	8,262	28.20	42.18	33.80
PolyglotNER	10,000	35.52	70.85	47.31
TweetNER7	576	43.46	48.62	45.90
WikiANN en	10,000	54.85	57.74	56.26
WikiNeural	11,597	73.93	87.53	80.16

Text classification

Dataset	Examples	Accuracy	Macro-F1	Weighted F1
SetFit/CR	376	91.22	90.45	91.20
SetFit/sst2	1,821	92.97	92.97	92.97
SetFit/sst5	2,210	44.34	40.33	43.39
stanfordnlp/imdb	25,000	93.94	93.93	93.93
SetFit/20_newsgroups	7,532	57.94	57.18	58.70
SetFit/enron_spam	2,000	97.95	97.95	97.95
AmazonScience/massive	2,974	71.32	69.98	71.93
PolyAI/banking77	3,080	70.97	70.55	70.55
mteb/financial_phrasebank	1,129	97.25	96.77	97.24
SetFit/ag_news	7,600	82.07	81.53	81.53
dair-ai/emotion	2,000	54.75	48.07	55.59
MoritzLaurer/cap_sotu	23,040	51.74	49.00	51.15
cornell-movie-review-data/rotten_tomatoes	1,066	86.68	86.68	86.68

Micro-F1 equals accuracy in these single-label runs. Reported prediction coverage is 97.15% for 20 Newsgroups and 100% for the other datasets. Summary scores retain the original reports' precision; means of the rounded rows can differ by 0.01.

Joint relation extraction

These runs use predicted entities. Gold counts are relation instances, not documents. Base and large were evaluated on different-sized subsets, so their relation scores are not a controlled comparison on identical examples.

Dataset	Gold relations	Precision	Recall	Micro-F1	Macro-F1
DocRED	6,003	29.90	8.13	12.78	3.64
CrossRE	1,926	22.76	8.72	12.61	11.72
FewRel	500	21.86	26.80	24.08	21.40
CoNLL04 zero-shot	677	38.47	33.53	35.83	35.32

CoNLL04 zero-shot typed F1, which also checks endpoint entity types, is 34.73%.

Multilevel structuring

Gold JSON depth	Order-free, boundary-tolerant F1
3	89.94
4	95.11
5	91.69
6+	92.80

Evaluation provenance and reproduction

The evaluation notes record base results under post1 and large results under glinext-large. The manuscript attributes those runs to GLiFormer-base and GLiFormer-large and supplies the corrected structuring results. The original GLiNExT naming in historical reports predates the framework rename. No synthetic placeholder scores are included here.

Evaluation entry points are in gliformer_eval. For example, after preparing the CrossNER files, run from the framework repository:

python gliformer_eval/eval_ner.py \
  --model knowledgator/gliformer-large-v1 \
  --data data/NER \
  --datasets CrossNER_AI CrossNER_literature CrossNER_music CrossNER_politics CrossNER_science \
  --output eval_results/gliformer_large_v1_ner.json

Each dataset directory must contain labels.json and test.json. The other task entry points are eval_classification.py, eval_relex.py, and eval_structuring.py; use --help for data paths and inference settings. Reproduction requires matching the original data subsets, schema labels, thresholds, and decoding settings.

Training and intended use

The checkpoint uses the backbone listed above with supervised task heads for information extraction, classification, structuring, and embeddings. See the manuscript for the documented multitask training mixtures. Full checkpoint-specific training provenance is not recorded in the saved evaluation reports.

Use this model for extracting labeled mentions, candidate classes, relations, and structured records from text, and for producing text similarity vectors. The available results cover English tasks. Quality on other languages, document-layout inputs, and embedding benchmarks is not established by the tables above.

Limitations

• Labels, schema wording, domain, input length, and thresholds affect predictions.
• Extraction can omit information, choose incorrect spans, or attach records to the wrong parent.
• Reported NER transfer groups do not establish that every evaluated domain was absent from training.
• Fixed record anchors and the configured span width constrain extraction capacity.
• This checkpoint has no dedicated vision, audio, or open relation head.