knowledgator/gliformer-base-v1

GLiFormer Base v1

One encoder for PDF layout processing, entity recognition, classification, relation extraction, structured records, and text embeddings.

knowledgator/gliformer-base-v1 is the 264.2M-parameter base release of GLiFormer. It accepts task labels and extraction schemas at inference time, with task heads sharing a DeBERTa encoder. The layout-aware architecture supports text and document-layout inputs; the usage examples and quality results below focus on text tasks.

Usage

pip install gliformer -U

Or install the GLiFormer framework in a Python 3.10+ environment from the source:

git clone https://github.com/Knowledgator/GLiFormer.git
cd GLiFormer
pip install -e .

Optional CUDA attention kernels are available with pip install -e ".[flash]". CPU inference uses eager attention. The following examples reuse model:

import torch
from gliformer import GLiFormer

model = GLiFormer.from_pretrained(
    "knowledgator/gliformer-base-v1",
    load_tokenizer=True,
)
model = model.to("cuda" if torch.cuda.is_available() else "cpu").eval()

For a local copy, replace the model ID with the checkpoint directory.

Named entity recognition

Specify the entity types at inference time:

text = "Alice works at Acme in London."
entities = model.predict_entities(
    text,
    ["person", "organization", "location"],
    threshold=0.5,
)
for entity in entities:
    print(entity["text"], entity["label"], entity["score"])

Each entity includes text, label, start, end, and score. Offsets are character positions with an exclusive end. Pass a list of texts and batch_size=8 for batched extraction.

Text classification

predictions = model.classify(
    "The new search feature is fast and easy to use.",
    ["positive", "negative", "neutral"],
    threshold=0.5,
)
print(predictions)  # Label dictionaries containing class_name and score.

Named groups are also supported, for example {"sentiment": ["positive", "negative"], "topic": ["product", "support"]}.

Joint relation extraction

Supply entity and relation labels together to use this checkpoint's joint relation head:

results = model.inference(
    "Alice works at Acme.",
    joint_relations={
        "employment": {
            "entities": ["person", "organization"],
            "relations": ["works_at"],
        }
    },
    threshold=0.5,
)
for relation in results["joint_relex"][0]:
    print(relation["head"]["text"], relation["relation"], relation["tail"]["text"])

inference returns a dictionary of task outputs, each containing one result per input text. The separate predict_relations convenience method requires an open relation head; use joint_relations for this model.

Structured extraction

Extract records directly into a Python dictionary:

records = model.structure(
    "Alice works at Acme.",
    {"employee": ["name", "company"]},
)
print(records)
# {'employee': [{'name': 'Alice', 'company': 'Acme'}]}

Nested Pydantic schemas support multilevel records:

from pydantic import BaseModel

class Employee(BaseModel):
    name: str
    role: str

class Department(BaseModel):
    name: str
    employees: list[Employee]

class Company(BaseModel):
    name: str
    departments: list[Department]

records = model.structure(
    "At Acme, Engineering includes Alice, a software engineer, and Bob, "
    "a designer. Sales includes Carol, an account manager.",
    {"company": Company},
    validate_output=True,
)
print(records)

The decoder assembles source-grounded fields and parent–child relationships into nested records. Predictions depend on the schema, input, and thresholds; Pydantic validation checks the output schema, not factual correctness.

Multiple tasks in one call

results = model.inference(
    "Alice joined Acme as a software engineer.",
    entities=["person", "organization"],
    classes=["business", "sports", "technology"],
    structures={"employee": ["name", "company"]},
)
print(results["ner"][0])
print(results["classification"][0])
print(results["structuring"][0])

Text embeddings

import torch.nn.functional as F

embeddings = model.embed_text([
    "A scientist works in a laboratory.",
    "A researcher conducts an experiment.",
])
print(embeddings.shape)  # torch.Size([2, 768])
print(F.cosine_similarity(embeddings[0:1], embeddings[1:2]).item())

Reported evaluation results

Task	Metric	Score
NER, 26 datasets / 131,156 examples	Mean dataset strict entity F1	50.45
CrossNER, 5 domains / 2,505 examples	Mean domain strict entity F1	65.10
Classification, 13 datasets / 79,828 examples	Mean dataset macro-F1	72.36
Multilevel structuring, 500 examples	Order-free, boundary-tolerant JSON F1	87.20

Dataset means weight datasets equally. NER requires both the entity span and type to match. Classification macro-F1 averages class F1 scores within each dataset. Structuring compares flattened JSON value paths after aligning records without requiring their original order and allowing the evaluator's limited boundary repairs; it is not exact JSON match.

Named entity recognition

Dataset	Examples	Precision	Recall	F1
ACE 2004	812	46.54	23.50	31.23
ACE 2005	1,060	39.69	17.93	24.70
AnatEM	3,830	31.98	35.80	33.78
bc2gm	5,000	44.96	54.83	49.41
bc4chemd	26,364	37.50	65.67	47.74
bc5cdr	4,797	59.96	67.65	63.57
Broad Tweet Corpus	2,000	54.44	71.43	61.79
CoNLL 2003	3,453	56.72	69.39	62.42
CrossNER_AI	431	64.43	50.11	56.38
CrossNER_literature	416	66.50	59.13	62.60
CrossNER_music	465	69.06	66.29	67.65
CrossNER_politics	650	71.08	71.64	71.36
CrossNER_science	543	71.86	63.67	67.51
FabNER	2,064	26.53	19.01	22.15
FindVehicle	20,777	41.33	52.97	46.43
GENIA_NER	1,854	44.25	54.76	48.95
HarveyNER	1,303	9.86	24.18	14.01
mit-movie	2,442	61.23	52.98	56.80
mit-restaurant	1,520	45.15	36.67	40.47
MultiNERD	10,000	56.43	88.66	68.97
ncbi	940	49.33	61.44	54.72
Ontonotes	8,262	30.77	35.86	33.12
PolyglotNER	10,000	34.93	68.31	46.23
TweetNER7	576	40.39	48.21	43.96
WikiANN en	10,000	55.22	58.60	56.86
WikiNeural	11,597	72.29	87.15	79.03

Text classification

Dataset	Examples	Accuracy	Macro-F1	Weighted F1
SetFit/CR	376	90.43	89.70	90.45
SetFit/sst2	1,821	90.44	90.44	90.44
SetFit/sst5	2,210	38.05	31.95	34.81
stanfordnlp/imdb	25,000	91.27	91.24	91.24
SetFit/20_newsgroups	7,532	48.67	47.38	48.56
SetFit/enron_spam	2,000	96.70	96.70	96.70
AmazonScience/massive	2,974	69.84	68.18	70.97
PolyAI/banking77	3,080	67.21	66.43	66.43
mteb/financial_phrasebank	1,129	95.31	94.90	95.24
SetFit/ag_news	7,600	82.59	82.26	82.26
dair-ai/emotion	2,000	53.55	48.29	54.63
MoritzLaurer/cap_sotu	23,040	51.11	48.63	51.15
cornell-movie-review-data/rotten_tomatoes	1,066	84.62	84.59	84.59

Micro-F1 equals accuracy in these single-label runs. Reported prediction coverage is 97.15% for 20 Newsgroups and 100% for the other datasets. Summary scores retain the original reports' precision; means of the rounded rows can differ by 0.01.

Joint relation extraction

These runs use predicted entities. Gold counts are relation instances, not documents. Base and large were evaluated on different-sized subsets, so their relation scores are not a controlled comparison on identical examples.

Dataset	Gold relations	Precision	Recall	Micro-F1	Macro-F1
DocRED	1,186	11.80	8.68	10.00	1.03
CrossRE	415	16.22	7.23	10.00	10.10
FewRel	100	18.23	33.00	23.49	26.48
CoNLL04 zero-shot	139	21.25	61.15	31.54	33.59

CoNLL04 zero-shot typed F1, which also checks endpoint entity types, is 29.68%.

Multilevel structuring

Gold JSON depth	Order-free, boundary-tolerant F1
3	85.95
4	91.41
5	86.89
6+	89.95

Evaluation provenance and reproduction

You can find more information on the evaluation methodology here: https://www.knowledgator.com/research

Evaluation entry points are in gliformer_eval. For example, after preparing the CrossNER files, run from the framework repository:

python gliformer_eval/eval_ner.py \
  --model knowledgator/gliformer-base-v1 \
  --data data/NER \
  --datasets CrossNER_AI CrossNER_literature CrossNER_music CrossNER_politics CrossNER_science \
  --output eval_results/gliformer_base_v1_ner.json

Each dataset directory must contain labels.json and test.json. The other task entry points are eval_classification.py, eval_relex.py, and eval_structuring.py; use --help for data paths and inference settings. Reproduction requires matching the original data subsets, schema labels, thresholds, and decoding settings.

Training and intended use

The checkpoint uses the backbone listed above with supervised task heads for information extraction, classification, structuring, and embeddings. See the manuscript for the documented multitask training mixtures. Full checkpoint-specific training provenance is not recorded in the saved evaluation reports.

Use this model for extracting labeled mentions, candidate classes, relations, and structured records from text, and for producing text similarity vectors. The available results cover English tasks. Quality on other languages, document-layout inputs, and embedding benchmarks is not established by the tables above.

Limitations

• Labels, schema wording, domain, input length, and thresholds affect predictions.
• Extraction can omit information, choose incorrect spans, or attach records to the wrong parent.
• Reported NER transfer groups do not establish that every evaluated domain was absent from training.
• Fixed record anchors and the configured span width constrain extraction capacity.
• This checkpoint has no dedicated vision, audio, or open relation head.