DOI: 10.60914/7jr7p-etd91
title: GSAP-ERE

description: GSAP-ERE Dataset

Introduction

GSAP-ERE is a dataset to train and evaluate models for Entity and Relation Extraction of machine learning related entities in scholarly publications (e.g., research papers). Find more information on the GSAP Project on data.gesis.org/gsap.

Data Citation

Please reference:

Wolfgang Otto, Lu Gan, Sharmila Upadhyaya, Saurav Karmakar, Stefan Dietze (2026) GSAP-ERE: Fine-Grained Scholarly Entity and Relation Extraction Focused on Machine Learning. AAAI2026.

Version Information

The annotation is finished on the 15th of April 2025 and can be used to reproduce the results in the connected publication Otto et al. 2026 (mentioned above).

Train/Dev/Test-Split

The dataset was partitioned into training, validation, and test sets with an 80% / 10% / 10% split, respectively, ensuring that all data points from a single publication remained within a single set to prevent data leakage.

Label Sets

Our 10 Named Entity Labels in 4 semantic grouped



  Method related:



MLModel

MLModelGeneric

ModelArchitecture

Method



Data related:



Dataset

DatasetGeneric

DataSource



Task related:



Task



Referencing:



ReferenceLink

URL




Our 18 Relation Labels (incl. domain and range) in 7 semantic groups



Model Design:



Method -usedFor-> Method|MLModel(Generic)

MLModel(Generic)|Method -architecture-> ModelArchitecture

MLModel(Generic) -isBasedOn-> MLModel(Generic)



Task Binding:



MLModel(Generic)|Method -appliedTo-> Task

Dataset(Generic) -benchmarkFor-> Task



Data Usage:



MLModel(Generic)|Method -trainedOn-> Dataset(Generic)

MLModel(Generic)|Method -evaluatedOn-> Dataset(Generic)



Data Provenance:



Dataset(Generic) -transformedFrom-> Dataset(Generic)

Dataset(Generic) -generatedBy-> Method

Dataset(Generic) -sourcedFrom-> DataSource



Data Properties:



Dataset(Generic) -size-> DatasetGeneric

Dataset(Generic) -hasInstanceType-> DatasetGeneric



Peer Relations:



<Any> -coreference-> <Same as Subject>

<Any> -isPartOf-> <Same as Subject>

<Any> -isHyponymOf-> <Same as Subject>

<Any> -isComparedTo-> <Same as Subject>



Referencing:



<Any> -citation-> ReferenceLink

<Any> -url-> URL




 

Format

The Files are encoded in the jsonl format, where each line represents the valid json of one publication.

Data field for each document

The data format of the jsonl files is compatible with many works in the field of entity and relation extraction (e.g., HGERE).

Each line of the jsonl file represents one document containing the following fields:

sentences: A list of sentences represented by a list of tokens (`[[<sentence_1_token_1_id>, <sentence_1_token_2_id>, ...],  [sentence_2_token_2id, ...], ...] (Resolve the word_ids based on the vocabulary given on our github project GSAP-ERE.)

ner: A list of named entities represented by a list of three elements: begin of entity, end of entity, label (e.g., [[<begin_idx>, <end_idx>, "MLModel"], ...] for each sentence. This includes stacked (i.e., overlapping) annotations.

relations :  A list of relation for each sentence. Each relation is represented by the begin and end of subject and object and the relation label for each sentence (e.g., `[<begin_idx_subject>, <end_idx_subject>, <begin_idx_object>, <end_idx_object>, "isPartOf"] `

clusters: This field exists for compatibility reasons. In this version no reference clusters are annotated. This will be reflected in future versions of the dataset.

doc_id: a unique identifier for each document

annotator: Id representing the initial annoator of the document (0 or 1) . During the refinement process other annotators might have corrected some of the annotations.