Omkar2118/datasets huggingface dataset:
EC-FUNSD

EC-FUNSD is introduced in  [[arXiv:2402.02379]](https://arxiv.org/abs/2402.02379) as a benchmark of semantic entity recognition (SER) and entity linking (EL), designed for the entity-centric robustness evaluation of pre-trained text-and-layout models (PTLMs).

In practical applications of document intelligence, PTLMs (e.g. the LayoutLM series) generally serve as the encoder of document layouts, similar to the role played by pre-trained contextualized language models (e.g. the BERT series) in NLP tasks.
Conventionally, information extraction (IE) ability of PTLMs is evaluated via SER and EL, esp. in a sequence-labeling manner. 
The performance of PTLMs on these tasks reflects the capacity of their layout embeddings to facilitate downstream IE tasks.
However, the prevailing benchmarks do not fully conform to the aforementioned evaluation pipeline, thereby diminishing the reliability of the assessment. Take FUNSD as an example, its block-level annotation falsely couples the annotations of segment and entity, which does not adequately represent semantic-driven entities and hinder the fair evaluation. 

The propose of EC-FUNSD aims to provide a fair and unbiased evaluation benchmark of IE ability of PTLMs. 
The construction of this dataset includes the revision of layout and IE annotations from FUNSD. 
First, the original layout annotation of FUNSD is cleaned, and multiple-row blocks are split into row-wise segments.
Second, the semantic entities are re-annotated together with their linking relationships, with the segment order preserved to ensure that each entity is represented as a continuous word span within layout, making the form of this dataset suitable for sequence-labeling models. 
The final dataset consists of 199 document samples including the image, layout annotation of segments and words, and labeled entities of 3 categories. 
For the detailed annotation process and statistics, please refer to the original paper.

Introducing paper: Rethinking the Evaluation of Pre-trained Text-and-Layout Models from an Entity-Centric Perspective
ROOR

ROOR is a reading order prediction (ROP) benchmark which annotates layout reading order as ordering relations. 

Layout reading order is typically formulated as a permutation of layout elements, i.e. a sequence containing all the layout elements. 
However, multiple cases have reflected that this formulation does not adequately convey the complete reading order information in the layout, which may negatively affect the utilization of this signal in downstream VrD tasks. 

Therefore, this work investigate the properties of layout reading order, conceptualizing it with terms `Immediate Succession During Reading`(ISDR) and `Generalized Succession During Reading`(GSDR), and formulate each of which as an ordering relation over layout elements. 
Then, ROOR provides the annotation of the ISDR relationship over layout segments, based on the layout annotation of [EC-FUNSD](https://paperswithcode.com/dataset/ec-funsd). 
Overall, ROOR comprises 199 samples including 10,662 segments, 31,297 words and 10,967 annotated reading order linking pairs.
We hope the construction of this benchmark could facilitate the development of automated ROP methods of the improved task form.

Introducing paper: Modeling Layout Reading Order as Ordering Relations for Visually-rich Document Understanding