Field Extraction from Forms with Unlabeled Data (2022)

Abstract: We propose a novel framework to conduct field extraction from forms with unlabeled data. To bootstrap the training process, we develop a rule-based method for mining noisy pseudo-labels from unlabeled forms. Using the supervisory signal from the pseudo-labels, we extract a discriminative token representation from a transformer-based model by modeling the interaction between text in the form. To prevent the model from overfitting to label noise, we introduce a refinement module based on a progressive pseudo-label ensemble. Experimental results demonstrate the effectiveness of our framework.

## introduction form-like documents, such as invoices, paystubs and patient referral forms, are very common in daily business workflows. a large amount of human effort is required to extract information from forms every day. in form processing, a worker is usually given a list of expected form fields (e.g., purchase_order, invoice_number and total_amount in figure 1 ), and the goal is to extract their corresponding values based on the understanding of the form, where keys are generally the most important features for value localization. a field extraction system aims to automatically extract field values from redundant information in forms, which is crucial for improving processing efficiency and reducing human labor. field extraction from forms is a challenging task. document layouts and text representations can be very different even for the same form type, if they are from different vendors. for example, invoices from different companies may have significantly different designs (see figure 3 ). paystubs from different systems (e.g., adp and workday) have different representations for similar information. recent methods formulate this problem as fieldvalue pairing or field tagging. majumder et al. ## broader impact this work is potentially useful for improving information extraction systems from forms. so, it has positive impacts including improving document processing efficiency, thus reducing human labor. reducing human labor may also cause negative consequences such as job loss or displacement, particularly amongst low-skilled labor who may be most in need of gainful employment. the negative impact is not specific to this work and should be addressed broadly in the field of ai research. we are securely using the in-invoice dataset internally. for the public data in the inv-cdip dataset, we made certain to consider their provenance and there are no restrictions on the use of the public data. all the annotations were conducted and carefully reviewed by the authors. as a result, and given the fact that the datasets contain forms without any personally identifiable data, we have relative confidence that the datasets are ethically sourced.