Metric-Type Identification for Multi-Level Header Numerical Tables in Scientific Papers (2021)

Abstract: Numerical tables are widely used to present experimental results in scientific papers. For table understanding, a metric-type is essential to discriminate numbers in the tables. We introduce a new information extraction task, metric-type identification from multi-level header numerical tables, and provide a dataset extracted from scientific papers consisting of header tables, captions, and metric-types. We then propose two joint-learning neural classification and generation schemes featuring pointer-generator-based and BERT-based models. Our results show that the joint models can handle both in-header and out-of-header metric-type identification problems.

## introduction tables are powerful tools for presenting data efficiently in row and column views. in scientific papers, numerical tables are commonly used to show experimental results for facilitating data analysis. examples of numerical tables in scientific papers are shown in figure 1 . tables have the ability to cover multiple categories written in table headers by incorporating several header sets in a hierarchical view, called multi-level header tables. scientific papers have strict guidelines about tables; for example, one states that a similar type of text is written in the same level of header. figure 1a shows a multi-level header example in the column part, with task type (task 1 and task 2) in the first header-level and metric-type (prec and rec) in the second. the table also has a row header specifying the model type (model a, model b, model c, and model d). in the real-world, this header-type information is limited due to the unknown table scheme. however, we assume tables in scientific papers follow the rule of to understand the numbers in the tables, metrictypes are important for discriminating the numbers. a comparison between numbers is applied for numbers in the same metric-type with different categories. for the table in figure 1a , we cannot compare the number 60 for model a in the first column with 60 in the second one because they have a different metric-type: prec and rec. computing numbers with different metric-types will result in inaccurate analysis. different tables may have different ways of writing their header name, such as using abbreviations like p, pre, or prec to refer to precision. due to the lexical diversity of header names, metric-type identification becomes more challenging. using a rule-based metric-type tagging or a limited set of metric-types in a dictionary is not enough to cover the diversity. since tables in scientific papers typically have logical captions and logical categorization of the header-level, we introduce a metric-type identification task that locates the metric-type in the headers by using the caption and header name as inputs. for the example shown in figure 1a , the metric-type is located in the second level of the column header. we also cover tables that do not mention metric-types in their header (out-of-headers), as shown in figure 1b . in these cases, the metric-types are identified in the caption. to cover metric-types located both in the headers and not in the headers, we propose a joint framework of metric-type location prediction and metric-type token generation for the metric-type identification task in multi-level header tables. our contributions are as follows: table information extraction is beneficial to cover unknown table schemes and understand the table contents. milosevic et al. (2019) proposed a framework for table information extraction in biomedical domains by defining rules for all possible variables. specifically, for numerical variables, they retrieved metric-types by searching a set of possible tokens in the dictionary. focusing on numerical tables, @xcite extracted metric-types in earning reports by using similarity scores between the corresponding non-numeric text for the leftmost cells and stored metric-types. the work closest to ours is the one by @xcite , who used tables from the experimental result section, combined with the title and abstract as document representations to extract triples of tasks, dataset, and metric for leaderboard construction. in our study, we represent the tables in more generic ways, preventing the original table structure in the multi-level headers form. we intend to retain the ability of a table to cover complex categorization in the headers and efficiently present all values. a previous study that also explored multi-dimensional tables was done by @xcite to automatically detect table structures from xml tables. our pointer-generator-based model in the metrictype generation scheme is inspired by the promising results of the pointer-generator network @xcite in the summarization task. the network deals with the out-of-vocabulary issue by joint copying from source texts and generating from vocabularies. recent studies have shown that pre-trained encoders can be successfully fine-tuned for downstream nlp tasks, thus avoiding the need to train a new model from scratch. a pre-trained encoder bert @xcite was trained on the bookscorpus (800m words) and wikipedia (2,500m words). for better-contextualized representation in the scientific domain, @xcite introduced a domain-specific bert model, scibert, which was trained on 1.14m papers from semantic scholar. @xcite implemented both bert and scibert on their models to solve the information extraction task and achieved significant performance gains. ## models we propose neural models to identify the metrictype for multi-level header tables by means of a joint model of metric-type location prediction and metric-type token generation. ## conclusion in this work, we provided multi-level header numerical table datasets extracted from scientific papers consisting of header tables, captions, and metrictypes. we introduced a metric-type identification task for multi-level header numerical tables, and proposed joint location prediction and generation models to solve the task. we have shown that our proposed model can identify metric-types from the multi-level header tables, both when the metrictypes are included in the headers and when they are not. our datasets only cover scientific papers in the computational linguistic domain. the generalization of our results beyond domain still remains an open question due to the difficulties of collecting comparable datasets in other domains without additional annotation by human experts.