Is this a bad table? A Closer Look at the Evaluation of Table Generation from Text
Authors: Pritika Ramu, Aparna Garimella, Sambaran Bandyopadhyay


Reviewer paper summaries: The paper introduces TABEVAL, a novel table evaluation strategy aimed at capturing the semantic quality of tables generated from text. Existing metrics often fail to assess the overall meaning conveyed by tables, leading to misjudgments in quality. TABEVAL addresses this by breaking down tables into natural language (NL) atomic statements using a technique called TabUnroll and then comparing these statements with ground truth using entailment-based measures. To validate the approach, the authors curated a diverse dataset named DESCTOTTO and conducted extensive experiments, showing that TABEVAL correlates more strongly with human judgments of table quality than existing metrics.
Summary of strengths: 1. TABEVAL evaluates tables by considering the overall semantics rather than independent cell values, providing a more reasonable assessment of table quality.
2. The use of TabUnroll to convert tables into NL statements and employing entailment-based measures is a novel approach that enhances the understanding of table content.
3. The newly curated DESCTOTTO dataset includes a wide range of table types and topics, allowing for comprehensive validation across different domains.
4. The experimental results demonstrate that TABEVAL has a higher correlation with human ratings of table quality, indicating its effectiveness and reliability.
Summary of weaknesses: I think the weaknesses of the paper are well summarized in the limitations section.
Comments, suggestions and typos: Despite the diversity, the dataset size of 1,250 tables may still be relatively small for certain applications, potentially limiting the generalizability of the findings.
Overall assessment scores: 4