Venue: NAACL 2025

# eC-Tab2Text: Aspect-Based Text Generation from e-Commerce Product Tables

Luis Antonio Gutierrez Guanilo\({}^{\blacklozenge}\) Mir Tafseer Nayeem\({}^{\blacklozenge,*}\)

**Cristian Lopez\({}^{\blacktriangle}\) Davood Rafiei\({}^{\blacklozenge,*}\)**

\({}^{\blacklozenge}\)University of Engineering and Technology (UTEC) University of Alberta

{mnayeem, drafiei}@ualberta.ca  {luis.gutierrez.g, clopezd}@utec.edu.pe

 Corresponding authors.

###### Abstract

Large Language Models (LLMs) have demonstrated exceptional versatility across diverse domains, yet their application in e-commerce remains underexplored due to a lack of domain-specific datasets. To address this gap, we introduce **eC-Tab2Text**, a novel dataset designed to capture the intricacies of e-commerce, including detailed product attributes and user-specific queries. Leveraging eC-Tab2Text, we focus on text generation from product tables, enabling LLMs to produce high-quality, attribute-specific product reviews from structured tabular data. Fine-tuned models were rigorously evaluated using standard Table2Text metrics, alongside correctness, faithfulness, and fluency assessments. Our results demonstrate substantial improvements in generating contextually accurate reviews, highlighting the transformative potential of tailored datasets and fine-tuning methodologies in optimizing e-commerce workflows. This work highlights the potential of LLMs in e-commerce workflows and the essential role of domain-specific datasets in tailoring them to industry-specific challenges1.

Footnote 1: Our code, dataset, evaluation, model outputs, and other resources are publicly available at eC-Tab2Text.

+
Footnote †: _Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Industry Track)_, pages 849–867

+
Footnote †: _Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Industry Track)_, pages 849–867

## 1 Introduction

E-commerce relies heavily on tabular data, such as product details and features, while user interactions, including assistant agents and Q&A, predominantly occur in natural language. This disparity underscores the need for models that can effectively parse tabular data and engage users through coherent, context-aware communication Zhao et al. (2023b). Table-to-text generation addresses this challenge by transforming structured data into natural language, enabling applications such as product reviews, personalized descriptions, and tailored summaries in e-commerce. Beyond e-commerce, this capacity extends to domains such as healthcare, where structured patient records are converted into concise summaries for doctors He et al. (2023), and finance, where tabular financial data is transformed into analytical reports Varshney (2024). However, generating text that is coherent, contextually relevant, and aligned with user-specific requirements remains a significant challenge, particularly for user- or query-centric tasks that demand domain-specific knowledge. Existing table-to-text datasets often focus on general-purpose applications and lack the depth required for specialized domains. For instance, datasets like QTSumm Zhao

Figure 1: Overview of **eC-Tab2Text**. Illustration of aspect-based text generation from e-commerce product tables, where an LLM generates summaries for user-specific aspects like “Camera” and “Design & Display.”

et al., 2023) offer tabular summaries unrelated to the product domain, limiting their relevance for generating attribute-specific product reviews. E-commerce text generation requires handling diverse attributes (e.g., battery life, display quality), reasoning across different attributes (e.g., battery life and display size) and adapting to various user intents, such as crafting targeted product reviews Mackova and Pilat (2024).

While Large Language Models (LLMs) excel in general-purpose text generation Touvron et al. (2023); Kabir et al. (2024), and fine-tuned models like LLama2Touvron et al. (2023), resulting in StructLMZhuang et al. (2024) have shown improved performance on table-based datasets, these approaches often struggle with the complexities of product-specific domains. Addressing these intricacies requires tailored datasets to capture the nuanced requirements of attribute-specific text generation. Table-to-text generation has benefited from datasets like WikiTableT Chen et al. (2021), TabFact Chen et al. (2020), and ROTOWIRE Wiseman et al. (2017). However, these datasets, designed for tasks like Wikipedia table descriptions, fact-checking, and sports summaries, lack the relevance for product-specific applications. Similarly, LogicNLG Chen et al. (2020) and ToTTo Parikh et al. (2020) emphasize logical inferences and refined sentence extraction but fall short in addressing the demands of e-commerce text generation He and Abisado (2023).

This paper introduces a tailored table-to-text dataset for the products domain and explores the potential of fine-tuned LLMs to bridge the gap between general-purpose capabilities and domain-specific needs. By leveraging domain-specific datasets and fine-tuning techniques, this work aims to empower e-commerce platforms to generate more precise and engaging product reviews given user aspects and tables (see Figure 1), enhancing customer satisfaction and business outcomes.

Our main contributions are as follows:

* We present **eC-Tab2Text**, a novel domain-specific dataset for table-to-text generation in the e-commerce domain. The dataset features attribute-rich product tables paired with user-specific queries and outputs.
* We fine-tune open-source LLMs on the **eC-Tab2Text** dataset, resulting in significant improvements in text generation performance across various metrics.
* We provide a detailed analysis of domain robustness by comparing models trained on **eC-Tab2Text** with those trained on QTSumm, highlighting the critical need for domain-specific datasets to achieve superior performance in e-commerce applications.

## 2 Related Work

Table-to-Text GenerationTable-to-text generation has advanced through datasets tailored to diverse domains and applications, as summarized in Table 1. Early efforts, such as WikiTableT Chen et al. (2021), focused on generating natural language descriptions from Wikipedia tables, while TabFact Chen et al. (2020) introduced factor-checking capabilities and ROTOWIRE Wiseman et al. (2017) generated detailed sports summaries. However, these datasets are limited in their relevance to product-specific domains. Later datasets like LogicNLG Chen et al. (2020) emphasized logical inference and reasoning, and ToTTo Parikh et al. (2020) supported controlled text generation by focusing on specific table regions. HiTab Cheng et al. (2022) extended these capabilities with hierarchical table structures and reasoning operators. Despite these advancements, none of these datasets provide the contextual and attribute-specific depth necessary for e-commerce applications, where generating meaningful descriptions requires reasoning across heterogeneous attributes, such as linking battery capacity to battery life or associating display size with user experience.

Query-Focused Summarization (QFS)Advances in text summarization have improved multi-document summarization through abstractive methods like paraphrastic fusion Nayeem and Chali (2017); Nayeem et al. (2018), compression Nayeem et al. (2019); Chowdhury et al. (2021), and diverse fusion models Fuad et al. (2019); Nayeem (2017), among others Nayeem and Chali (2017); Chali et al. (2017). These approaches lay the groundwork for query-focused summarization (QFS), which tailors summaries to user-specific queries. Initially formulated as a document summarization task, QFS aims to generate summaries tailored to specific user queries Dang (2006). Despite its potential real-world applications, QFS remains a challenging task due to the lack of datasets. In the textual domain, QFS has been explored in multicomponent settings Giorgi et al. (2023) and meeting summarization Zhong et al. (2021). Recent 

datasets like QTSumm [23] extend QFS to a new modality, using tables as input. However, QTSumm's general-purpose nature limits its applicability to product reviews, which require nuanced reasoning over attributes and user-specific contexts. Additionally, its queries are often disconnected from real-world e-commerce scenarios. In contrast, our proposed dataset, **eC-Tab2Text**, bridges this gap by providing attribute-specific and query-driven summaries tailored to e-commerce product tables.

## 3 eC-Tab2Text: Dataset Construction

To address the gap in table-to-text generation for user-specific aspects or queries, such as "Camera" and "Design & Display" (as illustrated in Figure 1), we developed the **eC-Tab2Text** dataset. This dataset comprises e-commerce product tables and is designed to facilitate aspect-based text generation by fine-tuning LLMs on our dataset. The pipeline for creating **eC-Tab2Text** is outlined in Figure 2 and described in detail below.

Data SourcesThe dataset was constructed using product reviews and specifications (i.e., tables) extracted from the Pricebaba website2. Pricebaba provides comprehensive information on electronic products, including mobile phones and laptops. For this study, the focus was exclusively on mobile phone data due to the richness of product specifications (attribute-value pairs) and the availability of detailed expert reviews as summaries. Additionally, the number of samples available for mobile phones is significantly larger than for laptops. Each sample includes feature-specific details such as camera performance, battery life, and display quality.

Footnote 2: [https://pricebaba.com](https://pricebaba.com), last accessed August 2024.

Data Extraction and FormatData extraction was performed using web scraping techniques, with the extracted data stored in JSON format to serialize the table structure and to ensure compatibility with modern data processing workflows. Two JSON files were generated (Appendix E): one containing aspect-based product reviews and the other containing product specifications. The review JSON file captures user aspects alongside their associated textual descriptions collected from the "Quick Review" section of the website, while the specifications JSON file stores key-value pairs for both key specifications and full technical details. The structures of the sample inputs and outputs are depicted in Figures 3 and 4 in the Appendix.

#### Data Cleaning, Normalization, and Integration

To ensure consistency, usability, and completeness, the extracted data underwent rigorous cleaning, normalization, and integration, similar to previous approaches [25, 26, 8]. The process includes **(1)** standardizing all text values to lowercase for uniformity, **(2)** replacing special

\begin{table}
\begin{tabular}{l l c c c} \hline \hline
**Dataset** & **Table Source** & \multicolumn{2}{c}{**\# Tables /**} & \multicolumn{2}{c}{**\# Words /**} \\  & & **Statements** & **Statement** & **Explicit Control** \\ \hline \hline \multicolumn{5}{c}{_Single-sentence Table-to-Text_} \\ \hline ToTTo [7] & Wikipedia & 83,141 / 83,141 & 17.4 & Table region \\ \hline LOGICNLG [9] & Wikipedia & 7,392 / 36,960 & 14.2 & Table regions \\ \hline HiTab [1] & Statistics web & 3,597 / 10,672 & 16.4 & Table regions \& reasoning operator \\ \hline \multicolumn{5}{c}{_Generic Table Summarization_} \\ \hline ROTOWIRE [10] & NBA games & 4,953 / 4,953 & 337.1 & ✗ \\ \hline SciGen [1] & Sci-Paper & 1,338 / 1,338 & 116.0 & ✗ \\ \hline NumericNLG [2] & Sci-Paper & 1,355 / 1,355 & 94.2 & ✗ \\ \hline \multicolumn{5}{c}{_Table Question Answering_} \\ \hline FeTaQA [11] & Wikipedia & 10,330 / 10,330 & 18.9 & Queries rewritten from ToTTo \\ \hline \multicolumn{5}{c}{_Query-Focused Table Summarization_} \\ \hline QTSumm [23] & Wikipedia & 2,934 / 7,111 & 68.0 & Queries from real-world scenarios \\ \hline \multirow{2}{*}{**eC-Tab2Text** (_ours_)} & e-Commerce & \multirow{2}{*}{1,452 / 3,354} & \multirow{2}{*}{56.61} & Queries from e-commerce products \\  & products & & & \\ \end{tabular}
\end{table}
Table 1: Comparison between **eC-Tab2Text** (_ours_) and existing table-to-text generation datasets. Statements and queries are used interchangeably. Our dataset specifically comprises tables from the e-commerce domain.

 

characters (e.g., & with "and") to improve readability, and **(3)** normalizing keys to maintain logical and contextual coherence. For example, the key Display & Design was transformed into Design and Display to improve readability and alignment with naming conventions.

To further enhance the dataset quality, irrelevant and redundant entries were removed through a systematic filtering process: **(1)** reviews lacking textual content in the text field were discarded, **(2)** specifications containing only generic or minimal information (e.g., entries labeled as General) were excluded, **(3)** overly simplistic reviews categorized as Overview were omited to maintain a focus on detailed and meaningful content.

Finally, the reviews and specifications JSON files were merged into a unified dataset by aligning entries based on their unique product URLs. This integration consolidated each product's reviews and specifications into a single, cohesive record, creating a streamlined and comprehensive dataset for downstream applications.

Our **eC-Tab2Text** dataset provides a comprehensive resource for table-to-text generation tasks based on user queries, as summarized in Table 2. The input JSON files contain attribute-rich product specifications, averaging 59.8 attribute-value pairs per table, with the largest entries containing up to 68 pairs. The dataset includes 3,354 queries, averaging 2.31 queries per table, with concise outputs averaging 56.61 words per query. This design supports query-specific training and evaluation of LLMs, enabling precise and contextually relevant text generation tailored to user queries.

## 4 eC-Tab2Text: Models

This section outlines the methodology for table serialization and provides details on the selection and fine-tuning of LLMs using our dataset.

Table SerializationThe representation of tabular data in machine learning has been addressed through various serialization techniques, including markdown format, comma-separated values (CSV), HTML [70, 71], and LaTeX [19]. However, for our specific problem involving semi-structured tables with nested structures, we adopt JSON serialization. This approach effectively addresses two critical needs: **(1)** representing the nested structures inherent in product tables and **(2)** enabling query-specific generation and evaluation [70].

In our eC-Tab2Text dataset, both input tables and query-specific outputs are serialized using JSON. The input JSON captures structured product specifications, while the output JSON aligns queries (e.g., "Design and Display" or "Battery") as keys and their corresponding generated texts as values. This unified representation facilitates efficient querying and maintains alignment between inputs and outputs, ensuring consistency across the dataset. Additional implementation details can be found in Appendix D (Listing 7 prompt).

Model Selection and CharacteristicsTo evaluate the effectiveness of the eC-Tab2Text dataset, we fine-tuned three open-source LLMs: **LLaMA 2-Chat 7B**[23], **Mistral 7B-Instruct**[18], and **StructLM 7B**[18]. These models were selected due to their distinct pretraining paradigms, which address diverse data modalities and tasks. Detailed descriptions of these models are provided in Appendix B and summarized below.

\begin{table}
\begin{tabular}{l l} \hline \hline
**Metric** & **Value** \\ \hline \hline _Input_ & \\ \hline \# Tables & 1,452 \\ \hline Avg \# Attribute-Value Pairs & 59.8 \\ \hline Max \# Attribute-Value Pairs & 68 \\ \hline _Output_ & \\ \hline \# Queries & 3,354 \\ \hline Avg \# queries/table & 2.31 \\ \hline Avg \# words/query & 56.61 \\ \hline \end{tabular}
\end{table}
Table 2: Statistics of our **eC-Tab2Text** dataset.

 

* **LLaMA 2-Chat 7B3**: This model, pretrained on 2 trillion tokens of publicly available text data, is fine-tuned on over one million human-annotated examples. It excels in general-purpose conversational and language understanding tasks Touvron et al. (2023). Footnote 3: Llama-2-7b-chat-hf
* **Mistral 7B-Instruct4**: Leveraging a mix of text and code during training, this model demonstrates strong performance in tasks that require natural language understanding and programming-related reasoning Jiang et al. (2023). Footnote 4: Mistral-7B-Instruct-v0.3
* **StructLM 7B5**: Pretrained on structured data, including databases, tables, and knowledge graphs, StructLM is optimized for structured knowledge grounding, making it particularly effective for domain-specific tasks Zhuang et al. (2024).

Footnote 5: StructLM-7B

Fine-Tuning ProcessThe fine-tuning process adapts these models to the e-commerce domain using the ec-Tab2Text dataset. This dataset focuses on attribute-specific and context-aware text generation tailored to user queries, such as detailed reviews of "Camera" or "Design & Display." The fine-tuning process follows best practices in instruction tuning and domain-specific dataset alignment Zhang et al. (2023); Chang et al. (2024). Optimization of hyperparameters ensured computational efficiency while maintaining high-quality performance, as detailed Table 4.

By leveraging these diverse models and aligning them with the eC-Tab2Text dataset, this work aims to advance the state-of-the-art in domain-specific language generation for e-commerce applications.

## 5 Evaluation

In this section, we evaluate the performance of the eC-Tab2Text models described in Section 4 along with several closed-source models, including GPT-4o-mini and Gemini-1.5-flash. The evaluation follows standard metrics commonly used in table-to-text generation, as outlined in Zhao et al. (2023). These metrics include BLEU Reiter (2018), the F-1 scores of ROUGE-1 and ROUGE-L Ganesan (2018), METEOR Dobre (2015), and BERTScore Zhang* et al. (2020), following Akash et al. (2023); Shohan et al. (2024). To assess the correctness, faithfulness, and fluency of the generated text, we employ PROMETHEUS 2 Kim et al. (2024) and an open-source LLM-based evaluator as an alternative to the closed-source G-Eval Liu et al. (2023). Our objective is to benchmark the performance of various LLMs under both zero-shot and fine-tuned settings using the proposed eC-Tab2Text dataset.

Experimental SetupThe fine-tuning process was conducted on a NVIDIA RTX 4070 Ti Super GPU with 16GB of VRAM, ensuring efficient training while managing memory-intensive operations. The AdamW optimizer Loshchilov and Hutter (2019) was configured with a learning rate of

\begin{table}
\begin{tabular}{|c|c|} \hline \multicolumn{1}{|l|}{**Hyperparameter**} & **Value** \\ \hline Learning Rate & \(2\times 10^{-4}\) \\ Batch Size & 2 \\ Epochs & 1 \\ Gradient Accumulation Steps & 1 \\ Weight Decay & 0.001 \\ Max Sequence Length & 900 \\ \hline \end{tabular}
\end{table}
Table 4: Hyperparameter settings for fine-tuning.

\begin{table}
\begin{tabular}{p{56.9pt} p{113.8pt} p{113.8pt}} \hline \hline
**Column Name** & **Data Type** & **Description** \\ \hline \multirow{3}{*}{table} & \multirow{3}{*}{Dictionary} & Contains structured data with headers and rows. \\ \cline{2-3}  & & Unique identifier for each dataset example. \\ \hline \multirow{3}{*}{table} & \multirow{3}{*}{String} & Textual description or query related to the dataset. \\ \cline{3-3}  & & Summary or explanation generated in response to the query. \\ \hline \multirow{3}{*}{www\_ids} & \multirow{3}{*}{Sequence of Integers} & Row indices corresponding to the data referenced in the table column. \\ \cline{3-3}  & & \\ \hline \hline \end{tabular}
\end{table}
Table 3: Structure of the QTSUMM Dataset.

\begin{table}
\begin{tabular}{|c c|} \hline \hline
**Hyperparameter** & **Value** \\ \hline bnb\_4bit\_compute\_dtype & float16 \\ \hline bnb\_4bit\_quant\_type & nf4 \\ \hline use\_nested\_quant & False \\ \hline \hline \end{tabular}
\end{table}
Table 5: Quantization settings used for fine-tuning.

 

\(2\times 10^{-4}\), chosen for its effectiveness in maintaining stability and convergence during training. To optimize resource usage, the _bitsandbytes_ library6 was employed for 4-bit quantization, reducing VRAM requirements without significant performance loss. Table 5 outlines the key parameters used, including 'float16' for computation data type and 'nf4' for quantization type. The 'use_nested_quant' option was set to 'False' to ensure compatibility across models.

Footnote 6: [https://github.com/bitsandbytes-foundation/bitsandbytes](https://github.com/bitsandbytes-foundation/bitsandbytes)

Detailed information on the evaluation metrics is included in Appendix A. Our eC-Tab2Text dataset was divided into training and testing subsets, using an 80%-20% split. This ensures a sufficient volume of data for training while preserving a reliable subset for evaluation.

### Robustness Evaluation

We evaluate the robustness of the models under domain differences, focusing on their performance with in-domain and out-of-domain training data. The primary objective is to analyze how models perform when fine-tuned on data from different domains and to emphasize the importance of our proposed eC-Tab2Text dataset for the e-commerce product domain. For this evaluation, we compare the performance of models fine-tuned on the QTSumm dataset Zhao et al. (2023a), which contains Wikipedia tables with queries, against those fine-tuned on our eC-Tab2Text dataset, which consists of product tables with user-specific queries.

QTSumm Dataset DetailsThe QTSumm dataset, obtained from Hugging Face7 provides structured data that facilitates query-specific text summarization tasks. The detailed structure of QTSumm is outlined in Table 3. This dataset's structure ensures a systematic alignment between the input queries, the corresponding structured data, and the generated summaries, making it a valuable benchmark for fine-tuning and evaluating the performance of LLMs in handling structured data. Its focus on query-specific summarization provided an excellent foundation for testing the robustness and adaptability of the proposed methodologies.

Footnote 7: [https://huggingface.co/datasets/yale-nlp/QTSumm](https://huggingface.co/datasets/yale-nlp/QTSumm)

For fine-tuning, we utilized the same models described in Section 4, employing identical hyperparameters. The models were trained using prompts structured consistently with those designed for the

\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline
**Mode** & **Models** & **BLEU** & **METEOR** & **ROUGE-I** & **ROUGE-L** & **BERTScore** & **Correctness** & **Faithfulness** & **Fluency** \\ \hline  & Llamaz & 1.39 & 3.59 & 5.57 & 4.09 & 66.49 & 32.18 & 37.68 & 32.47 \\  & StructLN & 6.21 & 11.96 & 20.09 & 15.34 & 82.56 & 64.30 & 70.08 & 63.10 \\
**Zero-Shot** & **Mistral** & 4.19 & 9.55 & 25.64 & 18.99 & 82.12 & 77.02 & 81.16 & 76.5 \\  & GPT-4o-mini & 7.14 & 16.12 & 29.44 & 19.47 & 83.75 & **80.89** & **83.92** & **80.81** \\  & Gemini-1.5-flash & 8.8 & 15.18 & 30.38 & 21.51 & 84.05 & 78.79 & 83.04 & 78.54 \\ \hline  & Llamaz & 29.36 & 40.2 & 48.36 & 39.25 & 90.05 & 61.38 & 63.78 & 61.47 \\
**Fine-tuned** & StructLM & 31.06 & 42.3 & 49.42 & 40.58 & 90.9 & 69.70 & 72.46 & 69.93 \\  & Mistral & **38.89** & **49.43** & **56.64** & **48.32** & **92.18** & 73.07 & 76.63 & 73.03 \\ \hline \end{tabular}
\end{table}
Table 6: Evaluation results of zero-shot and fine-tuned models on the **eC-Tab2Text** dataset. The best results are highlighted in **bold**, and the second-best results are underlined.

\begin{table}
\begin{tabular}{l l r|r r r r r r r} \hline
**Dataset Trained** & **Dataset Tested** & **Models** & **BLEU** & **METEOR** & **ROUGE-I** & **ROUGE-L** & **BERTScore** & **Perforches** & **Faithfulness** & **Fluency** \\ \hline \multirow{4}{*}{**QTSum**} & **QTSumm** & **Llamaz** & 13.32 & 32.38 & 26.3 & 19.22 & 86.47 & 51.09 & 57.30 & 48.98 \\  & (In-domain) & **StructUN** & 6.6 & 22.04 & 13.52 & 10.04 & 84.5 & 41.14 & 48.92 & 39.68 \\  & **Mistral** & 1.01 & 28.57 & 20.7 & 15.51 & 85.65 & 49.99 & 57.73 & 50.71 \\ \cline{2-11}  & **tC-Thab2Text** & **Llamaz** & 17.47 & 40.2 & 35.69 & 21.14 & 85.41 & 63.98 & 71.40 & 64.07 \\  & **structLN** & 3.73 & 17.42 & 10.41 & 6.77 & 82.91 & 36.69 & 60.81 & 37.03 \\  & **Mistral** & 13.97 & 26.88 & 28.58 & 17.08 & 84.83 & 58.35 & 69.81 & 58.95 \\ \hline \multirow{4}{*}{**eC-Tub2Text**} & **QTSumm** & **Llamaz** & 6.5 & 22.77 & 27.95 & 16.59 & 81.93 & 48.42 & 48.66 & 48.55 \\  & **structLN** & 10.15 & 30.59 & 30.59 & 23.04 & 85.13 & 58.71 & 56.60 & 58.26 \\  & **Mistral** & 10.39 & 18.11 & 30.27 & 24.24 & 84.23 & 64.83 & 61.14 & 64.51 \\ \cline{1-1} \cline{2-11}  & **tC-Thab2Text** & **Llamaz** & 29.4 & 40.21 & 48.43 & 39.52 & 90.05 & 61.38 & 63.78 & 61.47 \\ \cline{1-1} \cline{2-11}  & **tC-Thab2Text** & **StretLM** & 31.06 & 42.3 & 49.42 & 40.58 & 90.9 & 69.70 & 72.46 & 69.93 \\ \cline{1-1} \cline{2-11}  & **tC-Thab2Text** & **Mistral** & **38.89** & **49.43** & **56.64** & **48.32** & **92.18** & **73.07** & **76.63** & **73.03** \\ \hline \end{tabular}
\end{table}
Table 7: Robustness evaluation results on our **eC-Tab2Text** dataset and the QTSumm dataset Zhao et al. (2023a). The best results on our dataset, including both in-domain and out-of-domain scenarios, are highlighted in **bold**, while the best results on the QTSumm dataset, both in-domain and out-of-domain, are underlined.

 

eC-Tab2Text dataset. However, in the QTSumm setup, the prompts included row-level content tailored to the dataset's structure, as outlined in Appendix D (Listing 8). This alignment ensured methodological consistency while accounting for the unique characteristics of the QTSumm dataset. By highlighting these differences, our evaluation underscores the critical need for domain-specific datasets, such as eC-Tab2Text, to achieve robust and accurate performance in the product domain.

### Results & Analysis

Our experimental results, illustrated in Table 6, demonstrate that fine-tuning open-source 7B models on our dataset leads to substantial performance improvements. These fine-tuned models significantly outperform major proprietary models, such as GPT-4o-mini and Gemini-1.5-flash, across text-based metrics, including BLEU, ROUGE-1, ROUGE-L, METEOR, and BERTScore, while achieving competitive results in model-based metrics like faithfulness, correctness, and fluency, narrowing the gap with proprietary counterparts. This is significant given the relatively small size of our dataset compared to the much larger datasets used for training many proprietary models. Notably, Mistral_Instruct, fine-tuned on our dataset, excels by achieving the highest scores across all metrics, surpassing both zero-shot and fine-tuned models.

As highlighted in Table 7, the robustness of our dataset is further evidenced by comparing it against the QTSUMM dataset; models trained with our dataset consistently outperform those trained on QTSUMM across both in-domain and out-of-domain tasks, with Mistral_Instruct leading, followed closely by StructLM. Although both datasets share similar task objectives, the domain differences significantly affect the models' performance.

Outputs generated by different open-source models are presented in MISTAI (Listing 11), StructLM (Listing 14), and Llama2 (Listing 15), as well as by closed-source models GPT-4o-mini (Listing 13) and Gemini1.5-flash (Listing 12). Notably, the closed-source models tend to produce longer outputs compared to the open-source models, with their outputs often containing nested keys and detailed information.

## 6 Discussion and Future Directions

This section highlights the need for better numerical reasoning in table-to-text generation and improved evaluation methods.

Numerical ReasoningProduct tables, with their semi-structured and nested attributes (e.g., battery capacity in mAh, display size in inches), demand advanced numerical reasoning to generate meaningful text. Models must analyze relationships, such as how battery life depends on capacity and display size, or how display dimensions impact user experience. Unlike Wikipedia tables Zhao et al. (2023); Nahid and Rafiei (2024), which focus on factual text generation, our eC-Tab2Text dataset challenges models to integrate numerical reasoning with qualitative text generation Islam et al. (2024). This unique focus enables LLMs to synthesize structured data into nuanced, human-readable summaries, providing a benchmark for evaluating and improving reasoning capabilities in real-world applications Naeim abadi et al. (2023); Akhtar et al. (2023); Zhao et al. (2024). Future work could explore pushing the boundaries of LLMs capabilities in numerical and qualitative reasoning using our dataset.

EvaluationAlthough we evaluated the correctness, faithfulness, and fluency of the generated text using PROMETHEUS 2 Kim et al. (2024), attribute-specific text evaluation against product tables requires a more nuanced approach. Future evaluations could involve extracting attribute-value pairs from the generated text Shinzato et al. (2023); Brinkmann et al. (2024), verifying their correctness and contextual relevance, and comparing them with the corresponding values in the source tables to enable more fine-grained and precise assessments.

## 7 Conclusion

This work introduces **eC-Tab2Text**, a novel dataset for table-to-text generation in the e-commerce domain, addressing the limitations of existing general-purpose datasets. By fine-tuning open-source LLMs, we demonstrate substantial improvements in generating attribute-specific, contextually accurate product reviews. Our evaluation highlights the robustness of **eC-Tab2Text**, outperforming comparable datasets like QTSumm, and underscores the importance of domain-specific datasets for advancing LLM performance in industry-specific applications. This study lays the groundwork for future research in expanding dataset scope, evaluation methodologies, and enhancing numerical reasoning in e-commerce workflows 



### Limitations

In this work, we evaluated our proposed methods using a selection of both open-source and closed-source LLMs. We intentionally focused on cost-effective yet efficient closed-source models and open-source models deployable on consumer-grade hardware, considering the constraints of _academic settings_. The performance of more powerful, large-scale models remains unexplored; however, we encourage the broader research community to benchmark these models using our dataset. To support future research, we make our code, dataset, evaluation, model outputs, and other resources publicly available8.

Footnote 8: [https://github.com/Luis-ntonio/eC-Tab2Text](https://github.com/Luis-ntonio/eC-Tab2Text)

This study faced several system and resource constraints that shaped the methodology and evaluation process. For example, VRAM limitations required capping the maximum token length at 900 for the Mistral_Instruct model to ensure uniform hyperparameter settings across all models. While this standardization enabled consistent comparisons, it may have limited some models' ability to generate longer and potentially more nuanced outputs.

Our dataset focused exclusively on mobile phone data due to the richness of product specifications (attribute-value pairs) and the availability of detailed expert reviews as summaries. Future work could expand the dataset to include other domains, such as laptops, home appliances, and wearable devices, to assess the generalizability of the LLMs in e-Commerce domains.

Finally, the development of eC-Tab2Text has been exclusively centered on the **English language**. As a result, its effectiveness and applicability may differ for other languages. Future research could explore multilingual extensions to broaden its usability across diverse linguistic and cultural contexts.

## Ethics Statement

The data scraping process for this research was conducted with strict adherence to ethical guidelines and solely for non-commercial research purposes, under the Creative Commons Attribution-Non Commercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0)9. To minimize potential harm to the source website, measures were implemented to ensure controlled and responsible scraping practices. These safeguards were designed to avoid undue strain on the website's infrastructure, such as preventing Distributed Denial of Service (DDoS) attacks, thereby maintaining the integrity and functionality of the site.

Footnote 9: [https://creativecommons.org/licenseses/by-nc-sa/4.0/](https://creativecommons.org/licenseses/by-nc-sa/4.0/)

## Acknowledgements

We thank all the anonymous reviewers and the meta-reviewer for their valuable feedback and constructive suggestions. This research is supported by the Natural Sciences and Engineering Research Council of Canada (NSERC). Additionally, Luis Antonio Gutierrez Guanilo is supported by the Emerging Leaders in the Americas Program (ELAP), and Mir Tafseer Nayeem is supported by a Huawei PhD Fellowship.

## References

* Akkhtar et al. [2023] Abu Ubaida Akash, Mir Tafseer Nayeem, Faisal Tareque Shohan, and Tanvir Islam. 2023. Shironaam: Bengali news headline generation using auxiliary information. In _Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics_, pages 52-67, Dubrovnik, Croatia. Association for Computational Linguistics.
* Akhtar et al. [2023] Mubashara Akhtar, Abhilash Shankarampeta, Vivek Gupta, Arpit Patil, Oana Cocarascu, and Elena Simperl. 2023. Exploring the numerical reasoning capabilities of language models: A comprehensive analysis on tabular data. In _Findings of the Association for Computational Linguistics: EMNLP 2023_, pages 15391-15405, Singapore. Association for Computational Linguistics.
* Brinkmann et al. [2024] Alexander Brinkmann, Roee Shraga, and Christian Bizer. 2024. Extractgpt: Exploring the potential of large language models for product attribute value extraction. _Preprint_, arXiv:2310.12537.
* Chali et al. [2017] Yllias Chali, Moin Tanvee, and Mir Tafseer Nayeem. 2017. Towards abstractive multi-document summarization using submodular function-based framework, sentence compression and merging. In _Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_, pages 418-424, Taipei, Taiwan. Asian Federation of Natural Language Processing.
* Chang et al. [2024] Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, and Xing Xie. 2024. A survey on evaluation of large language models. _ACM Trans. Intell. Syst. Technol._, 15(3).



Mingda Chen, Sam Wiseman, and Kevin Gimpel. 2021. WikiTableT: A large-scale data-to-text dataset for generating Wikipedia article sections. In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_, pages 193-209, Online. Association for Computational Linguistics.
* Chen et al. (2020a) Wenhu Chen, Jianshu Chen, Yu Su, Zhiyu Chen, and William Yang Wang. 2020a. Logical natural language generation from open-domain tables. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 7929-7942, Online. Association for Computational Linguistics.
* Chen et al. (2020b) Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. 2020b. Tabfact: A large-scale dataset for table-based fact verification. In _International Conference on Learning Representations_.
* Cheng et al. (2022) Zhoujun Cheng, Haoyu Dong, Zhiruo Wang, Ran Jia, Jiaqi Guo, Yan Gao, Shi Han, Jian-Guang Lou, and Dongmei Zhang. 2022. HiTab: A hierarchical table dataset for question answering and natural language generation. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_, pages 1094-1110, Dublin, Ireland. Association for Computational Linguistics.
* Chowdhury et al. (2021) Radia Rayan Chowdhury, Mir Tafseer Nayeem, Tahsin Tasnim Mim, Md. Saifur Rahman Chowdhury, and Taufiqul jannat. 2021. Unsupervised abstractive summarization of Bengali text documents. In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_, pages 2612-2619, Online. Association for Computational Linguistics.
* Dang (2006) Hoa Trang Dang. 2006. DUC 2005: Evaluation of question-focused summarization systems. In _Proceedings of the Workshop on Task-Focused Summarization and Question Answering_, pages 48-55, Sydney, Australia. Association for Computational Linguistics.
* Social and Behavioral Sciences_, 180:305-312.
* a survey. _Transactions on Machine Learning Research_.
* Fuad et al. (2019)Tanvir Ahmed Fuad, Mir Tafseer Nayeem, Asif Mahmud, and Yllias Chali. 2019. Neural sentence fusion for diversity driven abstractive multi-document summarization. _Computer Speech & Language_, 58:216-230.
* Ganesan (2018) Kavita Ganesan. 2018. Rouge 2.0: Updated and improved measures for evaluation of summarization tasks. _Preprint_, arXiv:1803.01937.
* Gao et al. (2024) Chang Gao, Wenxuan Zhang, Guizhen Chen, and Wan Lam. 2024. Isontuning: Towards generalizable, robust, and controllable instruction tuning. _Preprint_, arXiv:2310.02953.
* Giorgi et al. (2023) John Giorgi, Luca Soldaini, Bo Wang, Gary Bader, Kyle Lo, Lucy Wang, and Arman Cohan. 2023. Open domain multi-document summarization: A comprehensive study of model britteless under retrieval. In _Findings of the Association for Computational Linguistics: EMNLP 2023_, pages 8177-8199, Singapore. Association for Computational Linguistics.
* He and Abisado (2023)Aixiang He and Mideth B. Abisado. 2023. Review on sentiment analysis of e-commerce product comments. _2023 IEEE 15th International Conference on Advanced Infocomm Technology (ICATT)_, pages 398-406.
* He et al. (2023) Kai He, Rui Mao, Qika Lin, Yucheng Ruan, Xiang Lan, Mengling Feng, and Erik Cambria. 2023. A survey of large language models for healthcare: from data, technology, and applications to accountability and ethics. _Preprint_, arXiv:2310.05694.
* Islam et al. (2024) Mohammed Saidul Islam, Raian Rahman, Ahmed Masry, Md Tahmid Rahman Laskar, Mir Tafseer Nayeem, and Enamul Hoque. 2024. Are large vision language models up to the challenge of chart comprehension and reasoning. In _Findings of the Association for Computational Linguistics: EMNLP 2024_, pages 3334-3368, Miami, Florida, USA. Association for Computational Linguistics.
* Jacovi and Goldberg (2020) Alon Jacovi and Yoav Goldberg. 2020. Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness? In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_, pages 4198-4205, Online. Association for Computational Linguistics.
* Jaitly et al. (2023) Sukriti Jaitly, Tanay Shah, Ashish Shugani, and Razik Singh Grewal. 2023. Towards better serialization of tabular data for few-shot classification with large language models. _Preprint_, arXiv:2312.12464.
* Jiang et al. (2023) Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulinier, Lelong Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothee Lacroix, and William El Sayed. 2023. Mitral 7b. Preprint_, arXiv:2310.06825.
* Kabir et al. (2024) Mohsinuul Kabir, Mohammed Saidul Islam, Md Tahmid Rahman Laskar, Mir Tafseer Nayeem, M Saiful Bari, and Enamul Hoque. 2024. BenLLM-eval: A comprehensive evaluation into the potentials and pitfalls of large language models on Bengail NLP. In _Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)_, pages 2238-2252, Torino, Italia. ELRA and ICCL 

Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungaie Lee, and Minjoon Seo. 2024. Prometheus 2: An open source language model specialized in evaluating other language models. In _Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing_, pages 4334-4353, Miami, Florida, USA. Association for Computational Linguistics.
* Lee et al. (2023) Seungjun Lee, Jungseob Lee, Hyeonseok Moon, Chanjun Park, Jaehyung Seo, Soyeong Eo, Seonmin Koo, and Heuiseko Lim. 2023. A survey on evaluation metrics for machine translation. _Mathematics_, 11(4).
* Liu et al. (2023) Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023. G-eval: NLG evaluation using gpt-4 with better human alignment. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_, pages 2511-2522, Singapore. Association for Computational Linguistics.
* Loshchilov and Hutter (2019) Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In _International Conference on Learning Representations_.
* Mackova and Pilat (2024) Katerina Mackova and Martin Pilat. 2024. Promap: Product mapping datasets. In _Advances in Information Retrieval: 46th European Conference on Information Retrieval, ECIR 2024, Glasgow, UK, March 24-28, 2024, Proceedings, Part II_, page 159-172, Berlin, Heidelberg. Springer-Verlag.
* Madsen et al. (2022) Andreas Madsen, Nicholas Meade, Vaibhav Adlakha, and Siva Reddy. 2022. Evaluating the faithfulness of importance measures in NLP by recursively masking allegedly important tokens and retraining. In _Findings of the Association for Computational Linguistics: EMNLP 2022_, pages 1731-1751, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
* Moosavi et al. (2021) Nafise Sadat Moosavi, Andreas Ruckle, Dan Roth, and Iryna Gurevych. 2021. Scigen: a dataset for reasoning-aware text generation from scientific tables. In _Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)_.
* abadi et al. (2023) Ali Naeim abadi, Mir Tafeser Nayeem, and Davood Rafiei. 2023. Product entity matching via tabular data. In _Proceedings of the 32nd ACM International Conference on Information and Knowledge Management_, CIKM '23, page 4215-4219, New York, NY, USA. Association for Computing Machinery.
* Nahid and Rafiefi (2024) Md Nahid and Davood Rafiei. 2024. TabSQLify: Enhancing reasoning capabilities of LLMs through table decomposition. In _Proceedings of the 2024 Science of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)_, pages 5725-5737, Mexico City, Mexico. Association for Computational Linguistics.
* Nan et al. (2022) Linyong Nan, Chiachun Hsieh, Ziming Mao, Xi Victoria Lin, Neha Verma, Rui Zhang, Wojciech Kryscinski, Hailey Schoelkopf, Riley Kong, Xiangru Tang, Mutthia Mutuna, Ben Rosand, Isabel Trindade, Renusree Bandaru, Jacob Cunningham, Caiming Xiong, Dragomir Radev, and Dragomir Radev. 2022. FeTaQA: Free-form table question answering. _Transactions of the Association for Computational Linguistics_, 10:35-49.
* Nayeem (2017) Mir Tafeser Nayeem. 2017. _Methods of Sentence Extraction, Abstraction and Ordering for Automatic Text Summarization_. Universtiy of Letbbridge, Department of Mathematics and Computer Science.
* Nayeem and Chali (2017a) Mir Tafeser Nayeem and Yllias Chali. 2017a. Extract with order for coherent multi-document summarization. In _Proceedings of TextGraphs-11: the Workshop on Graph-based Methods for Natural Language Processing_, pages 51-56, Vancouver, Canada. Association for Computational Linguistics.
* Nayeem and Chali (2017b) Mir Tafeser Nayeem and Yllis Chali. 2017b. Paraphrastic fusion for abstractive multi-sentence compression generation. In _Proceedings of the 2017 ACM on Conference on Information and Knowledge Management_, CIKM '17, page 2223-2226, New York, NY, USA. Association for Computing Machinery.
* Nayeem et al. (2018) Mir Tafeser Nayeem, Tanvir Ahmed Fuad, and Yllias Chali. 2018. Abstractive unsupervised multi-document summarization using paraphrastic sentence fusion. In _Proceedings of the 27th International Conference on Computational Linguistics_, pages 1191-1204, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
* Nayeem et al. (2019) Mir Tafeser Nayeem, Tanvir Ahmed Fuad, and Yllias Chali. 2019. Neural diverse abstractive sentence compression generation. In _Advances in Information Retrieval (ECIR)_, pages 109-116, Cham. Springer International Publishing.
* Nayeem and Rafiefi (2023) Mir Tafeser Nayeem and Davood Rafiefi. 2023. On the role of reviewer expertise in temporal review helpfulness prediction. In _Findings of the Association for Computational Linguistics: EACL 2023_, pages 1684-1692, Dubrovnik, Croatia. Association for Computational Linguistics.
* early insights and future directions. In _Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing_, pages 4813-4836, Miami, Florida, USA. Association for Computational Linguistics.
* Nayeem and Rafiefi (2024b) Mir Tafeser Nayeem and Davood Rafiefi. 2024b. Lfosum: Summarizing long-form opinions with large language models. _Preprint_, arXiv:2410.13037.
* Parikh et al. (2020) Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manai Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020. ToTTo: A controlled table-to-text 

generation dataset. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online. Association for Computational Linguistics, pp. 1173-1186. External Links: Link Cited by: SS4.
* E. Reiter (2018)A structured review of the validity of BLEU. Computational Linguistics44 (3), pp. 393-401. External Links: ISSN 0050-321X, Document, Link Cited by: SS4.
* K. Shinzato, N. Yoshinaga, Y. Xia, and W. Chen (2023)A unified generative approach to product attribute-value identification. In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, pp. 6599-6612. External Links: ISBN 978-1-4503-3213-1, Link, Document Cited by: SS4.
* F. S. Shohan, M. Tafseer Nayeem, S. Islam, A. Ubiadda Akash, and S. Joty (2024)XL-HeadTags: leveraging multimodal retrieval augmentation for the multilingual generation of news headlines and tags. In Findings of the Association for Computational Linguistics: ACL 2024, Bangkok, Thailand, pp. 12991-13024. External Links: Link, Document Cited by: SS4.
* A. Singha, J. Cambronero, S. Gulwani, V. Le, and C. Parnin (2023)Tabular representation, noisy operators, and impacts on table structure understanding tasks in LLMs. In NeurIPS 2023 Second Table Representation Learning Workshop, Cited by: SS4.
* L. Hulliyyatus Suadaa, H. Kamigaito, K. Funakoshi, M. Okumura, and H. Takamura (2021)Towards table-to-text generation with numerical reasoning. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online. Association for Computational Linguistics, Cited by: SS4.
* H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, B. Bhosale, D. Bilkel, L. Blecher, C. Canton Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, and et al. others (2023)LLMA 2: open foundation and fine-tuned chat models. PreprintarXiv:2307.09288. External Links: Link, 2307.09288 Cited by: SS4.
* T. Varshney (2024)Build an llm-powered data agent for data analysis. External Links: Link Cited by: SS4.
* S. Wiseman, S. Shieber, and A. Rush (2017)Challenges in data-to-document generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark, pp. 2253-2263. External Links: Link, Document Cited by: SS4.
* Y. Yao and A. Koller (2024)Predicting generalization performance with correctness discriminators. In Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, pp. 11725-11739. External Links: Link, Document Cited by: SS4.
* S. Zhang, L. Dong, X. Li, S. Zhang, X. Sun, S. Wang, J. Li, R. Hu, T. Zhang, F. Wu, and G. Wang (2023)Instruction tuning for large language models: a survey. ArXivabs/2308.10792. External Links: Link, 2308.10792 Cited by: SS4.
* T. Zhao, Y. Long, H. Liu, R. Kamoi, L. Nan, L. Chen, Y. Liu, X. Tang, R. Zhang, and A. Cohan (2024)DocMath-eval: evaluating math reasoning capabilities of LLMs in understanding long and specialized documents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 16103-16120. External Links: Link, Document Cited by: SS4.
* Y. Zhao, Z. Qi, L. Nan, B. Mi, Y. Liu, W. Zou, S. Han, R. Chen, X. Tang, Y. Xu, D. Radev, and A. Cohan (2023)QTSumm: query-focused summarization over tabular data. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore, pp. 1157-1172. External Links: Link, Document Cited by: SS4.
* Y. Zhao, H. Zhang, S. Si, L. Nan, X. Tang, and A. Cohan (2023b)Investigating table-to-text generation capabilities of large language models in real-world information seeking scenarios. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track, Singapore, pp. 160-175. External Links: Link, Document Cited by: SS4.
* M. Zhong, D. Yin, T. Yu, A. Zaidi, M. Mutuma, R. Jia, A. Hassan Awadallah, A. Celikyilmaz, Y. Liu, X. Qiu, and D. Radev (2021)QMSum: a new benchmark for query-based multi-domain meeting summarization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online. Association for Computational Linguistics, Cited by: SS4.
* A. Zhuang, G. Zhang, T. Zheng, X. Du, J. Wang, W. Ren, W. Huang, J. Fu, X. Yue, and W. Chen (2024)StructLM: towards building generalist models for structured knowledge grounding. In First Conference on Language Modeling 



**Supplementary Material: Appendices**

## Appendix A Evaluation Metrics

* **BLEU (Bilingual Evaluation Understudy)**: Commonly used in machine translation and natural language generation, BLEU measures the overlap of n-grams between generated and reference texts. Despite its popularity, BLEU has limitations, particularly in capturing semantic similarity and evaluating beyond exact matches Reiter (2018).
* **ROUGE (Recall-Oriented Understudy for Gisting Evaluation)**: Focuses on recall-oriented evaluation by comparing the overlap of n-grams, word sequences, and word pairs between generated summaries and reference texts. It is highly effective for summarization tasks Ganesan (2018).
* **METEOR (Metric for Evaluation of Translation with Explicit ORdering)**: Incorporates stemming, synonymy, and flexible matching, providing a more nuanced evaluation than BLEU. It strongly correlates with human judgments, especially in translation tasks Dobre (2015).
* **BERTScore**: Leverages contextual embeddings from pre-trained transformer models to measure semantic similarity between generated and reference texts. Unlike n-gram-based metrics, BERTScore captures meaning and context, offering a robust evaluation for text generation tasks Zhang* et al. (2020).

The reliability and faithfulness of generated text, particularly in applications requiring high accuracy, such as medical or financial domains is crucial. To identify inaccuracies, hallucination detection was conducted using Prometheus 2, a robust evaluation model designed for analyzing outputs of Large Language Models (LLMs) Kim et al. (2024). This framework helps evaluate three critical dimensions:

* **Faithfulness**: Ensures that the generated content aligns with the source data and avoids unsupported claims Madsen et al. (2022); Jacovi and Goldberg (2020).
* **Correctness**: Measures factual accuracy and checks for logical consistency in the output Yao and Koller (2024); Kim et al. (2024).
* **Fluency**: Evaluates the readability and linguistic quality of the text, ensuring it adheres to natural language norms Suada et al. (2021); Lee et al. (2023).

## Appendix B Models for Fine-tuning

* **LLaMA 2-Chat 7B**Touvron et al. (2023): LLAMA 2-Chat 7B is a fine-tuned variant of the LLaMA 2 series, optimized for dialogue applications. It employs an autoregressive transformer architecture and has been trained on a diverse dataset comprising 2 trillion tokens from publicly available sources. The fine-tuning process incorporates over one million human-annotated examples to enhance its conversational capabilities and alignment with human preferences for helpfulness and safety.
* **StructLM 7B**Zhuang et al. (2024): StructLM 7B is a large language model finetuned specifically for structured knowledge grounding tasks. It utilizes the CodeLlamad-Instruct model as its base and is trained on the SKGInstruct dataset, which encompasses a mixture of 19 structured knowledge grounding tasks. This specialized training enables StructLM to effectively process and generate text from structured data sources such as tables, databases, and knowledge graphs, making it robust in domain-specific text generation tasks.
* **Mistral 7B-Instruct**Jiang et al. (2023): Mistral 7B-Instruct is an instruction fine-tuned version of the Mistral 7B model, designed to handle a wide array of tasks by following diverse instructions. It features a 32k context window and employs a Rope-theta of 1e6, without utilizing sliding-window attention. This configuration allows Mistral 7B-Instruct to perform effectively in multi-modal and domain-adapted text generation scenarios, achieving state-of-the-art performance in various benchmarks.

## Appendix C Prometheus Evaluation

To evaluate model-based metrics, the Prometheus framework Kim et al. (2024) was employed, utilizing structured prompts for three key evaluation

criteria: fluency, correctness, and faithfulness. The primary framework leverages an Absolute System Prompt, which defines the role of the evaluator and ensures objective, consistent assessments based on established rubrics. This Absolute System Prompt, shown in Listing 1, forms the foundation for all evaluations across metrics.

```
Youareafairjudgeassistanttaskedwithprovidingclear,objectivefeedbackbasedonspecificcriteria,ensuringeachassessmentreflectstheabsolutestandardsestforperformance.
```

Listing 1: Absolute System Prompt

The task descriptions for evaluating fluency, correctness, and faithfulness share a similar structure, as shown in Listing 2,3. These instructions define the evaluation process, requiring detailed feedback and a score between 1 and 5, strictly adhering to a given rubric.

```
##TaskDescription: Aninstruction(mightincludeanInputinsideit),aresponsetoevaluate,areferenceanswerthatgetsacoresof5,andascorerubicrepresentingevaluationcriteria aregiven.
1Writeatdeatifiedfeedbackthatassessthequalityoftheresponsecstrictlybasedonthegivenscorerobirc,notevaluatingingeneral.
2Afterwritingafeedback,writeascorethatisanintegerbetween1and5.Youshouldrefertothescorreurbic.
3Theoutputformatshouldlookassfollows:"Feedback:(writeagfeedbackforciteria][RESULT][anintegernumberbetween1and5]"4Pleasedonotgenerateanyotheropening,closing,andexplanations.
5Onlyevaluateoncommonthingsbetweengeneratedanswerandreferenceanswer.Don'tevaluationethingswhicharepresentinferenceanswerbutnotingeneratedanswer.
```

Listing 3: Task description used for evaluation of fluency and correctness

Instructions for EvaluationPrometheus prompts are customized for each evaluation metric. Below are the specialized structures and rubrics for fluency, faithfulness, and correctness.

FaithfulnessThis metric ensures the generated response aligns with both the provided context and reference answers. The evaluation structure incorporates specific rubrics for relevance and information consistency.

```
##TaskDeccription: Aninstruction(mightincludeanInputinsideit),aresponsetoevaluate,areferenceanswerthatgetascoresof5,andascorerubicrepresentingevaluationcriteriaaregiven.
1Writeadetailedfeedbackthatassessthequalityoftheresponsestrictlybasedonthegivenscorerubric,notevaluatingingeneral.
2Afterwritingafeedback,writeaconstatisanintegerbetween1and5.Youshouldrefertothescorrebrict.
3Theoutportormasthouldooksfollows:"Feedback:(writeagfeedbackforciteria][RESULT][anintegernumberbetween1and5]"4Pleasedonotgenerateanyotheropening,closing,andexplanations.
```

Listing 4: Prompt structured correctness

```##Theinstructiontoevaluate: EvaluatethenfluencyofthegeneratedJSONanswer.
##Context: {Pompt}
##Existinganswer(Score5): {reference_answer}
##Generatesawertocwaluate: {response}
##CordeRubrics: ^<xorel_description>::''Ifthegeneratedanswerisnotmatchingwithanyofthereferenceanswersandalsonothavinginformationfromthecontext. ^*> ^<xorel_description>::''Ifthegeneratedanswerishavinginformationfromthecontexthasourishavainformationfromthecontextandsnowavinginformation.","score3_description>::''Ifthegeneratedanswerishavingrelevantinformationfromthecontextandsomeinformationfromexistinganswerbuthaveadditionalinformationthatdoonetsisintcontextandalsodontonextinalessure." ^<xorel_description>::''Ifthegeneratedanswerishavingrelevantinformationfromthecontextandsomeinformationfromexistinganswer.","score5_description>::''Ifthegeneratedanswerismattendingwiththeexistinganswerandalsobavingrindromothemotext." ##Feedback:



FluencyThis metric evaluates the grammatical accuracy and readability of the generated response.

```
##Theinstructiontoevaluate:EvaluatethefluencyofthegeneratedJ5ONanswer###Responsetoevaluate:(response) ##ReferenceAnswer(Score5): ##ScoreRubrics: "score_description":"ThegeneratedJSONanswerisnotfluentandidsdifficultounderstand.",'score2_description":"ThegeneratedJSONanswerhasseveralgrammaticalerrorsandawafabrasing.",'score3_description":"ThegeneratedJSONanswerismostlyfluentbutcontainssomegrammaticalerrorsorawafabrasing.",'score4_description":"ThegeneratedJSONanswerisfluentwithminorgrammaticalerrorsorawfabrhasing.",'score5_description":"ThegeneratedJSONanswerisperfectlyfluentwithnogrammaticalerrorsorawafabrased%%#Feedback:
```

Listing 6Prompt structured correctness

CorrectnessThis metric assesses the logical accuracy and coherence of the generated response compared to the reference.

```
##Theinstructiontoevaluate: Yourtaskistodevalautethegeneratedanswerandreferenceanswertothequery:(Prompt) ##Responsoteovaluate:{response} ##ReferenceAnswer(Score5): {reference_answer} ##ScoreRubrics: "criteria": "Isthemodelproficientingenerateacobherenceresponse",'scorel_description": "Ifthegeneratedanswerisnotmatchingwithanyofthereferenceansers.",'score2_description": "Ifthegeneratedansweriscordinoferenceanswerbutnotrelevanttouserquery.",'score3_description": "Ifthegeneratedanswerisrelevanttotheuserqueryandreferenceanswerbutcontainsmistakes.",'score4_description": "Ifthegeneratedanswerisrelevanttotheuserqueryandhastheexactsamemetricsasthereferenceanswer,butitisnotasconcise.",'score5_description": "Ifthegeneratedanswerisrelevanttotheuserqueryandfullycorrectaccordingtothereferenceanswer.
``` ##Feedback: ```
Listing 7Prompt structure for eC-Tab2Text

## Appendix D Fine-tuning models

The prompts outlined below utilized for training eC-Tab2Text models (Listing 7) and for the QTSumm dataset (Listing 8).
``` ^Givenfollowingjsonthatcontainsspecificationsofaproduct,generatareviewofthekeycharacteristicswithjsonformat.followthestructureonKeystowritedeOutput:
##Product:ProductorJSONspecifications
##Keys:CombinationofthekeysoftheJSONreviews
##Output:reviewsforJSONreviewsaccordinglytothe keys" ```
Listing 9Prompt structure for QTSumm
``` ^Givenfollowingjsonhatcontainspecificationsofaproduct,genericateviewofthe keycharacteristicstributionfromard.followthestructureonKeystowritetheoutput:
##Product:ColumntableofJSONspecifications
## Keys:ColumnQueryofthedataset Output:Columnssummaryofthedataset" ```

Listing 10: Prompt structural correctness 

"value1"... ], "subcategories2":[  "value1"... ],... ].... ]}, )

Listing 10: JSON Data Format reviews

``` ("url":{ "text":[  "Characteristic1":["Description1"  ],  "Characteristic2":["Description2"  ],... } ) 


* [150] M. C. B. K., and R. J. S. H. Lee. A survey of the data-driven approach to the detection of a new class of objects in the field. In _Proceedings of the 20th International Conference on Machine Learning Research (ICML)_, pages 349-367, 2008.
* [151] D. L. Miller, P. G. W. Anderson, and T. E. Barnes. The use of multiple images for image recognition. In _IEEE Computer Vision & Pattern Recognition Workshops (CVPR)_, volume 1, pages 1-10, 2007. IEEE, 2007.
* [152] Y. Li, X. Zhang, Z. Wang, and Q. Sun. Deep residual learning for object detection. In _NIPS 2015 (IEEE)_, pages 1-10, 2015.
* [153] I. Goodfellow, I. Sutskever, and V. Vanhoucke. Generative adversarial nets: deep neural networks. In _Advances in Neural Information Processing Systems (IJCNN)_, pages 1-10, 2015.
* [154] J. Jia, X. Liu, and J. Xu. Deep residual learning for image recognition with convolutional neural networks. In _ECCV 2015 (IEEE)_, pages 1-10, 2015.
* [155] J. Jia, X. Ren, and J. Sun. Deep residual learning for image recognition. In _International conference on computer vision and pattern recognition_, pages 1-10, 2015.

[MISSING_PAGE_POST]

 

* "SoftwareandConnectivity": "TecnoSparkGo2020willrunAndroidGoUIontopofAndroid16outofthebox.Fortheconnectivity,thensmartphonewillsupportfeatureslikeDualsimVolLTE(Micro-Micro),Wi-Fi802.11,b/g/n,3.5mmjack,Bluetooth4.2,10-pincharging,GPS,andUSB2.0..","DesignandDisplay": "TheTecnoSparkGo2020willreportedlyshowcasea6.52inchFTdisplaywitharesolutionf720x1600pixelsandapixeldensityof269ppi.Thedisplaywillsupposedlyhaveawaterdropnotchwithinminumbezels."," BatteryandPerformance": "TheTecnoSparkGo2020willbecmbeddedwithaeMediateKhelioA20chipsetandPowerVRGE8300GPU.Thedevicewillcomewith2GBRAMand32GBofinternalmemory,whichcanbeexpandedupto256GB.Fwellingthedevicewillbeahuge500mAhdattery." } ```
Listing 12: Output generated with Gemini1.5-flash.
``` { "Cameras":{ "MainCamera":{  "Description":"Single13MPmaincamerawithLEDflash,AutoFlash, and AutoFocus.Offersdigitalzoom.",  "Video":"Specificationsnotprovided." }, "FromCamera":{  "Description":"Single8MPmaincamera.",  "Video":"Specificationsnotprovided." } }, "BatteryandPerformance":{ "Battery":{  "Capacity":"4000mahiLi-ion battery",  "Charging":"Nofastchargingorwiirelesscharging." }, "Performance":{  "Processor":"Quad-core1.3GHzprocessor.SpecificchipsetandGPU informationnotprovided.",  "RAM":"4GB RAM",  "Storage":"64GBinternalstorage,expandableupt64GBviamicroSD card." } }, "DesignandDisplay":{  "Display":{  "Size":"6.53inches(16.59cm)IPS LCDwith720x1560pixels resolution(263ppi).",  "Design":"Waterdropnotch display." }, "Other":{  "Colors":"AvailableinBlueandPurple.",  "FingerprintSensor":"Nofingerprintssensor." }

 

* "SoftwareandConnectivity":{  "OperatingSystem":"Android10(0)withAndroidGocustomUI",  "NetworkCompatibility":"Supports2G,36,and4Gnetworks,makingit suitableforuseacrossvariouslocations",  "SIMSlots":"DualSIM(Nano+Nano),allowingforkflexibleconnectivity options",  "Wi-FiandBluetooth":"EquippedwithWi-Fi802.11b/g/nandBluetooth4.2 forseamlesswirelessconnections",  "GPS":"IcincludesA-GPSandGlonassforaccurateclocationtracking",  "FMRadioandChargingPort":"FeaturesFMAfadioandusesamicroUSB2.0 chargingport" },
*DesignandDisplay='{  "Dimensions":"165.6mmx76.3mmx9.1mm,makingitareasonablysized smartphone",  "Weight":"193g,providagingasolidfeelwithoutbeingoverlyheavy",  "DisplaySize":"6.52incheswithATFTdisplaythatoffersaresresolutionof720x1600pixels",  "PixelDensity":"269ppi,deliveringdecentralarityforeverydayusage",  "Design":"Waterdropnotheddesignaddsanodderntouch;availableincolouroptionsofaQuBlueandTceJadeite" },  "BatteryandPerformance":{  "BatteryCapacity":"500 mAh,ensuringlong-lastingperformancebetweencharges",  "BatteryType":"Li-ion,whichistandardforsmentphonesinthisrange",  "ChargingOptions":"Nowirelesschargingandnowfastchargingfeature,butprovidesadequatteryliffethroughoutthedy",  "Hardware":{  "Chipset":"MediaTekHelioA20",  "RAM":"2GB,sufficientforbasictasksandeverydayapplications",  "InternalStorage":"32GBexpandableupto256GBviacircOSDCard, allowingforallstudorageforappsandmedia"  }  }  } ```

Listing 14: Output generated with StructLM trained with eC-Tab2Text.

 

* [