DataVisT5: A Pre-trained Language Model for Jointly Understanding Text and Data Visualization
Authors: Zhuoyue WAN, Yuanfeng SONG, Shuaimin Li, Chen Jason Zhang


Reviewer paper summaries: This paper presents DataVisT5, a new pre-trained language model based on the T5 model specifically designed for jointly understanding text and data visualization (DV). The authors aim to address the limitations of existing general pre-trained language models for DV tasks by developing a model that can effectively handle cross-modal information between text and DV data.

The paper introduces a T5-based model (DataVisT5), which is further pre-trained on data visualization-related text (schema, code, etc.) and then fine-tuned for text-to-vis, vis-to-text, table-to-text, and vis-question-answering (FeVisQA) tasks. It then presents performance results on all four tasks for the introduced model and several baselines.
Summary of strengths: 1. Utilized a new pre-training approach to better understand DV data.
2. Conducted extensive experiments on various tasks to investigate the performance of the proposed model.

* The paper is written and structured well. 

* The problem is well-motivated. Usability is still the biggest problem with data systems, including data visualization tools. Empowering users to easily create data visualizations will have a high impact across domains. 

* The work covers a complete set of tasks relevant to data visualization.
Summary of weaknesses: i. The paper missed comparisons with various strong baselines, such as [https://arxiv.org/abs/2403.09028]([https://arxiv.org/abs/2305.14761] [https://arxiv.org/abs/2311.16483].

ii. The paper only relied on automatic metric-based evaluation, but missed qualitative evaluation.

iii. Currently, there are many LLMs proposed that have multimodal capabilities. However, the proposed system is only focused on non-image-based data. 

iv. The performance gain every after extensive pre-training and fine-tuning does not look that high in comparison to the base CodeT5. Also, why CodeT5 was not fine-tuned? More ablation tests should have been conducted.

* My concerns center around performance and evaluation. When I look at Tables 1 - 3, I see that the performance numbers of the base model CodeT5+ are close to those of the new model, DataVisT5. The significance of the delta needs to be clarified. This suggests that a user study evaluation is in order.

* I expect LLMs to perform relatively well on table-to-text and vis-to-text tasks, but they aren't included in the performance comparison for these tasks. Including a strong LLM baseline in the performance evaluation of the remaining three tasks (table-to-text, vis-to-text, and vis-question-answering) could help make a case for the model's contributions.
Comments, suggestions and typos: 1. May add some open-source LLMs for comparisons.

* Overall, the paper will benefit from addressing the concerns above. With some push, they can be addressed within the revision cycle.
Overall assessment scores: 3

2.5