CDTP: A Benchmark Dataset for Data-to-Text Task
Authors: Chengwei Wu


Reviewer paper summaries: This paper presented a new dataset of triplets-text pairs in Chinese.
The paper shows the data collection, preparation, and cleaning.
The paper shows the performance of some Chinese-focus LM models on two tasks Knowledge Base Completion Task and Text Generation Task.
Summary of strengths: - This paper proposed a massive dataset of Chinese Data-Text pairs, which enables pretraining LLM models to reduce hallucination and improve factuality involving real entities.
- The data collection, preparation, and cleaning are clear.
Summary of weaknesses: The paper may find a lot of benefit from a major revision, here are some critical issues:
- A lack of baselines in Table 2: the paper only shows the model's performance after the training on their additional data while the baselines are not included.
- Since the paper collects the data from Internet sources, a substantial amount would already be included in the pre-training data, the paper should investigate the proportion of the dataset covered in the pre-training data for each model.
- More space to cover many more experiments: the paper only uses approx 6.5 pages while submitting to a long paper track.
- The paper needs to be proofread due to a significantly high number of typos of many kinds.
Comments, suggestions and typos: (This is not an exhaustive list)
- L21-L22: Model names are incorrect: "IntermLM" -> "InternLM", baichuan -> Baichuan, chatGLM -> ChatGLM, moss -> MOSS
- L27: "Pre-training" -> "Pretrained"
- Figure 4 caption:  t-sen -> t-SNE
- L301: All the words should be capitalized
Overall assessment scores: 1.5
Metareview score: 2
Metareview: This paper presents a benchmark for data-to-text generation in Chinese, introducing a large and diverse dataset of triplets-text pairs, which aims to address the scarcity of strictly aligned Chinese data-to-text training data and evaluates the performance of several Chinese-focused language models on Knowledge Base Completion and Text Generation tasks.
Summary of reasons to publish: 1. The paper provides a valuable Chinese benchmark for data-to-text generation, which can facilitate further research and development in this area.
2. The dataset undergoes an automatic cleaning process to remove redundant triples and descriptions, which is a commendable effort to enhance data quality.