Source Attribution for Large Language Model-Generated Data
Authors: Xinyang Lu, Jingtan Wang, Zitong Zhao, Zhongxiang Dai, Chuan-Sheng Foo, See-Kiong Ng, Bryan Kian Hsiang Low


Reviewer paper summaries: This paper presents a framework on source attribution, WASA.  WASA identifies the data provider responsible for the generation of synthetic texts by a LLM. It works by assigning an identifying watermark to the training data of every data providers. The watermark is composed of unicode chars that are invisible to human readers. 
The LLM thus learn to include the watermarks in the generated texts, allowing to attribute the source.
Summary of strengths: The paper addresses a critical issue concerning the protection of intellectual property related to data used to train LLMs.
The proposed method is relatively simple to apply, and the experiments (apart from the issues discussed in next section) show improvement with respect to simple baselines.
Summary of weaknesses: This paper already had a review round, and in the current version many passages are rewritten/revised, but this revision still leaves some issues open.
In the case of scalability the authors reduced the strength of their claims, yet the experiments and results still show that a very large model is needed to perform better than bm25 when n=100. n=100 is a very low value if the method is in theory designed to watermark the training set of an LLM, with potentially hundreds of thousands or millions of sources.
The paper still suffers from an ambiguous definition of what is a source, and the main datasets presented in the paper are more representative of a categorization problem than a source attribution one.
Additional datasets from dbpedia and of news are added to address this aspect, yet it is only reported in the appendix, not revising the main paper. 
DbPedia is still more a categorization task. 
CC-News seems the dataset that better models the idea of source and source attribution and also a real application scenario. 
Both baseline and WASA perform worse than on the other datasets. The author claim that this is due to lower quality of text, yet this is just an unproven hypothesis, as it is also possibly due to a more hard problem of separating sources that are not categorical, which more relevant for WASA.
The CC-News is given marginal treatment while it should be in the main part of the paper.
Also, CC-News is limited to n=10, with the motivation of avoiding data imbalance and having all sources well represented in data. Imbalance and data scarcity are dimensions that are relevant to the problem of source attribution.
Comments, suggestions and typos: Move CC-News to the main paper.
Test n>>100.
Overall assessment scores: 2.5