ROME: Memorization Insights from Text, Logits and Representation
Authors: Bo Li, Qinghua Zhao, Lijie Wen


Reviewer paper summaries: This paper suggests a novel approach for investigating the memorization issue of LLMs using existing datasets instead of training corpora of the target LLM.
Since "block in, block out" sequences such as idioms and poetry need a whole sequence for their meaning, if LLM can reproduce it, we can say LLM might memorize that sequence.
For "block in, block out" data, this paper suggests three groups (context-independent, conventional, and factual) consisting of seven existing datasets.
With five open-sourced LLMs, this paper conducts a prompt-based completion task to distinguish memorized and unmemorized sequences.
Finally, they prove that memorized and unmemorized sequences in their proposed datasets significantly differ in text/logit/representation-based analysis.
Summary of strengths: - suggesting an alternative approach for testing memorized sequences not needing to know training corpora and allowing a possibility of examining grey/black-box LLMs
- three-dimensional analysis for the differences in memorized/unmemorized sequences
- suggesting candidate datasets for the proposed approach
Summary of weaknesses: - The assumption about "block in, block out" sequences
  - Success/failure of completion task can not ensure that this sequence is trained.
  - For example, we can imagine the following cases: trained but forgot, not trained but (fortunately) generated, and so on.
- Consider providing more information to convince readers. The bellows are examples.
  - In session 4.1, this paper only provides explanations and interpretations of (conducted) experiments, but it should be better to suggest concrete results and scores together.
  - For the completion task, when reading Table 6, a few shots are used for each dataset. It would be better to explain the reason for this task and provide details.
Comments, suggestions and typos: - In line 65, `“lock in block` should be `"block in block`
- In line 576, ` in (Berglund et al., 2023), ` should be `In Berglund et al. (2023)`
Overall assessment scores: 3.5
Metareview score: 2
Metareview: This paper studies non-compositional recall using new datasets of idioms, poems, and proper nouns or jargon. They show this behavior is correlated with perplexity and representational similarity metrics.
Summary of reasons to publish: The paper proposes studying "memorization" through a set of different released datasets, which may be of interest to researchers studying specific exact-match behaviors in settings like idiomatic language.