Dated Data: Tracing Knowledge Cutoffs in Large Language Models
Authors: Jeffrey Cheng
Marc Marone
Orion Weller
Dawn Lawrie
Daniel Khashabi
Benjamin Van Durme
Venue: COLM (2024)
Keywords: knowledge cutoffs, training data, temporal alignment
TL;DR: Singular knowledge cutoff dates do not capture the entirety of LLM training corpora, so we design a simple probing method using time spanning datasets and analyze a large set of open access pretraining corpora.

Abstract: Large Language Models (LLMs) are often paired with a reported cutoff date, the time at which training data was gathered. 
Such information is crucial for applications where the LLM must provide up-to-date information. However, a reported cutoff only scratches the surface. Do all sub-resources in the training data share the same cutoff? Does the model's demonstrated knowledge for these sub-resources closely align to their cutoff? We define the notion of an effective cutoff, which is distinct from the LLM's reported cutoff and differs between sub-resources. We propose a simple approach to estimate effective cutoffs of an LLM on the resource-level by probing across versions of the data. Crucially, our method does not require access to a model's pre-training data. Through our analysis, we find that effective cutoffs often drastically differ from reported cutoffs. To understand the root cause of this observation, we conduct a large-scale analysis on open pre-training datasets. Our analysis reveals two reasons for these inconsistencies: (1) temporal misalignments of CommonCrawl data due to non-trivial amounts of old data in new dumps; and (2) complications in LLM deduplication schemes involving semantic duplicates and lexical near-duplicates. Overall, our results show that cutoffs are not as simple as they have seemed and that care must be taken both by LLM dataset curators as well as practitioners who seek to use these models.

Published as a conference paper at COLM 2024
Dated Data: Tracing Knowledge Cutoffs in Large Language Models
Jeffrey Cheng
Mare Marone
Orion Weller
Dawn Lawrie
Daniel Khashabi
Benjamin Van Durme
Johns Hopkins University
j jcheng71@jhu. edu
Abstract
Large Language Models (LLMs) are often paired with a reported cutoff date, the time at which training data was gathered. Such information is cru- cial for applications where the LLM must provide up-to-date information. However, a reported cutoff only scratches the surface. Do all sub-resources in the training data share the same cutoff? Does the model's demonstrated knowledge for these sub-resources closely align to their cutoff? We define the notion of an effective cutoff, which is distinct from the LLM's reported cutoff and differs between sub-resources. We propose a simple approach to estimate effective cutoffs of an LLM on the resource-level by probing across versions of the data. Crucially, our method does not require access to model's pre-training data. Through our analysis, we find that effec- tive cutoffs often drastically differ from reported cutoffs. To understand the root cause of this observation, we conduct a large-scale analysis on open pre-training datasets. Our analysis reveals two reasons for these inconsistencies: (1) temporal misalignments of CommonCraw! data due to non-trivial amounts of old data in new dumps; and (2) complications in LLM deduplication schemes involving semantic duplicates and lexical near-duplicates. Overall, our results show that cutoffs are not as simple as they have seemed and that care must be taken both by LLM dataset curators well as practitioners who seek to use these models. We release our results and the code to replicate them at https:/ ggithub.com coam///eync////eee /aseync/daaedddat////
1 Introduction
Many Large Language Model (LLM) creators do not elect to release their training data due o a reported cutoff date for their model. When faced with a description that states, e.g. "this model has cutoff date of March 2024," does that mean all of its included resources share the exact same cutoff date? Even if the model provides an explicit reported cutoff for a resource (e.g. the Wikipedia dump date), does that imply that the model's knowledge of that resource, or effective cutoff,, is the same as the reported cutoff date? For LLM users, these questions can be crucial: imagine a layperson using an LLM for tax advice, without realizing that the effective cutoff of the tax code is 2022 and thus outdated - despite the fact that the reported cutoff is advertised as 2023 (Fig. 1).
to competitive reasons. In place of providing the exact pre-training data, they often provide
As a result, there has been push for researchers to document their data (Mitchell et al., 2018; Pushkarna et al., 2022; Gebru et al., 2021; Luccioni et al., 2022), identify what data is in these models with membership inference tests (Carlini et al., 2022; Piktus et al., 2023; Marone Van Durme, 2023), and otherwise reproduce data from their training set (Carlini et al., 2021; Ippolito et al., 2022; Nasr et al., 2023; Weller et al., 2023). However, these tests generally only check for static inclusion of the data, rather than identifying when the resources stopped being included. Even more difficult are cases where there exist multiple versions of a resource, where different versions can contain information that is updated, deleted, or even conflicting with the previous versions.
1

---

Published as a conference paper at COLM 2024
As of my knowledge cutoff in March 2024, the limit is $20,000
Do need to issue a 1099-K for $10,000?
2022 S IRS
2024 K FIRS
Form 1099-K issued for transactions only if the aggregate amount these transactions exceeded $20,000
Now single transaction exceeding $5000 can require the third party platform issue 1099-K.
Figure 1: LLMs may contain different versions of a dataset in their training data than what is specified in a "cutoff" date, misleading users and causing potential errors.
Given the crucial importance of these knowledge cutoffs and the lack of transparency from LLM creators, we seek to automatically determine the effective cutoff of models with respect to a given resource, without needing access to the model's ' training data. We measure the perplexity of LLMs over varying versions of resources, identifying the effective cutoffs as the minima of the perplexity over time measurements. Our contributions are follows: (1) we collect resource sets spanning long time frames and propose a simple method to determine effective cutoff of LLMs; (2) we show that for variety of models (particularly newer models), the resource-level effective cutoffs differ drastically from their reported cutoff date; and (3) we provide an analysis detailing the causes of these misaligmments, showing that pre-trainnnn pre-training datasets suffer from deduplication complications and that CommonCrawl dumps exhibit temporal misalignment from the dump dates.
2 Related Work
Documenting and Describing LLM Training Data As the size of the data in LLMs in- creases, there have been many calls for researchers to document datasets through additional documentation artifacts. These include Model Cards, Datasheets, and Data Cards - each focusing on documentation of a specific part of model data source (Mitchell et al., 2018; Gebru et al., 2021; Pushkarna et al., 2022). The open-access community has adopted versions of these (e.g. Huggingface model descriptions) but they do not provide fine-grained versioning that enables precise tracking of cutoffs. For example, it may not be clear whether scraped Common Crawl data contains additional versions of scraped Wikipedia.
Other research has focused on more fine-grained analysis, such as the role of filtering in LLM-data creation (Gururangan et al., 2022; Lucy et al., 2024) or how PII, toxic data, n-grams, and provenance play role in LLM data (Dodge et al., 2021; Elazar et al., 2023). These works have provided great insight into LLMs, but necessarily depend on the data being available to the public. In contrast, our work focuses instead on determining what temporal versions of data exist in a model, without access to the training data.
Membership Inference Many prominent LLMs do not provide their training data or even
descriptive information about them, leading people to wonder if their data is included in the model's training set. Techniques like membership inference attacks (Shokri et al., 2017) have been applied to LLMs. Many strategies have been proposed for this problem: they include using similar but synthetic data (Mattern et al., 2023), prompting calibration (Fu et al., 2023), and a variety of other techniques (Hisamoto et al., 2019; Shi et al., 2023; Faysse et al., 2024). While most strategies rely on the LLM's perplexity over potential data instances, there has also been effort in black-box attacks using cloze tasks (Chang et al., 2023). Overall, membership inference testing focuses on whether an instance was included in the LLMs training set (with the critical assumption that there was only one version). In contrast, our work focuses on which version(s) of the data was included in the LLMs training data.
Continual Learning in LLMs New written knowledge increases every day, but language models remain static. As re-training LLM is prohibitively expensive, it is infeasible for LLMs to keep up with living online resources. Thus, there exists large field of research on continual learning, or helping LLMs stay up to date without expensive re-training. This
For example, Wikipedia gets edited about every seconds
2

---

Published as a conference paper at COLM 2024
typically involves modeling approaches that perform limited continued training to keep model knowledge up to date (e.g. Hu et al. 2023; Kasai et al. 2023, among others). Our work also involves examining the temporal knowledge in these models, but differs by focusing on if there is temporal misalignment in the original static models and what knowledge they contain rather than techniques to align them.
3 Methodology
We seek to probe LLMs to determine their resource-level effective cutoffs, defining the effective cutoff date of model with respect to a resource as the date of the version of the resource that is most closely aligned with the model. This effective cutoff date can differ from the inclusion date of model's sub-resources that LLM creators sometime provide, which is typically the last timestamp of that resource but does not address additional sources of earlier data.2 2 This is relevant because given that a certain Wikipedia dump included in training data, it is reasonable to assume that the effective cutoff for those articles is the corresponding dump date. However, older versions of similar text may be present in web scrapes like Craw.
To perform our analysis, we construct long spanning (2016-2023) datasets and measure perplexity the data across variety of models. We then analyze the implications of these perplexity-at--iin curves and verify our results with the ground truth pre-training data.
3.1 Timm--Sanning Datasets
Broadly, online resources included in LLM training data be divided into three subsets: resources that involve frequent updates (egg. legal texts or Wikipedia), resources that are static but build over time (e.g. blogs or news articles), and purely static resources (e.g. books). We construct representative datasets that allow us to probe the first two types, choosing Wikipedia for the updating resource and New York Times for the building resource.
WIKISPAN Wikipedia is commonly used in pre-training (Gao et al., 2020), provides broad topic coverage, and changes fre- quently. To create the dataset, we collect the 50003 most edited documents by number of edits.4 4 For each of these, we use the Wiki API to collect version of the document at monthly intervals, from April 2016 to April 2023, Our final dataset therefore con- sists of the same 5000 documents changing monthly over this seven year period. We call this dataset WIKISPAN, and individual documents are the version of the Wikipedia document on topic T at month M.
5..
5.6
Jan 2016
2016
dap 2010
Jan 2017
may 2017
Jan 2010
tay for 2
paay yor 2010
jan 2022
pay deg lan 2022
del 2012
May
deg Jan
payy
Figure 2: Perplexity of the Wiki document "Liverpool" under Pythia. Each point is the perplexity of the document at that time.
NEWSSPAN We use New York Times (NYT) articles to represent our muilding-over-tting resource, as the documents contain long-form high quality text, are frequently included in pre-training data and CommonCrawl dumps, and provide long and steady stream of new documents. For our probing dataset, called NEWSSPAN, we collect all the articles with top level domain "nytimes.com dumps (Soldainí et al., 2024). We bucket the articles by month according to their publication date, and collect 500 articles from each bucket from January 2016 until July 2020. Note that due to copyright concerns, CommonCrawl removed NYT articles from recent dumps
from curated collection of the 20 most recent CommonCrawl
hww.gn/gtt....aaanaaaaaaeeeeeeee co/allenai/OMM, 7B reports using the January 2023 dump of Wikipedia We filter out 100 topics that did not exist all the way back to 2016. httees//eewwwwiiie http::///eewwiiiiiie...o wiki Wikipedia:Most frequently edited pages, from May 2023
3

---

Published as a conference paper at COLM 2024
Model Name
Pile
C4
RW
CC Dumps
Wiki Dump
CC Cutoff
Pythia (Biderman et al., 2023)
'20
GPT-Neo (Black et al., 2022)
'20
GPT-J (Wang & Komatsuzaki, 2021)
'20
RedPajamas (Computer, 2023)
5 (119'23)
Mar '23
Jan '23
Falcon (Almazrouei et al., 2023)
Feb '23
FalconRW (Almazrouei et al., 2023)
Feb '23
OLMo (Groeneveld et al., 2024)
20 (220-223)
Mar '23
June '23
LLaMA (Touvron et al., 2023a)
5 ('17-'20)
Aug '22
'20
Table 1: Different decoder-only LLMs and their corresponding pre-training data. The CommonCraw dumps are processed to various degrees. Frequently used datasets with their own columns include Refined (RW), C4, and the Pile. CC Cutoff indicates the last CommonCrawl dump included, unknown months are marked with a ?.
and have stopped scraping it. Unlike the documents in WIKISPAN, the documents in each bucket have no relation with one another.
3.2 Probing Methodology
Our goal is to determine the effective cutoff of the model for a given resource, rather than focusing on individual topics or documents. We do this by measuring perplexity on documents in each month bucket, using the first 512 tokens for Wikipedia and 256 for NYT (due to shorter documents). See Fig. 2 for example.
Normalization Occasionally some documents in our time-spanning datasets have drasti- cally different perplexity compared previous months (e.g. a Wikipedia page changing to become redirect page rather than content page for one month). As this distracts from understanding the relative model perplexity across months, we follow previous work and aggregate perplexities by taking an average of the measured perplexities after discarding the lowest and highest 2.5% of measurements in each month (Shi et al., 2023).
Perplexity measurements are not comparable across unrelated models due to data and train- ing differences. Thus, we normalize the averaged perplexities to 0-1 scale by performing min-max scaling over the entire time-span in order to compare their fluctuations across time. We call these relative perplexities. We take the time which relative perplexities are minimized to be the effective knowledge cutoff. Since the minima not always sharp, the effective cutoffs should be interpreted as distribution over time.
3.3 Mining from Pre-training Data
We hypothesize that similar documents in training impact model knowledge and relative perplexity measurements. For example, if parts of a resource were duplicated many times, we might expect perplexity on those documents to be particularly low. Prior work has shown the effects of document frequency LLM memorization (Carlini et al., 2022).
To better understand the perplexity curves, we search for documents similar to those in our time-spanning datasets. This retrieves old versions of the documents, near duplicates, and copied fragments all of which may impact information in the model and our perplexity measurements. We expect that the distribution mined from training data is inversely correlated with the perplexity trends - the counts of the versions of the retrieved Wikipedia- alike documents should be higher at the same months where perplexity is lower. The entire process consists of obtaining and indexing nearly 4T tokens from several LLM training sets.
Given a pre-training dataset, we first construct a BM25 index over it using Elasticsearch. For each topic, we use the first 512 words of the document as the query to find similar documents." We use the BM25 scores as first step filter in finding near duplicates.
We the version of Wikipedia from the model's dump date for queries
4

---

Published as a conference paper at COLM 2024
Using the top ten BM25 results, calculate the edit distance between the matched docu- ments and every version of the corresponding Wikipedia topic in WIKISPAN, normalizing by the character length of the matched document to avoid length biases. We classify matched documents as Wikipedia document only if the minimum of this normalized edit distance score is less than 0.2. With this subset of similar documents, we attribute each document to its closest matching month by edit distance (including ties). This then allows plot the ground truth distribution of all documents similar to the original by date. We provide more detailed description of this algorithm in Appendix A.
4 Experimental Setup
4.1 Pre-Training Datasets
The LLMs we evaluate were pre-trained on data derived from three major datasets: C4 (Raffel et al., 2020), the Pile (Gao et al., 2020), and Ref (Penedo et al., 2023), as well as additional CommonCrawl dumps. We describe their contents, as well how different LLMs used them during training, focusing on subcorporas that may contain Wikipedia or NYTimes content. We note that no open pre-training dataset ever includes a direct dump of NYT articles; they are only present in included CommonCrawl dumps.
C4 C4 is a single, heavily processed, open-access CommonCrawl dump from April 2019. It uses content-based filters to discard documents containing undesirable content such as obscene words, boilerplate templates, and code. C4 is deduplicated at a three sentence span level, and has an overall size of about 750 GB.
The Pile is an open-access curated dataset consisting of 22 sub-datasets and totals around 800GB. The relevant sub-datasets are Pile-CC and the Wikipedia dump. The Pile-CC is deduplicated at document level, and consists of 22 random chunks out of the 3679 extracted from CommonCrawl dumps between 2013-2020, The Wikipedia dump is from March 2020 and is up-sampled three times.
Pile
Reb Web (RW) consists of documents from CommonCrawl dumps span- ning 2008 to February 2023. Because RW was designed to be used in conjunction with other high quality data sources, URLs from specific top-level domains (including "wikipedia.org" are excluded. The public RW is only 600B token sample of the total 5T token dataset.
4.2 Models
We evaluate a variety of decoder-only Transformer LLMs with accessible (or described) data, as shown in Table 1. We provide speculation about closed-data models in Appendix D, but as we cannot verify correctness, we do not include these results in the main text.
Pythia (Biderman et al., 2023), GPT-Neo (Black al., 2022), and GPT-J (Wang & Komatsuzaki, 2021) were trained on the Pile, and represented early attempts to replicate closed-acced models by the open-source community. The Pythia suite was designed to provide a replica- ble training process for studying ideas like training dynamics and memorization in LLMs. The RedPajama model suite (Computer, 2023) was intended as an open-access replica of the LLaMA model (Touvron et al., 2023a), providing replicable pre-training data sourced from web-crawls. The Falcon suites are trained on RefinedWeb, a dataset consisting solely of web-scraped data (Almazrouei et al., 2023). Finally, OLMo (Groeneveld et al., 2024), and LLaMA (Touvron et al., 2023a) are the latest generation of LLMs, trained on datasets used by prior models and large amounts of CommonCtawl. These models span several iterations of LLM pre-training approaches (data curation, data size, and model size). Table 1 describes these models in terms of their corresponding training sets and data versions.
6 At most 20% of document can to be changed count as exact match version of the topic.
5

---

Published as a conference paper at COLM 2024
5 Results
In this section we discuss the results of probing language models to determine their effective cutoffs and compare the dates with the ground truth distributions from their training sets.
5.1 NEWSSPAN
As mentioned in Section 3.1, we use NYT data until mid-2020, and thus only evaluate models with pre-2021 CommonCrawl cutoffs due to the lack of evaluation data. Our results are shown in Fig. 3. For each of the models, the perplexity curve increases in early 2020, which is the date of the last CommonCrawi dump included in the Pile. Thus, we see that is possible to determine the effective knowledge cutoff of different articles posted over time using this method.
1.0
LLaMA
Pythia
Reletive Perplexities
0.0
GPT-J
GPT-Neo
0.6
0.4
0.0
Jan 2016
Apr 2616
2016
oct 2016
Jan 2017
Apr 2017
jul 2017
oct 2017
Jan 2018
Apr 2018
jut 2018
2018
Jan 2019
Apr 2019
2019
Oct 2019
Jan 2020
Apr 2020
Aul 2020
Figure 3: Relative perplexities models per month using the NEWSSPAN ($3.1) dataset (we
use relative exact perplexities not needed for determining effective cutoffs). We find that our approach identifies the effective cutoffs as the stated knowledge cutoff for NYT, as models have increased perplexity when their CommnonCrawl data dumps end in 2020.
5.2 WIKISPAN
As discussed in Section 4.1, there are three major categories of datasets that models are trained on: C4, the Pile, and Falcon RefinedWeb (RW) and we note that the datasets are only a subset of the training set for some models. Again only uncomparable relative perplexities are shown absolute perplexities between different models not relevant to our goal of determining knowledge-cutoffs. For each category, we also overlay the computed distribution of similar ground truth documents in light grey (when available) to show the correlation between the ground truth results and effective cutoffs.
Pile-based Models The three models GPT-Neo, GPT-J and Pythia are exclusively trained on the Pile. We show the results of the perplexity measurements in Fig. 4 (upper), where we see noticeable drop in perplexity around March 2020. This month corresponds exactly to the date of the Wikipedia included in the Pile, indicating the effective cutoff for Wikipedia of the models aligns with the reported Wikipedia cutoff. Appropriately, we also note that the distribution of ground truth versions highest at that month, corresponding to the 3x up-sampled Wikipedia dump in the Pile.
Falson Models FalconRW is exclusively trained on RW while Falcon incorpo- rates curated corpora from the Pile. Note that both models were trained on subset of the RW dataset, the exact subset which is not publicly described. We see in Fig, 4 (middle) that the perplexity curves have a low perplexity from late 2019 to early 2021 with mini- mum around January 2020. These results may seem surprising FalconRW has not seen Wikipedia training - however, as the overlayed ground truth and Section 6.1 shows, does contain Wikipedia content found other top-level domains.

---

Published as a conference paper at COLM 2024
Pile Derived Models
Reletive Per perplxiities
0.6
0.4
Pythia-6.98
0.2
GB
GPTNco-2.38
0.0
FalconRW Derived Models
1.0
Falcon-78
RalconRW-P
Relative Per plexities
0.6
0.4
0.0
C4 Derived Models
1.0
RedPajamas-7B
Relative Perplexities
0.8
OLMo-78
LLaMA-78
0.4
0.2
Apr 2016
2010
oct 2016
Jan 2017
Apr 2017
Jul 2017
Oct 2017
Jan 2018
2018
2016
Oct 2018
Jan 2019
Apr 2019
Jul 2019
2019
Jan 2020
Apr 2020
Jur 2020
oct 2020
Jan 2021
Apr 2021
2ct2 2021
Jan 2022
Apr 2022
Jul 2012
Oct 2022
Jan 2023
Apr 2023
Figure 4: Relative perplexities of models per month using the WIKISPAN ($3.1) dataset. Upper plot shows Pile derived models, middle shows FalconKW derived models, while lower shows C4 derived models. The light grey bars indicate the distribution of Wikipedia- alike documents, matched their closest version, as calculated in Section 3.3. In some cases the knowledge cutoff aligns with the model's effective cutoff (e.g. the Pile) while more recent models are aligned much earlier (e.g. RedPajamas to 2019, even though it has an explicit 2023 Wikipedia dump).
C4-based Models Fig. 4 (lower) shows results for C4-derived models: RedPajamas, OLMo, and LLaMa. While each model uses C4 during pre-training, it only comprises small portion of their respective training data. The more salient similarity is that each of the models consists of many independent CommonCrawl dumps, and the differences in effective cutoff dates of the three models can be explained by the CommonCrawi dumps included in their training data. LLaMA incorporates dumps from 2017 to 2020, and its cutoff date is thus in that range. In contrast, RedPajant ajamas incorporates 5 dumps from 2019 to 2023, and its effective cutoff a few months later. OLMo uses all 20 dumps from 2019 to 2023, and thus sees the latest effective cutoff. Nonetheless, the effect of C4 on these models is evidenced by the effective cutoffs being biased towards the C4 CommonCrawl dump date (April 2019). See Section 6.2 for breakdown of the entire training data of RedPajamas,
Impact of Scale We also consider the effect of model size our methods. We evaluate the perplexity of WIKISPAN under a suite of Pythia (460M, 1B, 2.8B, 6.BB, 12B) and LLaMA (7B, 13B, 65B) models. Fig. 5 shows that while the smaller models have more varied perplexity curve, they are still minimized at the expected date. This makes intuitive sense as the models are all trained on the same data.

---

Published as a conference paper at COLM 2024
08-
138
a.--
6.5E
78
2.88
2018
jel 2012
2012
Jan 2020
Jul 2030
2021
Jan 3822
Jul 7022
Jin 2013
Jan 2010
pin 2019
2ut
Jan 2010
Jal
Jin 2021
Jái 20/202
Jon 2022
Jin 2023
Jin
Jul
Jdl
Figure 5: Relative perplexities of models in the Pythia (left) and LLaMA (right) suites.
Darker colors indicate larger model size. While the smaller models have more variable perplexity curve, they are still minimized at the same effective cutoff date.
Number of Documents We lastly consider the effect of the number of documents mea- sured. In some domains, document collection may be difficult; as such, we evaluate our method by varying x, the number of documents considered (2500, 1000, 500, 250, 100, 50) for the three C4 derived models. Fig. shows for x > 50, the effective cutoffs the three models consistent with the full results. trend are less consistent, likely due to the increased variance in the data. This shows that our method is robust even when many versions of documents cannot be collected.
50 appears to be the threshold where the
RedPajamas
LLaMA
Relative Perplexíties
5000
2500
500
100
50
OLMo
Jan 2016
May 2016
Sep 2016
Jan 2017
2017
May 2017
Sep 2017
Jan 2018
May 2018
sep 2018
Jan 2019
May 2019
Sep 2019
Jan 2020
May 2020
Sep 2020
2021
May 2021
Cep 2021
lart 2022
May 2022"
Sep 2022
Jan 2023
lor
Figure 6: Relative perplexities of RedPajamas (top), LLaMA (middle), and OLMo (bottom) when varying x, the number of documents in each bucket. Darker colors indicate more documents, and the black lines corresponding to 5000 are the results shown in Fig. For small x, the perplexity curves are more variable due to the smaller sample size, but for x 50, the ablated results are consistent with the full results.
Why are models not aligned to their cutoff date?
In this section describe why model's effective cutoff and reported cutoff can differ. This mismatch due to two main factors: (1) deduplication pipelines that ignore semantically equivalent but lexically near duplicates and (2) temporal biases of CommonCrawl dumps.
8

---

Published as a conference paper at COLM 2024
6.1 Complications in Deduplication Pipelines
It is common practice in LLM training pipelines to deduplicate data. In the context of Wikipedia, when dataset undergoes fuzzy or exact deduplication, one might expect that different versions of Wikipedia pages (near duplicates) and copies of Wikipedia pages (exact duplicates), respectively, are removed from the dataset. However, we empirically find that there exist a large number of near and exact duplicates in training datasets, and provide examples for each. This shows that deduplication pipelines are unable to detect the extra copies and versions of Wikipedia documents present in CommonCrawl dumps.
FalconRW FalconRW removed all documents that had Wikipedia top level domain so they could use FalconRW in conjunction with curated versions in the future (as they did in for the Falcon dataset). They assumed this would deduplicate the data, however, we find that there are still near duplicate Wikipedia documents, as shown in Table 2.
C4 C4 was created from one CommonCrawl dump and "discarded all but one of any three-sentence span occurring more than once." However, we show an example in Table of a pair of documents which contain the same three-sentence span. We also observe that the documents are semantically equivalent, and differ only by whitespace characters.
RedPajamas We find that the RedPajamas CommonCrawl dump that was paragraph-level deduplicated contains exact duplicate documents. We show an example in Table 4.
Discussion Out of all the models we evaluated, only Pile derived models exhibit sharp alignment towards their reported Wikipedia dump date. This due to two main factors: the size of its CommonCrawl data (which is minor compared to other models) and the purposeful up-sampling of their Wikipedia dump to match their desired date. The massive amounts of CommonCrawl data that other models are trained on compounds their issues with deduplication, leading to many versions of Wikipedia documents which are not necessarily of the version of their reported dump date.
We confirm this hypothesis by comparing Pythia vs. Pythia-deduplicated.. Fig. 7 shows that the deduplicated Pythia, which removes the purposefully up-sampled Wikipedia documents, no longer has the sharp minimum of standard Pythia and instead has an earlier effective cutoff (due to the older CC documents). Thus, we see that the accidental duplicates and the lack of purposeful duplicates (of versions corresponding to the desired effective knowledge cutoff) creates this misalignment in the deduped models.
sempoeattes
Pytiia 6.58 Deduped
Pyth a 128 ayth Deduped
04
ac.
2012
2012016
Jnn 2019
Ae 2019
20z0
Asl 2010
2022
Aul 2011
2022
Jst 7022
Jan 2023
2010
Aul 2018
2015
2019
Jan 2020
All 2020
Jan 2021
All 2021
2022
jul 20z2
2023
Figure 7: Relative perplexities of models trained on Pile and Pile-dedup. We see that dedu-
plicating the 3x ap-sampled Wikipedia in the Pile results in an older temporal alignment due to the included Wikipedia documents from CommonCrawl.
6.2 Misalignment of CommonCrawl Dump Dates
All our evaluated models were trained on some portion of CommonCrawl data, with recent models using larger proportions of it. Our ground truth results in Fig. 4 (especially C4-derived models) confirm previous work from Dodge et al. (2021) that suggests a non- trivial amount of data inside of a CommonCrawl dump actually old data. In the context of Wikipedia, this means that a CommonCrawl dump in 2023 contains many versions of documents dating back to 2016. While models may include range of CommonCrawl

---

Published as a conference paper at COLM 2024
1.0
W Wikipedia in C4
Wikipedia Dump
Number of Versions
0.8-
Wikipedia in CommonCiawll
0.6-
0.4
0.2
0.0
Apr 2016
Jul 2016
2016
Jan 2017
Apr 2017
2017
oct2
2018
Apr 2018
2010
oct 2019
jen 2019
Apr 2019
2012
oct 2010
Jan 2020
Apr 2020
Jul 2020
oct 2020
Jan 2021
Apr 2021
Jul 2021
Oct 2021
Jan 2012
Apr 2022
Jul 2022
Oct 2022
Jan 2023
Apr 2023
Figure 8: Distribution of Wikipedia versions over the entire RedPajamas training set. Each
color represents a different supcorpora of the training set. The black line represents the relative perplexity curve of RedPajamas 7B over WIKISPAN.
dumps, the aggregated data will thus be extremely biased towards earlier dates. To illustrate this concretely for a "newer" style model composed of large amounts of CommonCrawl, we collect the ground truth from all the resources containing Wikipedia in Redmajamas (CommonCrawl, C4, and explicit Wikipedia) and overlay its perplexity curve in Fig. 8.7
We see that although the direct Wikipedia dump is included in the pre-training data, over 80% of the Wikipedia documents are from earlier versions (pre-2023). Moreover, versions from the earlier months can and often have duplicate versions as described previously, while the documents in the direct Wikipedia dump are typically not duplicated. We also see that perplexity minimized around the date of these CommonCraww Wikipedia versions that compose the majority, in mid-2019.
6.3 Summary
In our analysis of available pretraining datasets, we find that CommonCrawl dumps often include multiple copies of different versions of documents (e.g. Wikipedia). These extra copies and versions are frequently undetected by deduplication pipelines and moreover can consist of outdated information, biasing the effective cutoffs of language models. Thus, we see that there exists two reasons that contribute to the temporal mismatch of a language model's 'ss reported and effective cutoff: (1) failures of deduplication pipelines to control for semantic duplicates and (2) the use of newer CommonCrawl dumps to provide updated information when there is a significant amount older data in the dumps.
7 Conclusion
It is now common practice for Large Language Models to provide a "knowledge-cutoff" which intends to communicate users the date at which LLMs no longer have up to date information. However, this simple metric oversimplifies LLM training in detrimental manner usability; it leaves unanswered the questions of "is this knowledge cutoff specific for all resources in the model", "how many copies of my my resource are in the model" or "which versions of my corpus are included?" We propose a method to automatically determine the effective cutoff date of LLMs for given resource and show that although sometimes it does align with the reported cutoff, in many cases it does not. To determine why they fail to align, analyze the training data of open-data LLMs to discover that there are large quantities of near-duplicates in LLM training data (such as differing only in the citation numbers included in the text) despite efforts from LLM creators to deduplicate. Further, most LLMs rely on CommonCrawl dumps for data, despite the fact that a non-trivial amount of CommonCrawl data is much older than the reported dump date. We hope this analysis will provide insight for users of LLMs who need resource-specific knowledge cutoffs and for LLM-creators who seek to align their LLMs a given date.
Note that although we performed this analysis by binning documents by their most similar version, one can compute n-gram anaylsis with similar results (Appendix C).
10

---

Published as a conference paper at COLM 2024
8 Acknowledgements
We thank JHU HLTCOE for the support and resources used for this project. We additionally thank the students in JHU CLSP for the valuable feedback, particularly Nathaniel Weir, Kate Sanders, Zhengping Jiang, Jingyu (Jack) Zhang and Aleem Khan. OW is supported by the National Science Foundation Graduate Research Fellowship Program. Any opinion, findings, and conclusions recom commendationss expressed in this material are those of the authors(s) and do not necessarily reflect the views of the National Science Foundation.
References
Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxan- dra Cojocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, al. The falcon series of open language models. arXiv preprint arXiv:2311.16867, 2023.
Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, pp. 2397-2430.
PMLR, 2023.
Sid Black, Stella Biderman, Eric Hallahan, Quentin G. Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Martin Pieler, USVSN Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Benqi Wang, and Samuel Weinbach. Gpt-neox-20b: An open-source autoregressive language model. ArXiv, abs/2204.06745 2022.
URL https: /appi.appp.pp semanticscholarr CorpCCrp 248177997.
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert- Voss, Kather- ine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pp. 2633-2650 2021.
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramèr, and Chiyuan Zhang. Quantifying memorization across neural language models. ArXiv, abs/2202.076666, 2022. URL https: //ttps://ap...ee http//ppp...mmattcccccohool.as wwwpechoolarrrg/oorrrooee
Kent K. Chang, Mackenzie Cramer, Sandeep Soni, and David Bamman. Speak, memory: An archaeology of books known to chatgpt/ gpt-4, 2023. URL https://arxiv.org/ /bs/// 2015.00118.
Together Computer. Redpajama: an open dataset for training large language models, October 2023. URL
Data.
Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groen- eveld, Margaret Mitchell, and Matt Gardner. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. arXiv preprint arXio:2104.08758, 2021.
Yanai Elazar, Akshita Bhagia, Ian Magnusson, Abhilasha Ravichander, Dustin Schwenk, Alane Suhr, Pete Walsh, Dirk Groeneveld, Luca Soldaini, Sameer Singh, et al. What's in my big data? ar Xiv preprint arXiv:2310.2000 20707, 2023.
Manuel Faysse Patrick Fernandes, Nuno Guerreiro, António Loison, Duarte Alves, Caio Corro, Nicolas Boizard, João Alves, Ricardo Rei, Pedro Martins, et al. Croissantllm: A truly bilingual french-english language model. arXiv preprint ar Xrii:2202.00786, 2024.
Wenjie Fu, Huandong Wang, Chen Gao, Guanghua Liu, Yong Li, and Tao Jiang. Practical membership inference attacks against fine-tuned large language models via self-prompt calibration. arxiv preprint ar arXiv:2311.06062 2023.
11

---

Published as a conference paper at COLM 2024
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The pile: An 800gb dataset of diverse text for language modeling. ArXiv, abs/2101.000277, 2020. URL U wrwp:rrorr/oo..:2:300. 22: 230435736.
semanticscholes
Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé lii, and Kate Crawford. Datasheets for datasets. Communications of the ACM, 64(12):86-92 92, 2021.
Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, A. Jha, Hamish Ivison, lan Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Daniel Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richard- son, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A. Smith, and Hanna Hajishirzi. Olmo: Accelerating the science of language models. 2024. URL https://api. wwmanticscholar. org/ Corpus : 267365485.
Suchin Gururangan, Dallas Card, Sarah K. Dreier, Emily K. Gade, Leroy Z. Wang, Zeyu Wang, Luke Zettlemoyer, and Noah A. Smith. Whose language counts as high quality? measuring language ideologies text data selection, 2022. URL https://arxx org/ abs/2201. 10474.
Sorami Hisamoto, Matt Post, and Kevin Duh. Membership inference attacks on sequence-to- sequence models: Is my data in your machine translation system? Transactions of the Asso- ciation for Computational Linguistics, 8:49-63, 2019. URL https: http:p:://pp..eemmmiiccccl.... semanticscholar.
org/CorpusID 119302127.
Nathan J. Hu, Eric Mitchell, Christopher D. Manning, and Chelsea Finn. Meta-learning online adaptation of language models. ArXiv, abs 2305.15155 2023. URL https://ppp.
semanticscholar. org/ IDrpps 258866057.
Daphne Ippolito, Florian Tramèr, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher A. Choquette-Choo, and Nicholas Carlini. Preventing verbatim memo- rization in language models gives false sense of privacy. ArXiv, abs/2210.17546, 2022. org/CorppslDD......oo : 253237404.
URL https: /tpi.
Albert Qiaochu Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Deven- dra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L'elio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b. Arv, abs/ 2310..622, 2023. URL https: //api. commantccscoolee org/ IorpusID: 2633330444
Jungo Kasai, Keisuke Sakaguchi, yoichi takahashi, Ronan Le Bras, Akari Asai, Xinyan Ve- locity Yu, Dragomir Radev, Noah A. Smith, Yejin Choi, and Kentaro Inui. Realtime QA: What's the answer right now? In Thirty-sevnnth Conference Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL https: http:///ppeeeeeeeeee net/forumiid HfKOIPCvsv.
Alexandra Sasha Luccioni, Frances Corry, Hamsini Sridharan, Mike Ananny, Jason Schultz, and Kate Crawford. A framework for deprecating datasets: Standardizing documentation, identification, and communication. In Proceedings of the 2022 ACM Conference Fairness, Accountability, and Transparency, pp. 199-212, 2022.
Li Lucy, Suchin Gururangan, Luca Soldaini, Emma Strubell, David Bamman, Lauren Klein, and Jesse Dodge. Aboutme: Using self-descriptions in webpages to document the effects of english pretraining data filters. Xiv preprint ar Xix::24011 X0e:22401.06088, 2024.
Marc Marone and Benjamin Van Durme. Data portraits: Recording foundation model training data. ar Xiv preprint ar XrXi::3303.03919 2023.
12

---

Published as a conference paper at COLM 2024
Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Schölkopf, Mrinmaya Membership inference attacks against language
Sachan, and Taylor models via neighbourhood comparison. aiv preprint ar Xii::3305.18462, 2023.
Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. Proceedings of the Conference on Faimesss Accountabilltty, and Transparency, org/
2018. URL https: www.ste
52946140.
Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Chon Choo, Eric Wallace, Florian Tramèr, and Katherine Lee. Scalable extraction of training data from (production) language models. arXiv preprint (rXio:2311.17035, 2023.
Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. The Refined dataset for Falcon LLM: outperforming curated corpora with web data, and web data only, Xrv preprint ar Kfto:2306.01116,, 2023. URL https : :tt:://aaxxx //txx//xvvv////////////////aaaa/ org/ abs/
1306.01116,
Aleksandra Piktus, Christopher Akiki, Paulo Villegas, Hugo Laurenccon, Gérard Dupont, Alexandra Sasha Luccioni, Yacine Jernite, and Anna Rogers. The roots search tool: Data transparency for llms. In Annual Meeting of the Association for Compulational Linguistics, 2023. URL https: : //ppp/app..... semanticscholar, .rgg//arrrrr///ooooo Tngg/CorpusID.. 257219882.
Mahima Pushkarna, Andrew Zaldivar, and Oddur Kjartansson. Data cards: Purposeful and transparent dataset documentation for responsible ai. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pp. 1776-1826, 2022.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485-5551 2020.
Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. Detecting pretraining data from large language models. Xrviv preprint arXio:2310.16789, 2023.
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEÉE symposium on security and privacy (SP), PP. 3-18. IEEE, 2017.
Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Raghavi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, A. Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Daniel Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah A. Smith, Hanna Hajishirzi, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, and Kyle Lo. Dolma: an open corpus of three trillion tokens for language model pretraining research. 2024. URL https://api. www.stiesmcnooooo.....
org/ I 267364861.
Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Léonard Hussenot, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros, Ambrase Slone, Amélie Héliou, Andrea Tacchetti, Anna Bulanova, Antonia Paterson, Beth Tsai, Bobak Shahriari, Charline Le Lan, Christopher A. Choquette- Choo, Clément Crepy, Daniel Cer, Daphne Ippolito, David Reid, Elena Buchatskaya, Eric Ni, Eric Noland, Geng Yan, George Tucker, George-Christian Muraru, Grigory Rozhdestvenskiy, Henryk Michalewski, Ian Tenney, Ivan Grishchenko, Jacob Austin, James Keeling, Jane Labanowski, Jean-Baptiste Lespiau, Jeff Stanway, Jenny Brennan, Jeremy Chen, Johan Ferret, Justin Chiu, Justin Mao-Jones, Katherine Lee, Kathy Yu,
13

---

Published as a conference paper at COLM 2024
Katie Millican, Lars Lowe Sjoesund, Lisa Lee, Lucas Dixon, Machel Reid, Maciej Mikula, Mateo Wirth, Michael Sharman, Nikolai Chinaev, Nithum Thain, Olivier Bachem, Oscar Chang, Oscar Wahltinez, Paige Bailey, Paul Michel, Petko Yotov, Pier Giuseppe Sessa, Rahma Chaabouni, Ramona Comanescu, Reena Jana, Rohan Anil, Ross Mcllroy, Ruibo Liu, Ryan Mullins, Samuel Lmmihh,, Sebastian Borgeaud, Sertan Girgin, Sholto Douglas, Shree Pandya, Siamak Shakeri, Soham De, Ted Klimenko, Tom Hennigan, Vlad Feinberg, Wojciech Stokowiec, Yu hui Chen, Zafarali Ahmed, Zhitao Gong, Tris Warkentin, Ludovic Peran, Minh Giang, Clément Farabet, Oriol Vinyals, Jeff Dean, Koray Kavukcuoglu, Demis Hassabis, Zoubin Ghahramani, Douglas Eck, Joelle Barral, Fernando Pereira, Eli Collins, Armand Joulin, Noah Fiedel, Evan Senter, Alek Andreev, and Kathleen Kenealy. Gemma: Open models based on gemini research and technology, 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXio:2302.13971, 2023a.
Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, Daniel M. Bikel, Lukas Blecher, Cristian Cantón Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel M. Kloumann, A. V. Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, R. Subramanian, Xia Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zhengxu Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models. Arviv, /ttp//337.099228 2023b. URL https: /aapi. semanti cscholarr org/
CorpusID:2599508:
Ben Wang and Aran Komatsuzaki. GPT-J-6B: A 6 Billion Parameter Autoregressive Lan- guage Model.
http://hhht...oooooooot//t////aa www.eeesstttaaassoer-eeeee May 2021.
Orion Weller, Mare Marone, Nathaniel Weir, Dawn Lawrie, Daniel Khashabi, and Benjamin Van Durme. "acorrding to...:: Prompting language models improves quoting from pre-training data. ar Xiv preprint ar Xrio:2305.1325. 1
2023.
14

---

Published as a conference paper at COLM 2024
A Probing Pseudocode
We denote the pre-training dataset D. Let M* denote the month corresponding to the reported Wikipedia dump in D. Let Dr,M represent the version of topic at month M. EDIT refers to the Levenshtein distance between two strings.
Algorithm Counting versions of documents WIKISPAN
1: procedure RETRIEVE(D, M+)
2: counts < 0
3: for Topic T WIKISPAN do
4:
Q - Dr.Mz 512
>> Query with the first 512 tokens
5:
R - BM25( 10]
10 retrieval results
6:
for Matched Document D E R do dists +
7:
8:
for Version V Dr,start" ' Dr.M
end do
9:
dist + ())een((( dists.append(dist) end for min(dists) < 0.2 then
10:
11:
12:
13:
min_months argmin(dists) for m E min months do
>> ties identical doc versions
14:
15:
counts[m] <- counts[m] + 1/ /lenn(mit months)
16:
end for end if end for
18:
end for
20:
return counts
21:
end procedure
B Deduplication Complications
B.1
Falcon
By the end of the 17th century, the Chinese ecumamy had recovered from the devastation caused by the wars which the overthrown, and the resulting breakdown of order] [47] In the following century, markets continued expand as in the late Ming period, but with more trade between regions, a greater dependence on overseas markets and greatly increased population.
Ming dynasty
(1llaaaioonn.][_111.. [148][149] The government broadened land ownership by returning to large landowners in the late Ming period by families unable to pay the land tax[150] To give people more incentives to participate in the market, they reduced the tax burden comparison with the late Ming, and replaced corvée system with a head tax used hire laburers.[15] [15]] The administration of the Grand Canal made mure efficient, and transpurt opened to private merchants. [152) A system of monitoring grain prices eliminated severe shortages, and enabled the price of rice rise slowly and smoothly through the 18th century:[133] Wary of the power of wealthy merchants, Qing rulers limited their trading licenses and usually refused them permission to open new mines, except in poor areas
By the end of the 17th century, the Chinese economy had recovered from the devastation caused by the wars in which the Ming dynasty were overthrown, and the resulting breakdown of [cder[[188] In the following century, markets continued to expand as in the lafe Ming period, but with more trade between regions, greater dependence greatly increased population. (aapmn./119]] [149]][100] The government broadened land ownership by returning land that had been sold large landowners in the late Ming period families unable pay the land tax[151] give people more incentives participate in the market, they reduced the tax burden in comparison with the late Ming, and replaced the corvée system with a head tax used hire laborers. [152] The administration
markets and
the Grand Canal was made more efficient, and transport opened to private merchants (erchants[[153]] A system of monitoring grain prices eliminated severe shortages, and enabled the price of rice to rise slowly and smoothly through the 18th [e5tury.[15]] Wary of the power of wealthy merchants, Qing rulers limited their trading licenses and usually refused them permission to open mines, except poor
Table 2:
An example of near-duplicate Wikipedia documents that are semantically equiv- alent in the Ra dataset, differing only by the reference numbers. The documents contain different versions of the Wikipedia article "Qing Dynasty," and are located on lines 168922 and 97669 of the 5th and 3970th parquet files in the public release of FalconRW. The colored text indicates exact matches.
15

---

Published as a conference paper at COLM 2024
B.2 C4
Natalie Portman is an actress with dual American and Israeli citizenship. Her role was as an orphan taken in by a hitman the 1994 action film Léon: The Prufessional, but mainstream success came when she was cast Paamé Amidala in the Star Wars prequel trilogy (released in 1999, 2002 and 2005). In 1999, she enrolled at Harvard University to study psychology while still working as an actress. She completed her bachelor's degree in 2005. In 2001, Portman opened in New York City's Public Theater production of Anton Chekhov's s The Seagull. In 2005, Portman won Golden Globe Award and received Academy Award nomination for Best Supporting Actress for her performance in the drama Closer. She won Constellation Award for Best Female Performance and Saturn Award for Best Actress for starring role V for Vendetta (2006). She played leading roles in the historical dramas Goya's 's Chusb (2006) and The Other Boleyn Cirl (2008). In May 2008, she served as youngest member the 61st Annual Cannes Film Festival jury. Portman's directorial debut, Eve, opened the 65th Venice International Film Festival's shorts competition in 2008, Portman directed a segment of the collective film New York, I Love You. Portman is also known for her portrayal as Jane Foster, the love interest of Marvel superhero Thor, in the film adaptation Thor (2011), and its sequel, Thor: The Dark World ... (2013). In 2010, Portman starred in the psychological thriller Black Swan. Her performance received critical praise and eamned her second Golden Globe Award, the Screen Actors Guild Award, BAFIA Award, the Broadcast Film Critics Association Award and the Academy Award for Best Actress 2011.
Natalie Portman an actress with dual American and Israeli citizenship. Her first role was as an orphan taken by a hitman in the 1994 action film Léon: The Professional, but mainstream success came when she was cast as Padmé Amidala in the Star Wars prequel trilogy (released in 1999, 2002 and 2005). In 1999, she enrolled at Harvard University to study working as an actress, She completed her bachelor's 's degree 2003. |nin 2001, Portman opened
psychology while New York City's is Public Theater production of Anton Chekhov's ' The Seagull. In 2005, Portman won Golden Globe Award and received an Academy Award nomination for Best Supporting Actress for her performance in the drama Closer. She won Constellation Award for Best Female Peefformacce and a Saturn Award for Best Actress for her starring role in V for Vendetta (2006). She played leading roles in the historical dramas Goya's 's Ghosts (2006) and The Other Boleyn Girl (2008). In May 2008, she served as the youngest member of the 61st Annual Cannes Film Festival jury. Portman's directorial debut, Eve, opened the 65th Venice International Film Festival's ss shorts competition in 2008. Portman directed segment of the collective film New York, Love You. Portman also known for her portrayal as Jame Foster, the love interest Marvel superhero Thor, in the film adaptation Thor (2011), and its sequel, Thor: The Dark World... ... (2013)))nnn 2010, Portman starred in the psychological thriller Black Swan. Her performance received critical praise and carned her second Gulden Globe Award, the Sereen Actors Guild Award, the BAFTA Award, the Broadcast Film Crities Association Award and the Academy Award for Best Actress in 2011.
Table 3: An example of exact three sentence duplicates in C4, along with semantically equivalent text following. The two documents are versions of the Wikipedia article "Natalie Portman." " The colored text indicates exact matches.
B.3 RedPajamas
"Adam Richard Sandler (born September 1966) American actor, comedian, screenwriter, film producer, and musician. After becoming Saturday Night Live cast member, he went on to star in many Hollywood feature films that have grossed over billion at the box office combined..ondlle
well-known roles include Billy Madison (1995), Happy Gilmore (1996), The Waterboy (1998), The Wedding Singer (1998), Big Daddy (1999), Mr. Deeds (2002), 50 First Dates (2004), The Longest Yard (2005), Click (2006), Grown Ups (2010). Just Go with It (2011), Grown Ups 2 (2013), Blended (2014), and Murder Mystery (2019). He also voices Dracula in the Hotel Transylvan Ivanylvaniaa franchise 22012-present panned Jack and Jill, have been heavily criticized, culminating in a shared second place in the number of Raspberry Awards (3) and Raspberry Award nominations (11), in both cases second only Sylvester Stallone. Sandler ventured into dramatic terrttory with his roles in Punch-Drunk Love (2002), Spanglish (2004), Reign Over Me (2007), Funny People (2009), The Meyerowitz Stories (New and Selected) (2017), and Uncut Gems (2019), all of which earned him critical praise.""
Some of his films, such the widely
Table 4: An example of the exact document duplicates in RedPajamas. The documents are versions of the Wikipedia article " "Adam Sandler," " and is duplicated 10 times in the RedPajamas CommonCrawl training data.
C N-gram Analysis instead of Exact Match
How is perplexity affected when seeing two lexically similar texts? One hypothesis is that perplexity is most affected by exact and near-duplicates. Another hypothesis is that the actual text in a document is factor affecting perplexity. To illustrate the difference between these hypotheses, document that contains shuffled sentences from a version of a Wikipedia article (shuffling order of sentences) would not affect perplexity in the former case, but would in the latter case. The formeh hypothesis is the basis behind our algorithm for attributing matched documents to their closest versions. We test the latter hypothesis by proposing another way to attribute credit, directly counting the intersection of n-grams in matched documents with precomputed sets of n-grams sourced from WIKISPAN. We discount by the number of times an ngram appears actoss all months (similar to an inverse document frequency) in order to count ngrams that are distinct to a specific Wikipedia version. Our algorithm and results are described in Appendix C.1
16

---

Published as a conference paper at COLM 2024
C.1 Algorithm Pseudocode
We use the same notation as in Appendix A
Algorithm 2 Counting n-grams of documents in WIKISPAN
1: ngrams +- {}
2: for Month i E WIKISPAN do
D Compute all n-grams of all docs in month for E WIKISPAN])
3: ngrams((n) [m] <- Counter Counter([NGRAAT) (NGRAMS(DT..)
4: end for
5: common_ngrans <- M ngrams[[]]
6: procedure RETRIRVE(D,
7: counts (}
8: for Topic 7 E WIKISPAN do
9:
Q DI.M 512]
Query with the first 512 tokens b> 10 retrieval results
10:
R + (M25(0,
for Matched Document D R do
12:
for ngram NGRAMS D[: 512)) do for month mı E WIKISPAN do if n E ngrams m then counts m + counts|m [m] + ingganssm m[[[x] common.ngrams(zt
14:
15:
16:
end if end for end for end for
17:
18:
19:
20:
end for
21:
return counts
22:
end procedure
17

---

Published as a conference paper at COLM 2024
C.2
Results
We show the perplexity curves of our evaluated models and compare it with our new n-gram statistics.
Pile Derived Models
1.0
Pythia-6.98
GFT)-5B
Relative Perplexities
GFTNeo-2.38
0.6
0.4
0.2
0.0
FalconRW Derived Models
1.0
Felcon-78
Reletiive Perplexitees
0.6
0.4
0.0
C4 Derived Models
1.0
RedPajamas
Belesive Per peexliiiiess
0.8
LLaMA-78
OLMo-78
0.4
0.2
2016
Idl 2016
odt 2012
jan 2017
Apr 2017
2017
oct 2017
Jan 2018
Agr 2018
2010
Oct 2018
Jan 2019
Aor 2019
1/2019
2019
Jan Agr 2020
2020
jul 2020
oct 2020
jan 2021
Apr 2021
2021
Oct 2021
Jan 2022
Apr 2022
Jul 2022
Oct 2022
Jan 2023
Agr 2023
Jul Oct
Agr
Figure 9: Relative perplexities of models per month using the WIKISPAN ($3.1) dataset (we use relative as exact perplexities are not needed for determining effective cutoffs). Upper plot shows Pile derived models, middle shows FalconRW derived models, while lower shows C4 derived models. The light grey bars indicate the ground truth similar documents, matched to their closest version, as calculated in Appendix C. Note that these datasets are only a subset of the training set for some models. In some cases the knowledge cutoff aligns with the model's ss effective cutoff (e.g. the Pile) while for more recent models they are aligned much earlier (e.g. RedPajamas to 2019, even though it has an explicit 2023 Wikipedia dump).
18

---

Published as a conference paper at COLM 2024
D Closed Model Results
We evaluate our method on the closed-data models Gemma (Team et al., 2024), LLaMA-2 (Touvron et al., 2023b) and Mistral (Jiang et al., 2023) in Fig. 10. We see that 2, like many of the open-data models we analyze, has much earlier effective cutoff for Wikipedia. In contrast, we see that Gemma has much later effective cutoff, indicating their success at aligning Wikipedia to roughly 2021.
Closed Data Models
1.0
Relecive Perplexities
0.8
0.6
Mistral-78
Gemma-77
LLaMA2-78
0.2
0.0
Apr 2010
jul 2016
odte
Jan 2017
Apr 2017
Jul 2017
2017
Jan 2016
Apr 2019
jui
oct 2018
2010
2019
2012
2019
2020
2020
jut 2020
odt 2020
Jan 2021
2021
Jul 2021
2ct
2022
Agr 2022
jui 2022
adt 20z2
Jan 2023
Apr 2023
oct
Jan
Ap
oct
Jan
Aor
Are
fan
Figure 10: Relative perplexities of models per month using the WIKISPANdataset (we use relative as exact perplexities are not needed for determining effective cutoff).
19


Reviews:
[{"review_id": "SDAjfVXKCf", "reviewer": "Reviewer_Z85G", "summary": "The paper presents a technique and application for discovering the effective training data cutoff date, via perplexity.", "questions": "", "limitations": "", "rating": 9, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review and thoughtful comments! We appreciate that you found our work \"revealing\" and \"simple\" which were the goals of our analysis to provide transparency for users and creators.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The approach is clever and revealing.  The technique itself is extremely simple and therefore seemingly quite robust.  The experiments demonstrating the success are well-thought out and also very convincing.  The problem is important in LLM accountability, transparency for the user to hold the correct expectations of the model.", "reasons_to_reject": "None.", "questions_to_authors": "no questions", "extra_scores": {}}, {"review_id": "CzzJZRhNNW", "reviewer": "Reviewer_eFeV", "summary": "LLMs report a cutoff date which indicates the date till which the LLMs can be considered knowledgeable. This paper introduces the concept of a resource-specific \"effective cutoff date,\" for an LLM. The effective cutoff date, distinct from the \"reported cutoff date\" of training data for an LLM, is an approximate date of earlier versions of the resource in the training data from which LLMs draw their knowledge. The paper presents an automated technique designed to identify the effective cutoff date for a resource in LLM training data without needing explicit access to training data. Experiments on different LLMs reveal that there are discrepancies between the reported and effective cut-offs that largely stem from two reasons: incomplete deduplication resulting in multiple versions of the same resource in training data and inadvertent leakage of old data in new data dumps.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> My biggest exception … is the nebulous description of some concepts\n\nWe will clarify the definition in Sec. 3. We define the effective cutoff date with respect to a model and resource as the date of the version of that resource that most closely aligns with a model. Alignment can be measured in several ways, but we use minimum perplexity. \n\nIn the tax example, resource refers to a version of the tax code, not a specific section.  In general, **our approach is for resources (e.g. a set of docs) rather than for attribution to a single doc.** \n\nWe agree - as in your news example - that a resource can be more nebulous, and in those cases it would depend on if there are enough unique articles in that resource.  If not, then perhaps it’s not a good resource definition and should be re-defined (as AP news or news broadly).\n\n> I didn't follow how the docs were slotted into T topics\n\nA wikipedia topic is effectively the document title. We use this term to disambiguate between versions of an article and the subject of the article. Wikispan has 94 doc versions (1/month) for each of the 5k topics. \n\nOur pipeline filters more recently created topics as these are blank for certain timespans (e.g. the “Covid-19” page is blank in the year 2017).\n\n> What does \"average of the median 95% mean\"\n\n\nTo remove any outlier ppl values (resulting from unfiltered stub articles/redirects) we first take the mean and 95% confidence interval, also known as a [truncated mean](https://en.wikipedia.org/wiki/Truncated_mean). We will update the wording to make it more clear and include math equations to remove ambiguity!\n\n> if they coincide wouldn't the effective cutoff dates be the same\n\nInterpreting relatively flat basins in the curves indeed requires nuanced considerations. If the two minima coincided, that would be correct. However in Figure 6, while the min of the non-dedup curve coincides with a local minima of the dedup curve, the global min of the dedup curve is around a year earlier. \n\n> connection besides both \"examine temporal knowledge\"?\n\nThat is the connection, as our work explores **if** there is temporal alignment and continual learning explores **how to** aligning them. We will revise this section to make this more clear with our analysis - for example, our analysis could be used “in-the-loop” to determine that a model should undergo additional learning on more recent sources.\n\nWe will add the relevant citations!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The primary strength of the paper is shifting the notion of cutoff dates from a reported cutoff to an empirically calculated effective cutoff. The paper's proposed method for estimating the approximate effective cutoff for a resource is straightforward and easy to replicate. The empirical analysis backs up the thought that the effective date is different from the reported cutoff. A side-effect of the analysis in this paper is that the issues found around data deduplication can serve as valuable insights for further pre-training data curation. The experiments were detailed and provided a clear understanding. Overall, the reporting of a resource-level effective cutoff can increase the transparency of an LLM.", "reasons_to_reject": "My biggest exception to this paper is the nebulous description of some concepts. Though the paper advances the notion of effective cutoff date, it never formally defines this concept. We're only given a procedure of finding an effective cutoff date and a few examples to illustrate the concept. Similarly, it may not always be clear how one would define a resource within training data and if there is a need to have an effective cutoff date at the resource level or at some other level. In the paper, for example, news from a single media outlet (e.g., nytimes) is considered a resource. But the tax example given in the paper suggests the need for an effective date on a topic rather than a specific news outlet. Moreover, sometimes news articles are sourced and reproduced from common reporting agencies such as Reuters or Associated Press, making it difficult to truly isolate the boundaries of a resource.", "questions_to_authors": "- In section 3.1, under Wikispan, I didn't follow how the documents were slotted into T topics and which topics were filtered out. Why is the topic distinction even necessary?\n- In section 3.2, under normalization, it was hard for me to follow how the perplexities were aggregated. What does \"average of the median 95% mean\"? It would have been helpful to rigorously layout the calculations as math equations to remove any ambiguity.\n- In figure 6, the minima in the left panel for both the plots looks to be coinciding. I understand the minima for the dedup curve is not as sharp as for non-dedup, but if they coincide wouldn't the effective cutoff dates be the same with and without deduplication?\n- Possible additional citations \n  - \"Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection\" by Gururangan et. al. also discuss the tole of quality filters in training data selection\n  - \"Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4\" by Chang et. al. as an example of membership inference testing without the need to compute the LLMs perplexity\n- In section 2, under continual learning, the objective of continual learning is a modeling concern which is different from the analytic concern of this paper. What is the more specific connection between the two besides both \"examine temporal knowledge\"?", "extra_scores": {}}, {"review_id": "Dhjk9HziLJ", "reviewer": "Reviewer_VAWs", "summary": "This work proposes to reexamine the reported knowledge cutoff dates of LLMs by demonstrating that different \"sub-resources\" in LLM training might come with different cutoff dates. This work calculates perplexity over a temporally shifting domain corpora to illustrate the perplexity changes corresponding to document time stamps. Experiments found various artifacts of existing models for NYT and Wikipedia domains.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "> Do we need a lot of documents\n\nYou bring up an important point about the number of documents required for our method. We empirically found the perplexity of documents in a sub-resource can vary greatly (due to the inherent differences in perplexity in language). As such, we use the median 95% of the perplexity values and take larger samples to obtain more unbiased samples of the true distribution. \n\nUpon your suggestion we are currently running this experiment as we feel it would add to our paper. We are varying the number of docs per timestamp and will provide this in the appendix. It should be done in the next few days and we will post the results here.  Thank you for the suggestion!\n\n> methodology to quantitatively decide on a specific cutoff date\n\nWe do provide a baseline approach to quantitatively define the effective cutoff — we use the minima of the perplexity curve, after truncating the distribution to the median 95%. This is what is shown in Figures 3-7. The quantitative way to do this automatically is simply to select the global minima.\n\n> certain long-tail documents from a newer date were there in the training data, but were just not properly \"memorized\"\n\nThis could happen, and we have already attempted to prevent this by using the median 95% of the distribution when calculating values. Thus if the values are on the tails, they will not be included. If there are a large amount of these, then that would be a problem, however, we are unaware of any one collection that contains a large subset of documents that are hard to memorize.\n\n> qualitative analysis … before the reported cutoff date but after the \"actual\" cutoff date\n\nWe included a qualitative analysis of duplicated documents, but agree that some examples of perplexity/docs from before/after the cutoff would be helpful also.  We will update the paper to include examples in a new appendix section!\n\n> what could people do with those dates? \n\nGreat question! As discussed in the introduction and conclusion, as an initial step we believe it serves as a warning to both LLM creators and users to be mindful of the temporal biases of LLMs. \n\nFor what users could do with that date, it will have to be resource-specific – perhaps if you need the LM up to date, you either pick a different LM or try some techniques to improve its temporal alignment.  However, this paper focuses on finding and notifying the community of this problem, we leave the rest to future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "+ this work has the potential to facilitate greater transparency in training data\n+ the proposed approach is straightforward", "reasons_to_reject": "- To verify the actual cutoff date for a certain \"sub-resource\" or domain, the proposed methodology would need a collection of documents in the domain for a long time period. I wonder how dependent is the approach on those documents: Do we need a lot of documents for each time stamp, or are a few/under 10 often enough? It might be challenging to collect a lot of documents for a domain-specific application and use the proposed method.\n\n- In addition to visualizing the perplexity changes through time, it might be nice to have a methodology to quantitatively decide on a specific cutoff date. In this way, users could decide on whether to trust the LLM when their query concerns information after that specific time stamp. I wonder if the authors might have ideas about how to obtain a specific cutoff date estimation.\n\n- The approach focuses on perplexity: however, it might be the case that certain long-tail documents from a newer date were there in the training data, but were just not properly \"memorized\" thus leading to high perplexities. I wonder if the authors believe that this might be a confounding factor, and how the proposed methodology mitigates this.\n\n- It would be nice to include a qualitative analysis showing example documents that are before the reported cutoff date but after the \"actual\" cutoff date, along with perplexity scores/metadata etc.\n\n- After identifying the \"actual\" cutoff dates of different sub-resources, what could people do with those dates? There could be social aspects such as better issuing a disclaimer about domain-specific knowledge cutoffs, there could also be technical implications such as adapting the model to more recent data on certain domains, instead of training on new data indiscrimnitaly. It would be nice to propose some applications/recommendations that build on this methodology.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3RHqiEoU4Q", "reviewer": "Reviewer_6tHA", "summary": "This paper investigates the concept of knowledge cutoffs in Large Language Models, which is critical for applications requiring up-to-date information. The authors challenge the assumption that a reported cutoff date for an LLM implies uniformity across all training data and propose a method to estimate the effective cutoff at the resource level. Their approach does not require access to the model's pre-training data and reveals significant discrepancies between reported and effective cutoffs. The paper also conducts a large-scale analysis to understand the causes of these inconsistencies, identifying issues with data deduplication and temporal misalignments in CommonCrawl data.", "questions": "", "limitations": "", "rating": 9, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review and for finding our work \"impressive\" and a \"substantial finding that contributes to the broader understanding of LLM training data nuances\"!\n\n> The paper does not discuss the scalability of the proposed method\n\nWe show results for models from 7B to 65B in Figure 5, but agree that we did not test models beyond 65B parameters. We will add more discussion to Section 5.2 discussing this – we believe that the trends from 7B to 65B will continue, as they show consistent results.\n\nAs the resource size increases our method will work better as the sample size will be larger and less biased, but we agree that computationally, our method will increase linearly with the size of the dataset. \n\nWe will add these comments to Section 4.1 and 4.2. \n\n> How does the model architecture or size influence the identification of effective cutoffs? \n\nGreat questions! We evaluate a large variety of language models of varying transformer architectures and training schemes (see Table 1). Moreover, we also evaluate different sized models in Pythia and LLaMA suites (see Figure 5 in section 5.2). Overall, we find that that effective cutoffs are consistent between different sized models that were pre-trained on the same data. \n\n> Have you considered or tested the applicability of your approach to other types of resources\n\nIn our paper we showed results for two types of resources (Wikipedia and news articles, as you suggest – that is our NYT collection), as they represent the streaming and updating types of datasets (Section 3). Unfortunately due to paper space we could not add more datasets comprehensively, but our initial tests showed that it holds on other resources as well.  \n\nWe leave it to future work (and to LLM creators) to document these results for all other resources that users are interested in – there are a lot of resources out there and we agree this is an important area of future work!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper introduces a novel method for evaluating the temporal dynamics of knowledge within LLMs. By measuring the perplexity across different versions of datasets, the authors provide a granular understanding of knowledge cutoffs, which is a significant advancement in LLM analysis.\n\n- The large-scale analysis of pre-training datasets is particularly impressive. The identification of issues such as temporal misalignments in CommonCrawl data and deduplication challenges is a substantial finding that contributes to the broader understanding of LLM training data nuances.\n\n- The detailed examination of the reasons behind the misalignment of cutoff dates provides diagnostic value to both LLM creators and users. The insights can guide the development of better training protocols and inform users about the potential limitations of LLM knowledge bases.", "reasons_to_reject": "- The paper does not discuss the scalability of the proposed method to even larger models or datasets.\n- It is unclear how these findings generalize to LLMs trained with different data or methods.", "questions_to_authors": "- Have you considered or tested the applicability of your approach to other types of resources, such as news articles, books, or legal documents, which may have different update frequencies and versioning challenges?\n- How does the model architecture or size influence the identification of effective cutoffs? Would smaller models or those with different training methodologies exhibit similar discrepancies between reported and effective cutoffs?", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces the interesting concept of an "effective knowledge cutoff" in LLM training: the date associated with specific resources using during training.  This work exposes the conceptual ambiguity of "reported knowledge cutoffs" as reported by LLM developers by demonstrating how different pre-training sub-collections have different knowledge cutoffs, introduces a simple perplexity-based measure for assessing the effective cutoff of a resource, and carries out several analyses investigating the misalignment between effective cutoffs and reported ones in several LLMs.  Reviewers generally found this work to be creative, conceptually innovative and experimentally robust, with important consequences for data documentation in LLMs.
Decision: Accept