NewsEdits 2.0: Learning the Intentions Behind Updating News
Authors: Alexander Spangher, Kung-Hsiang Huang, Hyundong Justin Cho, Jonathan May


Reviewer paper summaries: The paper presents a study on predicting factual fluidity in news articles, focusing on the ability to anticipate which facts within an article are likely to be updated over time. The authors hypothesize that linguistic features can indicate such fluidity, allowing prediction without relying on external resources. They develop the NewsEdits 2.0 taxonomy, distinguishing fact updates from stylistic and narrative revisions, and annotate over 9,200 sentence revisions. Using this data, they train ensemble models that achieve high precision in identifying potential fact updates. These predictions are applied in a Language Model Question Asking (LLM-QA) abstention task, where the model learns to refrain from answering questions when the information is likely to become outdated, achieving near-oracle accuracy. The work contributes the NewsEdits 2.0 schema, a large corpus of revision histories labeled with edit intentions, and insights into how LLMs struggle with this predictive capability.
Summary of strengths: The NewsEdits 2.0 taxonomy provides a structured approach to categorizing edits, which aids in the systematic analysis of factual fluidity in news articles.

The integration of updates predictions into an LLM-QA abstention task demonstrates real-world utility, potentially improving the performance of language models in dynamic information environments.
Summary of weaknesses: The reliance on a specific dataset (NewsEdits) and the context of news articles may limit the generalizability of the findings to other text types or domains.

The prediction accuracy of LLMs on whether a fact will need an update in the future is very low. This result causes concerns about whether this is a task that could be solved in the future. Additionally, whether a fact needs to be updated in the future is a temporally subtle question, and the answer varies a lot in different scenarios. More clarification and experiments should be conducted to explore this question.
Comments, suggestions and typos: N/A
Overall assessment scores: 2.5