F ood S afe S um: Enabling Natural Language Processing Applications for Food Safety Document Summarization and Analysis (2025)

Abstract: Food safety demands timely detection, regulation, and public communication, yet the lack of structured datasets hinders Natural Language Processing (NLP) research. We present and release a new dataset of human-written and Large Language Model (LLM)-generated summaries of food safety documents, plus food safety related metadata. We evaluate its utility on three NLP tasks directly reflecting food safety practices: multilabel classification for organizing documents into domain-specific categories; document retrieval for accessing regulatory and scientific evidence; and question answering via retrieval-augmented generation that improves factual accuracy.We show that LLM summaries perform comparably or better than human ones across tasks. We also demonstrate clustering of summaries for event tracking and compliance monitoring. This dataset enables NLP applications that support core food safety practices, including the organization of regulatory and scientific evidence, monitoring of compliance issues, and communication of risks to the public.

## introduction the increasing prevalence of food safety incidents has recently attracted the interest of researchers, including the natural language processing (nlp) community. understanding the causes of these incidents and addressing contamination @xcite is a pressing challenge, prompting initiatives that are reshaping the landscape of food safety research. 1 additionally, governments worldwide have established regulatory frameworks to oversee the production, distribution, and consumption of food products, enabling the development of compliance-checking systems @xcite . food safety-related information, however, is fragmented across multiple sources, including regulatory reports, scientific publications, news articles, and government guidelines, making it challenging to extract, analyse, and act upon critical insights efficiently. the rapid growth of these textual resources presents an urgent need for automated, scalable nlp solutions to facilitate information retrieval, risk assessment, and decision-making in food safety @xcite . key attributes such as the topic of a document (e.g., contaminants, labelling, policies) or the hazard it reports (e.g., pathogens, residues, food fraud) are essential for structuring this information, yet, progress is hindered by the lack of publicly available and machineactionable datasets that provide such annotations for nlp research. while exceptions exist @xcite , we are still missing a comprehensive resource that integrates diverse sources of food safety information. to this end, we introduce foodsafesum, a novel dataset that contains manual and large language model (llm)-generated summaries of food safety-related documents collected from diverse sources. the dataset includes manually annotated metadata, such as relevant topics and document types, and automatically extracted hazard annotations, enabling structured analysis of food safety risks. using this dataset, we experiment with three core nlp tasks that mirror real-world food safety needs (examples shown in ## related work to date, publicly available datasets for nlp research in the food safety domain remain scarce. several agencies, including the u.s. food safety and inspection service (fsis) 2 and the european food safety authority (efsa), 3 provide diverse food safety data -such as monitoring results, occurrence data, and exposure assessments -that are primarily intended for regulatory and scientific use. while these datasets are valuable, their formats often require adaptation for nlp applications (e.g., text extraction, annotation), underscoring the need for dedicated nlp-focused food safety resources. to our knowledge, the only publicly available dataset specifically designed to advance nlp research in the food safety domain is the food recall incidents dataset @xcite . this dataset comprises 7,546 recall announcement 2 https://www.fsis.usda.gov/ 3 https://www.efsa.europa.eu/en titles, along with associated metadata (e.g., publication dates) and manual annotations of hazards and products at both coarse and fine granularity. recent studies have begun to leverage nlp to address various food safety challenges @xcite @xcite ). an early study by @xcite summarization plays a crucial role in extracting key insights from lengthy and complex food safety documents. prior research has explored summarization across various domains, including legal and medical texts, using extractive, abstractive, and hybrid techniques @xcite . large-scale datasets, such as billsum @xcite and eur-lex-sum @xcite , have facilitated advancements in legal summarization, while models incorporating rag and domain-aware adaptations have improved summary quality and relevance @xcite . ## the foodsafesum dataset foodsafesum is a structured dataset derived from 2,091 food safety-related documents collected by sgs digicomply foot_0 on may 24, 2024. these documents are sourced from news platforms, legal and regulatory agencies, food safety authorities and organizations, government guidance portals, scientific journals, and research platforms, where sgs food safety experts continuously monitored them and retrieved content relevant to food safety hazards. they span the years 2002-2023, with approximately 65.7% originally in english and the remainder in 33 other languages. non-english documents were automatically translated using google translate or deepl and subsequently curated by sgs experts. the documents were manually annotated by sgs experts with document type, covering news articles, regulations, governmental guidance, and scientific articles, as well as with one or more relevant topics. the topics span 12 high-level categories that reflect key thematic areas in food safety, including regulation and policy, scientific and technical aspects, consumer-focused issues, and broader systemic concerns. the hazard annotations were automatically extracted using a controlled vocabulary derived from the food recall incidents dataset @xcite . within the food safety domain, a hazard refers to any substance, object, or type of non-compliance detected or reported in a post. 5 this includes biological agents (e.g., salmonella), chemical substances (e.g., pesticide residues, allergens), physical contaminants (e.g., metal fragments), regulatory violations (e.g., exceeding per-mitted levels of additives), and food fraud. these metadata enable structured analyses of food safety risks and support downstream nlp tasks. foodsafesum also contains multiple document versions: manual summaries (manual-s) and titles (manual-t) created by sgs experts, as well as llm-generated summaries (ll70b-s) and titles (ll70b-t) using the instructiontuned meta.llama3-70b-instruct-v1:0 model (via amazon bedrock), where s and t denote summary and title versions, respectively. in summary, the publicly released dataset provides for each document: (i) manual annotations of document type and topics, (ii) automatically extracted hazard annotations, (iii) manual summaries and titles, and (iv) llm-generated summaries and titles. the original full documents (d) are included for internal experiments but are not publicly released. the dataset is publicly available on zenodo. foot_2 ## empirical analysis to assess the quality of the foodsafesum dataset's summaries relative to each other and to the original documents,, we experiment with three nlp tasks -ir, mlc, and qa -which serve as proxies for real world food safety applications. in addition, we demonstrate a use case of text clustering, to identify related events and recurring issues. these tasks, along with the clustering use case, are discussed in the following sections. we represented the data using contextual embeddings from stella 1.5b @xcite ), foot_3 opting for a model that achieves an optimal trade-off of performance on the mteb leaderboard, foot_4 memory usage, and the maximum token capacity. approximately 0.4% of the ds exceeded the model's maximum token limit (131,072) and were truncated. ## conclusions this work bridges a critical gap in nlp for food safety by releasing a machine-actionable dataset that is enriched with metadata and multiple summary versions, and by evaluating its utility on tasks directly aligned with food safety practice. our experiments show that multilabel classification supports the organization of documents by food safety topics; information retrieval improves practitioners' access to relevant regulatory and scientific information; and retrieval-augmented question answering addresses practical needs, such as identifying contamination causes or verifying compliance. the exploratory use case of clustering further demonstrates how related events can be tracked across document types, enabling compliance monitoring and early-warning applications. beyond highlighting the potential of modern llms for summarization, this study provides an open resource to foster research in automated food safety monitoring, risk detection, policy enforcement, and public health communication. we envision foodsafesum as a foundation for developing food safety ai systems, policy-driven nlp, and decision support tools for regulators, industry stakeholders, and researchers.