GenAI4ELab/papercli-papers huggingface dataset:
FORCE: A Benchmark Dataset for Foodborne Disease Outbreak and Recall Event Extraction from News
Authors: Sudeshna Jana; Manjira Sinha; Tirthankar Dasgupta
Venue: ACL 2024 (Proceedings of the 9th Social Media Mining for Health Research and Applications (SMM4H 2024) Workshop and Shared Tasks)

Abstract: The escalating prevalence of food safety incidents within the food supply chain necessitates immediate action to protect consumers. These incidents encompass a spectrum of issues, including food product contamination and deliberate food and feed adulteration for economic gain leading to outbreaks and recalls. Understanding the origins and pathways of contamination is imperative for prevention and mitigation. In this paper, we introduce FORCE Foodborne disease Outbreak and ReCall Event extraction from openweb). Our proposed model leverages a multi-tasking sequence labeling architecture in conjunction with transformer-based document embeddings. We have compiled a substantial annotated corpus comprising relevant articles published between 2011 and 2023 to train and evaluate the model. The dataset will be publicly released with the paper. The event detection model demonstrates fair accuracy in identifying food-related incidents and outbreaks associated with organizations, as assessed through cross-validation techniques.
Overview of #SMM4H 2024 – Task 2: Cross-Lingual Few-Shot Relation Extraction for Pharmacovigilance in French, German, and Japanese
Authors: Lisa Raithel; Philippe Thomas; Bhuvanesh Verma; Roland Roller; Hui-Syuan Yeh; Shuntaro Yada; Cyril Grouin; Shoko Wakamiya; Eiji Aramaki; Sebastian Möller; Pierre Zweigenbaum
Venue: ACL 2024 (Proceedings of the 9th Social Media Mining for Health Research and Applications (SMM4H 2024) Workshop and Shared Tasks)

Abstract: This paper provides an overview of Task 2 from the Social Media Mining for Health 2024 shared task (#SMM4H 2024), which focused on Named Entity Recognition (NER, Subtask 2a) and the joint task of NER and Relation Extraction (RE, Subtask 2b) for detecting adverse drug reactions (ADRs) in German, Japanese, and French texts written by patients. Participants were challenged with a few-shot learning scenario, necessitating models that can effectively generalize from limited annotated examples. Despite the diverse strategies employed by the participants, the overall performance across submissions from three teams highlighted significant challenges. The results underscored the complexity of extracting entities and relations in multi-lingual contexts, especially from the noisy and informal nature of user-generated content. Further research is required to develop robust systems capable of accurately identifying and associating ADR-related information in low-resource and multilingual settings.
Overview of the 9th Social Media Mining for Health Applications (#SMM4H) Shared Tasks at ACL 2024 – Large Language Models and Generalizability for Social Media NLP
Authors: Dongfang Xu; Guillermo Garcia; Lisa Raithel; Philippe Thomas; Roland Roller; Eiji Aramaki; Shoko Wakamiya; Shuntaro Yada; Pierre Zweigenbaum; Karen O’Connor; Sai Samineni; Sophia Hernandez; Yao Ge; Swati Rajwal; Sudeshna Das; Abeed Sarker; Ari Klein; Ana Schmidt; Vishakha Sharma; Raul Rodriguez-Esteban; Juan Banda; Ivan Amaro; Davy Weissenbacher; Graciela Gonzalez-Hernandez
Venue: ACL 2024 (Proceedings of the 9th Social Media Mining for Health Research and Applications (SMM4H 2024) Workshop and Shared Tasks)

Abstract: For the past nine years, the Social Media Mining for Health Applications (#SMM4H) shared tasks have promoted community-driven development and evaluation of advanced natural language processing systems to detect, extract, and normalize health-related information in publicly available user-generated content. This year, #SMM4H included seven shared tasks in English, Japanese, German, French, and Spanish from Twitter, Reddit, and health forums. A total of 84 teams from 22 countries registered for #SMM4H, and 45 teams participated in at least one task. This represents a growth of 180% and 160% in registration and participation, respectively, compared to the last iteration. This paper provides an overview of the tasks and participating systems. The data sets remain available upon request, and new systems can be evaluated through the post-evaluation phase on CodaLab.
Language-guided World Models: A Model-based Approach to AI Control
Authors: Alex Zhang; Khanh Nguyen; Jens Tuyls; Albert Lin; Karthik Narasimhan
Venue: ACL 2024 (Proceedings of the 4th Workshop on Spatial Language Understanding and Grounded Communication for Robotics (SpLU-RoboNLP 2024))

Abstract: Developing internal world models for artificial agents opens an efficient channel for humans to communicate with and control them. In addition to updating policies, humans can modify the world models of these agents in order to influence their decisions.The challenge, however, is that currently existing world models are difficult for humans to adapt because they lack a natural communication interface. Aimed at addressing this shortcoming, we develop *Language-Guided World Models* (LWMs), which can capture environment dynamics by reading language descriptions. These models enhance agent communication efficiency, allowing humans to simultaneously alter their behavior on multiple tasks with concise language feedback. They also enable agents to self-learn from texts originally written to instruct humans. To facilitate the development of LWMs, we design a challenging benchmark based on the game of MESSENGER (Hanjie et al., 2021), requiring compositional generalization to new language descriptions and environment dynamics. Our experiments reveal that the current state-of-the-art Transformer architecture performs poorly on this benchmark, motivating us to design a more robust architecture. To showcase the practicality of our proposed LWMs, we simulate a scenario where these models augment the interpretability and safety of an agent by enabling it to generate and discuss plans with a human before execution. By effectively incorporating language feedback on the plan, the models boost the agent performance in the real environment by up to three times without collecting any interactive experiences in this environment.
Learning Communication Policies for Different Follower Behaviors in a Collaborative Reference Game
Authors: Philipp Sadler; Sherzod Hakimov; David Schlangen
Venue: ACL 2024 (Proceedings of the 4th Workshop on Spatial Language Understanding and Grounded Communication for Robotics (SpLU-RoboNLP 2024))

Abstract: In this work, we evaluate the adaptability of neural agents towards assumed partner behaviors in a collaborative reference game. In this game, success is achieved when a knowledgeable guide can verbally lead a follower to the selection of a specific puzzle piece among several distractors. We frame this language grounding and coordination task as a reinforcement learning problem and measure to which extent a common reinforcement training algorithm (PPO) is able to produce neural agents (the guides) that perform well with various heuristic follower behaviors that vary along the dimensions of confidence and autonomy. We experiment with a learning signal that in addition to the goal condition also respects an assumed communicative effort. Our results indicate that this novel ingredient leads to communicative strategies that are less verbose (staying silent in some of the steps) and that with respect to that the guide’s strategies indeed adapt to the partner’s level of confidence and autonomy.
Collection of Japanese Route Information Reference Expressions Using Maps as Stimuli
Authors: Yoshiko Kawabata; Mai Omura; Hikari Konishi; Masayuki Asahara; Johane Takeuchi
Venue: ACL 2024 (Proceedings of the 4th Workshop on Spatial Language Understanding and Grounded Communication for Robotics (SpLU-RoboNLP 2024))

Abstract: We constructed a database of Japanese expressions based on route information. Using 20 maps as stimuli, we requested descriptions of routes between two points on each map from 40 individuals per route, collecting 1600 route information reference expressions. We determined whether the expressions were based solely on relative reference expressions by using landmarks on the maps. In cases in which only relative reference expressions were used, we labeled the presence or absence of information regarding the starting point, waypoints, and destination. Additionally, we collected clarity ratings for each expression using a survey.
Documenting the Unwritten Curriculum of Student Research
Authors: Shomir Wilson
Venue: ACL 2024 (Proceedings of the Sixth Workshop on Teaching NLP)

Abstract: Graduate and undergraduate student researchers in natural language processing (NLP) often need mentoring to learn the norms of research. While methodological and technical knowledge are essential, there is also a “hidden curriculum” of experiential knowledge about topics like work strategies, common obstacles, collaboration, conferences, and scholarly writing. As a professor, I have written a set of guides that cover typically unwritten customs and procedures for academic research. I share them with advisees to help them understand research norms and to help us focus on their specific questions and interests. This paper describes these guides, which are freely accessible on the web (https://shomir.net/advice), and I provide recommendations to faculty who are interested in creating similar materials for their advisees.
Example-Driven Course Slides on Natural Language Processing Concepts
Authors: Natalie Parde
Venue: ACL 2024 (Proceedings of the Sixth Workshop on Teaching NLP)

Abstract: Natural language processing (NLP) is a fast-paced field and a popular course topic in many undergraduate and graduate programs. This paper presents a comprehensive suite of example-driven course slides covering NLP concepts, ranging from fundamental building blocks to modern state-of-the-art approaches. In contributing these slides, I hope to alleviate burden for those starting out as faculty or in need of course material updates. The slides are publicly available for external use and are updated regularly to incorporate new advancements.
Industry vs Academia: Running a Course on Transformers in Two Setups
Authors: Irina Nikishina; Maria Tikhonova; Viktoriia Chekalina; Alexey Zaytsev; Artem Vazhentsev; Alexander Panchenko
Venue: ACL 2024 (Proceedings of the Sixth Workshop on Teaching NLP)

Abstract: This paper presents a course on neural networks based on the Transformer architecture targeted at diverse groups of people from academia and industry with experience in Python, Machine Learning, and Deep Learning but little or no experience with Transformers. The course covers a comprehensive overview of the Transformers NLP applications and their use for other data types. The course features 15 sessions, each consisting of a lecture and a practical part, and two homework assignments organized as CodaLab competitions. The first six sessions of the course are devoted to the Transformer and the variations of this architecture (e.g., encoders, decoders, encoder-decoders) as well as different techniques of model tuning. Subsequent sessions are devoted to multilingualism, multimodality (e.g., texts and images), efficiency, event sequences, and tabular data.We ran the course for different audiences: academic students and people from industry. The first run was held in 2022. During the subsequent iterations until 2024, it was constantly updated and extended with recently emerged findings on GPT-4, LLMs, RLHF, etc. Overall, it has been ran six times (four times in industry and twice in academia) and received positive feedback from academic and industry students.
Striking a Balance between Classical and Deep Learning Approaches in Natural Language Processing Pedagogy
Authors: Aditya Joshi; Jake Renzella; Pushpak Bhattacharyya; Saurav Jha; Xiangyu Zhang
Venue: ACL 2024 (Proceedings of the Sixth Workshop on Teaching NLP)

Abstract: While deep learning approaches represent the state-of-the-art of natural language processing (NLP) today, classical algorithms and approaches still find a place in NLP textbooks and courses of recent years. This paper discusses the perspectives of conveners of two introductory NLP courses taught in Australia and India, and examines how classical and deep learning approaches can be balanced within the lecture plan and assessments of the courses. We also draw parallels with the objects-first and objects-later debate in CS1 education. We observe that teaching classical approaches adds value to student learning by building an intuitive understanding of NLP problems, potential solutions, and even deep learning models themselves. Despite classical approaches not being state-of-the-art, the paper makes a case for their inclusion in NLP courses today.
Co-Creational Teaching of Natural Language Processing
Authors: John P. McCrae
Venue: ACL 2024 (Proceedings of the Sixth Workshop on Teaching NLP)

Abstract: Traditional lectures have poorer outcomes compared to active learning methodologies, yet many natural language processing classes in higher education still follow this outdated methodology. In this paper, we present, co-creational teaching, a methodology that encourages partnership between staff and lecturers and show how this can be applied to teach natural language processing. As a fast-moving and dynamic area of study with high interest from students, natural language processing is an ideal subject for innovative teaching methodologies to improve student outcomes. We detail our experience with teaching natural language processing through partnership with students and provide detailed descriptions of methodologies that can be used by others in their teaching, including considerations of diverse student populations.
Collaborative Development of Modular Open Source Educational Resources for Natural Language Processing
Authors: Matthias Aßenmacher; Andreas Stephan; Leonie Weissweiler; Erion Çano; Ingo Ziegler; Marwin Härttrich; Bernd Bischl; Benjamin Roth; Christian Heumann; Hinrich Schütze
Venue: ACL 2024 (Proceedings of the Sixth Workshop on Teaching NLP)

Abstract: In this work, we present a collaboratively and continuously developed open-source educational resource (OSER) for teaching natural language processing at two different universities. We shed light on the principles we followed for the initial design of the course and the rationale for ongoing developments, followed by a reflection on the inter-university collaboration for designing and maintaining teaching material. When reflecting on the latter, we explicitly emphasize the considerations that need to be made when facing heterogeneous groups and when having to accommodate multiple examination regulations within one single course framework. Relying on the fundamental principles of OSER developments as defined by Bothmann et al. (2023) proved to be an important guideline during this process. The final part pertains to open-sourcing our teaching material, coping with the increasing speed of developments in the field, and integrating the course digitally, also addressing conflicting priorities and challenges we are currently facing.
From Hate Speech to Societal Empowerment: A Pedagogical Journey Through Computational Thinking and NLP for High School Students
Authors: Alessandra Teresa Cignarella; Elisa Chierchiello; Chiara Ferrando; Simona Frenda; Soda Marem Lo; Andrea Marra
Venue: ACL 2024 (Proceedings of the Sixth Workshop on Teaching NLP)

Abstract: The teaching laboratory we have created integrates methodologies to address the topic of hate speech on social media among students while fostering computational thinking and AI education for societal impact. We provide a foundational understanding of hate speech and introduce computational concepts using matrices, bag of words, and practical exercises in platforms like Colaboratory. Additionally, we emphasize the application of AI, particularly in NLP, to address real-world challenges. Through retrospective evaluation, we assess the efficacy of our approach, aiming to empower students as proactive contributors to societal betterment. With this paper we present an overview of the laboratory’s structure, the primary materials used, and insights gleaned from six editions conducted to the present date.
Tightly Coupled Worksheets and Homework Assignments for NLP
Authors: Laura Biester; Winston Wu
Venue: ACL 2024 (Proceedings of the Sixth Workshop on Teaching NLP)

Abstract: In natural language processing courses, students often struggle to debug their code. In this paper, we present three homework assignments that are tightly coupled with in-class worksheets. The worksheets allow students to confirm their understanding of the algorithms on paper before trying to write code. Then, as students complete the coding portion of the assignments, the worksheets aid students in the debugging process as test cases for the code, allowing students to seamlessly compare their results to those from the correct execution of the algorithm.
Teaching LLMs at Charles University: Assignments and Activities
Authors: Jindřich Helcl; Zdeněk Kasner; Ondřej Dušek; Tomasz Limisiewicz; Dominik Macháček; Tomáš Musil; Jindřich Libovický
Venue: ACL 2024 (Proceedings of the Sixth Workshop on Teaching NLP)

Abstract: This paper presents teaching materials, particularly assignments and ideas for classroom activities, from a new course on large language modelsThe assignments include experiments with LLM inference for weather report generation and machine translation.The classroom activities include class quizzes, focused research on downstream tasks and datasets, and an interactive “best paper” session aimed at reading and comprehension of research papers.
Empowering the Future with Multilinguality and Language Diversity
Authors: En-Shiun Annie Lee; Kosei Uemura; Syed Mekael Wasti; Mason Shipton
Venue: ACL 2024 (Proceedings of the Sixth Workshop on Teaching NLP)

Abstract: The rapid advancements and the widespread transformation of Large Language Models, have made it necessary to incorporate these cutting-edge techniques into the educational curricula of Natural Language Processing (NLP) with limited computing resources. This paper presents an applied NLP course designed for upper-year computer science undergraduate students on state-of-the-art techniques with an emphasis on multilinguality and language diversity. We hope to empower learners to advance their language community while preparing for industry.
A Course Shared Task on Evaluating LLM Output for Clinical Questions
Authors: Yufang Hou; Thy Thy Tran; Doan Nam Long Vu; Yiwen Cao; Kai Li; Lukas Rohde; Iryna Gurevych
Venue: ACL 2024 (Proceedings of the Sixth Workshop on Teaching NLP)

Abstract: This paper presents a shared task that we organized at the Foundations of Language Technology (FoLT) course in 2023/2024 at the Technical University of Darmstadt, which focuses on evaluating the output of Large Language Models (LLMs) in generating harmful answers to health-related clinical questions. We describe the task design considerations and report the feedback we received from the students. We expect the task and the findings reported in this paper to be relevant for instructors teaching natural language processing (NLP).
A Prompting Assignment for Exploring Pretrained LLMs
Authors: Carolyn Jane Anderson
Venue: ACL 2024 (Proceedings of the Sixth Workshop on Teaching NLP)

Abstract: As the scale of publicly-available large language models (LLMs) has increased, so has interest in few-shot prompting methods. This paper presents an assignment that asks students to explore three aspects of large language model capabilities (commonsense reasoning, factuality, and wordplay) with a prompt engineering focus. The assignment consists of three tasks designed to share a common programming framework, so that students can reuse and adapt code from earlier tasks. Two of the tasks also involve dataset construction: students are asked to construct a simple dataset for the wordplay task, and a more challenging dataset for the factuality task. In addition, the assignment includes reflection questions that ask students to think critically about what they observe.
Teaching Natural Language Processing in Law School
Authors: Daniel Braun
Venue: ACL 2024 (Proceedings of the Sixth Workshop on Teaching NLP)

Abstract: Fuelled by technical advances, the interest in Natural Language Processing in the legal domain has rapidly increased over the last months and years. The design, usage, and testing of domain-specific systems, but also assessing these systems from a legal perspective, needs competencies at the intersection of law and Natural Language Processing. While the demand for such competencies is high among students, only a few law schools, particularly in Europe, teach such competencies. In this paper, we present the design for a Natural Language Processing course for postgraduate law students that is based on the principle of constructive alignment and has proven to be successful over the last three years.
Exploring Language Representation through a Resource Inventory Project
Authors: Carolyn Jane Anderson
Venue: ACL 2024 (Proceedings of the Sixth Workshop on Teaching NLP)

Abstract: The increasing scale of large language models has led some students to wonder what contributions can be made in academia. However, students are often unaware that LLM-based approaches are not feasible for the majority of the world’s languages due to lack of data availability. This paper presents a research project in which students explore the issue of language representation by creating an inventory of the data, preprocessing, and model resources available for a less-resourced language. Students are put into small groups and assigned a language to research. Within the group, students take on one of three roles: dataset investigator, preprocessing investigator, or downstream task investigator. Students then work together to create a 7-page research report about their language.
BELT: Building Endangered Language Technology
Authors: Michael Ginn; David Saavedra-Beltrán; Camilo Robayo; Alexis Palmer
Venue: ACL 2024 (Proceedings of the Sixth Workshop on Teaching NLP)

Abstract: The development of language technology (LT) for an endangered language is often identified as a goal in language revitalization efforts, but developing such technologies is typically subject to additional methodological challenges as well as social and ethical concerns. In particular, LT development has too often taken on colonialist qualities, extracting language data, relying on outside experts, and denying the speakers of a language sovereignty over the technologies produced.We seek to avoid such an approach through the development of the Building Endangered Language Technology (BELT) website, an educational resource designed for speakers and community members with limited technological experience to develop LTs for their own language. Specifically, BELT provides interactive lessons on basic Python programming, coupled with projects to develop specific language technologies, such as spellcheckers or word games. In this paper, we describe BELT’s design, the motivation underlying many key decisions, and preliminary responses from learners.