Teaching NLP with Bracelets and Restaurant Menus: An Interactive Workshop for I talian Students (2021)

Abstract: Although Natural Language Processing is at the core of many tools young people use in their everyday life, high school curricula (in Italy) do not include any computational linguistics education. This lack of exposure makes the use of such tools less responsible than it could be, and makes choosing computational linguistics as a university degree unlikely. To raise awareness, curiosity, and longer-term interest in young people, we have developed an interactive workshop designed to illustrate the basic principles of NLP and computational linguistics to high school Italian students aged between 13 and 18 years. The workshop takes the form of a game in which participants play the role of machines needing to solve some of the most common problems a computer faces in understanding language: from voice recognition to Markov chains to syntactic parsing. Participants are guided through the workshop with the help of instructors, who present the activities and explain core concepts from computational linguistics. The workshop was presented at numerous outlets in Italy between 2019 and 2020, both face-to-face and online.

## introduction have you used google this week? this question would kick off the activity that we describe in this paper every time we delivered it. and a number of follow-up comments would generally appear. what for? translating, getting some help for homework, looking for info, writing collaboratively -and getting spelling correction! in our workshops, we talk to groups of teenagers -even if someone has not personally used any of those tools on a daily basis, it is utmost unlikely that they have never interacted with a vocal assistant, wondered how their email spam filter works, used text predictions, or spoken to a chat-bot. also, applications that do not require a proactive role of the user are growing: most of us, for example, are subject to targeted advertising, profiled on the content we produce and share on social media. natural language processing (nlp) has grown at an incredibly fast pace, and it is at the core of many of the tools we use every day. 1 at the same time, though, awareness of its underlying mechanisms and the scientific discussion that has led to such innovations, and even nlp's very existence as a scientific discipline is generally much less widespread and is basically unknown to the general public @xcite . a concurrent cause to this lack of awareness resides in the fact that in more traditional high-school formal education systems, such as the italian one, "young disciplines" such as linguistics and computer science tend to be overlooked. grammar, that in a high-school setting is the closest field to linguistics, is rarely taught as a descriptive discipline; oftentimes, it is presented as a set of norms that one should follow in order to speak and write correctly in a given language. while this approach has its benefits, it is particularly misleading when it comes to what actual linguistic research is about. similarly, computer science is often misread by the general public as an activity that deals with computers, while aspects concerning information technology and language processing are often ne-glected. this often leads to two important consequences. first, despite the overwhelming amount of nlp applications, students and citizens at large lack the basic notions that would allow them to fully understand technology and interact with it in a responsible and critical way. second, high-school students might not be aware of computational linguistics as an option for their university degree. oftentimes, students that enrol in humanities degrees are mainly interested in literature and they only get acquainted with linguistics as discipline at university. at the same time, most computer science curricula in italian higher education rarely focus on natural language-based applications. as a result, computational linguistics as such is practically never taught before graduate studies. as members of the italian association for computational linguistics (ailc, www.ai-lc.it ) we have long felt the need to bridge this knowledge gap, and made dissemination a core goal of the association. as a step in this direction, we have developed a dissemination activity that covers the basic aspects of what it means to process and analyze language computationally. this is the first activity of its kind developed and promoted by ailc, and to the best of our knowledge, among the first in italy at large. this contribution describes the activity itself, the way it was implemented as a workshop for high school students in the context of several dissemination events, and how it can serve as a blueprint to develop similar activities for yet new languages. ## genesis and goals we set to develop an activity whose main aim would be to provide a broad overview of language modeling, and, most importantly, to highlight the open challenges in language understanding and generation. without any ambition to present and explain the actual nlp techniques to students, we rather focused on showing how language, which is usually conceptualized by the layperson as a simple and monolithic object, is instead a complex stratification of interconnected layers that need to be disentangled in order to provide a suitable formalization. in developing our activity, we took inspiration from the word salad linguistic puzzle, as published in @xcite : charlie and jane had been passing notes in class, when suddenly their teacher mr. john-son saw what was going on. he rushed to the back of the class, took the note charlie had just passed jane, and ripped it up, dropping the pieces on the floor. jane noticed that he had managed to rip each word of the message onto a separate piece of paper. the pieces of paper were, in alphabetical order, as follows: dog, in, is, my, school, the. most likely, what did charlie's note originally say? the problem includes a number of follow up questions that encourage the student to reflect upon the boundaries of sentence structure. in particular, we found that the word salad puzzle would give us the opportunity to introduce some of the core aspects of computational linguistics' research. approaching the problem with no previous knowledge helps raising some crucial questions, such as: what are the building blocks of our linguistic ability that allow us to perform such a task?, how much knowledge can we extract from text alone?, what does linguistic knowledge look like? since the workshop here presented is the first activity of this kind in the italian context, we took inspiration from games and problems such as those outlined in @xcite and used for the north american computational linguistics olympiads, similar to the ones described in van @xcite and @xcite . particularly, we were inspired by the institution of (computational) linguistic olympiads in making our workshop a problem-solving game with different activities, each related to a different aspect of computational language processing. linguistic olympiads are now an established annual event in many parts of the world since they first took place in moscow in 1965. in these competitions students (generally of high-school age) are faced with linguistic problems of varying nature, that require participants to use problem-solving abilities to uncover underlying patterns or rules in the data. for an in-depth discussion of the history and diffusion of linguistic olympiads in the world, see @xcite and @xcite . in the choice of algorithms to include in our dissemination activity, we decided to leave aside neural networks and instead focus on traditional statistical approaches, both for historical reasons and for the fact that these convey more clearly the distinction between different layers of linguistic information and their roles in language modeling. a brief discussion of most recent nlp technologies, including the application of neural networks, is included in the final part of the workshop (sec. 3.5). the activity is targeted at students in their last year of middle school (13 years of age) or older. while we believe 13 is a good entry point, there isn't an actual upper-bound, since the activity can be enjoyed by people of any older age (though in practice participation was mainly offered to schools, with the oldest students being 18-19). we thought this would be the appropriate target audience of this workshop for two main reasons. on the one hand, we believe that coming to the activity with a richer metalinguistic background, typically acquired during the first italian school cycle, would be beneficial for the attendees to better grasp the differences between the scientific approach to language and the more prescriptive approach they are exposed to in school. on the other hand we also conceived the activity as a way of helping students in their future study choices: we therefore included both students attending their last year of middle school and therefore about to choose a high school curriculum as well as high school students, the latter in order to provide them with more options for their university choices. ## the activity we planned our dissemination activity for a 90 minutes time slot, divided into five parts, as detailed in ## the workshop in action the activity -here outlined in its complete and original form -was presented in various outlets during the last year and a half. it was initially designed for the 2019 edition of the "bergamo scienza" science festival foot_3 , where it was presented live to over 450 participants in the course of two weeks. a simplified "print-and-play" version of the workshop was also presented at the 2020 edition of the sissa foot_4 student day. due to the covid-19 pandemic, all other presentations of the activity had to be moved online. transposing the workshop crucially meant striving to maintaining the interactive nature of the activities without the possibility of meeting face to face. to do so, we integrated our original presentation on google slides with the interactive presentation software mentimeter foot_5 -which we used for questions, polling and quizzes. the corpus and tokens were presented via a web interface created especially for this purpose 7 . the interface presented the masked corpus, complete with pos tags and syntactic annotations. for the bracelet activity participants were automatically assigned a number of tokens which they could use to build a sentence by simply dragging and dropping them. (figures 14 and 15 ). this online version was crafted in the first place to be presented at "festival della scienza" foot_7 (figure 16 ), a science festival primarily aimed at school students held each year in genoa, where multiple 45' sessions of the workshop were run over the course of two days. the fourth activity (section 3.4) involved "bootstrapping" syntactic rules based only on our color-based annotation. to simplify online interaction, we only used the unmasked italian version of the corpus and participants played collectively, helping each other to build sentences and grammatical rules: rules were collected through a mentimeter poll, while a sentence was generated in a guided demonstration by the presenter of the workshop. overall, the workshop transposed really well online, and was extremely well-received in this version as well. the online modality also allowed us to present it to a more vast and varied audience than just students: a version for the general public was presented at european researchers' night (bright night foot_8 ) at the university of pisa, thanks to a collaboration with the computational linguistics laboratory foot_9 and ilc-cnr foot_10 in november 2020, a dedicated session was run for the high school its buzzi foot_11 (prato, italy) in december 2020, and during the second edition of the science web festival foot_12 in april 2021. ## reusability: this activity as a blueprint as mentioned in section 1, our activity was inspired by a collection of english-language prob-lems created for students participating in the computational linguistics olympiad @xcite . we adapted the original activity to the italian language and context. while we tried to choose widely shared linguistic principles to communicate, the operation of adapting the game to a different language obviously requires language specific choices and details, which would have to be re-evaluated when porting the activity to yet new languages. particularly, since the materials have been developed for italian, it might be the case that transposition is not straightforward for some languages. however, we believe that the general structure of our workshop can serve as a blueprint for extension to new languages. for this reason all the relevant materials are made available in a dedicated repository. the repository includes both scripts to reproduce our activity as well as a general set of insights/recommendations regarding the structure and principles of the workshop. all of the scripts necessary to produce the materials used in the game's workflow in a different language are made available in our open-access repository foot_13 . to get the activity into production using the scripts provided, one only needs to create an annotated corpus in the target language. our specific choice of "snow white" as a corpus is motivated by the fact that the story can be phrased in a fairly repetitive formulation, with two advantages. one is that enough bigrams are present that enable the generation of new sentences with the bracelet method. the other one is that the story contains recognizable characters (e.g., the dwarfs, the evil queen etc.), so that, when the unmasked text is revealed, the process results intuitively transparent. such characteristics are desirable for the activity, and should be kept in mind when choosing a new text for a new language. in addition to sharing scripts, we are sharing here the core structure and principles the workshop relies on, which can be reproduced when replicating the activity also for a different language. parts 1&2 at the beginning of the workshop (see section 3.2) we show the limits of current technologies, in particular in terms of adaptability. to this end, we exploited diatopic variations, such as italian regional accents, since this is an aspect readily available to italian students. in the context of other languages, the same concept could be however shown by different means, such as the influence of jargon or minority languages, or simply differences in accuracy depending on, age or other socio-demographic and socio-cultural variables. part 3 the next part of the activity (section 3.3) is the most easily reproducible in a different language as it only exploits statistical co-occurrences as a cue for linguistic structure. the only prerequisite here is the availability of a sufficiently standardized tokenization for the language of interest. as described above, we masked the language through a transliteration system: this was achieved either substituting words with sequences of sym-bols, or with non-words. in the case of non-words, these were generated by manually defining a series of phonotactic constraints for italian, which should be adapted to the target language. the main message to be conveyed through this activity is that language is a complex system; we did this disentangling semantics from the purely symbolic tier, which is the one computers most commonly manipulate. ## looking back and ahead in the previous sections we described an interactive workshop designed to illustrate the basic principles of computational linguistics to young italian students. it is the first activity of its kind to be designed by the italian association for computational linguistics and among the first dissemination activities in italy for computational linguistics directed to young students. the activity had the broad aim of increasing awareness towards applications based on language technology, and introduce students to the study of language as a scientific discipline. we run the activity in both face-to-face and, due to covid-19, online form: generally speaking, we received enthusiastic feedback both from younger participants and from the more general public. we adapted the activity to a variety of formats and timeslots, ranging from 30 to 90 minutes: the amount of time required to approach the game and get acquainted with the concepts is of course variable, and depends on the participants' background and on the level of engagement that is expected of them. generally speaking 45 minutes are enough for a presentation including some interaction with the audience, especially in the online setting, but at least 90 minutes are needed for the participants to fully experiment with the hands-on activity. we want to specifically stress how time is a central ingredient in the activity. while the gamerelated aspects remained engaging and fun even in the shortened, online versions, in order to fully grasp the mechanisms underlying the presented algorithms longer sessions would be needed. in fact, we often felt that more time would be beneficial for a deeper discussion about language as an object of study itself, and about language as data on which theories can be built. in particular, shorter timeslots or less guided activities enhance the risk of participants approaching the challenge as a puzzle that they can solve regardless of linguistic knowledge. this is because the approach to language we are presenting is entirely new to our audience, and not only to the younger students. although we did not have a formal system in place to collect systematic feedback, the overall response across venues and conditions has been extremely positive. 15 curiosity and engagement of participants remained high both onsite and online, and we received many questions on several aspects of natural language processing and neural networks, as well as concerning its role in artificial intelligence at large. the participants' enthusiastic questions gave us many ideas for future dissemination activities. in fact, the technological world is advancing fast and we firmly believe that it is necessary to spread more awareness on the inner workings of ai-based technologies, to develop a society-level conscience to approach them in a critical way. the activity described in this paper was targeted at middle to high-school students as well as the general public. it would be interesting to engage with a younger audience as well, as communicating the study and (computational) modelling of language to them would raise awareness towards language studies as a scientific discipline. for our activity, we took inspiration from one of the problems proposed in @xcite . puzzles such as the ones presented at the computational linguistics competitions are a great way to introduce both important challenges and the methodology to solve them, as they stimulate students to investigate linguistic aspects in a bottomup fashion. organizing the competition in italy would represent a bigger project for our association, to be addressed in the coming years.