On Shortcuts and Biases: How Finetuned Language Models Distinguish Audience-Specific Instructions in Italian and English (2024)

Abstract: Instructional texts for different audience groups can help to address specific needs, but at the same time run the risk of perpetrating biases. In this paper, we extend previous findings on disparate social norms and subtle stereotypes in wikiHow in two directions: We explore the use of fine-tuned language models to determine how audience-specific instructional texts can be distinguished and we transfer the methodology to another language, Italian, to identify crosslinguistic patterns. We find that language models mostly rely on group terms, gender markings, and attributes reinforcing stereotypes.

text: Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 78–93 August 16, 2024 ©2024 Association for Computational Linguistics
Bias Statement
In this study, bias is defined as systematic differences in content and presentation of wikiHow articles that are tailored to different audiences, particularly in ways that can reinforce gender stereotypes or inequities. Such biases include the allocation of topics in a way that reinforces traditional gender stereotypes as well as the use of language that perpetuates hetero-normative gender roles.
Following Blodgett et al. (2020), we recognize that bias is not merely a technical issue but a deeply embedded social problem that reflects structural inequalities. This work analyzes social constructs, as described in collaboratively edited how-to guides, in which biases operate and which, when used as training data, can raise issues in NLP systems.
Potential harms of biased data, as defined above, include unequal access to information, exposure to content that can affect self-esteem and self-worth, as well as limiting individual aspirations. We identify sources of underlying biases in the data as a starting point for editors to create fairer content and for developers to foster more ethical AI systems. As such, our work aims to actively promote diversity and inclusion on a specific online platform and to generally contribute to a more nuanced understanding of origins of gender bias in NLP.

heading: 1 Introduction
text: Instructional texts aim to convey the necessary knowledge for readers to accomplish specific tasks. On the collaboratively edited online platform wikiHow, hundreds of thousands of instructional texts are available on a variety of topics and in multiple languages. The goal, or mission, of this vast repository is to democratize access to knowledge and skills across diverse subject matters.1 Among other works on wikiHow, prior research has explored in how far texts are formulated in linguistically inclusive terms and which adjustments are made for specific target audiences (Suhr and Roth, 2024; Fanton et al., 2023). However, these previous studies primarily relied on simple classifiers and focused exclusively on English texts, leaving a gap in understanding multilingual phenomena and if fine-tuning language models might exacerbate biases (see §2).
Acknowledging the limitations of prior research to English, we first compile a new dataset in a less resourced language, specifically Italian (see §3). Our initial research question investigates how texts for different target audiences in English and Italian vary in terms of the topics they address (see §4). This exploration directly contributes to the analysis of social biases in the data (see Table 1 for an example). To draw comparisons with previous
1https://www.wikihow.com/wikiHow:Mission
78
research, we then explore how articles for different target groups can be distinguished computationally and which characteristics are learned in this process (see §5). Unlike previous work, we employ fine-tuned language models and utilize a wellestablished interpretation method, integrated gradients (Sundararajan et al., 2017). This approach represents a recent advancement beyond simple classifiers to interpreting more sophisticated models that can provide deeper insights into language use and biases.
In short, we make the following contributions:
• We release a new data collection, wikiHowAudIT (short wHA-IT), and assess the audience-specific biases in how-to guides by a topic-based data analysis.
• We cross-lingually compare biases in wHAIT and in an existing English dataset, wikiHowAudiences (short wHA-EN; Fanton et al., 2023), by fine-tuning and analyzing language models for audience classification.

heading: 2 Related Work
text: In this section we briefly review three related areas: Our work continues a series of recent contributions dealing with the collection of data sets for Italian. While there exists little work on instructional texts for Italian, data in English has been examined and tested from different angles and perspectives in the NLP community. Finally, work on model-based data interpretation has received increasing attention, but almost no work studied biases in audiencespecific instructional texts.
Italian NLP datasets. Recent data collections for the Italian language include DIATOPIT (Ramponi and Casula, 2023a), a dataset representative in time and space on variations of non-Standard Italian. A new shared task for geo-locating the linguistic variation in Italy (Ramponi and Casula, 2023b) is based on this data collection. Another recent effort for the Italian language is IRMA, a data collection for studying misinformation (Carrella et al., 2023). In their paper, the authors curated a dataset from untrustworthy websites, and emphasized its significance for the less-represented language studied. Minnema et al. (2023) advance the task of responsibility perspective transfer, in the context of studying gender-based violence, and a dataset of sentences for Italian news about femicides. To
the best of our knowledge, there are no previous studies on how-to guides in the Italian language.
Instructional texts. Anthonio et al. (2020) introduced wikiHowToImprove, a data collection of original and revised sentences based on wikiHow articles and their revision histories. Kojima et al. (2021) contribute with a continual approach for instruction generation. Fanton et al. (2023) examine audience-specific wikiHow guides in English. They find traces of subtle biases, using shallow classifiers and qualitative analyses. In this work, we extend their findings to fine-tuned language models in two languages.
Interpreting Language Models. A number of methods have been proposed recently for interpreting fine-tuned language models. Our work makes use of Integrated Gradients (Sundararajan et al., 2017), which computes the gradients of a model’s output with respect to the input, based on (stepwise) back-propagation and summation as an approximation method. Falk and Lapesa (2022) employ a variant of Integrated Gradients for getting attributions and importance scores. They point to the capabilities of such method(s) “to uncover potential biases picked up by the model”. In their case, the reveal of these biases concerns how the model’s class prediction is influenced by sensitive words. Luu and Inoue (2023) propose the Counterfactual Adversarial Training (CAT) technique, with the broader goal of improving LMs’ robustness. They make use of Integrated Gradients in CAT for calculating tokens’ salience, before obtaining the counterfactual perturbations. This is then put into practice by changing the thus extracted important tokens. Other works that rely on Integrated Gradients include studies on irony detection in Dutch (Maladry et al., 2023) and gender-based violence in Italian (Minnema et al., 2023), among others.

heading: 3 Data
text: We first build a data collection to investigate our first research question, namely how texts for different target audiences in Italian vary in terms of the topics they address. As a starting point, we use how-to guides from publicly available wikiHow dumps2 for Italian. Out of 34,801 guides, 1,031 feature an indicator between round parentheses at the end of the title (see Table 1). For each guide featur-
2https://ftp.fau.de/kiwix/zim/wikihow/, we refer to this file: wikihow_it_maxi_2023-02.

heading: 5 self-care 5 18

heading: 6 school and work 15 8
text: ing a group indicator, we use wikiHow’s Esporta3 service to get the latest version. Following previous work (Fanton et al., 2023), we manually categorize the indicators into four target groups: Women (W), Men (M); Kids (K), Teens (T). Similar to previous work, we find that there is a lack of indicators for non-binary/other groups (see §A.1 for a complete list of common indicators), forcing us to consider only binary distinctions: Women–Men (W–M) and Kids–Tens (K–T).4 Table 2 comprises the distribution over audience groups for the wikiHowAudIT (wHA-IT) corpus, which comprises a total of 423 how-to guides, and for the corpus from previous work (wHA-EN). For training, validation and testing, we create stratified experimental partitions for each task with a proportion of 8 : 1 : 1 (see Table 12 in A.2 for details).

heading: 4 Data Analysis
text: We address our first research question, namely how texts for different target audiences in Italian vary in terms of the topics, by clustering articles according to their content. We describe the approach in §4.1 and findings in §4.2. For this part of our work, we
3https://www.wikihow.it/Speciale:Esporta 4Note that an article may target two groups, meaning that
some data points appear in both distinctions.

heading: C Cluster Name W M

heading: 6 clothes and style 12 11
text: focus exclusively on the TRAIN and DEV partitions of the data in Italian, so that the TEST part remains held-out for computational experiments (see §5).

heading: 4.1 Clustering Approach
text: Our approach makes use of agglomerative clustering, using embeddings for capturing the contents of each article. First, we embed the articles with a sentence-transformer model5 (Reimers and Gurevych, 2019). Second, we normalize the embeddings obtained. Third, we leverage the scikit-learn (Pedregosa et al., 2011) AgglomerativeClustering algorithm and default options to put into practice the clustering, with the distance threshold set to 1.5.6 Finally, we review the titles of the guides assigned to each cluster in order to find an overarching topic.
Inspired by Montariol et al. (2021), we perform an additional validation for topics as cluster names. Specifically, we collect all word tokens within the articles of a cluster and sort them according to their tf-idf scores, providing us with the tokens that seem most relevant for the cluster. In order to select the
5The LM used here for wHA-IT is nickprock/sentence-bert-base-italian-uncased with input size 512 tokens, for wHA-EN sentence-transformers/all-mpnet-base-v2 (384).
6The value of the distance threshold chosen is the default value implemented in the sentence-transformers library for the agglomerative clustering. For wHA-EN, we raised the threshold to 4 experimentally.
most discerning tokens, for each cluster we leave out the tokens featured in all the other clusters.
We execute agglomerative clustering for each task separately: one time for the task W–M and once for K–T. For cross-lingual comparison, we perform the same steps for the wHA-EN corpus introduced by Fanton et al. (2023).

heading: 4.2 Cluster Findings
text: For the task W–M in wHA-IT, we found 10 clusters. For the task K–T, we found 7 clusters. An overview of the clusters for both tasks are shown in Table 3 and 4, including topic-based cluster names and counts for each target group. For W–M we find a prevalent presence of body-related clusters (labelled with 1, 2, 3, 5, 8), as well as socially coded occupations (labelled with 0, 4, 6, 7, 9). Interestingly, there are two clusters (labelled with 5 and 8) that focus not only on physical aspect, but also more in detail about being fit. Additional details can be seen based on the highest-scoring tokens (“weight”, “calories”, “fat”), as summarized for all clusters in Table 5. For K–T, unlike the previous task, we find more behavioral and social activities.
In summary, our analysis on wHA-IT shows how the examined articles are clusterable by topical information across audiences, indicating that topics are not specific for a target group. Considered these
overlaps, we remark that there are less topical biases than we had assumed and it may be interesting to see which differences a computational model learns for distinguishing audiences in Italian.
In wHA-EN, we find 11 clusters for W-M, distributed over both target groups (see Table 7). For K-T, we find 8 clusters (Table 8). For W–M, we meta-group the clusters. The activities to perform in specific places, like in school or outside are labelled with 0, 5, 7, 8. Moreover, a further distinction is between activities in relation to others (labelled with 2, 10) and activities in relation to oneself (labelled with 1, 3, 6, 9). However, cluster 4 (appear and act) cannot unambiguously be allocated to activities in relation to others, nor to activities in relation to oneself, because it features subtly disparate guides. As examples, we show two titles per audience from that cluster:
W: “Be Drama Free”, “Eat Healthy Around Your Friends” M: “Look Handsome”, “Be More Socially Open”
The first example each might imply to work more on oneself rather than in direct relation to others, but it is not possible to conclude exactly so for the other two. That is to say, to eat in a certain way around other people, and to be more socially open, requires at least some relation to others.

heading: C Cluster Name W M
text: Table 6 shows the most discerning tokens for the clusters induced from the English data for W–M. We note that pronouns appear among the highest scoring tokens for several clusters (e.g. clusters 0, 2 and 10), which are at the same time large clusters that contain disproportionally many articles for one of the two target groups (cf. Table 7).
Cross-lingually, we find for W–M that relations to other people (e.g. relationships and friendships), as well as self-centered activities (self-care, personal care) are similarly present in English and Italian. In contrast, body-related topics only seem narrowly present in wHA-EN, whereas they are highly pervasive in wHA-IT. The topics for K–T are largely overlapping across languages. For instance, we find routines for both

heading: C Designation K T

heading: 3 parties 8 7

heading: 4 money (games) 7 5

heading: 5 relationships 7 13

heading: 6 holiday 10 12

heading: 7 crafting 28 9
text: languages. Similarly, we find relationships and friendships, clothes and style in wHAIT and relationships, lifestyle in wHA-EN. However, self-care emerges only from the Italian data, while crafting and money (games) are specific to the English data, which may point at a need for financial education of the younger generations (Lusardi, 2019).

heading: 5 Experimental Setup
text: Given the data and analyses of the previous sections, we next investigate what features and biases computational models learn when they are trained to distinguish articles for different audiences.

heading: 5.1 Models
text: As discussed in Section 2, previous work attempted to distinguish texts by target groups using simple classifiers. However, we take the results from our data analysis as an indicator that lexical and off-theshelf representations might not be fully sufficient
for this task. In order to find more nuanced biases, we propose to fine-tune language models. We test whether this leads to a higher distinction and which patterns are learned in the process. For comparability, we adopt the same setups for LMs fine-tuned on wHA-IT and wHA-EN.
We employ a set of LMs from Hugging Face (Wolf et al., 2020) and set up binary classification tasks based on the previously discussed data. Due to computational constraints, we use LMs with a maximum length of 512 tokens. For Italian, these include the monolingual language models Italian BERT cased/uncased (Schweter, 2020), UmBERTo cased/uncased (Parisi et al., 2020) as well the multilingual models mBERT cased/uncased (Devlin et al., 2019) and XLM-RoBERTa (Conneau et al., 2020). For English, we follow previous work and only tested BERT-cased/uncased (Devlin et al., 2019) and RoBERTa (Liu et al., 2019). For hyperparameter selection, we maximize the macro F1 on the DEV set. We perform 3 trials for each LM and for each task, W–M and K–T, using Optuna (Akiba et al., 2019) as the optimization framework. More details on the tested LMs and used hyperparameters are listed in Appendix A.4.

heading: 5.2 Attributions
text: Based on the F1 scores obtained for each task on the DEV sets, we select the best-performing LM for further analysis. We leverage the Transformers Interpret7 library to inspect which are the parts of the articles that are relevant in distinguishing the audience-specific guides. Specifically, we pass the fine-tuned LM, their tokenizer and the (truncated) articles as inputs to the SequenceClassificationExplainer. The output of each pass is a list of attributions: tokens with respective scores. For W–M, each text is explained with respect to the class label W. For K–T, explanations are taken with respect to the label K. For each task, we first collect attributes for each article and then summarize them for the full task data by averaging the scores found for each article.

heading: 6 Results
text: We first discuss results in terms of model performance for the classification task itself (§6.1) and then analyze the attributions of the models that perform best at distinguishing audiences (§6.2).
7https://github.com/cdpierse/ transformers-interpret

heading: 6.1 Performance
text: In Table 9, we report solely the performance of our best configuration (as determined on the development set) and comparison numbers from Fanton et al. (2023) on wHA-EN. Specifically, we use bert-base-italian-cased for both tasks on wHA-IT, and roberta-base and bert-base-uncased for W-M and K-T, respectively, on wHA-EN. More details on the experiments, i.e. the scores on the three experimental partitions for each corpus, can be found in Table 18 and Table 17 in A.4.
Cross-task comparisons. Considering the wHAIT column, the F1 score is higher for the W–M task than the one obtained for the K–T task. The same finding can be observed for the wHA-EN column. Intuitively, this result could be explained in that the categories of men and women are typically viewed by editors as more discrete than the categories of kids and teens, whose boundaries are continuous in general. This finding represents the opposite of previous work, where a lower score was obtained for the W–M task than for K–T (0.71 vs 0.78; Fanton et al., 2023). We note, however, that results are only partially comparable as Fanton et al. did not apply fine-tuned language models and their experimental setup did not account for stratified partitions.
Cross-language comparisons. We focus now on the W–M row. What emerges is that the performance of the LM finetuned for wHA-EN is slightly higher than the performance of the LM finetuned for wHA-IT (with a difference of about 3 percentage points). We observe a much larger difference for K–T, with a decrease in F1 of around 22 percentage points. Both differences could be explained by the data scarcity for Italian (see Table 2), which seems particularly problematic for the K–T task.
It is further worth pointing out that multilingual models performed consistently worse in our experiments than monolingual models, suggesting that cross-lingual training might not be promising (see also Table 17 in A.4). This finding is in line with findings on the task of responsibility perception prediction for gender-based violence in Italian
IT. Highlighted tokens indicate feminine (f.) and masculine (m.) grammatical gender. A more comprehensive list with scores is provided in the Appendix.
(Minnema et al., 2022), where better performance was also observed by monolingual models.

heading: 6.2 Attributions
text: Our final analysis concerns the attributions by the language models with the highest results on each task, which provide us with insights on generalizable patterns learned from the training data. Table 10 and Table 11 show the top-10 tokens, after filtering of punctuation and sub-word tokens, for each audience in wHA-IT and wHA-EN, respectively.
“Group terms”. We observe that many of the top features to be direct addresses of the reader in terms of their group membership (“even if you’re a kid”). The presence of such “group terms” was also found in the analysis of word-based logistic classification models by Fanton et al. (2023).
For all audiences, our model analysis consistently shows fewer group terms among the topranked and filtered tokens in Italian, as compared to English. For example, 6 out of 10 top tokens
for M in wHA-EN are group terms (‘men’, ‘guy’, ‘boy’, ‘man’ ‘boys’, ‘guys’), whereas for wHA-IT we only find uomini (“men”), uomo (“man”) and ragazzo (“boy”). Although we also observe such group terms for K–T in wHA-EN experiments (e.g. ‘kids’, ‘teen’), this is not the case for the experiments conducted with wHA-IT. If classifiers rely to a high degree on such “group terms” for classification, this finding might explain the low model performance for the Italian K–T data.
Negation. Another feature discussed in previous work concerns the presence of negations. Like in the case of English, we also find for wHA-IT that non (“not”) is among the 10 top-ranked features exactly for the audience W. As highlighted by Fanton et al. (2023), this might raise concerns as negations have been shown to be used in stereotypemaintaining function (Beukeboom et al., 2010, 2020). Consider the following example:

heading: Se stai cercando di farti notare da qualcuno di cui ti sei infatuata o ti trovi al primo appuntamento con lui, non con-
text: cederti troppo facilmente. “If you’re trying to get noticed by someone you’re infatuated with or you’re on a first date with him, don’t give in too easily.”
This extract is from the guide titled Apparire Bella Davanti al Tuo Ex Ragazzo (Solo Ragazze Adolescenti) (“Looking Beautiful In Front Of Your Ex Boyfriend (Teenage Girls Only)”). It reinforces gender-roles, as the targeted audience (Teenage Girls) is not at all encouraged to make the first
move according to their feelings, but rather to stay passive, and to conform to the stereotype about men’s agency (Ellemers, 2018). Moreover, instead of information about what to do, the instruction explicitly points out what “not” to do.
Grammatical gender. What is also interesting in the aforementioned example is the presence of heteronormativity, defined as heterosexuality as the norm (see Warner, 1991, and Vásquez et al., 2022). While this can already be inferred from the title, the explicit use of the masculine pronoun lui (“him”) in the excerpt leaves no space for ambiguity in the interpretation of the assumed gender of the referent.
We can argue that qualcuno (“someone”, m.), is encapsulating generic masculine (Silveira, 1980), as it is not qualcuna (“someone”, f.). Unlike English, Italian features grammatical gender, in terms of which we find a polarising situation: feminine tokens (80%) for W and of masculine tokens (100%) for M (data: wHA-IT). This might provide a shortcut for classifiers to distinguish the instances in the (Italian) W–M task. For K–T, in contrast, we only find traces of masculine gender for kids (30%). Nonetheless, it is worth noting that the usage of generic masculine in Italian, especially, from Table 10, dei (“of the”, m.) could capture cases of collective plurals, for which it is used a masculine plural to refer to groups of unknown genders (also to heterogeneous group in terms of gender).
Taglia i prati. Devi stabilire diverse tariffe in base alla dimensione del giardino. Fatti pubblicità nel quartiere attaccando qualche volantino alle porte dei vicini, ma cerca di essere discreto. “Cut lawns. You need to set different rates based on the size of the yard. Advertise in the neighborhood by sticking a few flyers on neighbors’ doors, but try to be discreet.”
The sentences above are extracted from Guadagnare dei Soldi (per Ragazzini) (“Earn Money (for Kids)”). From those, dei vicini (“of the neighbors”, m.) exemplifies masculine generics.
In summary, we find that grammatical gender in wHA-IT provides a shortcut for language models to distinguish instructions for different audiences. We provide additional attributions in a longer list in the appendix (see Table 21), containing also tokens that correspond to the same lemma: for example, sicura (“sure”, f.) for W versus sicuro (“sure”,
m.) for M. In comparison, the longer list of top attributions for wHA-EN (see Table 19 in A.5), features tokens that represent rather stereotypical attributes such as “makeup”, “pretty”, “pink” for W, and “gentleman”, “nerd”, “handsome” for M.

heading: 7 Conclusion
text: We introduced wikiHowAudIT, a dataset of instructional texts from wikiHow for different audiences in Italian. Our data analysis has shown that wikiHowAudIT contains different topics across audiences, which makes computational modeling difficult. In order to still learn what biases can be found in texts for different audiences, we fine-tuned language models and investigated which attributes rank highest for each target group. As a result, we found that models perform very well even with training on only 100 data points and that they capture more fine-grained differences in English than simpler models from previous work.
However, our analysis of the attributes also confirmed trends already observed with simpler methods: Regardless of language, models consistently learn that texts for different audiences can be distinguished with high effectiveness based on group terms, grammatical gender, negations and stereotype reinforcing references. Several of these points may represent critical issues, particularly given that wikiHow is one of the most visited websites on the internet.8 Our results further support existing findings on gender roles in other domains, such as in stories for children and educational resources for young age groups, where females are also associated with gender stereotypes (Adukia et al., 2022).
One reason for us to analyze texts regarding biases is that we want to understand assumptions structurally made about the readers and to what extent these potentially reflect actual characteristics. Future work should accordingly focus on how to identify and remove those biases that are inadequate (e.g. stereotypes) while maintaining adaptations that appeal to an audience (e.g. group terms). Future work could also include different languages and varieties in order to provide a wider understanding of the shortcuts and biases hereby highlighted. For deeper insights on the biases, we encourage future research that could, for example, mask shortcuts by LMs as identified in our study.
We believe that wikiHow is an ideal resource for 8https://www.wikihow.com/wikiHow:
About-wikiHow
such work because its collaborative nature makes it possible to put changes directly into practice and instructional texts in general would strongly benefit being easier accessible and more inclusive.

heading: Acknowledgements
text: This work is supported by the Ministry of Science, Research, and the Arts, Baden-Württemberg through the project IRIS3D (Reflecting Intelligent Systems for Diversity, Demography and Democracy, Az. 33-7533-9-19/54/5). Work by the second author was funded by the DFG Emmy Noether program (RO 4848/2-1).
Limitations
While the present work focuses on a less frequently studied language, namely Italian, in addition to English, the work is still limited culturally (i.e., to “western culture”). Critically, the considered audience attributes, gender and age, are subjected to a simplification that is for now lacking, in particular, intersectional perspectives (Crenshaw, 1991). Another limitation of this work lies in the focus on a single data source. For better generalizations over the instructional scenarios, it is important to contemplate other, different, data sources. The present work is by no means aimed at reinforcing representational bias. We conceive our research efforts as first steps towards inclusion, especially for queer identities, who can be audiences of instructions but are insufficiently accounted as such. With the present work, our hope is also to stimulate future work on instructions in other, especially under-represented, languages and cultures.

heading: A Appendix
text: A.1 Indicators (Italian) [(‘Android’, 135), (‘PC-o-Mac’, 115), (‘iPhone-o-iPad’, 103), (‘per-Ragazze’, 37), (‘Ragazze’, 22), (‘per-Donne’, 18), (‘Uomini’, 14), (‘Windows-e-Mac’, 14), (‘per-Ragazzi’, 13), (‘Per-Ragazze’, 13), (‘Adolescenti’, 12), (‘per-Adolescenti’, 12), (‘Windows’, 11), (‘Ragazzi’, 11), (‘PC-e-Mac’, 8), (‘perPrincipianti’, 8), (‘per-Uomini’, 8), (‘Ragazze-Adolescenti’, 7), (‘per-Bambini’, 7), (‘iPhone’, 7), (‘Cristianesimo’, 6), (‘per-Ragazze-Adolescenti’, 6), (‘per-ragazze’, 5), (‘Donne’, 4), (‘Windows-10’, 4), (‘USA’, 3), (‘Principianti’, 3), (‘per-Cristiani’, 3), (‘PC’, 3), (‘Per-Uomini’, 3), (‘SEO’, 2), (‘Per-RagazzeAdolescenti’, 2), (‘per-Preadolescenti’, 2), (‘Per-Ragazzi’, 2), (‘Jicama’, 2), (‘Windows-7’, 2), (‘MRI’, 2), (‘Per-gli-Uomini’, 2), (‘per-Ragazzini’, 2), (‘Per-Adolescenti’, 2), (‘RCP’, 2), (‘MRSA’, 2), (‘per-leDonne’, 2), (‘Yoga’, 2), (‘Per-Ragazze-Teenager’, 2), (‘per-le-Adolescenti’, 2), (‘Negli-Stati-Uniti’, 2), (‘per-i-Ragazzi’, 2), (‘LAN’, 2), (‘per-Bambine’, 2), (‘DOC’, 2), (‘Scuola-Media’, 2), (‘Teenager’, 2), (‘Per-Uomini-Gay’, 2), (‘Atletica-Leggera’, 2), (‘Bambini’, 2), (‘DPTS’, 2), (‘Per-Bambini’, 2), (‘iOS’, 2), ...]
A.2 Experimental Partitions
A.3 Clustering results for K–T data
A.4 Modeling
A.5 Attributions

heading: Future Work
text: With the present work, our hope is also to stimulate future work on instructions in other, especially under-represented, languages and cultures