Question: Does the cluster labeling prove to be very effective at pulling out the patterns?
Evidence: We applied this method to discover gender-associated words in both domains. In Table TABREF9, we present a sample of the most gender-associated nouns from the celebrity domain. Several themes emerge: for example, female celebrities seem to be more associated with appearance (“gown,” “photo,” “hair,” “look”), while male celebrities are more associated with creating content (“movie,” “film,” “host,” “director”). This echoes real-world trends: for instance, on the red carpet, actresses tend to be asked more questions about their appearance –- what brands they are wearing, how long it took to get ready, etc. –- while actors are asked questions about their careers and creative processes (as an example, see BIBREF31).
Table TABREF9 also includes some of the most gender-associated verbs and adjectives from the professor domain. Female CS professors seem to be praised for being communicative and personal with students (“respond,” “communicate,” “kind,” “caring”), while male CS professors are recognized for being knowledgeable and challenging the students (“teach,”, “challenge,” “brilliant,” “practical”). These trends are well-supported by social science literature, which has found that female teachers are praised for “personalizing” instruction and interacting extensively with students, while male teachers are praised for using “teacher as expert” styles that showcase mastery of material BIBREF32.
These findings establish that there are clear differences in how people talk about women and men – even with Bonferroni correction, there are still over 500 significantly gender-associated nouns, verbs, and adjectives in the celebrity domain and over 200 in the professor domain. Furthermore, the results in both domains align with prior studies and real world trends, which validates that our methods can capture meaningful patterns and innovatively provide evidence at the large-scale. This analysis also hints that it can be helpful to abstract from words to topics to recognize higher-level patterns of gender associations, which motivates our next section on clustering.
Table TABREF11 displays a sample of our results – we find that the clusters are coherent in context and the labels seem reasonable. In the next section, we discuss human evaluations that we conducted to more rigorously evaluate the output, but first we discuss the value of these methods toward analysis.
At the word-level, we hypothesized that in the celebrity domain, women were more associated with appearance and men with creating content. Now, we can validate those hypotheses against labeled clusters – indeed, there is a cluster labeled clothing that is 100% female (i.e. 100% words are female-associated), and a 80% male cluster labeled movie. Likewise, in the professor domain, we had guessed that women are associated with communication and men with knowledge, and there is a 100% female cluster labeled communication and a 89% male cluster labeled cognition. Thus, cluster labeling proves to be very effective at pulling out the patterns that we believed we saw at the word-level, but could not formally validate.
The clusters we mentioned so far all lean heavily toward one gender association or the other, but some clusters are interesting precisely because they do not lean heavily – this allows us to see where semantic groupings do not align exactly with gender association. For example, in the celebrity domain, there is a cluster labeled lover that has a mix of female-associated words (“boyfriend,” “beau,” “hubby”) and male-associated words (“wife,” “girlfriend”). Jointly leveraging cluster labels and gender associations allows us to see that in the semantic context of having a lover, women are typically associated with male figures and men with female figures, which reflects heteronormativity in society.
Thus, cluster labeling proves to be very effective at pulling out the patterns that we believed we saw at the word-level, but could not formally validate.
Answer: Yes, it does.