Question: What does the second dataset contain?
Evidence: Our first dataset contains articles from celebrity magazines People, UsWeekly, and E!News. We labeled each article for whether it was reporting on men, women, or neither/unknown. To do this, we first extracted the article's topic tags. Some of these tags referred to people, but others to non-people entities, such as “Gift Ideas” or “Health.” To distinguish between these types of tags, we queried each tag on Wikipedia and checked whether the top page result contained a “Born” entry in its infobox – if so, we concluded that the tag referred to a person.
Then, from the person's Wikipedia page, we determined their gender by checking whether the introductory paragraphs of the page contained more male or female pronouns. This method was simple but effective, since pronouns in the introduction almost always resolve to the subject of that page. In fact, on a sample of 80 tags that we manually annotated, we found that comparing pronoun counts predicted gender with perfect accuracy. Finally, if an article tagged at least one woman and did not tag any men, we labeled the article as Female; in the opposite case, we labeled it as Male.
Our second dataset contains reviews from RateMyProfessors (RMP), an online platform where students can review their professors. We included all 5,604 U.S. schools on RMP, and collected all reviews for CS professors at those schools. We labeled each review with the gender of the professor whom it was about, which we determined by comparing the count of male versus female pronouns over all reviews for that professor. This method was again effective, because the reviews are expressly written about a certain professor, so the pronouns typically resolve to that professor.
In addition to extracting the text of the articles or reviews, for each dataset we also collected various useful metadata. For the celebrity dataset, we recorded each article's timestamp and the name of the author, if available. Storing author names creates the potential to examine the relationship between the gender of the author and the gender of the subject, such as asking if there are differences between how women write about men and how men write about men. In this work, we did not yet pursue this direction because we wanted to begin with a simpler question of how gender is discussed: regardless of the gender of the authors, what is the content being put forth and consumed? Furthermore, we were unable to extract author gender in the professor dataset since the RMP reviews are anonymous. However, in future work, we may explore the influence of author gender in the celebrity dataset.
For the professor dataset, we captured metadata such as each review's rating, which indicates how the student feels about the professor on a scale of AWFUL to AWESOME. This additional variable in our data creates the option in future work to factor in sentiment; for example, we could study whether there are differences in language used when criticizing a female versus a male professor.
Our second dataset contains reviews from RateMyProfessors (RMP), an online platform where students can review their professors.
Answer: The second dataset contains reviews from RateMyProfessors (RMP), an online platform where students can review their professors.