User Profile Characterization Within a Brazilian Online Dispute Resolution Platform
Abstract
This work investigates consumer complaints shared on Consumidor.gov.br , a Brazilian Online Dispute Resolution platform.
Leveraging Natural Language Processing (NLP) techniques, we inquire to what extent complaints by male and female consumers are represented in that platform and whether differences in the language of their complaints can reveal patterns to be explored as linguistic indicators of gender authoring.
Our results show differences in the way males and females construe meanings in terms of word POS, psycholinguistic properties and emotions expressed.
Introduction
In recent years, Online Dispute Resolution (ODR) platforms have become an efficient alternative for consumers seeking to solve problems in services provided by all types of companies.
These platforms allow the resolution of consumer conflicts asynchronously within a short time.
In Brazil, consumers make use of ODR plataforms to mediate their disputes with companies when products or services do not meet their expetations, Consumidor.gov.br 1 being among the most popular ones (Schmidt-Kessen et al., 2019).
By 20th April, 2022, Consumidor. gov.br had over 5 million resolved complaints and over 3.5 million registered consumers.
These numbers have been steadily increasing and companies feel pressed to provide quick and personalized solutions for solving conflicts with their customers.
Bearing upon the fact that ODR platforms play a major role in consumer rights and that equal access is desirable for both male and female consumers, it is relevant to inquire to what extent complaints by male and female consumers are represented in those platforms and whether the language of their complaints differs.
Hence the main goal of this work is to characterize consumer complaints shared on Consumidor.gov.br , investigating whether there are patterns in linguistic choices for complaints made by males and females.
To that end, we used used part of speech tagging, the Brazilian Portuguese version of LWIC (Balage Filho et al., 2013) for psychometric properties of words and the Goemotion model trained on
Brazilian Portuguese for automatic emotion detection (Hammes and de Freitas, 2021).
We found similarities and diferences in the way male and female consumers formulate their complaints.
Our results are expected to inform the design of personalized support systems aimed at providing the most suitable answer to a consumer’s complaint.
Related Work
Our work addresses the following research concerns.
Consumer Complaint Characterization .
Authors in (Ganesan and Zhou, 2016) studied linguistic features in instances of complaint and praise in user reviews from various sources including TripAdvisor, Yelp, Walmart and Sephora.
They investigated sentence length, nouns, adjectives, verbs and negatives, among others, and found that sentences categorized as complaints and praise were on average at least 50% longer than those classed either as praise or complaint.
Moreover, compliments tended to use more adjectives and intensifiers compared to complaints.
In (Preotiuc-Pietro et al., 2019), authors used NLP techniques to successfully predict whether a consumer text is a complaint.
Author profiling .
In (Hsieh et al., 2018), researchers presented preliminary results of an author profiling task in Facebook texts by Brazilians.
They focused on gender and age as well as religion and IT background prediction.
Authors in (Dias and Paraboni, 2020) trained gender classifiers in
Brazilian Portuguese texts in multiple domains by making use of word- and psycholinguistics-based features.
Our work contributes to studies on gender characterization by exploring complaints in Portuguese shared on a Brazilian Online Dispute Resolution
Plataform.
To the best of our knowledge, this is the first study of that kind and is expected to motivate further research on other corpora.
Methodology 3. ODR Dataset
We gathered data from Consumidor.gov.br 3 , a Brazilian ODR platform for the resolution of consumer disputes arising from online and offline transactions.
Our data is split into four sets: (i) consumer complaint metadata including information on company’s market segment, problem category, complainant’s age and gender 4 and date of conflict resolution 5 ; (ii) consumer complaints; (iii) company replies; and (iv) consumer evaluation.
We collected around 1.5 million complaints and 2 million metadata, from January 1st, 2019 to December 31st, 2020.
Regarding consumer gender metadata, the proportion of complaints made by males is around 56.72% and by females around 43.27%.
Only 0.01% selected "prefer not to declare".
3. Gender Identification
In our data, consumer complaints are separate from their metadata.
To identify the consumer’s gender, we applied the following heuristic.
Examining the answers provided by the companies, we found that the majority of them used a form of addressing the consumer which signalled gender, such as “Mrs.{author’s name}” or “Ms.{author’s name }”.
We applied regular expression extraction techniques on the complaints’ responses to assign the author’s gender based on form of address.
We ran the aforementioned technique in our dataset and we assigned a gender label to over 500 thousand complaints, 36 .
4% of complaints to females and 63 .
6% to males.
3. Part of Speech (POS) Tagging
To analyze the grammatical class of each word in a complaint, we applied Part of Speech (POS) tagging (Petrov et al., 2011) using a model trained with the dataset UD Portuguese GSD 6 and STANZA library (Qi et al., 2020).
We calculated Spearman’s correlation 7 and Jaccard similarity (Chung et al., 2019) for the top50 most frequent words to quantify the power of word grammatical class to discriminate complaints made by males and females.
Spearman’s correlation takes into account the ranking position of each word.
The Jaccard similarity compares the words in the two sets to see which members are shared and which are distinct, regardless of their position in the ranking.
We focused on words tagged as verbs, nouns, proper nouns and adjectives.
3. Psycholinguist Features
We studied the psychometric properties of the words in the complaints, relying on the Linguistic
Inquiry and Word Count (LIWC) lexicon (Tausczik and Pennebaker, 2010) to categorize words in each complaint according to linguistic style and affective and cognitive attributes.
We identified 62 attributes out of the 64 available in LIWC’s Portuguese dictionary.
We normalized the frequency of each attribute by the total number of words in the complaint.
To verify whether the differences between attributes were statistically significant, we applied the non-parametric Kruskal-Wallis (Kruskal and
Wallis, 1952) test with α = 0 .
05 , using the SciPy (Virtanen, 2020) library.
3. Emotion Analysis
We investigated emotions expressed in complaints applying the Emotions Classifier (EC) model proposed in (Hammes and de Freitas, 2021).
Roughly speaking, EC model is a fine-tuning version of the
BERTimbau model (Souza et al., 2020) based on the GoEmotions dataset (Demszky et al., 2020) translated into Portuguese.
For the entire text in each complaint, the EC model calculates an emotion prediction score for each of the 28 classes in GoEmotions, ranging from 0 to 1 .
To determine the frequency of a particular emotion in our data, we considered complaints with [Table] score ≥ 0 .
3 , following the methodology proposed in (Hammes and de Freitas, 2021)
Results
POS Tagging Analysis : Table 1 shows the Jaccard similarity and Spearman’s correlation values considering the 50 most frequent words used by males and by females.
The Jaccard similarity values are high, indicating a large intersection between the set of the chosen words.
However, Spearman’s correlations are low, revealing a weak correlation between the used words’ rankings.
Although males and females use a very similar set of words in their complaints, the frequency of their usage is different.
For instance, the verb “paid” ranked 18th in texts of female authorship, but is not listed among the top 50 verbs in texts of male authorship.
Psycholinguist Analysis : We found 55 attributes for which differences in frequency of use in complaints written by males and females are statistically significant.
Due to space constraints, Table 3 reports the top-5 attributes with the largest differences and examples of words related to those attributes.
Interestingly, all 5 attributes are more frequent in complaints made by females.
Emotion Analysis : Figure 1 shows the percentage of the 19 most expressed emotions in the complaints.
The high percentage of complaints classed as neutral can be ascribed to cases where no particular emotion was assigned by the model.
Frequency is high for complaints expressing remorse and other negative emotions, such as disappointment , disapproval and sadness , expected in the dispute resolution domain.
Consumers also expressed desire and curiosity .
Male complainants expressed more remorse and optimism than females, 79% and 10% in proportion respectively, while female complainants conveyed more fear , desire , and gratitude , with 38%, 25%, and 18% respectively more than male.
Discussion
We found 56.7% complaints by males and 43.3% by females approximately.
A POS analysis re-
[Table]
vealed no significant correlation between the rankings of the top-50 terms among verbs, nouns, adjectives and proper nouns.
However, we found differences in the frequency of words used, which is aligned with findings in (Ishikawa, 2015) about female participants tending to use adjectives and verbs, whereas men use nouns.
We found statistically significant differences for the averages ofout of 64 LIWC attributes.
Emotion classification revealed that texts by men had a higher number of neutral labels, whereas those by women had more labels for desire and sadness .
This finding is in line with the literature, which suggests there are gender differences in emotion expression. Attribute Examples
[Table]
Cognitive
[Table]