# Emotion Detection in Non-native English Speakers' Text-Only Messages by Native and Non-native Speakers

Ari Hautasaari, Naomi Yamashita

► To cite this version:

Ari Hautasaari, Naomi Yamashita. Emotion Detection in Non-native English Speakers' Text-Only Messages by Native and Non-native Speakers. 15th Human-Computer Interaction (INTERACT), Sep 2015, Bamberg, Germany. pp.183-200, 10.1007/978-3-319-22701-6_13. hal-01599638

HAL Id: hal-01599638

https://inria.hal.science/hal-01599638

Submitted on 2 Oct 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Emotion Detection in Non-Native English Speakers' Text-Only Messages by Native and Non-Native Speakers

Ari Hautasaari¹ and Naomi Yamashita¹

¹NTT Communication Science Laboratories, Kyoto, Japan
ari.hautasaari@lab.ntt.co.jp
naomiy@acm.org

**Abstract.** When people from different language backgrounds communicate, they need to adopt a common shared language, such as English, to set up the conversation. In conversations conducted over text-only computer-mediated communication (CMC) mediums, mutual exchange of socio-emotional information is limited to the use of words, symbols and emoticons. Previous research suggests that when message receivers share the same native language with the authors, they are more accurate at detecting the emotional valence of messages based on these cues compared to non-native speaking receivers. But is this still true when the messages are written by non-native speakers? Moreover, what message properties influence the accuracy of emotional valence detection? In this paper, we report on an experiment where native English speakers and Japanese non-native English speakers rate the emotional valence of text-only messages written by Japanese non-native English speaking authors. We analyze how three message properties, grammatical correctness, fluency of language and use of symbols and emoticons, influence emotional valence detection for native and non-native speakers. Based on our results, we propose theoretical and practical implications for supporting multilingual socio-emotional communication in text-only CMC.

**Keywords:** Computer-mediated communication · text-only message · non-native speaker · emotion detection

## 1 Introduction

In face-to-face meetings, or during conversations on the phone, people are in general able to judge their conversational partners' emotional states based only on their facial expressions or the tone of their voice [1, 2]. However, geographical distribution, social dynamics and diverse language backgrounds of interlocutors often lead them to rely on text-only computer-mediated communication (CMC) mediums, such as email or instant messaging, to set up the conversation [3, 4, 5, 6]. These lean communication mediums lack the vocal and non-verbal cues available in richer mediums, such as video conferencing [7], making it more difficult for message receivers to determine the author's emotional states and tone [8, 9, 10, 11].
Exchange of socio-emotional information in current text-only CMC mediums is limited to the use of words, symbols and emoticons as emotional cues [8], [11], which inhibits the mutual exchange of emotional and interpersonal information between interlocutors [12]. Moreover, text-only CMC is often characterized as fragmented, agrammatical and incoherent [13], which may further impede detection of emotional information in the messages. In an effort to overcome these challenges and accurately convey their intended emotional tone, message authors often employ symbols and emoticons as emotional cues to enhance the verbal emotional content [14, 15, 16]. However, misunderstandings regarding the emotional tone of a message can sometimes occur even between conversational partners who share the same native language [17]. This problem is more salient when the message receivers are non-native speakers of the author's native language [18]. That is, non-native speaking receivers are less accurate than native speaking receivers at detecting the emotional valence of native speaking authors' messages. But, is this still true when the message author is a non-native speaker?

One pivotal aspect in multilingual communication over text-only CMC mediums is that non-native authors are generally less competent than native authors in using their second language. They rarely reach a native-level of grammatical correctness or the fluency of language when authoring messages [19]. They also often lack in experience using their non-native language outside a classroom [20, 21], which may in turn limit their competence in using symbols and emoticons to enhance the verbal content of their second language messages [22, 23]. Yet, previous research offers little indication how these limitations affect the exchange of socio-emotional information between non-native authors, and native and non-native receivers in text-only CMC.

In this paper, we will firstly explore how different message properties may affect socio-emotional communication in text-only CMC between native and non-native speakers. We focus on grammatical correctness, fluency of language, and the use of symbols and emoticons in relation to the verbal content of text-only messages authored by non-native speakers of English. Secondly, we aim to answer whether non-native speakers are still less accurate than native English speakers at detecting the emotional valence of messages when they share similar first and second language background with the authors. Answering these questions provides both theoretical and practical implications for supporting multilingual socio-emotional communication over text-only CMC mediums.

# 2 Related Work

## 2.1 Emotion Detection in Text-Only CMC

According to Social Presence theory, problems in detecting emotions in text-only CMC stem from the lack of non-verbal and vocal cues, which are available in richer mediums such as in video conferencing [7, 9]. Lack of these cues inhibits the mutual exchange of socio-emotional and interpersonal information between conversational partners [9]. However, according to Social Information Processing theory, emotional
information is still available in text-only CMC environments, but it takes longer for
interlocutors to detect this information [10].

The Social Information Processing view of emotion detection has been largely supported by recent literature on human-computer interaction and interpersonal communication in text-only CMC. Previous research suggests that interlocutors are able to distinguish between message author's positive and negative emotions in a text chat based only on the limited emotional cues [8], [14]. The emotional information for detecting some complex emotions, such as joy and anger, is also available in longer texts, such as online blogs [17], [24].

Exchange of emotional information in text-only CMC is in general limited to the use of words, symbols and emoticons used as emotional cues [14, 15, 16]. Besides explicit statements referring to the author's emotional state, such as "I am happy", the verbal and structural characteristics of a message may give receivers clues about the author's intended emotional tone. For example, authors expressing negative emotions use fewer words, more affective words and words conveying negative feelings and negations [8], [14]. Higher level of socio-emotional transmissions is also associated with conversational partners' tendency to use more emoticons and symbolic marks in their messages [25]. Emoticons indicate pauses in the emotional expression, such as laughter, that would occur in spoken dialogue in richer mediums or face-to-face settings [26]. Including emoticons in text-only messages strengthens the intensity of the verbal message, especially when the message is intended to carry a negative emotional tone [16], [27].

However, previous works focus largely on emotion and emotional valence detection based on the available cues by interlocutors who share the same native language, usually English. Indeed, misunderstandings about the emotional tone of text-only messages occur even between conversational partners who share the same native language [17], [25]. Yet, multinational organizations and intercultural working groups deal with communication problems caused by language diversity on a daily basis (e.g., [28]). In these multilingual environments, misunderstandings between native and non-native speaking conversational participants regarding the emotional tone of text-only messages, such as in email exchange, may be more salient.

## 2.2 Exchange of Socio-Emotional Information between Native and Non-Native Speakers

To date, scarce research has focused on multilingual socio-emotional communication, particularly in text-only CMC, where conversational partners from different language backgrounds have to adopt a common shared language to set up a conversation, usually English. In these communicative situations, message receivers who do not share the same native language with the message author may experience much greater difficulties in detecting the author's intended emotional tone. Besides empirical and anecdotal evidence suggesting that non-native English speakers have only limited exposure to their second language outside a classroom [20, 21], communicative norms and display rules may be very different for native and non-native speakers [29, 30]. Ab-
sence of shared norms and display rules regarding emotional expression may cause
receivers to misperceive the author's emotional tone in text-only CMC [31].

Besides different norms and rules regarding emotional expression, native and non-native speakers may experience the emotionality of words and phrases differently. In general, people experience the greatest emotional weight of emotionally arousing phrases in their native language [32]. Further, the social context of second language acquisition may mediate the emotionality of a language, where emotionality decreases the later the language is acquired [33, 34]. More detailed inquiries on the perceptions that interlocutors have regarding emotion and emotion-laden words in their native and non-native languages were reported by Russell [35], Romney et al. [36] and Moore et al. [37]. While they argued for the support of a shared model of semantic structure of emotion terms, they also highlighted discrepancies in how native and non-native speakers perceive the same emotion words in their shared language. In addition to emotion words and phrases, conversational participants make use of symbols and emoticons as emotional cues in text-only CMC [14, 15, 16]. However, previous research has found discrepancies in how interlocutors from different language and cultural backgrounds perceive these emotional cues [38]. Further, the adoption rate of different types of emoticons can vary greatly between languages and language groups, such as between Asia and North America [22, 23], [38], and non-native speakers may not be familiar with the emoticons commonly used in the shared language adopted with a native speaking conversational partner [23]. Altogether, these previous studies suggest that non-native conversational participants perceive the emotional nuances in words, symbols and emoticons differently from native speakers in text-only CMC, which may inhibit the mutual exchange of socio-emotional information.

Results reported in Hautasaari et al. [18] corroborate the findings in previous literature. Their results suggested that while non-native speakers are as accurate as native speakers at detecting the relevant emotional cues in messages written by native speaking authors, they are unable to reach similar accuracy when detecting the intended emotional valence of a message based on these cues. However, previous research gives little indication on whether this holds true when the message authors share similar first and second language background with the non-native speaking receivers. These discrepancies between native and non-native speaking conversational partners may, firstly, affect how accurately emotional valence is detected by receivers, and secondly, influence what emotional cues native and non-native authors choose to adopt to convey their emotional tone. Moreover, previous research has not addressed how different message properties, such as grammatical correctness of messages, affect the accuracy of emotional valence detection in text-only CMC between native and non-native speakers.

## 2.3 Grammatical Correctness and Fluency of Language in Text-Only CMC

In the previous section, we discussed how native and non-native speakers differ in their perception on the emotional nuances carried in words [32], [35, 36, 37], symbols and emoticons [22], [33], [38]. However, text-only CMC is also characterized as fragmented, agrammatical and often incoherent due to limitations imposed by the
available mediums [13], which may inhibit the mutual exchange of socio-emotional information between interlocutors. This may be particularly problematic in text-only conversations between native and non-native speakers.

While previous research suggests that non-native English speakers consider achieving grammatical correctness in vocal and text-only CMC conversations a high priority [39, 40], non-native speakers rarely reach a native-level of grammatical correctness or fluency of language when authoring messages [19]. Further, even if native and near-native (i.e., non-native speakers with very high language proficiency) speakers of a language appear equivalent in terms of language proficiency, they can still have markedly divergent perceptions on grammatical aspects of their shared language [41]. Non-native speakers may also end up with very different set of grammatical rules than native speakers, partly due to the second language education system [42].

Besides grammatical correctness, text-only messages are characterized by the fluency of language. Fluency refers to the clarity, consistency and naturalness of utterances or messages in relation to meaning and context. For non-native speakers', fluency in their second language generally increases as they progress as second language learners, and if they are able to use the non-native language in communicative contexts [44]. However, non-native speakers often lack this experience [20, 21], which may also further limit their competence in using symbols and emoticons to enhance the verbal content of their messages in text-only CMC [22, 23].

But, how do these properties of text-only messages influence the exchange of socio-emotional information between native and non-native conversational participants? That is, is emotional information more readily available in grammatically correct messages? How does the fluency of language used (i.e., the choice of words to express emotional tone) influence the detection of this information? Furthermore, how does the use of symbols and emoticons to enhance the verbal emotional content affect the transfer of emotional information? And lastly, considering the effects of these properties, how accurately native and non-native speaking receivers are able to detect the emotional valence of non-native speaking authors' text-only messages?

## 3 Current Study

In the current study, we explore how grammatical correctness, fluency of language, and use of symbols and emoticons affect emotional valence detection in text-only CMC between native and Japanese non-native speakers of English. Secondly, we compare native and non-native speakers' accuracy of detecting the emotional valence of non-native English speaking authors' messages. As a dataset for the analysis, we extracted 99 English public Facebook status updates from Japanese non-native English speakers. We use quantitative and qualitative data analysis to examine the following research questions.
**3.1 Research Questions**

Grammatical correctness of text-only messages may improve the comprehensibility of the verbal content. However, previous works imply that native and non-native speakers may possess different sets of grammatical rules for their shared language [41]. Furthermore, message authors who are non-native speakers rarely reach a level of grammatical correctness comparable to native speaking authors [19]. But, previous works have yet to answer how grammatical correctness of messages influences the exchange of socio-emotional information between non-native authors, and native and non-native receivers. Thus, we ask the following research question:

*RQ1: Does the increase in grammatical correctness of non-native speakers' text-only messages increase the accuracy of emotional valence detection for native and non-native receivers?*

Fluency refers to the clarity, consistency and naturalness of language used in a message. As non-native speakers advance as second language learners they are in general more fluent at expressing themselves [43]. Further, non-native speakers report being most fluent in their second language in text-only CMC mediums [6]. But, how does the fluency of language affect the exchange of socio-emotional information between non-native authors, and native and non-native receivers? We ask the following research question:

*RQ2: Does the increase in fluency of language in non-native speakers' text-only messages increase the accuracy of emotional valence detection for native and non-native receivers?*

Symbols and emoticons enhance the emotionality of the verbal content of a message [16], [27]. However, depending on their first language background, non-native speaking conversational participants may have different perceptions on the emotional nuances carried in these cues [22, 23], [38], and they may lack the experience in using them in text-only CMC [20, 21, 22, 23]. How appropriately the symbols and emoticons are linked to the verbal socio-emotional information may affect emotional valence detection accuracy for native and non-native receivers [18]. Thus, we ask the following research question:

*RQ3: How does the appropriateness of symbols and emoticons used to enhance the verbal content of non-native speakers' text-only messages affect the accuracy of emotional valence detection for native and non-native receivers?*

Native English speakers are able to detect the emotional valence of text-only messages written by other native English speakers more accurately than non-native speakers [18]. However, non-native receivers who share the same first and second language background with non-native authors may be more familiar with the emotional cues used by the authors [22, 23], [32], [35, 36, 37, 38], and the norms and display rules to express emotions in their shared non-native language [29, 30]. Thus, we ask the following research question:
RQ4: Are non-native speakers more or less accurate than native speakers at detecting the emotional valence in text-only messages written by non-native authors with similar first and second language background?

# 4 Method

## 4.1 Overview

To answer our research questions, we tested whether language background (native vs. non-native), and level (low vs. medium vs. high) of grammatical correctness, fluency of language, and appropriateness of symbols and emoticons in relation to verbal content of a message influenced participants’ emotional valence detection in messages authored by Japanese non-native English speakers. In this experiment, native and Japanese non-native English speaking participants rated the emotional valence in a set of text-only messages (public Facebook status updates) written by Japanese non-native English speakers. Each message in the dataset was categorized for the level of grammatical correctness, fluency of language, and appropriateness of symbols and emoticons by native English speaking annotators.

## 4.2 Authors

The non-native message authors (N=5) were all Japanese native speakers and spoke English as a non-native language. Their average TOEIC<fnref n="1" /> score (M=888.75, SD=107.81) and self-evaluated English language proficiency (M=4.80, SD=1.10: 1 = not fluent at all, 7 = very fluent) indicated that they speak English as a second language, but not fluently. Two of the non-native authors had resided in an English-speaking country, but lived in Japan at the time of the study. The rest of the authors had resided outside Japan for at least a year, where they primarily communicate in English. All authors had posted at least 20 public Facebook status updates in English.

The authors were contacted via email including an explanation of the study and a request to participate as message contributors. They were then asked to rate the emotional valence of their own English language messages on a 7-point Likert scale (1 = very negative, 4 = neutral, 7 = very positive). By doing this, we got a gold standard (i.e., author rating) for comparing the accuracy of emotional valence detection between native and non-native speaking participants.

## 4.3 Dataset

We extracted 99 messages (per author: M=19.8, SD=2.17) from Facebook with each author’s consent, which formed the initial message pool for our experiment materials. All messages were public status updates written by Japanese non-native English-speaking authors. None of the messages included any other additional content besides

<footnote n="1">¹ Test of English for International Communication</footnote>
English text (e.g., photos, hyperlinks). We fully anonymized all messages by exclud-
ing any names and affiliations.

**Message Categorization.** We hired three native English speakers to categorize the initial message pool for analysis. The three annotators were presented with the set of messages (N=99) from Japanese non-native English speaking authors in a randomized order. The annotators were then asked to read and rate each message on three 7-point scales (1 – Not accurate/fluent/appropriate at all, 7 – Very accurate/fluent/appropriate):

* Grammatical correctness (Accuracy): How correct the grammar and vocabulary in the message is?

* Fluency of language (Fluency): How clear, concise and natural the language used in the message is?

* Appropriateness of symbols and emoticons: How appropriately symbols (e.g., exclamation marks) and emoticons are used in the message in relation to the verbal content?

We averaged the ratings from the three annotators to categorize each message on these three scales. We then divided the messages to ordinal categories (low vs. medium vs. high) based on the average rating scores for grammatical correctness, fluency of language, and appropriateness of symbols and emoticons used in the message, and randomized the order to form the final dataset presented to the participants.

## 4.4 Participants

We hired 20 native English speakers and 20 non-native English speakers for this study. All native speaking participants had received their primary education (from the age of 6 to 18, elementary school to high school) in English speaking countries, and reported English as their only native language. All native speakers had lived in Japan for over two years at the time of the study (M = 9.98, SD = 6.65).

The non-native speaking participants in this study were all Japanese native speakers. They all had spent less than two years in English speaking countries. We required a minimum score of 750 in the TOEIC¹ English proficiency test for all non-native participants (M = 854.75, SD = 66.69). Their TOEIC¹ score and self-evaluated English language proficiency (M = 3.92, SD = 1.07; 1 = not fluent at all, 7 = very fluent) indicated that they speak English as their second language at similar level of fluency as the non-native English speaking message authors.

## 4.5 Procedures

We presented the native and non-native speaking participants a set of text-only messages on a laptop computer, and asked them to read and rate the emotional valence of each message on a 7-point Likert scale (1 = very negative, 4 = neutral, 7 = very positive). The participants were not informed of the authors’ language background. Depending on the participant’s speed, the experiment took 2 to 2.5 hours including 15
minutes reserved for instructions for both native and non-native speaking participants. The task instructions for non-native speaking participants were given in Japanese. In order to reduce fatigue during the experiment, each participant was asked to take a 10 to 15-minute break once they had rated half of the messages.

## 4.6 Measures

*Participants' emotional valence detection in the message:* native and non-native speaking participants rated their evaluation of the emotional valence of each message on a 7-point Likert scale (1 = very negative, 4 = neutral, 7 = very positive). The rating score reflects the participants’ emotional valence detection in the message. The inter-rater reliability was lower among native (Krippendorff’s α = .58) than non-native speakers (Krippendorff’s α = .66), which indicated that native speakers had a more diverse perception than non-native speakers regarding the emotional valence in the non-native authors’ messages.

# 5 Results

In the next sections, we explore how increase in grammatical correctness, fluency of language, and appropriateness of symbols and emoticons in relation to verbal content correspond to emotional valence detection accuracy for native and non-native speakers. We calculated the participants’ emotional valence detection accuracy as a correlation (Spearman’s correlation coefficient) with author ratings. We then compared the average correlations of native and non-native speakers.

## 5.1 Grammatical Correctness of Non-Native Speaking Authors' Messages

To answer our *RQ1*, we conducted a 3 (grammatical correctness: low vs. medium vs. high) × (language background: native vs. non-native) mixed ANOVA analysis on native and non-native speakers emotional valence detection accuracy in non-native speaking authors' messages (Fig. 1). The average grammatical correctness was 4.06 (SD=0.87) on a 7-point scale (1 – Not accurate (grammatically correct) at all, 7 – Very accurate (grammatically correct)).

There was a significant main effect of grammatical correctness on emotional valence detection accuracy (F[2, 37] = 44..22, p < .001), but no significant main effect for language background (F[1, 38] = 0.01, p = n.s.). The interaction effect between grammatical correctness and language background was also not significant (F[2, 37] = 0.68, p = n.s.). Bonferroni corrected post hoc tests showed that emotional valence detection accuracy differed significantly between all three levels of grammatical correctness (p < .016). These indicated that increase in grammatical correctness in non-native speaking authors' messages corresponded to increase in emotional valence detection accuracy for both native and non-native speaking raters (*RQ1*).
<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Accuracy of emotional valence detection for native and non-native speakers in three levels of grammatical correctness of non-native authors' messages (N=99). Error bars represent standard error of the mean.</figcaption></figure>

## 5.2 Fluency of Language in Non-Native Speaking Authors' Messages

To answer our *RQ2*, we conducted a 3 (fluency: low vs. medium vs. high) × (language background: native vs. non-native) mixed ANOVA analysis on native and non-native speakers emotional valence detection accuracy in non-native speaking authors' messages (Fig. 2). The average fluency was 4.49 (SD=0.68) on a 7-point scale (1 – Not fluent at all, 7 – Very fluent).

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Accuracy of emotional valence detection for native and non-native speakers in three levels of fluency in non-native authors' messages (N=99). Error bars represent standard error of the mean.</figcaption></figure>

There was a significant main effect of fluency on the emotional valence detection accuracy (F[2, 37] = 58.75, p < .001), but no significant main effect for language background (F[1, 38] = 0.27, p = *n.s.*). The interaction effect between fluency and language background was also not significant (F[2, 37] = 0.16, p = *n.s.*). Bonferroni corrected post hoc tests showed that emotional valence detection accuracy differed between all three levels of language fluency (p < .016), but decreased between medium-high levels. These indicated that increase in language fluency in non-native au-
thors' messages did not correspond to increase in emotional valence detection accuracy for native or non-native speaking raters (RQ2).

## 5.3 Appropriateness of Symbols and Emoticons in Non-Native Speaking Authors' Messages

To answer our *RQ3*, we conducted a 3 (appropriateness of symbols and emoticons: low vs. medium vs. high) × (language background: native vs. non-native) mixed ANOVA analysis on native and non-native speakers emotional valence detection accuracy in non-native speaking authors' messages (Fig. 3). The average appropriateness of symbols and emoticons was 4.61 (SD=0.70) on a 7-point scale (1 – Not appropriate at all, 7 – Very appropriate).

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Accuracy of emotional valence detection for native and non-native speakers in three levels of appropriateness of symbols and emoticons in non-native authors' messages (N=99). Error bars represent standard error of the mean.</figcaption></figure>

There was a significant main effect of appropriateness of symbols and emoticons on the emotional valence detection accuracy (F[2, 37] = 58.75, p < .001), but no significant main effect for language background (F[1, 38] = 0.91, p = *n.s.*). However, the interaction effect between appropriateness of symbols and emoticons and language background was significant (F[2, 37] = 3.49, p < .05). Bonferroni corrected post hoc tests showed that emotional valence detection accuracy differed between all three levels of appropriateness of symbols and emoticons (p < .016). Furthermore, planned pairwise comparisons showed that while native and non-native speakers' emotional valence detection accuracy did not differ significantly at low (F[1, 38] = 0.39, p = *n.s.*) and medium (F[1, 38] = 1.18, p = *n.s.*) levels of appropriateness of symbols and emoticons, native speakers (M=0.73, SD=0.06) were significantly more accurate than non-native speakers (M=0.66, SD=0.07) at high level of appropriateness (F[1, 38] = 10.70, p < .01).

These indicated that increase in appropriateness of symbols and emoticons used in relation to the verbal content in non-native authors' messages corresponded to the increase in emotional valence detection accuracy for both native and non-native
speaking raters. However, native speakers' accuracy surpassed that of non-native speakers when the appropriateness of symbols and emoticons increased (RQ3). These results also answered our RQ4. There was no significant difference between native and non-native speakers in their emotional valence detection accuracy in non-native authors' messages.

# 6 Discussion

The goal of this study was to increase our understanding on the factors that influence the success of socio-emotional communication between native and non-native speakers in text-only CMC, such as via email or instant messaging. Taken together, our results demonstrate how grammatical correctness, fluency of language, and appropriateness of symbols and emoticons in relation to verbal emotional content affect emotional valence detection accuracy in non-native English speaking authors' messages.

Grammatical correctness of messages in text-only CMC, such as in the context of our dataset, is generally low [13]. While non-native authors tend to strive for writing grammatically correct messages [39], they rarely reach the level of native speaking authors [19]. Our results suggest that lower grammatical correctness in non-native authors' messages may have detrimental effects on the exchange of socio-emotional information between the author and receivers, regardless whether a receiver is a native or non-native speaker of the shared language (Example 1). That is, both native and non-native speakers were less accurate at detecting the emotional valence of non-native authors' messages as the grammatical correctness deteriorated (Fig. 1).

<table><caption>**Example 1.** Non-native author's message with low level of grammatical correctness.</caption><thead><tr><th>Non-native author's message</th><th>Evaluation of the emotional valence</th></tr></thead><tbody><tr><td>Even after i speak perfect japanese i was mistaken as a foreigner lololol wtf going on?? Lol at tea place :)</td><td>Author: Positive<br>Native raters: Neutral<br>Non-native raters: Neutral</td></tr></tbody></table>

The second message property we examined was the fluency of language used in the text-only messages, which refers to the clarity, consistency and naturalness of the language. While the fluency of language in non-native authors' messages is often lower than that of native authors [19], our results implied that increase in the level of fluency does not correspond to higher accuracy of emotional valence detection (Fig. 2). One possible explanation for this is that messages with higher fluency of language may lack some syntactic or orthographic emotional cues, such as abbreviations and use of capital letters as emphasis, which could carry information about the emotional tone (Example 2). However, fluency in second language generally increases as the non-native speakers become more experienced in communicating with it [43], but our results do not address how their messages might change over time. This question should be answered in future work.
<table><caption>**Example 2.** Non-native author's message with low level of fluency.</caption><thead><tr><th>Non-native author's message</th><th>Evaluation of the emotional valence</th></tr></thead><tbody><tr><td>BUUUUUUUSSSSSSSSSYYYYYYYYY but fiiiiiiinally weeeeeekend!!!</td><td>Author: Positive<br>Native raters: Positive<br>Non-native raters: Positive</td></tr></tbody></table>

Our third message property of interest considered how appropriately symbols and emoticons are used in relation to the verbal content of the message. Our findings indicated that increase in the appropriateness of symbols and emoticons corresponded to increase in emotional valence detection accuracy for both native and non-native speakers. For one, appropriate use refers to enhancing text-only messages with emoticons in the same valence as the verbal emotional content of the message (Example 3).

<table><caption>**Example 3.** Non-native author's message with high appropriateness of symbols and emoticons.</caption><thead><tr><th>Non-native author's message</th><th>Evaluation of the emotional valence</th></tr></thead><tbody><tr><td>WHAT! "Your timeline foes live on April 28"?<br>No, no, I'd rather use the current/previous version of<br>Facebook!! I feel like I'm forced to use it. :</td><td>Author: Slightly Negative<br>Native raters: Negative<br>Non-native raters: Slightly Negative</td></tr></tbody></table>

Furthermore, employing the use of emoticons in the same valence as the verbal emotional content in the message may increase receivers' emotional valence detection accuracy even in lower levels of grammatical correctness (Example 4).

<table><caption>**Example 4.** Non-native author's message with high appropriateness of symbols and emoticons and medium level of grammatical correctness.</caption><thead><tr><th>Non-native author's message</th><th>Evaluation of the emotional valence</th></tr></thead><tbody><tr><td>I never say it doesn't suit me, I make it suitable for me :)) That's my spirit :P</td><td>Author: Very Positive<br>Native raters: Positive<br>Non-native raters: Positive</td></tr></tbody></table>

However, inappropriate use of symbols and emoticons in a message can make it increasingly difficult for receivers to detect the emotional valence, particularly in messages with low grammatical correctness (Example 5). That is, if the valence of the emoticons used as emotional cues does not appropriately reflect the verbal emotional content.
<table><caption>**Example 5.** Non-native author's message with low appropriateness of symbols and emoticons and low grammatical correctness.</caption><thead><tr><th>Non-native author's message</th><th>Evaluation of the emotional valence</th></tr></thead><tbody><tr><td>Insects r coming after me. Animals and kids r starting at me and i play with them :) And weirdos r ... Well above all i have been horrified by harmful insects which r coming after me and also mosquitos lol any solution even I was safe but soooo scary to have experience some big bees r back of ur neck inside of ur hair lol Still haven't being bite except for mosquitos type though... Kowaiiii lol</td><td><strong>Author:</strong> Very Positive<br><strong>Native raters:</strong><br>Slightly Negative<br><strong>Non-native raters:</strong><br>Slightly Negative</td></tr></tbody></table>

Furthermore, even if text-only messages have high grammatical correctness, omitting emoticons as emotional cues may lead the message receivers to misperceive the non-native author's intended emotional valence (Example 6).

<table><caption>**Example 6.** Non-native author's message with low appropriateness of symbols and emoticons and high grammatical correctness.</caption><thead><tr><th>Non-native author's message</th><th>Evaluation of the emotional valence</th></tr></thead><tbody><tr><td>Tomorrow, I will take a flight to Japan. Hot, Hot... melting Tokyo & Kyoto.</td><td><strong>Author:</strong> Slightly Positive<br><strong>Native raters:</strong><br>Slightly Negative<br><strong>Non-native raters:</strong><br>Slightly Negative</td></tr></tbody></table>

Our results also revealed that when non-native authors employed symbols and emoticons as emotional cues more appropriately, the native speaking raters surpassed non-native raters in their emotional valence detection accuracy (Fig. 3). One possible explanation for this is that in these messages, the use of Western symbols and emoticons by the Japanese non-native authors was closer to how native English speaking authors would use them as emotional cues. Previous works suggest that non-native receivers may have trouble identifying the connection between the verbal emotional content and symbols and emoticons in native English speaking authors messages [15]. Hence, it is possible that non-native receivers have trouble detecting the emotional information in messages from non-native authors who have more experience communicating in English [20, 21], and are more accustomed to using Western symbols and emoticons [22, 23].

Finally, how does a shared first and second language background affect the exchange of socio-emotional information in text-only CMC? Hautasaari et al. [18] found implications that native English speaking receivers are more accurate than non-native receivers at detecting the emotional valence of text-only messages written by
native English speaking authors. Our results suggest that when the non-native receivers share the same first and second language background with the authors, they are as accurate as native receivers at detecting the emotional valence. In part, this discrepancy may be explained by shared understanding on nuances in emotional cues [22, 23], [32], [35, 36, 37, 38], and the norms and display rules to express emotions in the authors' and receivers' shared non-native language [29, 30]. However, it is important to note that combined these two results do not answer whether non-native speakers write more emotionally arousing messages, or whether it is in general easier for receivers to detect the socio-emotional information in non-native authors' messages. Answering these questions requires future research.

**6.1 Design Implications**

A large number of text-only CMC mediums are already fitted with automatic spell checkers, and our results suggest that increase in grammatical correctness of non-native authors' messages increases the accuracy of emotional valence detection. However, there is a need for a balanced approach that would potentially help non-native authors to produce messages with similar levels of grammatically correctness as native authors [19], while not forcing or suggesting them to omit any syntactic or orthographic emotional cues. For example, a system that would detect grammatical errors and inform the non-native authors about the potentially low emotional expressivity of their message might improve the quality of socio-emotional communication with native/non-native receivers in text-only CMC.

In previous work by Hautasaari et al. [18], the authors suggested a system for translating the emotional nuances in symbols and emoticons in native authors' messages for non-native receivers. Our results indicate that the connection between symbols and emoticons to the verbal content of a message is crucial in socio-emotional communication between non-native authors and native/non-native receivers. However, non-native authors may not be familiar with the nuances carried in these emotional cues in their second language [22, 23], [38]. For example, Asian non-native speakers of English may often be unaware of the emotional nuances carried in Western emoticons, and fail to connect them appropriately in relation to the verbal emotional content of their message. A system that would (1) detect the presence/absence of emotion and emotion-laden words in a text-only message, and (2) suggest including/removing symbols and emoticons in the same/opposite valence could help non-native speakers to author more emotionally coherent messages.

**6.2 Limitations and Future Directions**

Mining a larger set of English messages written by a large number of non-native speakers from different first language backgrounds (e.g., Twitter) could provide a more generalizable sample of non-native speakers' socio-emotional messages. In future studies, we are interested in exploring how our results may apply to various conversational contexts, such as multilingual group communication via email, or when the author and receiver are familiar with each other.
# 7 Conclusion

We examined how grammatical correctness, fluency of language, and appropriateness of symbols and emoticons influence the exchange of socio-emotional information in text-only CMC between non-native English speaking authors, and native and non-native English speaking receivers. Our results suggested that increase in grammatical correctness of text-only messages corresponds to increase in accuracy of emotional valence detection in the message for both native and non-native receivers. However, increase in fluency of language used in the message did not correspond to increase in the accuracy of emotional valence detection. A likely explanation is that messages with higher fluency lack syntactic or orthographic emotional cues, such as abbreviations and use of capital letters as emphasis. Increase in the appropriateness of how symbols and emoticons are used in relation to the verbal content of a message corresponded to increase in emotional valence detection accuracy. However, the accuracy of native speaking receivers' emotional valence detection surpasses that of non-native receivers as the non-native authors' use of symbols and emoticons as emotional cues borders that of native speaking authors.

Lastly, we found that while non-native English speakers are less accurate at detecting the emotional valence in native authors' messages, they are as accurate as native speakers when message authors are other non-native speakers with similar first and second language background. These findings have implications for our theoretical understanding of socio-emotional communication between native and non-native speakers in text-only CMC, and practical implications for communication tools to support multilingual socio-emotional communication.

**Acknowledgements.** We would like to extend our gratitude to the message authors who agreed to let us use their public Facebook status updates as part of our dataset. We would also like to thank the anonymous reviewers for their insightful comments and suggestions for improving this paper.

# References

1. Buchanan, T.W., Lutz, K., Mirzazade, S., Specht, K., Shah, N.J., Zilles, K., Jäncke, L.: Recognition of Emotional Prosody and Verbal Components of Spoken Language: an fMRI Study. Cognitive Brain Research 9, 3, 227–238 (2000)

2. Ekman, P.: Emotion in the Human Face (2nd ed.). New York: Cambridge University Press (1982)

3. Harzing, A-W., Köster, K., Magner, U.: Babel in Business: The Language Barrier and Its Solutions in The HQ-Subsidiary Relationship. Journal of World Business 46, 3, 279-287 (2011)

4. Mok, D., Carrasco, J.A., Wellman, B.: Does Distance Still Matter in the Age of the Internet?. Urban Studies 46, 13, 2747-2783 (2009)

5. Olson, G.M., Olson, J.S.: Distance Matters. Human-Computer Interaction 15, 2, 139-178 (2000)
6. Setlock, L.D., Fussell, S.R.: What's it Worth to You? The Costs and Affordances of CMC Tools to Asian and American Users. In: Proceedings of 2010 ACM Conference on Computer Supported Cooperative Work, pp. 341-349, ACM (2010)

7. Dennis, A.R., Kinney, S.T.: Testing Media Richness Theory In The New Media: The Effects Of Cues, Feedback, And Task Equivocality. Information Systems Research 9, 3, 256-274 (1998)

8. Hancock, J.T., Gee, K., Ciaccio, K., Lin, J.M-H.: I'm Sad You're Sad: Emotional Contagion In CMC. In: Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work, pp. 295-298, ACM, New York, NY (2008)

9. Short, J., Williams, E., Christie, B.: The Social Psychology of Telecommunications. New York: Wiley (1976)

10. Walther, J.B.: Interpersonal Effects In Computer-Mediated Interaction: A Relational Perspective. Communication Research 19, 52-90 (1992)

11. Walther, J.B., Loh, T., Granka, L.: Let Me Count The Ways: The Interchange of Verbal and Nonverbal Cues In Computer-Mediated and Face-To-Face Affinity. Journal of Language and Social Psychology 24, 1, 36-65 (2005)

12. Murphy, K. L., Collins, M. P.: Development Of Communication Conventions In Instructional Electronic Chats. Journal of Distance Education 12, 1-2, 177-200 (1997)

13. Herring, S.: Interactional Coherence in CMC. Journal of Computer-Mediated Communication 4, 4 (1999)

14. Hancock, J.T., Landrigan, C., Silver, C.: Expressing Emotion In Text-Only Communication. In: Proceedings of 2007 ACM Conference on Human Factors in Computing Systems, pp. 929-923, ACM (2007)

15. Aman, S., Szpakowicz, S.: Identifying Expressions Of Emotion In Text. In: Proceedings of the 10th International Conference on Text, Speech and Dialogue (TSD-2007, LNAI 4626), pp. 196-205 (2007)

16. Walther, J.B., D'Addario, K.P.: The Impacts Of Emoticons On Message Interpretation In Computer-Mediated Communication. Social Science Computer Review 19, 324-347 (2001)

17. Gill, A.J., Gergle, D., French, R.M., Oberlander, J.: Emotion Rating From Short Blog Texts. In: Proceedings of 2008 ACM Conference on Human Factors in Computing Systems, pp. 1121-1124, ACM (2008)

18. Hautasaari, A., Yamashita, N., Gao, G.: "Maybe It Was a Joke": Emotion Detection in Text-Only Communication by Non-Native English Speakers. In: Proceedings of 2014 ACM Conference on Human Factors in Computing Systems, pp. 3715-3724, ACM (2014)

19. Medgyes, P.: Native or Non-Native: Who's Worth More?. ELT Journal 46, 4, 340-349 (1992)

20. Benson, M.J.: Attitudes And Motivation Towards English: A Survey Of Japanese Freshmen. RELC Journal 22, 1, 34-48 (1991)

21. Hyland, F.: Learning Autonomously: Contextualising Out-Of-Class English Language Learning. Language Awareness 13, 3, 180-202 (2004)

22. Nishimura, Y.: Linguistic Innovations and Interactional Features of Casual Online Communication in Japanese. Journal of Computer-Mediated Communication 9, 1 (2006)

23. Park, J., Barash, V., Fink, C., Cha, M.: Emoticon Style: Interpreting Differences In Emoticons Across Cultures. In: Proceedings of the 7th International AAAI Conference on Weblogs and Social Media (ICWSM'13), pp. 466-475, AAAI Press (2013)

24. Gill, A.J., French, R.M., Gergle, D., Oberlander, J.: The Language Of Emotion In Short Blog Texts. In: Proceedings of 2008 ACM Conference on Computer Supported Cooperative Work, pp. 299-302, ACM (2008)
25. Kato, S., Kato, Y., and Akahori, K.: Study On Emotional Transmissions In Communica-
tion Using A Bulletin Board System. In: Proceedings of World Conference on E-Learning
in Corporate, Government, Healthcare, and Higher Education (ELEARN) 2006, pp. 2576-
2584, Chesapeake, VA: AACE (2006)

26. Provine, R.R., Spencer R.J., Mandell, D.L.: Emotional Expression Online: Emoticons Punctuate Website Text Messages. Journal of Language and Social Psychology 26, 3, 299-307 (2007)

27. Derks, D., Bos, A.E.R., von Grumbkow, J.: Emoticons And Online Message Interpretation. Social Science Computer Review 26, 3, 379-388 (2008)

28. Harzing, A-W., Köster, K., Magner, U.: Babel In Business: The Language Barrier And Its Solutions In The HQ-Subsidiary Relationship. Journal of World Business 46, 3, 279-287 (2011)

29. Meierkord, C.: Interpreting Successful Lingua Franca Interaction: An Analysis Of Non-Native-/Non-Native Small Talk Conversations In English. Linguistik Online 5, 1 (2000)

30. Seidlhofer, B.: Closing A Conceptual Gap: The Case For A Description Of English As A Lingua Franca. International Journal of Applied Linguistics 11, 2, 133-158 (2001)

31. Byron, K.: Carrying Too Heavy A Load? The Communication And Miscommunication Of Emotion By Email. Academy of Management Review 33, 2, 309-327 (2008)

32. Dewaele, J-M.: The Emotional Weight Of I Love You In Multilinguals' Languages. Journal of Pragmatics 40, 10, 1753-1780 (2008)

33. Harris, C.L., Gleason, J.B., Aycicegi, A.: When Is A First Language More Emotional? Psychophysiological Evidence From Bilingual Speakers. In A. Pavlenko (Ed.), Bilingual Minds: Emotional Experience, Expression, And Representation. Clevedon, United Kingdom: Multilingual Matters, 257-283 (2006)

34. Schumann, J. The Neurobiology of Affect in Language. Malden, MA: Blackwell (1997)

35. Russell, J.A.: Pancultural Aspects Of The Human Conceptual Organization Of Emotions. Journal of Personality and Social Psychology 45, 6, 1281-1288 (1983)

36. Romney, A.K., Moore, C.C., Rusch, C.D. Cultural Universals: Measuring The Semantic Structure Of Emotion Terms In English And Japanese. In: Proceedings of National Academy of Sciences 94, 10, pp. 5489-5494 (1997)

37. Moore, C.C., A. Romney, K., Hsia, T-L., Rusch, G.D.: The Universality Of The Semantic Structure Of Emotion Terms: Methods For The Study Of Inter- And Intra-Cultural Variability. American Anthropologist 101, 3, 529-546 (1999)

38. Kayan, S., Fussell, S.R., Setlock, L.D.: Cultural Differences In The Use Of Instant Messaging In Asia And North America. In: Proceedings of the 2006 ACM Conference on Computer Supported Cooperative Work, pp. 525-528, ACM (2006)

39. Lee, C.K.M.: Text-Making Practices Beyond the Classroom Context: Private Instant Messaging in Hong Kong. Computers and Composition 24, 285-301 (2007)

40. Zakura, N.: Focus on the Form Versus Focus on the Message in Lingua Franca Conversations. Eesti Rakenduslilingvistika Uhingu Astaraamat 8, 275-287 (2012)

41. Coppieters, R.: Competence Differences between Native and Near-Native Speakers. Language 63, 3 (1987)

42. Dabrowska, E.: Different Speakers, Different Grammars: Individual Differences in Native Language Attainment. Linguist. Approach. Biling. 2 (2012)

43. Housen, A., Kuiken, F.: Complexity, Accuracy and Fluency in Second Language Acquisition. Applied Linguistics 30, 4, 461-473 (2009)