<figure><img src="image_1.png" /></figure>

HAL
open science

# Helsinki Digital Humanities Hackathon 2024 Portfolio

Natchanun Sanitdee

► To cite this version:

Natchanun Sanitdee. Helsinki Digital Humanities Hackathon 2024 Portfolio. University of Helsinki. 2024. hal-05090001

HAL Id: hal-05090001

https://hal.science/hal-05090001v1

Submitted on 29 May 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
# In search of civility online

## What does civil discourse look like on the Internet and where can it be found?

We investigated these questions by examining 1) the Reddit r/ChangeMyView community and 2) instances where someone saw the need to intervene to turn an uncivil discussion into a civil one.

**CMV** user • 5y ago

Yes, or as someone else has said, stop arguing emotionally in this discussion. You can feel whatever, but this is a place for civil discourse (and I should know, since I have had posts removed because of this).

<figure><img src="image_2.png" /></figure>

## Defining civility

**Respect of different views and others** - orientation toward understanding, not agreement.

**Openness to discussion** - seeking mutually acceptable outcomes.

**Adherence to contexts** - willingness to follow socially approved rules.

**Staying on topic** - civil discourse has “an eye toward...relevance”.

(Hauser, 1999; Kingwell, 1995; Massaro & Stryker, 2012; Papacharissi, 2004)

► **The rules of the Reddit r/ChangeMyView community match these principles, making it a good basis for finding civil discourse online.**

## CMV as a community

* Centered around challenging each other's views.

* Rules enforce respectful and impactful discussion.

* Users give deltas (Δ) to comments that changed their mind.

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /><figcaption>Top 20 communities of comments and replies in CMV</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Popular Topics</figcaption></figure>

## Close-reading civility on Reddit

<table><caption>Annotation Scheme</caption><thead><tr><th>Initial Identification</th><th>Category</th><th>Sub-category</th><th>Sub-subcategory</th></tr></thead><tbody><tr><td rowspan="6">Civility</td><td rowspan="3">Politeness</td><td>Face-saving</td><td><strong>Mitigation (MD)</strong><br>Correct me if I'm wrong...<br><br><strong>Indirect disagreement (ID)</strong><br>What about ...?<br><br><strong>Partial disagreement (PD)</strong><br>You're right but...</td></tr><tr><td>Degree Formality (FM)</td><td></td></tr><tr><td>Acknowledgement (AK)<br>That's a great point.</td><td></td></tr><tr><td rowspan="3">Public-mindedness</td><td>Clarification (CT)<br>Do you mean that ...?</td><td></td></tr><tr><td>Moral Civility (MC)<br>respecting fundamental rights</td><td></td></tr><tr><td>Justificatory Civility (JC)<br>refer to law/objective rules,<br>balance of values</td><td></td></tr><tr><td>Incivility</td><td>Impoliteness</td><td>Impolite Word (IM)</td><td></td></tr><tr><td>Neutral</td><td>Direct (total) disagreement (DD)<br>No, this is not the case...</td><td>Insults (IS)</td><td></td></tr></tbody></table>

<figure><img src="image_6.png" /><figcaption>Inter-annotator agreement by civility features</figcaption></figure>

* Disagreement strategies are challenging to distinguish.
* Public-mindedness appears in longer comments, which are rare online. When detected, it is hard to rate how much is present.
* Agreement on insults suffers due to difficulties judging sarcasm.
* For downstream use, annotations collapsed into a binary civility indicator based on feature combinations (also alleviating uncertainty caused by annotation disagreements).

Validate with annotations

66%
Precision
and recall

<figure><img src="image_7.png" /><figcaption>A random forest classifier was trained using a small subsample of the civil set and features and validated with the annotated data.</figcaption></figure>

## Theory meets data

<figure><img src="image_8.png" /></figure>

Apart from indirect disagreement and formality, many of the civility features turned out to be rare in online environments.

## Extracting an exemplary civil discourse subset from CMV

Criteria:

* Comment needs to be in a delta-awarding thread (persuasive discussion likely to be civil).

* There needs to be at least one back-and-forth exchange before the delta is awarded (requirement of discussion).

* No parts of the discussion can have more downvotes than upvotes (filter out potential incivility).

* Comment needs to be less than 10 deep in a thread (through exploration, it was determined that long threads deteriorate).

► **Even this quite strict filtering still leaves 164,592 comments as potential training data if the set turns out to be good!**

## Quality of the data-driven civil set

<figure><img src="image_9.png" /></figure>

93% of comments in the data-driven civil discourse set human-evaluated as civil

► **Should be good for training models, if structural/linguistic features differentiate the subset from other data.**

## Features differ between the civil set and other data

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

<figure><img src="image_12.png" /></figure>

## A classifier can separate the sets

<figure><img src="image_13.png" /></figure>

* Comments (esp. delta-awarded) on CMV as compared to random comments tend to be longer, linguistically more complex, and contain more quotes (17% of comments vs. 2%).

* Users who have been awarded many (>150) deltas tend to communicate in a structurally similar way on other subreddits.

* Threads with deltas are semantically more uniform.

## Interventions as civility

**Interventions** steer conversations to address perceived norm violations. We manually classified instances of *stop arguing*, a common intervention phrase (Tanskanen, 2021).

### Classification of stop arguing

**Genuine interventions**
* “Hey let's just *stop arguing* because we agree.”
* “I'll *stop arguing* this point, because I don't think I can change your view...”

**Not interventions**
* “A friend of mine posted on FB how white people need to *stop arguing* ...”
* “If you're willing to admit that, I'll *stop arguing*.”

**Ambiguous interventions**
* “Please *stop arguing* with me, you're unintelligent.”
* “Now *stop arguing* something you know nothing about and do the research yourself if you really care.”

* Interventions uphold r/CMV rules: they stop arguments, refocus discussions, and call out fallacious arguments.

* Civility features predict genuine interventions, but many interventions also contain incivility, thus remaining ambiguous.

## Where do interventions usually occur?

<figure><img src="image_14.png" /></figure>

1. Semantically divergent threads likely face intervention.

2. When discussion is off-topic or goes to extreme viewpoint.

Clearly justified interventions more successful.

3. When discussion is complex or unproductive.

Interventions more successful if supported by facts.

4. How often *stop arguing* is a genuine intervention differs by reddit.

<figure><img src="image_15.png" /></figure>

r/changemyview
53% (n=397)

<figure><img src="image_16.png" /></figure>

r/todayilearned
68% (n=50)

<figure><img src="image_17.png" /></figure>

r/unpopularopinion
52% (n=50)

<figure><img src="image_18.png" /></figure>

r/teenagers
44% (n=50)

## What happens after a genuine intervention?

Topic modeling reveals that post-intervention comments encompass **broader themes**, suggesting that interventions may deter off-topic and extreme viewpoints.

<figure><img src="image_19.png" /></figure>

<figure><img src="image_20.png" /></figure>

## References

* Bonotti, M., & Zech, S. T. (2021). *Recovering civility during COVID-19*. Palgrave Macmillan.

* Hauser, G. (1999). *Vernacular voices: The rhetoric of publics and public spheres*. University of South Carolina Press.

* Kingwell, M. (1995). *A civil tongue: Justice, dialogue and the politics of pluralism*. Pennsylvania State University Press.

* Massaro, T. M., & Stryker, R. (2012). Freedom of speech, liberal democracy, and emerging evidence on civility and effective democratic engagement. *Arizona Law Review*, 54(375).

* Papacharissi, Z. (2004). Democracy online: civility, politeness, and the democratic potential of online political discussion groups. *New Media & Society*, 6(2), 259-283.

* Tanskanen, S-K. (2021). "Stop arguing": Interventions as metapragmatic acts in discussion forum interaction. in M. Johansson, S-K. Tanskanen & J. Chovanec (eds.), *Analyzing Digital Discourses: Between Convergence and Controversy*. Palgrave Macmillan, 219-244.