Inclusivity (Online Discussions/Discussion Quality)
Authors: Heinbach Dominique 
Published: November 2022

Inclusivity is a key dimension to assess the deliberative quality of online discussions. In quantitative content analyses, this dimension measures the openness and accessibility of and the equality and diversity within a discussion.
Field of application/Theoretical foundation
Most studies on online discussions draw on deliberative norms to measure the quality of their discourse (e.g., Esau et al., 2017; Friess et al., 2021; Rowe, 2015; Ziegele et al., 2020; Zimmermann, 2017). Deliberation is an important concept for the study of (political) online discussions (Ziegele et al., 2020). It focuses on a free and equal exchange of arguments to bridge social differences and legitimize political decisions (Dryzek et al., 2019; Fishkin, 1991, Habermas, 2015). Inclusivity or open participation is one of the central criteria of Habermas’ discourse ethics. Deliberative discussions should be open to everyone and all participants should be able to express their attitudes, desires, and needs (Habermas, 2015; Steenbergen et al., 2003). Inclusivity occurs on two levels: On the one hand, it is a matter of open and free access for all citizens, which precedes the actual discussion process (input, Friess & Eilders, 2015). This precondition is often referred to as universalism or openness (Engelke, 2019; Kersting, 2008). In the discussion process itself (troughput, Friess & Eilders, 2015), all voices should have an equal opportunity to be heard and responded to, regardless of factors such as gender, race, or social background. Inclusivity usually implies opinion diversity, since one-sided discussions carry the risk of marginalizing other positions (Habermas, 2006; Manin, 1987; Zimmermann, 2017).
 
References/Combination with other methods
Besides quantitative content analyses, the (deliberative) quality of online discussions is examined with qualitative content analyses and discourse analyses (e.g., Graham & Witschge, 2003; Price & Capella, 2002). Furthermore, participants’ perceptions of the quality of online discussions are investigated with qualitative interviews (e.g., Engelke, 2019; Ziegele, 2016) or a combina­tion of qualitative interviews and content analy­sis (Díaz Noci et al., 2012).
 
Cross-references
Inclusivity is one of five dimensions of deliberative quality in this database written by the same author. Accordingly, there are overlaps with the entries on rationality, interactivity, explicit civility, and storytelling regarding the theoretical background, references/combinations with other methods, and some example studies.
 
Information on Stromer-Galley (2007)
Author: Jennifer Stromer-Galley
Research question: The aim of the paper was developing a coding scheme for academics and practitioners of deliberation to systematically measure what happens during group deliberations (p. 1; p. 7).
Object of analysis: The author conducted a secondary analysis of online group discussions (23 groups with 5-12 participants) in an experiment called “The Virtual Agora Project” at Carnegie Mellon Unversitiy in Pittsburgh, Pennsylvania. Participants attended the discussions from dormitory rooms that were equipped with a computer, headphones, and microphone. The group discussions were recorded and transcribed for analysis (pp. 7-8). Although strictly speaking the study does not analyze media content, the coding scheme has provided the basis for numerous other studies on the deliberative quality of online discussions (e.g., Rowe, 2015; Stroud et al., 2015; Ziegele et al., 2020).
Time frame of analysis: Three weeks in July 2004 (p. 7).
Info about variables
Level of analysis: Equality was measured on the level of the group discussion as well as on the level of the thought. Coders segmented each speaking contribution into thought units as first stage of the coding process. “A thought is defined as an utterance (from a single sentence to multiple sentences) that expresses an idea on a topic. A change in topic signaled a change in thought. A second indicator of a change in thought was a change in the type of talk. The distinct types of talk that this coding captured were the following: talk about the problem of public schools, talk about the process of the talk, talk about the process of the deliberation, and social talk” (p. 9).
Variables and values: For measuring the variable equality, the number of speakers within a group was counted. Furthermore, the thoughts were counted for the number of words per thought.  Additionally, the total number of thoughts spoken in a given group was counted (p. 15).
Reliability: “Two coders spent nearly two months developing and training with the coding scheme. The intercoder agreement measures […] were established from coding 3 of the 23 groups, which were randomly selected. […] The coders of the unitizing process achieved a statistically significant correlation of .86 (p < .001)” (p. 14).
Codebook: in the appendix (pp. 22-33)
Information on Zimmermann (2017)
Author: Tobias Zimmermann
Research question: Which role do online reader comments play for a deliberative-democratic understanding of a digital public sphere? (p. 11)
Object of analysis: To compare discursive participation online and offline, the author conducted a full-sample content analysis of online reader comments (N = 1.176) and letters to the editor (N = 381) from German local newspapers on three similar conflicts in local politics concerning the renaming of streets and squares. Because the coding scheme was based on the discourse quality index (DQI), only contributions that contained a demand were included in the analysis, that is, “a proposal on what decision should or should not be made” Steenbergen et al., 2003, p. 27). Only then, a speech act is considered relevant from a discourse ethics perspective.
Time frame of analysis: June 2012 to May 2013
Info about variables
Level of analysis: see Table 1
Variables: Following Stromer-Galley (2007) and Bächtiger et al. (2010), the author operationalizes participation (egalitarian openness) based on frequency and volume of the comments. Furthermore, the study assigns the comments to a pro or contra side in regard to their content. This allows conclusions regarding the equality of different positions (pp. 161-163). Additionally, based on the DQI (Steenbergen et al., 2003), he included the variable common good reference, because reasoning oriented to common interests represents the most inclusive form of reasoning (pp. 190-191).
Reliability: Intracoder reliability was tested on a subset of 100 comments. The variable “common good reference” reached a Krippendorff’s Alpha of .71 (p. 201).
Codebook: pp. 159-185 (in German)
Table 1: Variables, values and level of analysis (Zimmermann, 2017, p. 163; p. 191)

Indicator

Category

Definition

Level of analysis

Egalitarian openness

Egalitarian openness (a)

Length of a comment (or letter to the editor)

Individual contribution

 

Egalitarian openness (b)

Number of contributions per participant

Discussion

 

Egalitarian openness (c)

Number of contributions per thematic position

Discussion

Common good reference

No common good reference

No reference to the common good is explicitly made

Individual contribution

 

Explicit common good reference

The contribution includes at least one explicit reference to the common good (utilitarian or disadvantaged-oriented)

Individual contribution

 
Example studies
Ruiz, C., Domingo, D., Micó, J. L., Díaz-Noci, J., Meso, C. & Masip, P. (2011). Public Sphere 2.0? The Democratic Qualities of Citizen Debates in Online Newspapers. The International Journal of Press/Politics, 16, 463–487.
Stromer Galley, J. (2007). Measuring Deliberation's Content: A Coding Scheme. Journal of Public Deliberation, 3(1), Article 12.
Ziegele, M., Quiring, O., Esau, K. & Friess, D. (2020). Linking News Value Theory With Online Deliberation: How News Factors and Illustration Factors in News Articles Affect the Deliberative Quality of User Discussions in SNS’ Comment Sections. Communication Research, 47(6), 860-890. https://doi.org/10.1177/0093650218797884
Zimmermann, T. (2017). Digitale Diskussionen: Über politische Partizipation mittels Online-Leserkommentaren [Digital discussions: On political participation trough online reader comments]. Edition Politik: Bd. 44. transcript Verlag. http://www.content-select.com/index.php?id=bib_view&ean=9783839438886
Further references
Bächtiger, A., Shikano, S., Pedrini, S. & Ryser, M. (2010). Measuring Deliberation 2.0: Standards, Discourse Types, and Sequentialization. University of Konstanz and University of Bern. https://ash.harvard.edu/files/ash/files/baechtiger_0.pdf
Díaz Noci, J., Domingo, D., Masip, P., Micó, J. L. & Ruiz, C. (2012). Comments in news, democracy booster or journalistic night­mare: Assessing the quality and dynamics of citizen debates in Catalan online new­spapers. #ISOJ, 2(1), 46–64. https://isoj.org/ wp-content/uploads/2016/10/ISOJ_Jour­nal_V2_N1_2012_Spring.pdf#page=46
Dryzek, J. S., Bächtiger, A., Chambers, S., Cohen, J., Druckman, J. N., Felicetti, A., Fishkin, J. S., Farrell, D. M., Fung, A., Gutmann, A., Landemore, H., Mansbridge, J., Marien, S., Neblo, M. A., Niemeyer, S., Setälä, M., Slothuus, R., Suiter, J., Thompson, D. & Warren, M. E. (2019). The crisis of democracy and the science of deliberation. Science (New York, N.Y.), 363(6432), 1144–1146. https://doi.org/10.1126/science.aaw2694
Engelke, K. M. (2019). Enriching the Conversation: Audience Perspectives on the Deliberative Nature and Potential of User Comments for News Media. Digital Journalism, 8(4), 1–20. https://doi.org/10.1080/21670811.2019.1680567
Esau, K., Friess, D. & Eilders, C. (2017). Design Matters! An Empirical Analysis of Online Deliberation on Different News Platforms. Policy & Internet, 9(3), 321–342. https://doi.org/10.1002/poi3.154
Fishkin, J. S. (1991). Democracy and deliberation: New directions for democratic reform. Yale University Press. http://www.jstor.org/stable/10.2307/j.ctt1dt006v https://doi.org/10.2307/j.ctt1dt006v
Friess, D. & Eilders, C. (2015). A systematic review of online deliberation research. Policy & Internet, 7(3), 319–339. https://doi.org/10.1002/poi3.95
Friess, D., Ziegele, M. & Heinbach, D. (2021). Collective Civic Moderation for Deliberation? Exploring the Links between Citizens’ Organized Engagement in Comment Sections and the Deliberative Quality of Online Discussions. Political Communication, 38(5), 624–646. https://doi.org/10.1080/10584609.2020.1830322
Graham, T. & Witschge, T. (2003). In Search of Online Deliberation: Towards a New Method for Examining the Quality of Online Discussions. Communications, 28(2). https://doi.org/10.1515/comm.2003.012
Habermas, J. (2006). Political communication in media society: Does democracy still enjoy an epistemic dimension? The impact of normative theory on empirical research. Communication Theory, 16(4), 411–426.
Habermas, J. (2015). Between facts and norms: Contributions to a discourse theory of law and democracy (Reprinted.). Polity Press.
Kersting, N. (2008). Innovative Partizipation: Legitimation, Machtkontrolle und Transformation. Eine Einführung [Innovative participation. Legitimation, control of power, and transformation. An introduction]. In N. Kersting (Hrsg.), Politische Beteiligung: Einführung in dialogorientierte Instrumente politischer und gesellschaftlicher Partizipation (S. 11–39). VS Verlag für Sozialwissenschaften.
Manin, B. (1987). On Legitimacy and Political Deliberation. Political Theory, 15(3), 338–368. https://doi.org/10.1177/0090591787015003005
Price, V. & Cappella, J. N. (2002). Online deliberation and its influence: The Electronic Dialogue Project in Campaign 2000. IT&Society, 1(1), 303–329. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.9.5945&rep=rep1&type=pdf
Rowe, I. (2015). Deliberation 2.0: Comparing the Deliberative Quality of Online News User Comments Across Platforms. Journal of Broadcasting & Electronic Media, 59(4), 539–555. https://doi.org/10.1080/08838151.2015.1093482
Steenbergen, M. R., Bächtiger, A., Spörndli, M. & Steiner, J. (2003). Measuring Political Deliberation: A Discourse Quality Index. Comparative European Politics, 1(1), 21–48. https://doi.org/10.1057/palgrave.cep.6110002
Stroud, N. J., Scacco, J. M., Muddiman, A., & Curry, A. L. (2015). Changing Deliberative Norms on News Organizations' Facebook Sites. Journal of Computer-Mediated Communication, 20(2), 188–203. https://doi.org/10.1111/jcc4.12104
Ziegele, M. (2016). Nutzerkommentare als Anschlusskommunikation: Theorie und qualitative Analyse des Diskussionswerts von Online-Nachrichten [The Discussion Value of Online News. An Analysis of User Comments on News Platforms]. Springer VS.