# Twitter users' perception of AI in remote proctoring

Elen Le Foll, Lisa Tilz

► To cite this version:

Elen Le Foll, Lisa Tilz. Twitter users' perception of AI in remote proctoring. Artificial Intelligence in Public Discourse, Institute of Cognitive Science, Osnabrück University, pp.59-77, 2022, 10.48693/151.hal-04070478

HAL Id: hal-04070478

https://hal.science/hal-04070478

Submitted on 2 Nov 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
Artificial
Intelligence in
Public Discourse

Studies by Michael Alexandrovsky, Hanna Algedri, Micaela Barkmann, Tim Bax, Antonia Becker, Kyra Breidbach, Rabea Breininger, Christian Burmester, Eddie Charmichael, Alina Deuschle, Dana Dix, Mekselina Doganc, Kai Dönnebrink, Luisa Drescher, Kristof Engelhardt, Milan Ewert, Franziska Gellert, Kristin Gnadt, Nikolai Godt, Thiago Goldschmidt, Michelle Görlitz, Isabel Grauwelman, Yannick Hardt, Deborah Häuser, Jara Herwig, Malte Heyen, Till Holzapfel, Fabian Imkenberg, Virginia Jagusch, Ali Jandaghi, Joline Janz, Maximilian Kalcher, Karina Khokhlova, Paula Kirmis, Johanna Kopetsch, Ralf Krüger, Julia Laudon, Lina Lazik, Elen Le Foll, Sönke Lülf, Anna Ricarda Luther, Clara Matheis, Richard Matschke, Louisa Maubach, Sascha L. Mühlinghaus, Pia Münster, Felix Naujoks, Janeke Nemitz, Thimo Neugarth, Sarah Neuhoff, Till Nicke, Alina Ohnesorge, Lilith Okonnek, Cosima Oprotkowitz, Elisa Palme, Florian Pätzold, Tom Pieper, Daniel Pietschke, Ivan Polivanov, Muhammad Faraz Rajput, Janine Reichmann, Clara Schier, Rebekka Schlenker, Sabine Scholle, Archana Singh, Lennard Smyrka, Julia Stormborn, Konstantin Strömel, Johanna Tamm, Kim Targan, Hedye Tayebi, Franka Timm, Lisa Titz, Lea Tiyavorabun, Katharina Trant, Nikola Tsarigradski, Eva von Butler, Anita Wagner, Jasmin L. Walter, Nele Werner, Christoph Werries & Inga Wohlert, co-edited by Jacqueline Näther, Matthias Richter, Marcel Ruland & Anna Wiedenroth

Edited by

Tobias Thelen

PICS – Publications of the Institute of Cognitive Science

Universität Osnabrück
Editorial board:

Annette Hohenberger

Simone Pika

Gordon Pipa

Achim Stephan

Tobias Thelen
Artificial
Intelligence in
Public Discourse

Studies by Michael Alexandrovsky, Hanna Algedri, Micaela Barkmann, Tim Bax, Antonia Becker, Kyra Breidbach, Rabea Breininger, Christian Burmester, Eddie Charmichael, Alina Deuschle, Dana Dix, Mekselina Doganc, Kai Dönnebrink, Luisa Drescher, Kristof Engelhardt, Milan Ewert, Franziska Gellert, Kristin Gnadt, Nikolai Godt, Thiago Goldschmidt, Michelle Görlitz, Isabel Grauwelman, Yannick Hardt, Deborah Häuser, Jara Herwig, Malte Heyen, Till Holzapfel, Fabian Imkenberg, Virginia Jagusch, Ali Jandaghi, Joline Janz, Maximilian Kalcher, Karina Khokhlova, Paula Kirmis, Johanna Kopetsch, Ralf Krüger, Julia Laudon, Lina Lazik, Elen Le Foll, Sönke Lülf, Anna Ricarda Luther, Clara Matheis, Richard Matschke, Louisa Maubach, Sascha L. Mühlinghaus, Pia Münster, Felix Naujoks, Janeke Nemitz, Thimo Neugarth, Sarah Neuhoff, Till Nicke, Alina Ohnesorge, Lilith Okonnek, Cosima Oprotkowitz, Elisa Palme, Florian Pätzold, Tom Pieper, Daniel Pietschke, Ivan Polivanov, Muhammad Faraz Rajput, Janine Reichmann, Clara Schier, Rebekka Schlenker, Sabine Scholle, Archana Singh, Lennard Smyrka, Julia Stormborn, Konstantin Strömel, Johanna Tamm, Kim Targan, Hedye Tayebi, Franka Timm, Lisa Titz, Lea Tiyavorabun, Katharina Trant, Nikola Tsarigradski, Eva von Butler, Anita Wagner, Jasmin L. Walter, Nele Werner, Christoph Werries & Inga Wohlert, co-edited by Jacqueline Näther, Matthias Richter, Marcel Ruland & Anna Wiedenroth

Edited by

Tobias Thelen
Tobias Thelen (ed.). 2022. *Artificial Intelligence in Public Discourse* (PICS – Publications of the Institute of Cognitive Science 2022, Number 03). Osnabrück: Osnabrück University, Institute for Cognitive Science.

This title can be downloaded at:

https://osnadocs.ub.uni-osnabrueck.de/handle/urn:nbn:de:gbv:700-20211125625

© 2022, the authors

Published under the Creative Commons Attribution-ShareAlike 4.0 Licence (CC BY-SA 4.0): http://creativecommons.org/licenses/by-sa/4.0/

PICS Editorial Board: Annette Hohenberger, Simone Pika, Gordon Pipa, Achim Stephan, Tobias Thelen

Layout and workflows based on awesome templates and tools by Language Science Press, https://langscipress.org

Cover and concept of design: Ulrike Harbort

Fonts: Libertinus, Arimo, DejaVu Sans Mono

Typesetting software: Xe_LATEX

Published by:

Institute of Cognitive Science
Osnabrück University

49069 Osnabrück
Germany

https://ikw.uni-osnabrueck.de

Storage and cataloguing done by Osnabrück University, Universitätbibliothek.
# Contents

Vorwort v

Preface xi

1 Introduction
Tobias Thelen 3

2 The common ontology used in this book
Deborah Häuser & Archana Singh 13

I The general public

3 How do non-experts discuss AI?
Michael Alexandrovsky, Mekselina Doganc, Virginia Jagusch, Julia
Stormborn & Daniel Pietschke 27

4 What impact have experts on the public discourse about AI? - An
analysis of reactions to newspaper articles on "ZEIT-Online"
Sascha L. Mühlinghaus 39

5 Discourse about AI on Reddit
Richard Matschke & Till Nicke 49

6 Twitter users' perception of AI in remote proctoring
Elen Le Foll & Lisa Titz 59

7 How do Twitter users receive AI-related field research conducted by
police?
F. K. & F. P. 79
## II AI and subject experts

8 How do scientists explain AI to the broad public? — A TED talks analysis

Antonia Becker, Michelle Görlitz, Yannick Hardt & Clara Schier

95

9 How is AI perceived amongst experts of different disciplines?

Maximilian Kalcher

113

10 How does Elon Musk portray the dangers and the future of AI?

Alina Deuschle & Joline Janz

127

11 How is AI in healthcare perceived by physicians in 2020? — A qualitative analysis of recent opinions in YouTube videos

Janine Reichmann & Ali Jandaghi

137

## III The media

12 How AI transforms public discourse — An analysis of the impact of AI in public discourse as portrayed in major news outlets

Nikolai Godt, Ivan Polivanov, Karina Khokhlova & Muhammad Faraz

Rajput

155

13 Does AI in public discourse change with different political and socio-economic systems? An analysis of the AI debate in newspapers in the emergent AI Superpowers: USA, China and Germany

Sabine Scholle, Konstantin Strömel, Archana Singh, Louisa

Maubach, Johanna Kopetsch, Kyra Breidbach, Kristin Gnadt, Anna

Ricarda Luther, Lea Tiyavorabun & Hedye Tayebi

177

14 Ethical concerns and AI: Analysing British news articles about Alexa

Kim Targan, Clara Matheis & Kristof Engelhardt

209

15 How is AI explained to children? A qualitative analysis of educational videos for children

Franziska Gellert, Julia Laudon, Pia Münster & Rebekka Schlenker

237

16 How is AI portrayed in Netflix' documentary "The Social Dilemma" and how do newspapers react to it?

Micaela Barkmann, Dana Dix & Kai Dönnebrink

257
17 A modern god complex - Doctor Who? An analysis of (un-)specialized German news articles on AI in medical diagnostics
Isabel Grauwelman, Cosima Oprotkowitz & Katharina Trant 279

18 AI in healthcare – Expectation vs. reality of breast cancer detection
Tim Bax, Milan Ewert, Florian Pätzold & Franka Timm 309

19 Should robots take care of the elderly? – Comparing ethical guidelines to real life experiences
Rabea Breininger, Luisa Drescher, Lilith Okonnek & Inga Wohlert 321

20 Asian reactions to AI supremacy in the game of Go
Sarah Neuhoff, Ralf Krüger & Nikola Tsarigradski 339

IV Politics, governments and non-government organizations

21 AI made in Germany
Christian Burmester, Thiago Goldschmidt, Felix Naujoks & Tom Pieper 353

22 A comparison of different governmental approaches to prepare the public for the age of AI
Jara Herwig, Lina Lazik, Sönke Lülf & Elisa Palme 375

23 Ethical guidelines in the European judicial system
Hanna Algedri & Till Holzapfel 397

24 AI as part of the energy transition – A comparison of the portrayal of AI from the big energy group E.ON and the non-profit organisation Germanwatch
Eva von Butler, Janeke Nemitz & Nele Werner 409

25 How AI in the form of content filters for social media is discussed in the German parliament
Eddie Charmichael & M.S. 425

V Movies and literature

26 Portrayal of AI in popular movies
Fabian Imkenberg, Paula Kirmis, Johanna Tamm & Christoph Werries 443
Contents

27 AI Fatale – An analysis of AI characters focused on gender depiction and inflicted harm in movies from 2000–2020
Thimo Neugarth, Alina Ohnesorge, Lennard Smyrka & Jasmin L. Walter 461

28 AI in literature: An investigation of the science fiction novel QualityLand
Malte Heyen, F.R. & Anita Wagner 483
<figure><img src="image_2.png" /></figure>

# Twitter users' perception of AI in remote proctoring

Elen Le Foll & Lisa Titz

As a consequence of the COVID-19 pandemic, many educational institutions have turned to AI-based remote proctoring solutions to monitor students during online examinations. The deployment of such software has caused some concerns: reports of unfair treatment and accusations of racist and ableist systems have widely circulated in social media. Twitter has been at the forefront of these debates. The present study thus investigates how AI in online proctoring is portrayed by different stakeholders (e.g. companies, students, and representatives of educational institutions) in Twitter discourse. To this end, we compiled a small, highly specialised corpus of tweets (N = 467) which we manually tagged following a Qualitative Content Analysis (QCA) approach. The annotated data allows for the fine-grained analysis of the terminology used to describe AI in the context of remote proctoring in combination with the benefits, issues and emotions that Twitter users associate with such software. Our results suggest that general terms are preferred over more specialised ones. We also show that some terms are used inconsistently by the general public and sometimes even misleadingly by online proctoring companies themselves.

**Keywords:** Remote proctoring | E-proctoring | Online examinations | Anti-cheating software | Twitter | Suspicious behaviour | Facial recognition

## 1 Introduction

Not only has the COVID-19 pandemic resulted in a sudden switch to online and hybrid forms of teaching and learning, it has also forced schools, universities, and other educational institutions to find new ways of conducting examinations that respect social distancing guidelines. One of the major challenges that educational institutions have faced in organising remote examinations concerns
the academic integrity of online assessment forms (Milone et al. 2017). Thus, in
this new context of online distance learning, many educational institutions, and
in particular universities, have opted for remote, online exam proctoring tools
(Harwell 2020b). As a result, 2020 has seen an almost exponential increase in
the demand for online proctoring solutions and “anti-cheating” software (Mar-
kets 2021). These services are usually provided by for-profit companies (Kharbat
& Abu Daabes 2021). In spite of their high costs and widespread concerns over
privacy issues (Harwell 2020b), their attractiveness is undoubtedly due to their
ease of implementation as such systems enable the near replication of in-person
exam conditions. They provide solutions that eliminate the need for the physi-
cal presence of humans proctors. Traditionally, human proctors supervise large
groups of students answering previously undisclosed questions, most commonly
with pen and paper, in a limited amount of time and with access to no, or only
very limited, additional tools or resources.

Remote proctoring providers provide three types of remote proctoring solu-
tions, all three of which make it possible to implement examination formats that
are very similar or even identical to in-person assessments:

1. Live proctoring, which involves human proctors monitoring examinees using the video and audio streams from students' computers, thus simply removing the location constraint from traditional in-person examinations;

2. Fully automated proctoring, which involves AI-systems trained to identify fraud by recognising and classifying examinees' identity and activities as captured in real-time by the students' webcam and microphones; and

3. AI-supported live proctoring, which combines the two methods by hav-
ing algorithms flagging and recording suspicious behaviour, which human
proctors are to subsequently check to make an informed decision as to
whether a student has indeed cheated, attempted to cheat, or whether the
AI triggered a false alarm (c.f. LeewayHertz 2020, Singh 2021, Swauger
2020a,b).

In the second and third types of online, remote proctoring software, artificial
intelligence systems are deployed for voice and facial recognition (e.g. to check
that one and the same person is taking the test and that no one is assisting them),
pattern recognition (e.g. to detect (potentially) suspicious head, eye, and com-
puter mouse movements), speech-to-text (e.g. to monitor what is said) and object
recognition (e.g. to ensure that test-takers are only using authorised additional
tools and resources). Fully automated and AI-supported service providers and
their proponents (e.g. Scott 2019) argue that the use of AI in remote proctoring
improves the detection rate of dishonest behaviours by flagging suspicious pat-
terns that humans may not necessarily be able to identify. Further, it has been
hailed as a solution for making online examinations "credible and secure" (Scott
2019) since actions, such as excluding a dishonest student from an exam, can be
taken immediately.

However, as the use of online proctoring tools has boomed, so has criticism
of this new technology grown. Ever since the first wave of lockdowns in 2020,
a number of grievances by students who claim to have been falsely accused of
cheating by such software have gone viral on social media, spurring wider pub-
lic interest in the matter (Harwell 2020a). One prominent example was college
student and TikTok user @_daynuh_ who posted a tearful video on the popular
video-based social media platform TikTok in which she reported:

[...] my professor is giving me a zero, because the Review+<fnref n="1" /> said I was talking when I was just, like, re-reading the question so I could better understand it (as cited in Harris 2020).

Within days, the video had reached more than three million views with many
calling the AI “ableist” (Harris 2020) as it appears to unfairly disadvantage stu-
dents with learning disabilities such as dyslexia. There have also been reports of
racial discrimination with AI proctoring systems seemingly unable to detect the
facial movements of people of colour (Swauger 2020b, Feathers 2021). A tweet by
Alivardi Khan was cited in news sites from across the world and sparked outrage
beyond the student community.

The @ExamSoft software can't 'recognize' me due to 'poor lighting' even though I'm sitting in a well-lit room. Starting to think it has nothing to do with lighting. Pretty sure we all predicted their facial recognition software wouldn't work for people of color. @DiplomaPriv4All (as cited in Chin 2021).

In light of the growth and demand for (partially) automated, and therefore cheaper,
online proctoring AI-based solutions and as a result of the reactions to these
developments on social media, this study investigates the roles of AI in online
proctoring as perceived by Twitter users. Twitter was chosen because it is the
mouthpiece of a cross-section of different remote proctoring stakeholders. First,

<footnote n="1">¹One of the products offered by the remote proctoring firm ProctorU, which advertises it as "The Economical Solution To DETECT Proxy Testing, Cheating & Content Theft [...] for low-to-mid-stakes quizzes and exams" (ProctorU 2021).</footnote>
many online proctoring companies actively use this social media platform to ad-
vertise their products. Second, it is the home of a vibrant educational technology
(#edtech) community. Indeed, many of the recent controversies around remote
proctoring have either originated on Twitter, or quickly migrated to the platform
(as in the case of the discussion around @.daynuh._'s experience). And, third, it
is also popular with students, some of whom have rallied to publish their experi-
ences of online proctoring on specially dedicated accounts such as @Procteario
and @ProcterroU. We seek to explore the language used to describe artificial
intelligence systems in remote proctoring systems in the tweets of companies
providing such services, representatives of educational institutions, and students
who are proctored by or with such tools. Section 2 outlines our data collection,
data wrangling and manual annotation processes. Our quantitative and qualita-
tive results are presented in Section 3. We conclude with a discussion of our main
findings and an outlook for future developments in this fast-evolving field.

## 2 Data and methods

In the following, we describe our data extraction and processing processes, as
well as the cyclical development and evaluation of a custom tagset for the qualitative analysis of the harvested tweets.

### 2.1 Data extraction

As our research question focuses on the perceived roles, benefits and dangers of
AI in remote proctoring as communicated by Twitter users, the first step con-
sisted in automatically downloading potentially relevant tweets using sets of
keywords. We used the R (R Core Team 2020) rtweet package (Kearney 2019)
to formulate and send requests to Twitter's REST API. Given the fairly low num-
bers of tweets involved and the fact that all tweets were to be subsequently man-
ually sorted and annotated, the API queries were optimised for maximal recall
rather than precision. We thus opted for a large set of API queries consisting
of combinations of two keywords: one from a list of keywords related to online
proctoring (I.), and a second keyword from a list of words and phrases poten-
tially referring to some form of AI (II.). For each API query, two keywords were
joined by the Boolean operator AND to form one query: e.g. "ProctorU AND
algorithm"<fnref n="2" />. With our final selection of 12 proctoring keywords (I.) and 30 po-
tentially AI-related keywords (II.), a list of all 360 possible combinations of the

<footnote n="2">²All queries were case-insensitive</footnote>
two keyword lists was computed and subsequently used to automatically query
the Twitter API.

I. Proctoring keywords: proctoring, online exam, online exams, remote examination, remote examinations, ProctorU, Examus, Proctortrack, AIProctor, ExamMonitor, Proview, Honorlock

II. AI keywords: AI, artificial intelligence, algorithm, algorithms, scanning, scans, scan, scanned, detection, detecting, detected, detect, recognition, recognise, recognize, recognises, recognizes, recognising, recognizing, recognised, recognized, automated, automation, automate, automates, machine learning, flag, flags, flagged, flagging

Up to 5,000 tweets (including tweet replies) per query were extracted on a weekly basis over a period of a month from 4 January to 1 of February 2021. In practice, the upper limit of 5,000 was never reached. The weekly intervals were chosen because, at the time the data was mined, the Twitter REST API granted access to “recent tweets” from the past 6-9 days (Kearney 2019) for non-commercial research purposes. Duplicates which were scraped more than once were eliminated from the dataset. In addition, regular expressions were employed to eliminate near-duplicates (e.g. tweets which were identical in their wording except for slightly different distributions of white spaces, or links or hashtags placed in a different order), which are typically sent out by companies or bots. This process resulted in 375 tweets for manual sorting and tagging. Given the modest size of the dataset, we decided to also specifically target nine Twitter accounts known to frequently contribute to Twitter discourse on remote proctoring and AI, and which we identified by searching for relevant keywords on Twitter. Six of these target accounts belong to companies that market online proctoring services with AI (@AIProctor, @conductexam, @examity, @ProctorExam, @proctorio and @ProctorU), two are anonymous accounts of detractors of online proctoring that mostly retweet student experiences and complaints (@Procteario and @ProcterrorU), and the ninth target account (@Linkletter) is of a learning technology specialist from the University of British Columbia who has been at the forefront of Twitter discourses on the use of AI in remote proctoring ever since he revealed the inner workings of Proctorio to the wider public in late August 2020 (Chin 2020). Although all the tweets published between 1 September 2020 and 2 February 2021 by these accounts were originally retrieved, only those that included at least one of the keywords from the AI list (II) were subsequently retained. This resulted in 200 unique tweets. When the two datasets were merged and duplicates removed, a total of 467 tweets remained.
## 2.2 Qualitative Content Analysis

As the next step, we developed a tagset based on the ontology collaboratively established by the authors of this book and the specifics of our dataset. Following an iterative Qualitative Content Analysis (QCA) approach (Kuckartz & McWhertor 2014), we evaluated our tagset on a random sample of tweets before applying it to our full dataset.

### 2.2.1 Development of the tagset

To do so, we first examined a random subset of 100 tweets from our full dataset. Having identified their authors' roles, concerns, and the terminology frequently used to refer to AI in remote proctoring by Twitter users, we derived guiding questions to describe our tagset. On this basis, we subsequently developed tags and tag definitions in an iterative process. The resulting tagset is outlined in Table 1. It is subdivided in nine categories which are briefly outlined below.

The first category of tags "Tweet: Properties of the tweet" (Table 1, first section) was designed to filter out irrelevant or uninterpretable tweets. Additional tags of this category which are not included in the table are: Tweets assigned the tag T_unrelated, which were deemed to be unrelated to our topic, the tag T duplicate, which excludes near-duplicate tweets, e.g. those that only differ in their combinations of hashtags, and the tag T_difflang, which indicates that the contribution is not (fully) written in English. Finally, T_unclear was used when the content was not clearly about AI-based online proctoring services.

The second category in the tagset is concerned with the Twitter users involved in the discourse and aims to identify the distribution of contributions and opinions among students, educators, and companies.

The third category, definition, captures the aspects of the artificial intelligent systems that are discussed and the specific terms used to describe the technology. Both characterise and guide the understanding of AI within the discourse. We distinguish between the terms: "AI", "machine learning", "deep learning", "algorithm" and the words derived from the verb "automate". Tagged aspects of AI are behavioural patterns, such as hand or head movements, or turning around (D_behaviour) and, as special cases, eye movement detection (Deye) and face recognition (DFace). In addition, we tagged discourse concerned with the amount of data gathered (D_data) and room scans (D_scan).

The next category focuses on issues associated with AI-based remote proctoring. The tag Iagainst describes a general negative attitude towards online proctoring and is assigned whenever none of the other tags within this category
are appropriate, or whenever several issues are mentioned within one tweet, and
at least one argument is not covered by the more specific issue tags. An exam-
ple of a specific issue tag is: software failure (I_failed), i.e. tweets about systems
falsely detecting cheating or demanding a human review. For face recognition
problems of people of colour, the specific tag I_racism is used. Similarly, issues
revolving around the unfair treatment of people with disabilities are tagged as
I_disability. I_treatment is assigned to tweets that more generally refer to stu-
dents having to follow strict rules, e.g. focus their gaze on the screen, or have
their exams interrupted by the system to carry out room scans.

In contrast, the benefits category is concerned with discourse highlighting the
beneficial aspects of remote proctoring, whereby the use of the B_helpful tag
follows the rules of IAgainst and thus depicts a general positive attitude. B_fair-
ness is assigned to tweets that contend that online proctoring makes examination
situations fairer by detecting cheaters.

To investigate which emotions are related to remote proctoring, the tags E_positive and Eiris capture expressions that do not feature one of the more specific emotions listed in Table 1. Further, we also analyse language use with a specific tag for sarcasm (L_sarcasm), as well as tags to indicate that the current need for online proctoring solutions is construed as a consequence of the COVID-19 pandemic (C_covid and C_covidtrigger). The final category is concerned with the question of whether AI-based online proctoring should (continue to) be used in the future. Among those, the tag F_research is assigned to demands of increased investments and/or further research in the technology.

2.2.2 Evaluation and application of the tagset

First, both authors independently tagged 20 tweets using an earlier version of
the tagset. We discussed the outcome and agreed on the addition, exclusion and
refined definitions of certain tags. A second set of tweets was then tagged by
both authors to evaluate the inter-rater agreement on this refined tagset. In total,
just over 10% of tweets (52 out of 467) were tagged for this purpose. The 218
tags assigned to these 52 tweets by the two authors independently of each other
were subsequently compared. Inter-rater agreement was found to be very high
(94.2%). As a result, the second author tagged the remaining 415 tweets following
the tagset outlined in Table 1. In the context of this qualitative tagging process,
out of the various data fields retrieved from the Twitter API, only the tweeted
text including any emojis and hashtags, the user screen name and, whenever
possible, any linked web addresses were considered. Whenever necessary, we
assigned several tags to any one tweet.
# 3 Results

In total, 467 tweets were manually tagged using the tagset laid out in Section 2.2. Of those, 215 tweets were excluded because they were tagged as either T_unrelated (N = 158), T_duplate (N = 32), T_difflang (N = 16) or T_unclear (N = 9). Consequently, the final dataset analysed in the following consists of 252 manually annotated tweets.

<figure><img src="image_3.png" /><figcaption>Figure 1: Frequency of tweets analysed (from 1 Sep 2020 to 2 Feb 2021, aggregated using three-hour intervals)</figcaption></figure>

Figure 1 shows when these 252 tweets were published across the data collection period, which spanned from 1 September 2020 to 2 February 2021 (cf. Section 2.1). The outlier peak corresponds to the publication on 5 January 2021 of an article in The Verge on face detection issues with the online proctoring software ExamSoft, which was widely shared and discussed in the Twitter sphere (Chin 2021).

The distribution of the tags is depicted in Table 1 as counts and relative frequencies. The relative frequencies add up to more than 100 % because many tweets were assigned more than one tag. The distribution of identified user groups is listed in the first segment of the table. In total, 111 tweets were assigned to one of the three groups of users listed in Table 1, of which 65 tweets were identified as being published by a student, 28 tweets by a teacher or lecturer, and 18 tweets
by a company providing AI-based online proctoring solutions. Second, we analysed which aspects of AI are most often discussed. As listed in the third segment of Table 1 tallying the terms used to describe AI, the most often assigned tags are D_ai (N = 63), DFace (N = 42), D_behaviour (N = 29), and D_scan (N = 25). In terms of definitions of the AI systems themselves, proctoring technology is most frequently referred to as involving “AI” (N = 63), whereas “machine learning” (N = 10) is used significantly less often and “deep learning” is not mentioned at all in our corpus. More general terms, such as “algorithm” (N = 14) and “automated/automation” (N = 16), were applied more often than the more specialist terms “deep learning” and “machine learning”. Users seem not to associate the term “machine learning” with their descriptions of tracking behavioural responses (pattern recognition), room scans (object recognition), or face detection and recognition. The only exception to this are two tweets quoting software promotions by Proctorio – an online proctoring service provider widely discussed throughout the tweets captured in the present dataset:

> "Proctorio is the first and only proctoring solution that combines facial recognition technology and machine learning to eliminate any human error or bias" [...] (Tweet 447)

Only one tweet explicitly refers to biased algorithms by addressing the underlying mechanisms stating:

> [...] Aaaand yup it's a training-set problem ::angry face::<fnref n="3" /> [...] (Tweet 117)<fnref n="4" />

Instead, the verb “flagging” is used particularly frequently by students to describe the fact that the software detects certain behavioural patterns, facial features, and/or object, and identifies them as inappropriate within the examination context. The following tweet exemplifies the use of the verb “flag” and the concerns of many students:

> I have to use online proctoring software for my exam next week and I'm so scared it'll flag me for something stupid ::crying:: (Tweet 169)

Moreover, face recognition is not only discussed as a software feature, but also to define the technology itself. The following two tweets serve to summarise the extensive Twitter-based debate on the extent to which the term “face recognition” can be used to define online proctoring software that rely on AI.

<footnote n="3">³Words bracketed by double colons represent emojis that were used in the original tweet.</footnote>

<footnote n="4">⁴For data privacy reasons, tweets are only referred to by their position number in our dataset.</footnote>
A misconception about Proctorio is that we use facial recognition, but we never do. There is an important distinction between this & facial detection. Our choice to use facial detection means our software cannot be used to identify a test taker ever. https://bit.ly/3jVU0oX (Tweet 462)

The following tweet was published in response:

Proctorio has now deleted the tweet where they said they did NOT use facial recognition. Yesterday they deleted the tweets where they said they DO use facial recognition. Can you see why people are confused? [...] (Tweet 443)

Within the investigated discourse, the definitions of face recognition vs. face detection are fiercely debated. Hence, not only do users of online proctoring AI-based software not apply these terms according to consistent definitions, we also observe that one of the leading companies in the sector uses inconsistent terminology, seemingly in an attempt to defend its system.

<table><caption>Table 1: Tagset</caption><thead><tr><th>Topic</th><th>Tags</th><th>N</th><th>Rel.F.</th></tr></thead><tbody><tr><td colspan="4">Tweet: Properties of the tweet</td></tr><tr><td>Tweet covers topic, but cannot be matched to any other tag.</td><td>T_discourse</td><td>41</td><td>16.27%</td></tr><tr><td colspan="4">Users: Are there common opinions among student/ teachers/ companies?</td></tr><tr><td rowspan="3">To which group does the author belong?</td><td>UCompany</td><td>18</td><td>7.14%</td></tr><tr><td>UStudent</td><td>65</td><td>25.79%</td></tr><tr><td>UTeacher</td><td>28</td><td>11.11%</td></tr><tr><td colspan="4">Definition: How is AI defined?</td></tr><tr><td>"artificial intelligence" / "AI"</td><td>D_ai</td><td>63</td><td>25.00%</td></tr><tr><td>"algorithm" / "algorithmic"</td><td>D_algorithm</td><td>14</td><td>5.56%</td></tr><tr><td>"automated" / "automation" / "automatic"</td><td>D_automation</td><td>16</td><td>6.35%</td></tr><tr><td>System detects behaviour.</td><td>D_behaviour</td><td>29</td><td>11.51%</td></tr><tr><td>Amount of data is gathered</td><td>D_data</td><td>16</td><td>6.35%</td></tr><tr><td>"deep learning"</td><td>DDL</td><td>0</td><td>0</td></tr><tr><td>System detects eye movement.</td><td>D_eye</td><td>12</td><td>4.76%</td></tr><tr><td>System uses face recognition.</td><td>DFace</td><td>42</td><td>16.67%</td></tr><tr><td>"machine learning" / "ML"</td><td>D_ML</td><td>10</td><td>3.97%</td></tr><tr><td>System scans the room.</td><td>D_scan</td><td>25</td><td>9.92%</td></tr><tr><td colspan="4">Issues: What issues come with remote proctoring?</td></tr><tr><td>Generally against remote proctoring</td><td>I_against</td><td>34</td><td>13.49%</td></tr><tr><td>Generally biased system</td><td>I biased</td><td>16</td><td>6.35%</td></tr><tr><td>Privacy issues</td><td>I_contraprivacy</td><td>20</td><td>7.94%</td></tr><tr><td>System is inappropriate for people with disabilities</td><td>I_disability</td><td>12</td><td>4.76%</td></tr><tr><td>Face recognition problems</td><td>IFaceprob</td><td>21</td><td>8.33%</td></tr></tbody></table>
6 Twitter users' perception of AI in remote proctoring

<table><thead><tr><th>Topic</th><th>Tags</th><th>N</th><th>Rel.F.</th></tr></thead><tbody><tr><td>Software failed or is expected to fail</td><td>I_failed</td><td>44</td><td>17.46%</td></tr><tr><td>System makes more mistakes than humans.</td><td>I_moreerrors</td><td>0</td><td>0</td></tr><tr><td>Algorithm lacks transparency</td><td>I_opaque</td><td>13</td><td>5.16%</td></tr><tr><td>Racist software</td><td>I_racism</td><td>15</td><td>5.95%</td></tr><tr><td>How students are treated by system</td><td>I_treatment</td><td>23</td><td>9.13%</td></tr><tr><td colspan="4">Benefits: What are beneficial aspects of remote proctoring?</td></tr><tr><td>Allows access to education.</td><td>B_accessedu</td><td>5</td><td>1.98%</td></tr><tr><td>System supports fairness.</td><td>B_fairness</td><td>10</td><td>3.97%</td></tr><tr><td>System makes fewer mistakes than humans.</td><td>BFewererrors</td><td>1</td><td>0.40%</td></tr><tr><td>Generally described as a helpful tool</td><td>B_helpful</td><td>11</td><td>4.37%</td></tr><tr><td>System respects privacy.</td><td>B_proprivacy</td><td>10</td><td>3.97%</td></tr><tr><td>Support education</td><td>B_supported</td><td>7</td><td>2.78%</td></tr><tr><td>Algorithm is transparent.</td><td>B_transparent</td><td>1</td><td>0.40%</td></tr><tr><td>Generally unbiased</td><td>B_unbiased</td><td>2</td><td>0.79%</td></tr><tr><td colspan="4">Emotions: Which emotions are associated with remote proctoring?</td></tr><tr><td rowspan="4">Positive emotions</td><td>E_hope</td><td>1</td><td>0.40%</td></tr><tr><td>E_lesstanxiety</td><td>1</td><td>0.40%</td></tr><tr><td>E_positive</td><td>10</td><td>3.97%</td></tr><tr><td>E_discomfort</td><td>1</td><td>0.40%</td></tr><tr><td rowspan="3">Negative emotions</td><td>E_fear</td><td>8</td><td>3.17%</td></tr><tr><td>E_negative</td><td>52</td><td>20.63%</td></tr><tr><td>E_worry</td><td>22</td><td>8.73%</td></tr><tr><td colspan="4">Language: What kind of language is used?</td></tr><tr><td>Sarcasm</td><td>L_sarcasm</td><td>9</td><td>3.57%</td></tr><tr><td colspan="4">Covid: Is the discourse addressing the pandemic?</td></tr><tr><td>Pandemic mentioned</td><td>C_covid</td><td>6</td><td>2.38%</td></tr><tr><td>Pandemic as trigger to use online proctoring</td><td>C_covidtrigger</td><td>0</td><td>0</td></tr><tr><td colspan="4">Future: Should online proctoring be used in the future?</td></tr><tr><td>Against</td><td>F_against</td><td>16</td><td>6.35%</td></tr><tr><td>For</td><td>F_for</td><td>6</td><td>2.38%</td></tr><tr><td>Further research demanded</td><td>F_research</td><td>10</td><td>3.97%</td></tr></tbody></table>

We sought to investigate whether users' contributions highlight positive or negative aspects of remote proctoring and thus to determine the extent to which AI is addressed in a positive or negative way in Twitter discourse. Our analysis of the tags pertaining to the issues associated with online proctoring reveals that 198 tweets mention (potential) drawbacks of the technology. By contrast, only 47 tags refer to beneficial aspects of online proctoring. Negative emotions are expressed almost seven times as often as positive ones. Unsurprisingly, whenever user role could be identified, negative emotive language was exclusively found in the tweets of students, as well as, perhaps less expectedly educators.
The following section investigates some of these negative aspects in greater detail. They are succinctly summarised in the following tweet:

> [I] had to take an exam with a proctoring software for the first time & that shit is so invasive & anxiety-provoking... when I take exams I'm not a robot... I have to move around, look around, & mouth words/talk to myself which are all things that could be flagged as cheating. dumb! (Tweet 375)

As made clear in the last part of the tweet above, students are concerned with the algorithm's categorisation of certain behavioural patterns or objects as suspicious (tag: I_failed). Additionally, students report that behaviour which interferes with face detection (e.g. touching their face) or eye movement detection (e.g. crying) is frequently and unfairly flagged as suspicious behaviour. Consequently, how students are treated within an exam situation (tag: I_treatment) is discussed very emotionally on Twitter. Concerns are frequently raised about the fact that these kinds of behaviours may result in being (temporarily) excluded from the assessment or in automatically failing the exam. Emotions repeatedly linked to these concerns include anxiety and anger towards the AI-based proctoring software (cf. Tweet 375).

The second part of Tweet 375 touches on the issue of privacy (tag: I_contraprivacy). The range and quantity of data gathered (tag: D_data) is the subject of much debate among opponents of the technology – with, here too, emotions of anxiety and anger frequently voiced, e.g.:

> I have to use proctorio for one [of] my online classes and I am honestly so terrified to use it. It basically scans your whole room and monitors what you are doing while taking an exam, which is a massive invasion of privacy and causes unnecessary stress. (Tweet 376)

Although the issue was raised by the press (cf. Section 1), racism was not a prominent issue in Twitter discussions on AI-based online proctoring during the surveyed period (N = 15 for I_racism). In the context of our analysis, the racism tag refers to reports or fears of AI systems' inability to correctly recognise the faces of people of colour. The topic was addressed in no uncertain terms by opponents of the technology, though without explicitly mentioning strong negative emotions, e.g.:

> In general, technology has a pattern of reinforcing structural oppression like racism and sexism. Now these same biases are showing up in test proctoring software that disproportionately hurts marginalized students. [...] #edtech #equity #racism #sexism (Tweet 321)
<figure><img src="image_4.png" /><figcaption>Figure 2: Heat map showing co-occurrences of the category Definitions of AI (D_) with the category Emotions associated with remote proctoring (E_)</figcaption></figure>

To better visualise the co-occurrence of the tags that define AI with several other tag categories, heat maps were plotted. In the following, only the most frequently used definition tags, D_ai, DFace, D_behaviour and D_scan, are further explored.

Fig. 2 depicts how often certain definitions of AI are mentioned in combination with emotive language. The tags DFace, D_behaviour and D_scan are most predominantly used together with negative emotions (E_negative, E_worry and E_fear). In line with this observation, we also found that the most frequently used emoji was a crying face (N = 8).

Benefits of remote proctoring with AI-based solutions are almost exclusively discussed by company tweets (out of the tweets whose user role could be determined). Interestingly, as Fig. 3 shows, those descriptions mainly made use of superficial definitions of AI ("AI" and "algorithm") and variations of the word
<figure><img src="image_5.png" /><figcaption>Figure 3: Heat map showing co-occurrences of the category Definitions of AI (D_) with the category Benefits of remote proctoring (B_)</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Figure 4: Heat map showing the co-occurrence of the category Definitions of AI (D_) with the category Issues with remote proctoring (I_)</figcaption></figure>
“automate”.

Further, we found that issues with online proctoring are exclusively discussed by students and educators (teachers and lecturers). As Fig. 4 visualises, the tags `D_ai`, `DFace`, `D_behaviour` and `D_scan` are applied frequently while users position themselves generally against online proctoring. Most of the problems and concerns with regard to face detection and recognition accused the AI of making racist and/or ableist decisions. Further, users report problems concerning pattern recognition of their behaviour, such as head or hand movements, and thereby discuss how they feel they are being treated while using such software. Additionally, examinees' improper treatment was frequently debated in the context of room scans.

## 4 Discussion

In our analysis, we saw that Twitter discourse is notable for its lack of clear definitions of the terminology used to describe AI in online proctoring software (Fig. 1, category definition). Contrary to our assumptions, companies do not lead the conversation around those terms and do not position themselves as experts in the field. This becomes especially clear in the Twitter-based debate on whether the software provider Proctorio applies “face detection” or “face recognition”. Here, the company itself changed their statement several times without providing a clear definition or explanation of the terms or of the technology deployed in their marketed solutions. In the end, they claimed that they only use “face detection”. To clarify (see Dwivedi 2018), face recognition builds on face detection as the algorithms involved in face recognition must first detect a face in order to proceed further. Face detection tracks the position of a human face in an image based on common human facial features. However, the face is not identified as belonging to a particular individual. According to Proctorio's web page, they offer “machine learning-enabled ID verification [which] automatically analyzes and captures images of test-takers and their ID's” (Proctorio 2021). In other words, they compare the facial features of individuals and thereby clearly apply face recognition algorithms. The information provided on their website thus contradicts the statements the company published on Twitter. Indeed the company tweets analysed as part of this study (e.g. Tweet 462) seem to have been purposefully written to, if not outright mislead the public, at least muddy the waters. As shown in Tweet 443, these corporate tweets led to genuine confusion about the correct use and definition of AI-related terminology among Twitter users.

While highlighting the beneficial aspects of their technology, companies only use superficial definitions of AI and make vague statements using terms such
as “automation”. This imprecise usage of terminology (Fig. 1, category definition) was also observed in the tweets of students. Students describe the roles of AI very well by using expressions like “flagging” (see Tweet 169) and “monitors what you are doing” (see Tweet 376). When they refer to the technology at all, they mostly use the general term “AI”, and only rarely use more specific terms such as “machine learning”, “object detection” or “face detection”. Based on our findings, however, we cannot conclude whether the students in our dataset understand the underlying structures of AI-based proctoring systems but are not aware of more specific terms to describe them, or if they are choosing terms such as “flagging” as a means of self-identifying with the Twitter community of critics of the technology.

Our analysis revealed that tweets of students predominantly discuss negative aspects of AI-based proctoring systems. Students describe being scared of using these systems (e.g. Tweets 375 and 376). This seems to be foremost caused by the systems' lack of transparency. For instance, users are not informed about which specific behavioural patterns may cause the system to flag suspicious behaviour; it is not clear to the students why AI-based proctoring systems seemingly proscribe many legitimate, routine movements. Similarly, students' anxiety is often triggered by the invasiveness of AI-based proctoring systems which cause a range of privacy concerns (e.g. Tweet 375). The tweets analysed in the present study make clear that examinees proctored with AI-based software commonly cannot reproduce decisions taken by the algorithms. This inevitably casts doubts as to the legitimacy of such decisions. Test-takers' anxiety is no doubt exacerbated by the fact that the underlying mechanisms of online proctoring technologies are not clearly communicated by the tech companies themselves.

# 5 Conclusion

Our study has confirmed that Twitter is a highly relevant sphere of public discourse in investigating the roles, benefits and drawbacks associated with AI in remote proctoring. Our analysis includes tweets from a range of stakeholders: including service providers, educators, representatives of educational institutions and, last but not least, students.

Our examination of perceptions of AI in online proctoring suggests that educational institutions would do well to consider the concerns that students frequently voice about unfair treatment and privacy violations before deploying online AI-based proctoring tools (cf., e.g. Kharbat & Abu Daabes 2021). Even in the context of a global pandemic, students' concerns should feed into discussions
of possible alternative assessment forms (cf. Guangul et al. 2020). We find that companies marketing AI-based proctoring systems currently do not communicate how artificial intelligence systems are deployed in their software. As a result, their clients, both educational institutions as buyers of the services, as well as students and lecturers as the end-users are largely ill- or uninformed about what the AI-powered technology genuinely entails.

We suggest that greater transparency would likely benefit everyone. Companies could outline the steps they have undertaken to address known issues of unfair treatment and discrimination, and educational institutions could make informed decisions as to whether, when, and in what contexts relying on such systems may be appropriate (cf. Milone et al. 2017). Greater transparency would also empower test-takers to become more informed users of remote proctoring tools. Students would then be more likely to competently handle the technology without needing to face the fear and anxiety that the tools' opaque and non-reproducible mechanisms are currently provoking.

## References

Chin, Monica. 2020. *An ed-tech specialist spoke out about remote testing software — and now he's being sued.* https://www.theverge.com/2020/10/22/21526792/proctorio-online-test-proctoring-lawsuit-universities-students-coronavirus (28 March, 2021).

Chin, Monica. 2021. *Examsoft's proctoring software has a face-detection problem.* https://www.theverge.com/2021/1/5/22215727/examsoft-online-exams-testing-facial-recognition-report (28 March, 2021).

Dwivedi, Divyansh. 2018. *Face detection For beginners.* https://www.vice.com/en/article/7k9ag4/schools-are-abandoning-invasive-proctoring-software-after-student-backlash (28 March, 2021).

Feathers, Todd. 2021. Schools are abandoning invasive proctoring software after student backlash. *VICE*. https://www.vice.com/en/article/7k9ag4/schools-are-abandoning-invasive-proctoring-software-after-student-backlash (28 March, 2021).

Guangul, Fiseha M., Adeel H. Suhail, Muhammad I. Khalit & Basim A. Khidhir. 2020. Challenges of remote assessment in higher education in the context of COVID-19: a case study of Middle East College. *Educational Assessment, Evaluation and Accountability* 32(4). 519–535. DOI: 10.1007/s11092-020-09340-w. https://doi.org/10.1007/s11092-020-09340-w (28 March, 2021).
Harris, Margot. 2020. A student says test proctoring AI flagged her as cheating when she read a question out loud. Others say the software could have more dire consequences. *Insider*. https://www.insider.com/viral-tiktok-student-fails-exam-after-ai-software-flags-cheating-2020-10 (28 March, 2021).

Harwell, Drew. 2020a. Cheating-detection companies made millions during the pandemic. Now students are fighting back. *Washington Post*. https://www.washingtonpost.com/technology/2020/11/12/test-monitoring-student-revolt/ (19 November, 2020).

Harwell, Drew. 2020b. Mass school closures in the wake of the coronavirus are driving a new wave of student surveillance. *Washington Post*. https://www.washingtonpost.com/technology/2020/04/01/online-proctoring-college-exams-coronavirus/ (28 March, 2021).

Kearney, Michael W. 2019. Rtweet: collecting and analyzing Twitter data. *Journal of Open Source Software* 4(42). R package version 0.7.0, 1829. DOI: 10.21105/joss.01829. https://joss.theoj.org/papers/10.21105/joss.01829.

Kharbat, Faten F. & Ajayeb S. Abu Daabes. 2021. E-proctored exams during the COVID-19 pandemic: A close understanding. *Education and Information Technologies*. DOI: 10.1007/s10639-021-10458-7. http://link.springer.com/10.1007/s10639-021-10458-7 (28 March, 2021).

Kuckartz, Udo & Anne McWhertor. 2014. *Qualitative text analysis: a guide to methods, practice & using software*. Los Angeles: SAGE. 173 pp.

LeewayHertz. 2020. *Remote Proctoring Using Artificial Intelligence*. LeewayHertz - Software Development Company. https://www.leewayhertz.com/remote-proctoring-using-ai/ (28 March, 2021).

Markets, Research and. 2021. *Global $1.18 Billion Online Exam Proctoring Market to 2027 with COVID-19 Impact Analysis*. https://www.prnewswire.com/news-releases/global-1-18-billion-online-exam-proctoring-market-to-2027-with-covid-19-impact-analysis-301218806.html (28 March, 2021).

Milone, Anna S., Angela M. Cortese, Rebecca L. Balestrieri & Amy L. Pittenger. 2017. The impact of proctored online exams on the educational experience. *Currents in Pharmacy Teaching and Learning* 9(1). 108–114. DOI: 10.1016/j.cptl.2016.08.037. https://www.sciencedirect.com/science/article/pii/S1877129715200056 (28 March, 2021).

Proctorio. 2021. *ID Verification*. https://proctorio.com/platform/id-verification (30 March, 2021).

ProctorU. 2021. *Review+. https://www.proctoru.com/services/review-plus* (28 March, 2021).
R Core Team. 2020. *R: A Language and Environment for Statistical Computing*. R Foundation for Statistical Computing. Vienna, Austria. https://www.R-project.org/.

Scott, Aileen. 2019. *Artificial intelligence is making online proctoring safe and secure*. https://medium.com/@aileenscott604/artificial-intelligence-is-making-online-proctoring-safe-and-secure-9b03845602da (28 March, 2021).

Singh, Rajesh. 2021. Remote online exams made possible with the use of AI-based proctoring solutions. *MarylandReporter.com*. https://marylandreporter.com/2021/01/06/remote-online-exams-made-possible-with-the-use-of-ai-based-proctoring-solutions/ (28 March, 2021).

Swauger, Shea. 2020a. *Software That Monitors Students during Tests Perpetuates Inequality and Violates Their Privacy*. MIT Technology Review. https://www.technologyreview.com/2020/08/07/1006132/software-algorithms-proctoring-online-tests-ai-ethics/ (28 March, 2021).

Swauger, Shea. 2020b. What's worse than remote school? Remote test-taking with AI proctors. *NBC News*. https://www.nbcnews.com/think/opinion/remote-testing-monitored-ai-failing-students-forced-undergo-it-ncna1246769 (28 March, 2021).