Modeling the Social Acceptability of Technologies
Using Twitter Data
Hidetsugu Nanba
Faculty of Science and Engineering
Chuo University
Tokyo, Japannanba@kc.chuo-u.ac.jp

Katsuya Yamamoto
Faculty of Science and Engineering
Chuo University
Tokyo, Japan

Satoshi Fukuda
Faculty of Science and Engineering
Chuo University
Tokyo, Japan
fukuda.satoshi.3238@kc.chuo-u.ac.jp

Hiroko Shoji
Faculty of Science and Engineering
Chuo University
Tokyo, Japan
hiroko@kc.chuo-u.ac.jp

Masayoshi Tanishita
Faculty of Science and Engineering
Chuo University
Tokyo, Japantaninaho@gmail.com

Yasushi Kyutoku
Research and Development InstituteChuo University
Tokyo, Japan
qworkdog@gmail.com

Mitsuru Yamashina
Faculty of Literature
Chuo University
Tokyo, Japanm-yama@tamacc.chuo-u.ac.jp

Abstract—Our society is built on the benefits of technologicaldevelopment, but this development also entails risks. When theconvenience of a new technology outweighs the sense of anxiety
about its risks, we can consider that this technology is sociallyaccepted. In this study, we use the social media to analyze thegeneral public's anxiety feelings toward various technologies
and model the social acceptability of the new technologies basedon this analysis. In this study, 6,452,730 tweets about threetechnologies: “automated driving,” “electronic currency,” and“drones” were collected, and the social acceptability of thetechnologies was examined using emotion classification, text
categorization, and topic analysis techniques. As a result, weconcluded that the anxiety about unemployment can be one
perspective for analyzing the social acceptability of a technology.
Keywords—Twitter,emotionclassification,
acceptability, text categorization, topic analysis

socialI. INTRODUCTION
Our society is built on the benefits of technologicaldevelopment, but there are always risks involved. For
example, automated driving technology could make better useof time for people living in suburbs and commuting to cities,
and could improve the lives of people living in depopulatedareas without public transportation. On the other hand,
everyone must have felt the uneasiness about the safety of
automated driving technology. When the convenience of a
new technology outweighs such concerns, the technology canbe considered socially accepted. In this study, we use thesocial media to analyze the anxiety feelings of the general

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

public toward various types of technology, and model thesocial acceptability of the technologies based on this analysis.
It is considered that the anxiety toward new technologies
can transform over time. For example, if you search forautomated driving on Twitter, tweets around 2010 are
centered on vague concerns such as “I'm quite worried about
automated driving buses,” but by 2021, tweets are more
specific such as “I'll be happy when automated driving cars
become common, but I guess it will be a long time before
automated driving is possible even on snowy roads.” As in thelatter tweet, the concern has become more concrete. The
person who posted this tweet believes that automated driving
technology can be used in everyday life to some extent unlessunder special situations such as snowy roads. It can beassumed that the type of anxiety changed between 2010 and
2021, and that somewhere in that time, automated driving
technology became socially accepted.
In this research, we first collect the anxiety that people
have about three technologies, which are “automated driving,”“electronic currency,” and “drones,” from Twitter using
emotion classification. Next, we will use text categorizationand topic analysis techniques to clarify when thesetechnologies can be said to have become socially accepted.
The contributions of this paper are as follows.
l Modeling social acceptability of technologies byanalyzing a total of 6,452,730 tweets,l Highly reproducible analysis of tweets using state-of-theart natural language processing techniques,l Proposal of training data construction method forautomatically categorizing tweets in a very short time
(2,465 tagged-tweets in one day).
II. RELATED WORKS
Fukuda et al. [1] classified the tweets about thecoronavirus vaccine by emotion, and found out how thepercentage of tweets with each emotion changed in Japan, theUnited States, the United Kingdom, Canada, Australia, andIndia. We also analyze the temporal trend of tweets, but differ
in that we model the social acceptability of the technologies
based on the analysis results.
Dahal et al. [2] analyzed a large-scale dataset of geotagged
tweets containing specific keywords related to climate changeusing text mining techniques such as volume analysis,
sentiment analysis, and topic modeling. The content of thedebate and the differences between countries were clarified.
They used Latent Dirichlet Allocation (LDA) [3] as a topic
modeling method, but LDA ignores the semantic relationships
between words and has the problem of not being able tointerpret the meaning of words in consideration of the context.
To address this problem, text embedding methods
represented by BERT [4] have been rapidly spreading inrecent years. Grootendorst [5] showed that a new topic
modeling method, BERTopic, which utilizes embedded
representations of text and class-based TF-IDF, can classify
with high accuracy. Thus, we use BERTopic for topic analysis.
However, since BERTopic uses HDBSCAN for text
clustering, there may be a problem that outliers (when only
one document belongs to one cluster) increase. Therefore, we
use the k-means method instead of HDBSCAN as the text
clustering method.
III. MODELING THE SOCIAL ACCEPTABILITY OF
TECHNOLOGIESIn order to determine the social acceptability of newtechnologies, it is necessary to understand the public's
concerns and to identify how these concerns have changed
over time. In this study, we model the social acceptability ofa technology through the following three steps.
(A) Collection of anxiety tweets(B) Classification of anxiety tweets(C) Modeling of social acceptability of technology by
topic analysis of anxiety tweets
The details of each procedure are described below.
A. Collection of Anxiety Tweets
We use the Twitter API to collect tweets written in
Japanese about “automated driving,” “electronic currency,”
and “drones.” The search queries used for collection are
shown below.
l 自動運転 (automated driving): 自動運転 (automateddriving) OR ドライバーレスカー (driverless car) OR

1

https://github.com/cl-tohoku/bert-japanese

セルフドライビングカー (self-driving car) OR 無人
運転 (driverless)l 電 子 マ ネ ー (electronic currency): 電 子 マ ネ ー(electronic currency) OR キ ャ ッ シ ュ レ ス 決 済(cashless payment) OR おサイフケータイ (mobile
wallet) OR スマホ決済 (smartphone payment) OR ス
マートフォン決済 (smartphone payment) OR QR 決
済 (QR payment) OR コード決済 (code payment) OR
バーコード決済 (barcode payment) OR 非接触型決済 (contactless payment) OR タ ッ チ 決 済 (touchpayment) OR モバイルウォレット (mobile wallet)
OR デジタルウォレット (digital wallet) OR 交通系
IC (transportation IC)l

ドローン (drone): ドローン (drone)

Next, we classify the collected tweets by emotion to
extract anxiety tweets using BERT 1 [4] and T5 2 [6] toconstruct the emotion classifier. In this study, the model of
BERT used is a Japanese pre-trained model from Inui and
Suzuki Laboratory of Tohoku University, which was finetuned with 1,593,552 tweets collected from Twitter.
We also use a model for T5 that has been pre-trained on a
Japanese corpus of approximately 100 GB. We construct
two types of emotion classifiers: one that learns each emotion
type independently, and one that learns all emotions at once.
The training data used to construct the emotion classifiers
is WRIME [7], a tagged corpus for constructing emotion
classifiers using machine learning. In this corpus, eight typesof emotions, “joy,” “sadness,” “expectation,” “surprise,”“anger,” “fear,” “disgust,” and “trust,” were assigned to
40,000 tweets written in Japanese with an emotion intensity
value of 0-3.
After constructing the two emotion classifiers, we
compare the accuracy of both classifiers Next, we use themodel with higher accuracy to extract tweets containing
anxiety. We selected “fear” and “disgust” as the emotion types
related to anxiety among 8 emotions, extracting only 3 out of
0-3 for the emotion intensity as tweets containing anxiety.
B. Classification of Anxiety Tweets
The emotion classifier constructed in the previous section
is used to classify anxiety tweets. A report on key issues
related to safety and security science and technology was
written by the Committee on Safety and Security Science andTechnology [8]. In this report, the factors that threaten safety
and security are organized into three tiers. The first tierconsists of 11 categories: “crime,” “accidents,” “natural
disasters,” “war,” “cyberspace issues,” “health issues,” “foodissues,” “social life issues,” “economic issues,” “political andadministrative issues,” and “environmental and energy issues.”
We use the 9 categories, excluding “health issues” and “foodissues,” which are less relevant for our analysis, and the tweetsthat do not belong to these 9 categories are classified as“others,” for a total of 10 categories.
2

https://github.com/sonoisa/t5-japaneseGenerally, constructing a text classifier by machinelearning requires a large amount of manually categorized text.
However, preparing such data is time-consuming. Therefore,
we create a manually-tagged corpus for machine learning in avery short time. The procedure consists of two steps. In step
1, we manually select 27 tweets, 3 per category, from the setof tweets collected in the previous section. In step 2, we use
these 27 tweets as the training data and apply few-shotlearning to automatically categorize a total of 2,465 tweets.
We manually check these 2,465 tweets and correct if theautomatically assigned category is wrong. Using these results,
we build a classifier that classifies tweets into 9 categories.
ChatGPT 3 (GPT-3.5) and T5 are used in steps 1 and 2,
respectively.
Using this classifier, we categorize all tweets collected in
the previous section. The tweets that are not categorized in anyof the 9 categories are assigned the “other” category. It istheoretically possible to categorize all the tweets using the
few-shot learning in step 1, but the problem is that the
ChatGPT API is a paid service and it is very costly tocategorize all the large number of tweets.

Evaluation Measures
The Quadratic Weighted Kappa (QWK) [20] was used toevaluate the emotion classifiers.
Experimental Results
Table I shows the experimental results. The highest
accuracy for each emotion is shown in bold. The T5 model,
which was trained for each emotion, obtained the highest
accuracy for five of the eight emotions, and the highest values
were also obtained for “fear” and “disgust,” which wereselected as the types related to anxiety in this study. The
highest values were also obtained for “fear” and “disgust,”
which were selected as types related to anxiety. Therefore, theT5 model trained for each emotion is used to extract anxiety
tweets.
TABLE I.
Emotion

EXPERIMENTAL RESULT OF EMOTION CLASSIFICATION
BERT

T5 (learning all
emotion types
at once)T5(learningeach emotion
type
independently)

C. Modeling of Social Acceptability of Technology by Topic
Analysis of Anxiety Tweets
After classifying all tweets into ten categories using themethod in the previous section, the tweets in each category are
analyzed using BERTopic [5]. This tool visualizes the numberof tweets per cluster over time. Using this tool, we find theclusters that are related to the social acceptability of thetechnology. The details of the analysis are described in sectionV.

Joy

0.68710.71850.7142

Sadness

0.59310.5916

0.6180

Anticipation

0.66500.66210.6882

Surprise0.57320.57020.5906Anger

0.37940.29770.2934Fear

0.54800.5532

0.5869Disgust0.47040.47290.5662

IV. EXPERIMENT
We conducted experiments to confirm the effectiveness ofthe emotion classification method proposed in Section III-A
and the category classification method proposed in Section
III-B.

Trust

0.23370.30320.3031

Average

0.51750.52120.5451

A. Emotion Classification of Tweets
Experimental Data
To construct the emotion classifier, we use WRIME, adataset for estimating emotion intensity in Japanese. In thisexperiment, we use 30,000 training data, 2,500 validation data,
and 2,500 evaluation data without duplication, following thedivision of Kajiwara et al. [7].
Alternative Methods
Experiments were conducted using the following threemethods.
l BERTl T5 (learning all emotion types at once)l T5 (learning each emotion type independently)

3

https://openai.com/blog/chatgpt

Discussion
The results of learning by BERT and evaluation by QWK
in the experiment by Kajiwara et al. [7] were 0.386 for fear
and 0.348 for disgust. Compared to these results, the BERT
model used in our experiment showed a significant
improvement in evaluation. This may be due to the fact that
fine-tuning using tweet data allowed us to specialize in short
sentences, which is a characteristic of data on social media.
B. Anxiety Tweet Classification
Experimental Data
The classifier was constructed using ChatGPT and amanually created anxiety tweet dataset; 1,725 out of 2,465
were used as training data, 370 as validation data, and 370 as
evaluation data. Table II is a breakdown of the 2,465 tweets
by category.

TABLE II.

EXPERIMENTAL RESULT OF TWEET CLASSIFICATION BY
ANXIETY CATEGORY

Anxiety Category

Numberof tweets

Anxiety Category

Number of
tweetsCrime

201

Social life issues

397

Accidents

334

Economic issues

81

Natural disasters

27

Political andadministrative issues

217

War

73

Environmental andenergy issues

55

Cyberspace issues

75

Other

1,005

Evaluation Measures
Precision and recall were used to evaluate the anxiety
tweet classifier.
Experimental Results
Table III shows the experimental results, which are goodconsidering the fact that the classification is a multi-classclassification of 10 categories, with an average precision andrecall of more than 0.53.
TABLE III.

EXPERIMENTAL RESULT OF TWEET CLASSIFICATION BY
ANXIETY CATEGORY

Anxiety Category

Precision

Recall

Crime

0.48

0.64

Accidents

0.61

0.75

Natural disasters

0.33

0.50

War

0.60

0.40

Cyberspace issues

0.38

0.33

Social life issues

0.48

0.57

Economic issues

0.67

0.57

Political
andadministrative issues

0.68

0.72

Environmental
energy issues

0.40

0.20

Other

0.68

0.59

Average

0.53

0.53

and

agree in advance that all the responsibility goes to thepassenger? It can do whatever it wants! It's too scary!
This tweet should be categorized as either “accidents” or“political and administrative issues,” but in the results of thisevaluation, it is categorized as “Other.” The precision andrecall could be improved by multi-labeling these problems.
V. ANALYSIS OF ANXIETY TWEETS ABOUT THE THREETECHNOLOGIES AND THEIR SOCIAL ACCEPTABILITY
This section presents the results of extraction,
classification, and topic analysis of anxiety tweets for the threetechnologies of automated driving, electronic currency, anddrones.
A. Data Used for Analysis
Table IV shows the collection period for each technology,
the total number of tweets, and the number of extracted
anxiety tweets. Before the analysis, we excluded tweets
containing URLs for all technologies, and tweets containing“applications” and “campaigns” for electronic currency.
TABLE IV.
Technologies

Collected
periodAutomated
driving

DATA USED FOR ANALYSIS
Numbertweetsof

Numberofanxiety tweets

2006.3-2022.5

2,086,612

79,827

Electronic
currency

2006.3-2022.5

1,619,275

81,239Drones2015.62022.12

2,746,843

112,393

B. Temporal Trends in the Probability of Occurrence ofAnxiety Categories
Automated driving
For each technology we calculated the probability ofoccurrence by dividing the number of tweets in each anxiety
category by the number of anxiety tweets on a yearly basis.
Fig. 1 shows the trends in the probability of occurrence ofautomated driving by category.

Discussion
Regarding the evaluation results of the T5 classifier,although it was generally able to classify with high precision,for some categories, both precision and recall were below 0.3.
In order to further improve precision, multi-label
classification is considered necessary. The proposed method
uses single-label classification, with one category assigned toeach tweet, but in many cases, one label is not sufficient. The
following is an example of a case where multiple labels arenecessary.
l It occurred to me that AI can run over and kill people inorder to protect passengers in automated driving. Whowould be responsible in such a case? Do we have to

Fig. 1. Temporal trend in the probability of occurrence of anxiety
categories (automated driving)As can be seen from Fig. 1, the percentage of anxiety about“accidents” is always high, and that anxiety will account for
more than 25% even in 2022. It can be inferred that anxiety
has not been dispelled even with the spread of automateddriving cars. Regarding “political and administrative issues,”
there were many opinions about who would be responsible forthe accident and what kind of system the licensing would be,
tweeted many times since 2013. Many tweets about driverunemployment are categorized as “social life issues,” and thepercentage is even higher now that Level 3 automated driving
is seen across the city.
Electronic currency
Fig. 2 shows the trends in the probability of occurrence ofelectronic currency by category.
Fig. 3. Temporal trend in the probability of occurrence of anxiety
categories (drone)

The ratio of each category has changed greatly dependingon the period, and in recent years, anxiety about “war” hastaken up a fairly large proportion. Looking at the tweets, theuse of drones in the Ukraine war was a big topic, and there
were many mixed opinions about the impact of technologicalprogress on the war.

Fig. 2. Temporal trend in the probability of occurrence of anxiety
categories (electronic currency)

From Fig. 2, the probability in “social life issues” is quite
high and is increasing gradually. The next most common
result was classified as “crime.” Concerns about personal
information and privacy were still present today. A similar
opinion was also seen on “cyberspace issues.” “Political andadministrative issues” tended to increase around 2020.
Looking at the tweets during the period, there were many
voices calling for an extension and concerns about theeconomy in response to the end of the government's cashless
payment point return business in the recession.

Since 2018, there has been an increase in political andadministrative issues, such as opinions on flight area andweight regulations. The topic of “accidents” has been
declining since 2016. One of the factors is the decrease in“accidents” due to stricter regulations. On the other hand,
there were many doubts, anxieties, and harsh opinions about
how to use the technology, contrary to the progress of thetechnology. As for “cyberspace issues,” there were many
voices of anxiety and worries about hacking and cyberattacks
of drones, which appeared for automated driving as well.
C. Social Acceptability of Technology
Topic analysis using BERTopic clarified thechronological changes in anxiety factors within each category.
Some of the topic analysis results for each technology are
shown below.
Automated driving
Fig. 4 shows the results of the temporal trend analysis ofunemployment in the “social life issues” category.

DronesFig. 3 shows the trends in the probability of occurrence ofdrones by category.

Fig. 4. Temporal trend of topic: Unemployment (automated driving)

As the topic of automated driving cars becomes more popular,
we can see that the following unemployment-related topics

are increasing. All these tweets were written in Japanese, but
we translated them into English.

years, this landscape will be wiped out by unmanneddelivery vehicles or drones. ....

l When automated driving cars become fully practical, what
will happen to the employment of taxis and bus drivers?
I'm worried about other people.

l Combined with automated driving, it seems that homedelivery will become unmanned. It seems to be fasterthan a drone. But if I open the door and these guys are
there, I would be scared.

l It would be nice to build a normal automated driving
private car first, so why build a taxi that can carrypassengers first? Do you really want to increase thenumber of unemployed by stealing drivers' jobs?
l First, you will lose your driver's license. It will be fully
automatic. In other words, the job of a driver will
disappear. This is the future one inch ahead. Companies
that do not anticipate this at this stage will go bankrupt.
surely.
Many automakers are setting out to develop automateddriving cars, which has increased since 2015, and several US
states have allowed automated driving cars to be tested on
public roads. In 2016, the SAE (Society of Automotive
Engineers) defined automated driving levels as six categories,
and there were major movements related to automated driving
during this period. The number of tweets about
unemployment has started to increase around this time. In thisway, concerns about unemployment against a new technology
seem to indicate that the technology is being socially accepted.
Electronic currency
Regarding electronic currency, there were not many
tweets about unemployment seen in automated driving, butthere were opinions that worried about unemployment, suchas the following.
l If convenience stores are unmanned and cashless
payments are made, there is no need to restrain
employees even if they are open 24 hours a day.
l If all payments are cashless, the working hours of a store
cashier will be reduced by about 4 hours per day.

l There are unmanned convenience stores in China, anddrone delivery experiments are being conducted in the
United States. There is also a trend toward unmanned
operations in Japan. It's really dangerous if you don't
build up your strength before it's unmanned. I am gettingstronger every day.
From the above analysis, we believe that the anxiety about
unemployment can be one perspective for analyzing the social
acceptability of technologies, although some tweets could not
be collected enough to draw a graph of the transition
depending on the technology.
VI. CONCLUSIONSIn this research, we examined the social acceptability oftechnologies using emotion classification, text classification,and topic analysis techniques, targeting 6,452,730 tweets
about three technologies: automated driving, electronic
currency, and drones. As a result, we concluded that anxiety
about unemployment can be one of the perspectives foranalyzing the social acceptability of technologies.
REFERENCES
[1]

[2]

[3]
[4]

DronesFig. 5 shows the results of the temporal trend analysis ofunemployment in the “social life issues” category.

[5]
[6]

[7]

Fig. 5. Temporal Trend of topic: Unemployment (drones)

In this cluster, we saw the following tweets about whether
drones would make people unemployed.
l At noon, lunch delivery men run around the Shanghai
office on electric motorcycles, but in the next five to ten[8]

S. Fukuda, H. Nanba, and H. Shoji, Analysis of International Public
Emotional Responses Toward the COVID-19 Vaccine, Proceedings ofthe 12rd International Conference on Advanced Applied Informatics(IIAI AAI 2022), 2022.
B. Dahal, S.A.P. Kumar, and Z. Li, Topic Modeling and Sentiment
Analysis of Global Climate Change Tweets, Social Network Analysis
and Mining, Vol. 9, No. 24, 2019, pp.1-20.
D.M. Blei, A.Y. Ng, and M.I. Jordan, Latent Dirichlet Allocation,
Journal of Machine Learning Research, 3, 2003, pp.993-1022.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, BERT: Pretraining of Deep Bidirectional Transformers for Language
Understanding, Proceedings of the 2019 Conference of the North
American Chapter of the Association for Computational Linguistics:
Human Language Technologies, 2019, pp.4171-4186.
M. Grootendorst, BERTopic: Neural Topic Modeling with a Classbased TF-IDF Procedure, arXiv:2203.05794 [cs.CL], 2022.
C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y.
Zhou, W. Li, and P. J. Liu, Exploring the Limits of Transfer Learningwith a Unified Text-to-Text Transformer, Journal of Machine Learning
Research, 21(140), 2020, pp.1-67.
T. Kajiwara, C. Chu, N. Takemura, Y. Nakashima, and H. Nagahara,
WRIME: A New Dataset for Emotional Intensity Estimation with
Subjective and Objective Annotations, Proceedings of the 2021
Conference of the North American Chapter of the Association for
Computational Linguistics: Human Language Technologies, 2021,
pp.2095-2104.
Science and Technology Council, Research Planning and Evaluation
Subcommittee, Committee on Science and Technology for Safety andSecurity, On Important Issues Related to Safety and Security Scienceand Technology, Ministry of Education, Culture, Sports, Science andTechnologyofJapan,
2011.
(inJapanese)https://www.mext.go.jp/b_menu/shingi/gijyutu/gijyutu2/016/houkoku/__icsFiles/afieldfile/2012/08/01/1323913_01_1.pdf (accessed on
April 3, 2023)