EVENT REPORT

Report on the 2nd International Workshop on
Transforms in Behavioral and Affective
Computing (THECOG 2022) at CIKM 2022
Georgios Drakopoulos

Eleanna Kafeza

Ionian University
GreeceZayed University
UAE

c16drak@ionio.gr

eleana.kafeza@zu.ac.ae

Abstract
Human decision making is central in many functions across a broad spectrum of fields including marketing, investments and smart contracts, digital health, political campaigns, logistics,
and strategic management to name only a few. Computational behavioral science, the focusof the second consecutive iteration of the international workshop on transforms in behavioral
and affective computing (THECOG) which was held in conjunction with CIKM 2022, notonly studies the various psychological, cultural, and social factors contributing to decision
making besides reasoning, but it also seeks to construct robust, scalable, and efficient computational models imitating or extending decision making processes. This year the keynote
speech focused on affective robotics and their expected advantages in substantially improving
the quality of human life. Moreover, the accepted papers had a considerable topical variety covering among others smart cities, speech emotion recognition, deepfake discovery, andhow smart coupons may influence online consumer behavior. THECOG 2022 for a second
continuous year was a central meeting point where new results were presented.
Date: 21 October, 2022.
Website: https://www.cikm2022.org/workshops.

1

Introduction

Decision making, even when only incomplete data are available, is a fundamental process integrated into literally every aspect of human activity. Thus, sheding light on the various judgement
mechanisms humans consciously or subconsciously employ as well as on their interaction is ofparamount importance [Costa et al., 2019]. Lately it has been proposed that emotions and intrinsic data completion operations akin to numerical inverse problem solution actually play a central
role in decison making [Cartwright, 2018]. It should be highlighted that under computational
behavioral science the rationality axioms of classical economic theory are not negated but instead
extended. Computational behavioral science can evaluate the individual or collective decision
making processes in massive populations with signal estimation or deep learning techniques basedon a wide array of attributes ranging from social media posts [Drakopoulos et al., 2021] andACM SIGIR Forum

1

Vol. 56 No. 2 December 2022

multimedia [Mittal et al., 2021; Zhao et al., 2021] to physiological signs [McDuff, 2021], gaming[Drakopoulos et al., 2020b], and neuroimaging results [Drakopoulos et al., 2020a]. Additionally,time dependent decision making processes can be understood and processed in a signal processing context and even tracked with input-output or state space models. As disposition towards
alternative decisions may well change over time and as new data become available, this is a majoradvantage compared to traditional static decision making analysis. So far the primary findingsin the field are concepts like bounded rationality [Simon, 1990; Geng et al., 2022] and perceived
risk [Hasan et al., 2021; Li et al., 2019], while results include optimal strategies for various levels
of information awareness [Arnott and Gao, 2019] and action strategies based on perceived loss
aversion principles [Yechiam, 2019] which have been successfully applied to many situations [Foxxet al., 2022; Khazanov et al., 2022]:
 Affective state in social media beyond single polarity.
 Behavioral computing and deep learning.
 Bounded rationality estimation techniques and computational models.
 Case studies of behavioral economics.
 Computational affective models.
 Data-driven approaches for strategy recommendation.
 Deep learning strategies for training nudge theory models.
 Distributed approaches to computing public sentiment at massive scale.
 Graph neural networks for assessing sentiments in social graphs.
 Higher order sentiment metrics.
 Information augmentation through social media and crowdsourcing.
 Semantics for Barnum statement and affective state discovery.
 Natural language processing for aspect mining.
 Nudge theory based strategies for brand loyalty and social media campaigns.
 Risk estimators based on perceptive criteria and incomplete information.

The objectives of THECOG were the following. First, to become a major focal point for
researchers and practitioners of diverse backgrounds in order to obtain strong interdisciplinary
results in the field of behavioral economics. Second, to become progressively a reference point forhigh quality papers and case studies. This will be achieved by inviting top researchers both fromacademia and industry, maintaining a strong peer review process, posting the CFP to high profile
mailing lists and in general maintaining an active social media presence throughout the academic
year with emphasis in special dates such as the beginning of the academic year and right after falland spring breaks, and by keeping a balance between case studies and research papers.
The three mid- to long-terms goals of THECOG include the extension of the computer science methodologies to behavioral economics, allowing not only the application of state-of-the-art
techniques to a new field of study but also the creation of new and diverse datasets, the development of a strong demo and application-oriented culture among contributors and participants
alike, and finally the presentation of software which can solve problems in the field. One of themain outcomes of THECOG is expected to be, as stated earlier, the progressive creation of aconcrete digital base of case studies, best practice guides, and online resources which will cover
many and diverse needs. This will be instrumental in creating a visible and unified conceptual
framework for common understanding between field practitioners. Although currently there is
ACM SIGIR Forum2

Vol. 56 No. 2 December 2022

actually a considerable number of digital guides, dictionaries, and glossaries available online, they
do not cover the field in a satisfactory depth and there are few real examples which can truly help
an interested reader, whether a newcomer or a seasoned professional.
The intended audience includes researchers with a strong background in: Deep learning, especially distributed optimization and one shot learning.
 Crowdsourcing, mainly with respect to incentive theory.
 Signal processing, especially in estimation theory, adaptive filters, and risk functions.
 Computational theory and algorithmic design with an emphasis on bounded rationality.
 Natural language processing, affective computing, and social media analysis.
 Game theory, market mechanism and auction design, marketing, and microeconomics.

The remainder of this work is structured as follows. In section 2 the keynote speech of
THECOG 2022 is summarized. Then in section 3 the abstracts of the accepted papers are presented. Finally, in section 4 this article is concluded with a brief overview of the workshop as well
as of the field of computational behavioral sciences.

2

Keynote Speech: Affective Human-Robot Interaction

This year the keynote speech was delivered by Professor Anthony Tzes, Program Head of the Electrical Engineering (EE) program at New York University Abu Dhabi (NYUAD). He is also the
Director of the Center for Artificial Intelligence and Robotics. Prior to this, he was a member of theGoverning Council of University of Patras (UPAT) in Greece and Professor and Head of the Electrical and Computer Engineering at UPAT. He was the director of UPAT’s graduate program in
Biomedical Engineering (2015-7). He is a graduate of UPAT (1985) and has received his doctorate
from the Ohio State University in 1990. From 1990 till 1999 he was (tenured associate professor)
with NYU’s Tandon School of Engineering. He has been the national representative (2006-9) toEU’s FP7’s thematic area “Regions of Knowledge, Research Potential and Coherent Developmentof Policies”. He has served in various positions (Program Chairman (MIM ’00), Organizing Committee Chairman (ECC’07), General Chairman (MED2011 and ICUAS2020), Program Chairman(MED2015)), and as IPC-member at several international conferences.
In brief, the major points of the keynote speech were the following: Social robots soon will be everywhere.
 Interaction relies heavily on facial expressions.
 Human expressions are associated with emotional states.
 Physiological indicators include also voice and body pose.
 They can be combined with EEG readings.
 Multimodal data fusion is a primary challenge.

ACM SIGIR Forum

3

Vol. 56 No. 2 December 2022

3

Papers And Presentations

In this section the abstracts of the accepted papers are presented. As it can be seen, they cover abroad array of topics in the field of computational behavioral sciences and, therefore, contributeto the general discussion in the field:
 Drakopoulos and Kafeza, “Estimating the MBTI type of Twitter accounts withgraph neural networks over Neo4j”. Intelligent agents are indispensable and flexible
autonomous tools for efficiently mining large graphs for heterogeneous knowledge. Twitter isa prime case in point with structural and functional attributes such as original multimedia
content posting and retweeting revealing important affective information about accounts.
Additionally, this can be facilitated by including hashtag emotional polarity and reactions
to political, social, or even historical events. Further insight can be gained by moving one
step forward from individual emotional reactions to integrated personality estimations suchas the MBTI taxonomy. An intelligent agent has been developed with a stochastic accountvisiting policy based on preferential attachment, an optional evolving forgetting factor forpenalizing vertices appearing too frequently, and the capability to yield an MBTI estimatebased on a graph neural network. The results indicate the superior performance of the
proposed heuristic based on evaluation criteria including community size distribution andhashtag coherency.
 Karavokyris and Sioutas, “Graph neural networks for affective social media: A
comprehensive overview”. Social media have become the main platforms for expressing
and supplementing nuanced human activity such as engaging in public and private conversations, creating and sharing multimedia content, participating to digital culture events,
and recently describing emotions about events, places, or even products. In this survey, we
provide a comprehensive overview of graph mining and machine learning on affective social
media through graph neural networks (GNNs). The latter are capable of performing a variety of tasks, such as graph and vertex classification, link prediction, and graph clustering
using vertex information, edge information, and topological structure. These capabilities
are critical in harnessing the vast emotional information available in social media in orderto generate meaningful and scalable affective analytics.
 Iqbal, Abbasi, Javed, and Jalil, “Deepfake audio detection via feature engineering and machine learning model”. With the advancement of technologies in synthetic
speech generation, audio deepfake is becoming the most common source of deception. As aresult, distinguishing between fake and real audio is becoming increasingly difficult. Several
studies were conducted based on machine learning approaches using ASVSpoof or AVSpoofto deal with these challenges. This study experiments on the latest fake or real (FoR) dataset.
The audio samples of this dataset are generated using the best text-to-speech (TTS) models. The proposed approach is based on optimal feature engineering and selecting the bestmachine learning models to detect fake or real audios. The feature engineering approach
employs various methods for extracting features from audio. In contrast, the feature selection method employs the best-performing minimum features, which are then fed to machinelearning classifiers. The experiments used six ML classifiers and three subsets of the FoR
dataset. The experimental results show that the proposed approach can accurately detect

ACM SIGIR Forum

4

Vol. 56 No. 2 December 2022

real or fake audio. The proposed method outperforms the baseline method by an accuracy
gain of 26%.
 Liu, Albuquerque, Hung, Gabbar, Fantinato, and Iqbal, “Towards a real-time
emergency response model for connected and autonomous vehicle”. Recently technological advancements in the automobile and transportation sector have gained significant
interest from governments, industry leaders, and citizens. Together with Autonomous Vehicles (AV) and Connected Vehicles (CV), Connected-Autonomous Vehicles (CAV) have madea revolution in these sectors. Emergency Vehicles (EVs), such as ambulances, fire trucks,
and patrol cars, are essential to our daily traffic life. Each of EVs has a different purpose,but all have their urgency and importance, and any time passing may cause the death oflife. Thus, whenever other vehicle drivers encounter an EV on the road, they must yield
to the EVs. Therefore, a CAV system that can detect EVs will significantly improve theseissues. According to the Society of Automotive Engineers International (SAE), in today’s
autonomous vehicles, most of them are less than Level 5, and car manufacturers assume thedriver will take back control. Still, most autonomous vehicles mainly rely on their vision
sensor instead of their sound sensor. Thus, when the system notifies the driver that theEVs are already close to them, it may be dangerous for the driver, pedestrians, and passengers in the vehicle. This paper proposes a conceptual framework and discusses a related
methodology to support such a real-time emergency response model for CAV.
 Drakopoulos and Mylonas, “Extreme learning machines for efficient speech emotion estimation in Julia” [short]. Speech is a mainstay of communication across literally
all human activities. Besides facts and statements speech carries substantial information regarding experiences, thoughts, and emotions, therefore adding significant context. Moreover,
non-linguistic elements such as pauses add more to the message. The field of speech emotion
recognition (SER) has been developed precisely to develop algorithms and tools performing
what humans learn to do from early on. One promising line of research comes from applying
deep learning techniques trained on numerous audio attributes to discern between various
emotions as dictated by a given model of fundamental human emotions. Extreme learning
machines (ELMs) are neural network architectures achieving efficiency through simplicity
and can potentially operate akin to a sparse coder. When trained by a plethora of audio
attributes, such as cepstrum coefficients, zero crossing rate, and autocorrelation, then it canclassify emotions in speech based on the established emotion wheel model. The evaluation,
done with the Toronto emotional speech set (TESS) on an ELM implemented in Julia, isquite encouraging.
 Liu, Chiu, and Ho, “Prediction of mobile coupon use: Data analytics of influencing factors” [short]. As the Online to Offline (O2O) business model has entered a
stage of rapid development, mobile coupons have attracted the attention of scholars andthe industry as a preferential means to attract customers to pay online. This research uses
algorithms and models in machine learning to explore the consumption records of the mobile coupon data set. Our results show that the characteristics of merchants and coupons
significantly affect the use of mobile coupons. Based on our findings, a Customer CRM
system architecture suitable for merchants to manage coupons was designed to optimize thebusiness management of coupons and users.

ACM SIGIR Forum

5

Vol. 56 No. 2 December 2022

4

SummaryThis article focuses on describing the scope of and the results presented at the international
workshop on transforms in behavioral and affective computing (THECOG) which was held inconjunction with CIKM 2022 for the second year in a row. The keynote address highlighted thepotential of affective robotics, whereas the papers addressed diverse topics including intelligent
agents operating on affective principle, deepfake discovery, and the coordination of autonomous
emergency response vehicles in smart cities. It is worth mentioning that each paper presentation
led to a lively follow up conversation.
As a more general remark, recent advances in deep learning can have considerable impact oncomputational behavioral sciences such as the development of data driven algorithms, sophisticated representations for human emotion spaces, and of real time tracking of affective state ofsocial media accounts. Combined with technologically refined biomedical equipment which provide high resolution data with low noise and distortion levels, the prospects of major advances in
the field look bright.

Acknowledgments
We would like to thank the authors and the workshop participants for their contributions. Moreover, special thanks go to the keynote speaker for his considerable contribution and for sharing
his views on affective robotics.

References
David Arnott and Shijia Gao. Behavioral economics for decision support systems researchers.
Decision Support Systems, 122, 2019.
Edward Cartwright. Behavioral economics. Routledge, 2018.
Daniel Fonseca Costa, Francisval de Melo Carvalho, and Bruno César de Melo Moreira. Behavioral
economics and behavioral finance: A bibliometric analysis of the scientific fields. Journal of
Economic Surveys, 33(1):3–24, 2019.
Georgios Drakopoulos, Ioanna Giannoukou, Phivos Mylonas, and Spyros Sioutas. On tensor
distances for self organizing maps: Clustering cognitive tasks. In DEXA, volume 12392 of
Lecture Notes in Computer Science, pages 195–210. Springer, 2020a. doi: 10.1007/978-3-030-5
9051-2 13.
Georgios Drakopoulos, Yorghos Voutos, and Phivos Mylonas. Annotation-assisted clustering ofplayer profiles in cultural games: A case for tensor analytics in Julia. BDCC, 4(4), 2020b. doi:
10.3390/bdcc4040039.
Georgios Drakopoulos, Eleanna Kafeza, Phivos Mylonas, and Spyros Sioutas. A graph neuralnetwork for fuzzy Twitter graphs. In Gao Cong and Maya Ramanath, editors, CIKM companion
volume, volume 3052. CEUR-WS.org, 2021.
ACM SIGIR Forum

6

Vol. 56 No. 2 December 2022

Ricarda K Foxx, Danielle E Jake-Schoffman, Anne E Mathews, JeeWon Cheong, and Ali M
Yurasek. Relative reinforcement from physical activity in real-world environments: A novel
application of behavioral economics. Journal of Behavioral Medicine, pages 1–11, 2022.
Baocheng Geng, Qunwei Li, and Pramod K Varshney. Human decision making with bounded
rationality. In ICASSP, pages 5493–5497. IEEE, 2022.
Rajibul Hasan, Riad Shams, and Mizan Rahman. Consumer trust and perceived risk for voicecontrolled artificial intelligence: The case of Siri. Journal of Business Research, 131:591–597,
2021.
Gabriela Kattan Khazanov, Shari Jager-Hyman, Joseph Harrison, Molly Candon, Alison Buttenheim, Matteo F Pieri, David W Oslin, and Courtney Benjamin Wolk. Leveraging behavioral
economics and implementation science to engage patients at risk for suicide in mental healthtreatment: A pilot study protocol. Pilot and feasibility studies, 8(1):1–12, 2022.
Mengyao Li, Brittany E Holthausen, Rachel E Stuck, and Bruce N Walker. No risk no trust:
Investigating perceived risk in highly automated driving. In Proceedings of the 11th International
Conference on Automotive User Interfaces and Interactive Vehicular Applications, pages 177–
185, 2019.
Daniel McDuff. Camera measurement of physiological vital signs. ACM Computing Surveys(CSUR), 2021.
Trisha Mittal, Puneet Mathur, Aniket Bera, and Dinesh Manocha. Affect2mm: Affective analysis
of multimedia content using emotion causality. In CVPR, pages 5661–5671, 2021.
Herbert A Simon. Bounded rationality. In Utility and probability, pages 15–18. Springer, 1990.
Eldad Yechiam. Acceptable losses: The debatable origins of loss aversion. Psychological research,
83(7):1327–1339, 2019.
Sicheng Zhao, Xingxu Yao, Jufeng Yang, Guoli Jia, Guiguang Ding, Tat-Seng Chua, Bjoern W
Schuller, and Kurt Keutzer. Affective image content analysis: Two decades review and new
perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.

ACM SIGIR Forum

7

Vol. 56 No. 2 December 2022