Reviewerly/exHarmony huggingface dataset:
Quantifying the narrative flow of imagined versus autobiographical stories

Lifelong experiences and learned knowledge lead to shared expectations about how common situations tend to unfold. Such knowledge of narrative event flow enables people to weave together a story. However, comparable computational tools to evaluate the flow of events in narratives are limited. We quantify the differences between autobiographical and imagined stories by introducing sequentiality, a measure of narrative flow of events, drawing probabilistic inferences from a cutting-edge large language model (GPT-3). Sequentiality captures the flow of a narrative by comparing the probability of a sentence with and without its preceding story context. We applied our measure to study thousands of diary-like stories, collected from crowdworkers, about either a recent remembered experience or an imagined story on the same topic. The results show that imagined stories have higher sequentiality than autobiographical stories and that the sequentiality of autobiographical stories increases when the memories are retold several months later. In pursuit of deeper understandings of how sequentiality measures the flow of narratives, we explore proportions of major and minor events in story sentences, as annotated by crowdworkers. We find that lower sequentiality is associated with higher proportions of major events. The methods and results highlight opportunities to use cutting-edge computational analyses, such as sequentiality, on large corpora of matched imagined and autobiographical stories to investigate the influences of memory and reasoning on language generation processes.
Explainable AI (XAI) In Biomedical Signal and Image Processing: Promises and Challenges

Artificial intelligence has become pervasive across disciplines and fields, and biomedical image and signal processing is no exception. The growing and widespread interest on the topic has triggered a vast research activity that is reflected in an exponential research effort. Through study of massive and diverse biomedical data, machine and deep learning models have revolutionized various tasks such as modeling, segmentation, registration, classification and synthesis, outperforming traditional techniques. However, the difficulty in translating the results into biologically/clinically interpretable information is preventing their full exploitation in the field. Explainable AI (XAI) attempts to fill this translational gap by providing means to make the models interpretable and providing explanations. Different solutions have been proposed so far and are gaining increasing interest from the community. This paper aims at providing an overview on XAI in biomedical data processing and points to an upcoming Special Issue on Deep Learning in Biomedical Image and Signal Processing of the IEEE Signal Processing Magazine that is going to appear in March 2022.
Object-Centric and Memory-Guided Normality Reconstruction for Video Anomaly Detection

This paper addresses video anomaly detection problem for videosurveillance. Due to the inherent rarity and heterogeneity of abnormal events, the problem is viewed as a normality modeling strategy, in which our model learns object-centric normal patterns without seeing anomalous samples during training. The main contributions consist in coupling pre-trained object-level action features prototypes with a cosine distance-based anomaly estimation function, therefore extending previous methods by introducing additional constraints to the mainstream reconstruction-based strategy. Our framework leverages both appearance and motion information to learn object-level behavior and captures prototypical patterns within a memory module. Experiments on several well-known datasets demonstrate the effectiveness of our method as it outperforms current state-of-the-art on most relevant spatio-temporal evaluation metrics.
Topic modeling and sentiment analysis of Chinese people’s attitudes toward volunteerism amid the COVID-19 pandemic

The COVID-19 pandemic has created an urgent need for volunteers to complement overwhelmed public health systems. This study aims to explore Chinese people’s attitudes toward volunteerism amid the COVID-19 pandemic. To this end, we identify the latent topics in volunteerism-related microblogs on Weibo, the Chinese equivalent of Twitter using the topic modeling analysis via Latent Dirichlet Allocation (LDA). To further investigate the public sentiment toward the topics generated by LDA, we also conducted sentiment analysis on the sample posts using the open-source natural language processing (NLP) technique from Baidu. Through an in-depth analysis of 91,933 Weibo posts, this study captures 10 topics that are, in turn, distributed into five factors associated with volunteerism in China as motive fulfillment ( n = 31,661, 34.44%), fear of COVID-19 ( n = 22,597, 24.58%), individual characteristic ( n = 17,688, 19.24%), government support ( n = 15,482, 16.84%), and community effect ( n = 4,505, 4.90%). The results show that motive fulfillment, government support, and community effect are the factors that could enhance positive attitudes toward volunteerism since the topics related to these factors report high proportions of positive emotion. Fear of COVID-19 and individual characteristic are the factors inducing negative sentiment toward volunteerism as the topics related to these factors show relatively high proportions of negative emotion. The provision of tailored strategies based on the factors could potentially enhance Chinese people’s willingness to participate in volunteer activities during the COVID-19 pandemic.
Granular Content Distribution for IoT Remote Sensing Data Supporting Privacy Preservation

Facilitated by the Internet of Things (IoT) and diverse IoT devices, remote sensing data are evolving into the multimedia era with an expanding data scale. Massive remote sensing data are collected by IoT devices to monitor environments and human activities. Because IoT devices are involved in the data collection, there are probably private data contained in the collected remote sensing data, such as the device owner information and the precise location. Therefore, when data analysts, researchers, and other stakeholders require remote sensing data from numerous IoT devices for different analyses and investigations, how to distribute massive remote sensing data efficiently and regulate different people to view different parts of the distributed remote sensing data is a challenge to be addressed. Many general solutions rely on granular access control for content distribution but do not consider the low computational efficiency caused by the huge file size of the remote sensing data or certain IoT devices only have a constrained computational performance. Therefore, we propose a new granular content distribution scheme, which is more lightweight and practical for the distribution of multimedia remote sensing data with the consideration of the large data size to avoid complicated operations to the data. Furthermore, a dual data integrity check (hash summary and watermark) designed in our scheme can detect tampering or forgery from encrypted remote sensing data before decrypting it and validate it again after decryption. The security analyses and experimental results manifest that our new scheme can maintain high computational efficiency and block tampering and forgery during the granular content distribution for IoT remote sensing data.
Investigating the impact of fourth-generation (4G) mobile technologies on student academic performance

The fourth generation (4G) wireless communications and technologies has improved our way of life in many areas, such as health, communications, and education. This study sought to explore the impact of fourth-generation mobile technologies in education; more specifically, its impact on university students’ academic performance. The study used the quantitative research method using a closed-ended questionnaire to determine the impact of fourth-generation technologies on students' academic performance in a higher education institution (HEI). The study measured and understood the students’ performance through the research questions and specifically the question part 3 (question 4) which concentrated on the impact of 4G mobile technology on students’ academic performance. It was found that 4G mobile technologies improve students’ mobility, Internet quality, faster Internet connection, and online resources than going to the library. It also improves communication with other students and sharing of academic materials which directly impact students’ academic performance and many more.
Code, quality, and process metrics in graduated and retired ASFI projects

Recent work on open source sustainability shows that successful trajectories of projects in the Apache Software Foundation Incubator (ASFI) can be predicted early on, using a set of socio-technical measures. Because OSS projects are socio-technical systems centered around code artifacts, we hypothesize that sustainable projects may exhibit different code and process patterns than unsustainable ones, and that those patterns can grow more apparent as projects evolve over time. Here we studied the code and coding processes of over 200 ASFI projects, and found that ASFI graduated projects have different patterns of code quality and complexity than retired ones. Likewise for the coding processes – e.g., feature commits or bug-fixing commits are correlated with project graduation success. We find that minor contributors and major contributors (who contribute <5%, respectively >=95% commits) associate with graduation outcomes, implying that having also developers who contribute fewer commits are important for a project's success. This study provides evidence that OSS projects, especially nascent ones, can benefit from introspection and instrumentation using multidimensional modeling of the whole system, including code, processes, and code quality measures, and how they are interconnected over time.
Underwater Object Detection Based on Improved SSD with Convolutional Block Attention

With the ever-increasing demand for marine products with high nutritional value in the sea, rapid and accurate detection of target organisms is highly desired. In this context, a novel underwater object detection algorithm based on improved Single Shot MultiBox Detector (SSD) is proposed to improve the detection accuracy and real-time performance in complex underwater environment. The proposed algorithm employs ResNeXt-50 instead of VGGNet-16 as the backbone network to improve the ability to obtain details and position information of the target object. To be more specific, a channel-spatial attention mechanism is incorporated in the deep layers of the improved XC-SSD model to enhance semantic information of high-level feature maps and reduce missed detection rate and false detection rate. Comprehensive experiments on the URPC-2019 dataset show that the detection accuracy and speed of the proposed algorithm are 79.76% and 18.95 FPS, respectively, which are 3.49% and 2.8 FPS higher than the original SSD algorithm. Experimental results clearly demonstrate that the proposed algorithm is highly efficient and suitable for real-time object detection.
Latent Space Semantic Supervision Based on Knowledge Distillation for Cross-Modal Retrieval

As an important field in information retrieval, fine-grained cross-modal retrieval has received great attentions from researchers. Existing fine-grained cross-modal retrieval methods made several improvements in capturing the fine-grained interplay between vision and language, failing to consider the fine-grained correspondences between the features in the image latent space and the text latent space respectively, which may lead to inaccurate inference of intra-modal relations or false alignment of cross-modal information. Considering that object detection can get the fine-grained correspondences of image region features and the corresponding semantic features, this paper proposed a novel latent space semantic supervision model based on knowledge distillation (L3S-KD), which trains classifiers supervised by the fine-grained correspondences obtained from an object detection model by using knowledge distillation for image latent space fine-grained alignment, and by the labels of objects and attributes for text latent space fine-grained alignment. Compared with existing fine-grained correspondence matching methods, L3S-KD can learn more accurate semantic similarities for local fragments in image-text pairs. Extensive experiments on MS-COCO and Flickr30K datasets demonstrate that the L3S-KD model consistently outperforms state-of-the-art methods for image-text matching.
A Preprocessing Perspective for Quantum Machine Learning Classification Advantage in Finance Using NISQ Algorithms

Quantum Machine Learning (QML) has not yet demonstrated extensively and clearly its advantages compared to the classical machine learning approach. So far, there are only specific cases where some quantum-inspired techniques have achieved small incremental advantages, and a few experimental cases in hybrid quantum computing are promising, considering a mid-term future (not taking into account the achievements purely associated with optimization using quantum-classical algorithms). The current quantum computers are noisy and have few qubits to test, making it difficult to demonstrate the current and potential quantum advantage of QML methods. This study shows that we can achieve better classical encoding and performance of quantum classifiers by using Linear Discriminant Analysis (LDA) during the data preprocessing step. As a result, the Variational Quantum Algorithm (VQA) shows a gain of performance in balanced accuracy with the LDA technique and outperforms baseline classical classifiers.
Multimodal Human Activity Recognition for Smart Healthcare Applications

Human Activity Recognition (HAR) has emerged as a potential research topic for smart healthcare owing to the fast growth of wearable and smart devices in recent years. The significant applications of HAR in ambient assisted living environments include monitoring the daily activities of elderly and cognitively impaired individuals to assist them by observing their health status. In this research, we present a deep learning-based fusion approach for multimodal HAR that fuses the different modalities of data to obtain robust outcomes. Here, Convolutional Neural Networks (CNNs) retrieve the high-level attributes from the image data, and the Convolutional Long Short Term Memory (ConvLSTM) is utilized to capture significant patterns from the multi-sensory data. Finally, the extracted features from the modalities are fused through self-attention mechanisms that enhance the relevant activity data and inhibit the superfluous and possibly confusing information by measuring their compatibility. Lastly, extensive tests have been performed to measure the efficiency and robustness of the developed fusion approach using the UP-Fall detection dataset. It is evident from the experimental findings that the proposed fusion technique outperforms the existing state-of-the-art and achieves relatively better performance.
Provably throttling SQLI using an enciphering query and secure matching

Web applications, which dominate the internet, act as communication media between customers and service providers. Web applications are an internet innovation that provide customer services such as e-banking, e-commerce and e-booking. Developing web applications has become increasingly complicated because of security threats and service issues that involve valuable information. Attack methods such as structured query language (SQL) injection insert malicious code within user input data requests to gain unauthorised access, and then the attacker targets a database to manipulate information. In this paper, we propose a prevention method against SQL injection attacks through cryptography and searchable encryption. The proposed method uses a cryptography technique to encrypt all database information, where each piece of user information is encrypted with a separate key. The rest of the database information is ciphered with secret keys, and a searchable encryption technique is used for other database operations to preserve privacy. The login process compares the ciphered username from the database and user entry to authenticate the user. The proposed method is implemented on the PHP and MySQL databases, which are open-source applications. The results show efficient prevention of SQL injection, and the database remains protected against SQL injection attacks
Adaptive social recommendation combined with the multi-domain influence

Social relationships help to model user's potential preferences and improve recommendation accuracy. In social recommendation, user decision-making will be affected by his own historical interaction items and social friends. Most social recommendations consider these two aspects separately, but users are also driven by both when making decisions. To address the above issue, this paper proposes an adaptive social recommendation method based on attention mechanism, Adaptive Social Recommendation combine with Multi-domain influence (ASRM). Specifically, we defined three domains, item domain, social domain, and common domain to measure the possible impact of social relations and item interactions on users. We propose an adaptive attention module to simulate users being influenced by social relations and historical interaction items in these three domains, and obtain user's potential preferences. In addition, we jointly optimize by calculating the loss of each domain, resulting in more accurate recommendations. A large number of experiments on three real datasets have demonstrated the effectiveness of our method. The code is available at https://github.com/qinkaili/ASRM.
An autonomous decision-making framework for gait recognition systems against adversarial attack using reinforcement learning

Gait identification based on Deep Learning (DL) techniques has recently emerged as biometric technology for surveillance. We leveraged the vulnerabilities and decision-making abilities of the DL model in gait-based autonomous surveillance systems when attackers have no access to underlying model gradients/structures using a patch-based black-box adversarial attack with Reinforcement Learning (RL). These automated surveillance systems are secured, blocking the attacker’s access. Therefore, the attack can be conducted in an RL framework where the agent’s goal is determining the optimal image location, causing the model to perform incorrectly when perturbed with random pixels. Furthermore, the proposed adversarial attack presents encouraging results (maximum success rate = 77.59%). Researchers should explore system resilience scenarios (e.g., when attackers have no system access) before using these models in surveillance applications.
Deep Transformer Language Models for Arabic Text Summarization: A Comparison Study

Large text documents are sometimes challenging to understand and time-consuming to extract vital information from. These issues are addressed by automatic text summarizing techniques, which condense lengthy texts while preserving their key information. Thus, the development of automatic summarization systems capable of fulfilling the ever-increasing demands of textual data becomes of utmost importance. It is even more vital with complex natural languages. This study explores five State-Of-The-Art (SOTA) Arabic deep Transformer-based Language Models (TLMs) in the task of text summarization by adapting various text summarization datasets dedicated to Arabic. A comparison against deep learning and machine learning-based baseline models has also been conducted. Experimental results reveal the superiority of TLMs, specifically the PEAGASUS family, against the baseline approaches, with an average F1-score of 90% on several benchmark datasets.
The impact of virtual agents’ multimodal communication on brain activity and cognitive load in Virtual Reality

Related research has shown that collaborating with Intelligent Virtual Agents (IVAs) embodied in Augmented Reality (AR) or Virtual Reality (VR) can improve task performance and reduce task load. Human cognition and behaviors are controlled by brain activities, which can be captured and reflected by Electroencephalogram (EEG) signals. However, little research has been done to understand users’ cognition and behaviors using EEG while interacting with IVAs embodied in AR and VR environments. In this paper, we investigate the impact of the virtual agent’s multimodal communication in VR on users’ EEG signals as measured by alpha band power. We develop a desert survival game where the participants make decisions collaboratively with the virtual agent in VR. We evaluate three different communication methods based on a within-subject pilot study: 1) a Voice-only Agent, 2) an Embodied Agent with speech and gaze, and 3) a Gestural Agent with a gesture pointing at the object while talking about it. No significant difference was found in the EEG alpha band power. However, the alpha band ERD/ERS calculated around the moment when the virtual agent started speaking indicated providing a virtual body for the sudden speech could avoid the abrupt attentional demand when the agent started speaking. Moreover, a sudden gesture coupled with the speech induced more attentional demands, even though the speech was matched with the virtual body. This work is the first to explore the impact of IVAs’ interaction methods in VR on users’ brain activity, and our findings contribute to the IVAs interaction design.
Unleashing the power of pseudo-code for binary code similarity analysis

Abstract Code similarity analysis has become more popular due to its significant applicantions, including vulnerability detection, malware detection, and patch analysis. Since the source code of the software is difficult to obtain under most circumstances, binary-level code similarity analysis (BCSA) has been paid much attention to. In recent years, many BCSA studies incorporating AI techniques focus on deriving semantic information from binary functions with code representations such as assembly code, intermediate representations, and control flow graphs to measure the similarity. However, due to the impacts of different compilers, architectures, and obfuscations, binaries compiled from the same source code may vary considerably, which becomes the major obstacle for these works to obtain robust features. In this paper, we propose a solution, named UPPC (Unleashing the Power of Pseudo-code), which leverages the pseudo-code of binary function as input, to address the binary code similarity analysis challenge, since pseudo-code has higher abstraction and is platform-independent compared to binary instructions. UPPC selectively inlines the functions to capture the full function semantics across different compiler optimization levels and uses a deep pyramidal convolutional neural network to obtain the semantic embedding of the function. We evaluated UPPC on a data set containing vulnerabilities and a data set including different architectures (X86, ARM), different optimization options (O0-O3), different compilers (GCC, Clang), and four obfuscation strategies. The experimental results show that the accuracy of UPPC in function search is 33.2% higher than that of existing methods.