Reviewerly/exHarmony huggingface dataset:
Vehicle State Monitoring and Fault Detection System for Unmanned Ground Vehicles (UGV) using Markov Models

This research presents a novel fault detection and diagnostics system for unmanned ground vehicles (UGVs) by combining Markov models representing the vehicle's navigation, kinematic behavior, and vehicle dynamics systems. Existing studies do not specifically address the challenges related to UGVs and their complex subsystems or the incorporation of weather and environmental condition data. The proposed system leverages environmental and weather condition data to monitor the UGV's state and detect anomalies in its behavior. By predicting the probability of faults such as collisions, sensor damage, and other malfunctions, the system aims to enhance the safety, reliability, and performance of UGVs. The research will demonstrate the effectiveness of the proposed methodology through case studies and performance evaluation, highlighting its potential application in various real-world scenarios. This work contributes to the ongoing research in prognostics and health management, particularly for autonomous systems, by providing a new approach to fault detection and diagnostics in UGVs.
An Efficient Hardware Implementation of CNN Generic Processor for FPGA

Convolutional Neural Networks (CNNs) have gained significant popularity in recent years due to their exceptional performance in classification tasks. However, the large number of computational operations that are needed for these models can limit their practicality. Field Programmable Gate Arrays (FPGAs) can be a promising solution for accelerating CNNs as they give the ability to create custom hardware designs. In this paper, we present an efficient hardware implementation for the CNN processor, in which a generic processor is designed and implemented to accelerate CNNs on hardware with the minimum number of operations and hardware resources. Our target is to reduce the number of needed memory and Digital Signal Processor blocks (DSP) to allow full implementation in FPGA on-chip memory. Multiple optimization techniques were used, such as loop unrolling, weights pruning, and quantization, to achieve our target. The processor is evaluated on the LeNet5 architecture using the MNIST dataset on the Vertix-7 FPGA, which showed a 17.9x reduction in weights size and very competitive performance-to-resources efficiency when compared with literature.
Extractive Text Summarization of Long Documents Using Word and Sentence Encoding

Automatic Text Summarization (ATS) for long documents is a very challenging task. A long document includes more than one topic, so it is required to construct a summary that covers the most important contextual information from the different topics in the input document. There are many ATS techniques, but the produced summaries are much less accurate than the human summaries. This paper proposes a supervised extractive Neural-based summarization model that focuses on long documents summarization by integrating both the local and global contexts of the document. The proposed system employs two bidirectional Gated Recurrent Units (GRUs) as word and sentence encoders, respectively, and uses Word2Vec embeddings as word representation. Two ensembles are proposed by applying the average ensemble method to the proposed system with two other Neural-based systems. The proposed system uses the PubMed dataset to evaluate the two ensembles using the ROUGE evaluation metrics. The results of the evaluation show that there are promising and remarkable improvements of 0.14% and 0.97% for ROUGE-1, 0.33% and 1.12% for ROUGE-2, and 0.25% for ROUGE-L.
Detection of Cyberbullying in Arabic using Machine Learning and ChatGPT

Cyberbullying is one of the complicated problems in Arabic communities. Cyberbullying detection is the main key to control racism and hate speech over the Internet especially on social media. Detection of cyberbullying in Arabic is challenging due to the variety of expressions and different Arabic dialects. Many researches discussed this topic using only one dialect and achieved good results. In this paper, we consolidated datasets with multiple Arabic dialects from different social media platforms. Text pre-processing enhanced the performance of the used models. We classified the comments in the dataset using Naive Bayes model, AraBERT model and ChatGPT for comparison purposes. The experiments showed that AraBERT outperformed the other classifiers for our multi-dialect dataset showing its potential for effectively monitoring Arabic content over the Internet.
Dynamic Causal Modelling and Predictive Analysis for the COVID-19 Pandemic

Amidst the global rampage of the Coronavirus disease 2019 (COVID-19), discussions regarding COVID-19-related topics have gained significant attention from netizens worldwide. The continuous co-evolution and interaction of public sentiments towards these topics present a complex phenomenon. Uncovering the underlying causal relationships among these topics holds practical significance in promptly understanding and accurately predicting public sentiments and opinions. To address this, we present a causality-driven, co-evolutionary, and interpretable framework called Dynamic Causal Modelling and Prediction (DCMP). By detecting and quantifying the nature of causal relationships, DCMP surpasses traditional competitive methods. Moreover, the results obtained from DCMP offer comprehensive interpretations from both trend perspective and content perspective of online topics pertaining to the COVID-19 pandemic. These findings provide valuable insights for policymakers and researchers to comprehend the concealed mechanisms governing sentiment dynamics related to COVID-19 topics and devise effective policies.
An Analytical Study of Visual Attention Behavior in Viewing Panoramic Video

Panoramic video offers an immersive viewing experience in which viewers can actively explore 360-degree motion pictures and engage with the narrative. Studying user visual attention behavior could help us to have a better understanding of video processing, semantic learning, and coding in 360-degree videos. In this paper, we developed two attention visualization toolkits, visual saliency map and semantic attention annotation, for collecting ROI data. The practice-based analytical methodology is employed to discuss user behavior while viewing panoramic shorts. We gathered viewing behavior data from 23 participants and visualized attention saliency to analyze the viewers' visual attention behavior and narrative cognition process. According to the collected data, we summarize attention distribution regulations and derive practical insights into the aspects of learning decision-making for panorama production.
An Overview of 3GPP Standardization for Extended Reality (XR) in 5G and Beyond

In recent years, aiming to enhance and extend user experiences beyond the real world, Extended Reality (XR) has emerged to become a new paradigm that enables a plethora of applications [1], e.g., online gaming, online conferencing, social media, etc. XR refers to the human-machine interactions that combine real and virtual environments with the support of computing/communications technologies and wearable devices. The XR content is generated by providers or other users, including audio, video and other metadata. In general, the generated XR content is transmitted to XR devices and rendered into XR scenes (i.e., to generate an image from a 2D or 3D model by means of a computer program), where users can experience a hybrid experience of the real and virtual worlds.
Unleashing Immersive Experiences: The Power of Gesture-Based VR Interaction

This forum features game-practitioner perspectives on the interaction design process, techniques, and evaluation involved in creating playful experiences. We focus on how technology advancement, infrastructure, and constraints shape the player experience. --- Pejman Mirza-Babaei, Editor
Digital Images Authentication Technique Based on Multi-Blocks DCT and Singular Value Decomposition

The use of digital photographs and their uses in contemporary life have undergone a revolutionary change. It is now quite simple to change the content of an image without leaving any noticeable traces due to the widely available image editing software. Manipulated images that are generated by image splicing are visually hard to notice. Several methods have previously been developed to detect image splicing. Existing algorithms have certain intrinsic weaknesses such as large false-positive rates, poor precision, and features with high dimension. This work introduces a method to detect image splicing tampering in digital images. This algorithm is based on applying the Singular Value Decomposition (SVD) as a feature extraction to the coefficients of the Multi-Block Discrete Cosine Transform (MBDCT). In addition, four types of statistical features are computed and then merged in a feature vector. SVM is employed for classification. Two publicly available image datasets are used to assess the proposed technique: CASIA v1.0 and CASIA v2.0. The results of the studies demonstrate how effective the suggested algorithm is in identifying spliced images.
Multi-Modal Multi-Task Joint 2D and 3D Scene Perception and Localization

The talk will cover several important computer vision tasks within the context of visual 2D/3D scene understanding, including scene depth estimation, joint learning of scene depth and scene parsing, and visual deep SLAM. Specifically, the talk presents how to design effective deep learning models to deal with these challenging problems while using the least effort of human annotations, including the introduction of a transformer network structure for deep multi-task learning, a weakly supervised model for scene parsing, and self-supervised joint learning of scene depth, camera pose, and moving object discovery. The talk will also cover explorations of large-scale implicit scene modeling in self-supervised manners which benefits various human-centric tasks and applications.
Training Data Optimization in Human-Centric Analysis

Training data are a pillar in computer vision applications. While existing works typically assume fixed training sets, I will discuss how training data optimization complements and benefis state-of-the-art computer vision models. In particular, this talk focuses on a few human-centric applications: person re-identification, multi-object tracking , and micro-expression recognition. For person re-identification, I will discuss how to search for training data that match the target domain distribution. For tracking, I will introduce our research utilizing synthetic human data to learn association policies. Finally, I will introduce a protocol to synthesize large amounts of micro-expression training images leading to a state-of-the-art recognition system. I conclude the talk with trends and new problems in data-centric computer vision.
Boosting Domain-Specific Question Answering Through Weakly Supervised Self-Training

Question answering systems have emerged as an important research area in the field of natural language processing, enabling the provision of accurate answers to user queries in a more efficient and user-friendly manner. These systems have much practical significance especially in security-related applications, where intelligence analysts can easily access pertinent information from diverse sources to accelerate the decision-making process. In the context of security-related scenarios, users tend to interact with a question answering system with specific domain-oriented purposes. However, most existing question answering systems focus on open-domain situations with abundant labeled data as well as structured data, and domain-specific question answering methods are in urgent need. Domain-specific question answering faces the challenge of the low-resource issue caused by data scarcity. To address this challenge, in this paper, we propose a weakly supervised self-training method for domain-specific question answering based on the Retriever-Reader framework. For the retriever module, during the self-training process, we develop two strategies for generating pseudo-labels to augment the labeled dataset, including high confidence sampling and random negative sampling. For the reader module, we adopt the pre-trained language model and fine-tune the generative reader using limited labeled datasets. To evaluate our proposed method, we construct the first Chinese financial question answering dataset of textual document. Experimental results demonstrate that our proposed method can significantly improve the performances of the baseline method through the self-training process.
Learning to Listen and Listening to Learn: Spoofed Audio Detection Through Linguistic Data Augmentation

Spoofed audio, both human or machine generated, causes deception and disinformation and as such is a societal challenge. This study advances the detection of spoofed audio through a novel approach that augments knowledge about audio data by incorporating linguistic information. Using perceptual methods, for English audio samples, experts in sociolinguistics listened for audio cues, and used binary labels to indicate the perceived authenticity of a set of speech samples, based on phonetic and phonological features that occur frequently in spoken English. These Expert Defined Linguistic Features (EDLFs) were then used in supervised spoofed audio detection methods to augment AI models. An ensemble method based on multi-domain features both from the audio data itself and the EDLFs was also created to evaluate the spoofed audio detection, and to demonstrate how EDLFs can improve traditional methods of spoofed audio detection. We found that augmenting the audio data with expertinformed linguistic annotation increased the accuracy of spoofed audio detection significantly in both the training and testing datasets across the evaluated single and ensemble models. Our findings indicate the promising avenue of augmenting audio data with perceptual linguistic techniques, as a method of human discernment, to enhance AI-based approaches for spoofed audio detection. These features also establish a foundation for direct linguistic annotations on new audio clips for robust spoofed audio detection.
Adversarial Topic-Aware Memory Network for Cross-Lingual Stance Detection

Stance detection is an important research area in social media analytics and text mining and has a number of security-related applications, which aims to identify the user's viewpoint towards specific targets in domains such as social security, politics and public events. Previous research on stance detection has mainly focused on monolingual scenarios with the limited number of targets, while little attention was paid to cross-lingual stance detection. In contrast to the abundant labeled data in source language (typically English), the labeled data are often scarce in the non-English target language. Meanwhile, the diverse expressions of target further complicate the task and bring additional challenges. In this paper, we focus on cross-lingual stance detection in practical applications, where target is generally expressed as a topic and no labeled data are available for the target language. To tackle the above challenges, we propose an adversarial topic-aware memory network (ATOM) for cross-lingual stance detection. Specifically, our method first mines the generalized topic representations across source and target languages and utilizes them as the guidance to transfer knowledge from the high-resource source language to the low-resource target language. We further develop an iterative memory network to facilitate knowledge transfer across languages, which adaptively generates language-invariant topic-aware clues via adversarial training. Experimental results on three multilingual datasets in the politics domain demonstrate the effectiveness of our proposed method.
Functas Usability for Human Activity Recognition using Wearable Sensor Data

Recent advancements in data science have introduced implicit neural representations as a powerful approach for learning complex, high-dimensional functions, bypassing the need for explicit equations or manual feature engineering. In this paper, we present our research on employing the weights of these implicit neural representations to characterize and classify batches of data, referred to as 'functas.' This approach eliminates the need for manual feature engineering on raw data. Specifically, we showcase the efficacy of the 'functas' method in the domain of human activity recognition, utilizing output data from sensors such as accelerometers and gyroscopes. Our results demonstrate the promising potential of the 'functas' approach, suggesting a potential shift in the paradigm of data science methodologies.
Not So Atomic Payments: A Supply Chain Example

The advent of blockchain as a technology is revolutionizing the supply chain domain with its tamper-proof and traceable architecture. However, the blockchain must be fed with accurate data to actualize its immutability, especially in supply chains where data origin is outside the blockchain. When the data sources (e.g., IoT devices) are managed by the same entity that intends to prove the proposition built on top of those data causes a conflict of interest and undermines the data’s credibility. Therefore, the independence of data oracles is crucial, and it allows supply chain stakeholders to focus on their work as well. Using the payment channel network (PCN) concept, we propose a low-cost payment protocol for supply chains with an independent and trusted data oracle environment. It ensures the atomicity of the end-to-end payment from a consumer to the manufacturer, including other participants and data oracles. Further, if the primary transaction fails, the Oracle environment is still compensated regardless, and the consumer is not held liable for that payment. We present how to achieve this unique type of atomicity requirement and discuss its potential, security, and future direction.
A Deep Reinforcement Learning Perspective on Adaptive Federated Dropout

Federated learning is a distributed learning solution that achieves high-quality machine learning models while ensuring privacy and collaboration among various end devices. However, different kinds of end devices can lead to an unstable training process due to limited and dynamic communication and computation resources. Federated Dropout is an important technique for mitigating such resource bottlenecks by randomly removing a fixed percentage of activations of model components. However, most of the existing FL dropout methods cannot effectively utilize the distinguished characteristics of different clients or fail to automatically adapt to end device heterogeneity. In this paper, we thus propose Adaptive Federated Dropout with Reinforcement Learning (AFD-RL). AFD-RL employs reinforcement learning technology to adapt to the clients’ unbalanced computation and communication resources and provide a personalized dropout strategy for each client. By automatically selecting suitable model components for each client in each communication round, AFD-RL achieves better accuracy with fewer communication rounds. Extensive experiments validate the effectiveness of the proposed approach in improving training efficiency and inference accuracy.
Time-Series Misalignment Aware DNN Adversarial Attacks for Connected Autonomous Vehicles

Connected Autonomous Vehicle (CAV) technique allows each autonomous vehicle to analyze time-series driving signals (e.g., speed, steering wheel angle) sent by its nearby vehicle using the Deep Neural Network (DNN). This scenario could attract adversarial attacks on the DNNs. To fool the victim's DNN model to make misclassification in the black-box scenario, an attacker generates a small perturbation series on time-series signals to generate an adversarial input. However, as the attacker does not know the start time stamp of each input to the DNN, the start point of the perturbation may not match that of real target input and the misalignment can make the attack unsuccessful. To address this problem, we propose three time-series Misalignment Aware DNN Adversarial Attacks (MA <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> ) for CAVs. The first two methods estimate the start point of the target input, while the third method creates a perturbation for each maneuver that misclassifies all inputs with all time stamps as the start point. Both of our experimental evaluations based on real driving data and real experiments show that MA <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> produces up to 63.49% higher success rate, 56% less perturbation generation time, and 63.16% less perturbation amount compared with previous attack method. Our source code is publicly available.
Designing for Uncertain Futures: An Anticipatory Approach

research-article Open Access Share on Designing for Uncertain Futures: An Anticipatory Approach Authors: Tim Moesgen Aalto University Aalto UniversitySearch about this author , Antti Salovaara Aalto University Aalto UniversitySearch about this author , Felix A. Epp Aalto University Aalto UniversitySearch about this author , Camilo Sanchez Aalto University Aalto UniversitySearch about this author Authors Info & Claims InteractionsVolume 30Issue 6November - December 2023pp 36–41https://doi.org/10.1145/3624698Published:01 November 2023Publication History 0citation0DownloadsMetricsTotal Citations0Total Downloads0Last 12 Months0Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteView all FormatsPDF
Can we integrate color and depth information for richer captions?

While image captioning models have successfully produced high-quality descriptions, most of the research has focused on providing a single sentence for 2D images. This study explores whether adding depth information to RGB images can improve captioning and generate better descriptions. We propose a Transformer-based encoder-decoder model to generate a multi-sentence description of a 3D scene. Our framework takes an RGB image and its corresponding depth map as input and combines them to create a more comprehensive understanding of the input scene. We investigated various fusion strategies for combining RGB and depth images. We conducted experiments using the NYU-v2 dataset. However, while working with the NYU-v2 dataset, we discovered inconsistent labeling, which undermines the benefit of utilizing depth information to improve the captioning process. As a result, the results were considerably worse than when using RGB images alone. Therefore, we suggest a more accurate and consistent version of the NYU-v2 dataset. Our results demonstrate that the proposed framework effectively benefits from depth information and produces better captions.