OpenRaiser/Intern-Atlas huggingface dataset:
Leveraging driver vehicle and environment interaction: Machine learning using driver monitoring cameras to detect drunk driving
Venue: arXiv cs.HC 2023
Year: 2023

Excessive alcohol consumption causes disability and death. Digital interventions are promising means to promote behavioral change and thus prevent alcohol-related harm, especially in critical moments such as driving. This requires real-time information on a person's blood alcohol concentration (BAC). Here, we develop an in-vehicle machine learning system to predict critical BAC levels. Our system leverages driver monitoring cameras mandated in numerous countries worldwide. We evaluate our system with n=30 participants in an interventional simulator study. Our system reliably detects driving under any alcohol influence (area under the receiver operating characteristic curve [AUROC] 0.88) and driving above the WHO recommended limit of 0.05g/dL BAC (AUROC 0.79). Model inspection reveals reliance on pathophysiological effects associated with alcohol consumption. To our knowledge, we are the first to rigorously evaluate the use of driver monitoring cameras for detecting drunk driving. Our results highlight the potential of driver monitoring cameras and enable next-generation drunk driver interaction preventing alcohol-related harm.
REDAffectiveLM: Leveraging Affect Enriched Embedding and Transformer-based Neural Language Model for Readers' Emotion Detection
Venue: arXiv cs.CL 2023
Year: 2023

Technological advancements in web platforms allow people to express and share emotions towards textual write-ups written and shared by others. This brings about different interesting domains for analysis; emotion expressed by the writer and emotion elicited from the readers. In this paper, we propose a novel approach for Readers' Emotion Detection from short-text documents using a deep learning model called REDAffectiveLM. Within state-of-the-art NLP tasks, it is well understood that utilizing context-specific representations from transformer-based pre-trained language models helps achieve improved performance. Within this affective computing task, we explore how incorporating affective information can further enhance performance. Towards this, we leverage context-specific and affect enriched representations by using a transformer-based pre-trained language model in tandem with affect enriched Bi-LSTM+Attention. For empirical evaluation, we procure a new dataset REN-20k, besides using RENh-4k and SemEval-2007. We evaluate the performance of our REDAffectiveLM rigorously across these datasets, against a vast set of state-of-the-art baselines, where our model consistently outperforms baselines and obtains statistically significant results. Our results establish that utilizing affect enriched representation along with context-specific representation within a neural architecture can considerably enhance readers' emotion detection. Since the impact of affect enrichment specifically in readers' emotion detection isn't well explored, we conduct a detailed analysis over affect enriched Bi-LSTM+Attention using qualitative and quantitative model behavior evaluation techniques. We observe that compared to conventional semantic embedding, affect enriched embedding increases ability of the network to effectively identify and assign weightage to key terms responsible for readers' emotion detection.
My Actions Speak Louder Than Your Words: When User Behavior Predicts Their Beliefs about Agents' Attributes
Venue: arXiv cs.HC 2023
Year: 2023

An implicit expectation of asking users to rate agents, such as an AI decision-aid, is that they will use only relevant information -- ask them about an agent's benevolence, and they should consider whether or not it was kind. Behavioral science, however, suggests that people sometimes use irrelevant information. We identify an instance of this phenomenon, where users who experience better outcomes in a human-agent interaction systematically rated the agent as having better abilities, being more benevolent, and exhibiting greater integrity in a post hoc assessment than users who experienced worse outcome -- which were the result of their own behavior -- with the same agent. Our analyses suggest the need for augmentation of models so that they account for such biased perceptions as well as mechanisms so that agents can detect and even actively work to correct this and similar biases of users.
E$^3$Pose: Energy-Efficient Edge-assisted Multi-camera System for Multi-human 3D Pose Estimation
Venue: arXiv cs.CV 2023
Year: 2023

Multi-human 3D pose estimation plays a key role in establishing a seamless connection between the real world and the virtual world. Recent efforts adopted a two-stage framework that first builds 2D pose estimations in multiple camera views from different perspectives and then synthesizes them into 3D poses. However, the focus has largely been on developing new computer vision algorithms on the offline video datasets without much consideration on the energy constraints in real-world systems with flexibly-deployed and battery-powered cameras. In this paper, we propose an energy-efficient edge-assisted multiple-camera system, dubbed E$^3$Pose, for real-time multi-human 3D pose estimation, based on the key idea of adaptive camera selection. Instead of always employing all available cameras to perform 2D pose estimations as in the existing works, E$^3$Pose selects only a subset of cameras depending on their camera view qualities in terms of occlusion and energy states in an adaptive manner, thereby reducing the energy consumption (which translates to extended battery lifetime) and improving the estimation accuracy. To achieve this goal, E$^3$Pose incorporates an attention-based LSTM to predict the occlusion information of each camera view and guide camera selection before cameras are selected to process the images of a scene, and runs a camera selection algorithm based on the Lyapunov optimization framework to make long-term adaptive selection decisions. We build a prototype of E$^3$Pose on a 5-camera testbed, demonstrate its feasibility and evaluate its performance. Our results show that a significant energy saving (up to 31.21%) can be achieved while maintaining a high 3D pose estimation accuracy comparable to state-of-the-art methods.
Nichtverbales Verhalten sozialer Roboter: Bewegungen, deren Bedeutung und die Technik dahinter
Venue: arXiv cs.RO 2023
Year: 2023

Nichtverbale Signale sind ein elementarer Bestandteil der menschlichen Kommunikation. Sie erfüllen eine Vielzahl von Funktionen bei der Klärung von Mehrdeutigkeiten, der subtilen Aushandlung von Rollen oder dem Ausdruck dessen, was im Inneren der Gesprächspartner vorgeht. Viele Studien mit sozial-interaktiven Robotern zeigen, dass vom Menschen inspirierte Bewegungsmuster ähnlich interpretiert werden wie die von realen Personen. Dieses Kapitel erläutert daher die wichtigsten Funktionen, welche die jeweiligen Bewegungsmuster in der Kommunikation erfüllen, und gibt einen Überblick darüber, wie sie auf Roboter übertragen werden können. -- Non-verbal signals are a fundamental part of human communication. They serve a variety of functions in clarifying ambiguities, subtly negotiating roles, or expressing what is going on inside the interlocutors. Many studies with socially-interactive robots show that human-inspired movement patterns are interpreted similarly to those of real people. This chapter therefore explains the most important functions that the respective movement patterns fulfill in communication and gives an overview of how they can be transferred to robots.
Champion Solution for the WSDM2023 Toloka VQA Challenge
Venue: arXiv cs.CV 2023
Year: 2023

In this report, we present our champion solution to the WSDM2023 Toloka Visual Question Answering (VQA) Challenge. Different from the common VQA and visual grounding (VG) tasks, this challenge involves a more complex scenario, i.e. inferring and locating the object implicitly specified by the given interrogative question. For this task, we leverage ViT-Adapter, a pre-training-free adapter network, to adapt multi-modal pre-trained Uni-Perceiver for better cross-modal localization. Our method ranks first on the leaderboard, achieving 77.5 and 76.347 IoU on public and private test sets, respectively. It shows that ViT-Adapter is also an effective paradigm for adapting the unified perception model to vision-language downstream tasks. Code and models will be released at https://github.com/czczup/ViT-Adapter/tree/main/wsdm2023.
Performance Study of YOLOv5 and Faster R-CNN for Autonomous Navigation around Non-Cooperative Targets
Venue: arXiv cs.RO 2023
Year: 2023

Autonomous navigation and path-planning around non-cooperative space objects is an enabling technology for on-orbit servicing and space debris removal systems. The navigation task includes the determination of target object motion, the identification of target object features suitable for grasping, and the identification of collision hazards and other keep-out zones. Given this knowledge, chaser spacecraft can be guided towards capture locations without damaging the target object or without unduly the operations of a servicing target by covering up solar arrays or communication antennas. One way to autonomously achieve target identification, characterization and feature recognition is by use of artificial intelligence algorithms. This paper discusses how the combination of cameras and machine learning algorithms can achieve the relative navigation task. The performance of two deep learning-based object detection algorithms, Faster Region-based Convolutional Neural Networks (R-CNN) and You Only Look Once (YOLOv5), is tested using experimental data obtained in formation flight simulations in the ORION Lab at Florida Institute of Technology. The simulation scenarios vary the yaw motion of the target object, the chaser approach trajectory, and the lighting conditions in order to test the algorithms in a wide range of realistic and performance limiting situations. The data analyzed include the mean average precision metrics in order to compare the performance of the object detectors. The paper discusses the path to implementing the feature recognition algorithms and towards integrating them into the spacecraft Guidance Navigation and Control system.
RIS-assisted Cell-Free MIMO with Dynamic Arrivals and Departures of Users: A Novel Network Stability Approach
Venue: arXiv cs.IT 2023
Year: 2023

Reconfigurable Intelligent Surfaces (RIS) have recently emerged as a hot research topic, being widely advocated as a candidate technology for next generation wireless communications. These surfaces passively alter the behavior of propagation environments enhancing the performance of wireless communication systems. In this paper, we study the use of RIS in cell-free multiple-input multiple-output (MIMO) setting where distributed service antennas, called Access Points (APs), simultaneously serve the users in the network. While most existing works focus on the physical layer improvements RIS carry, less attention has been paid to the impact of dynamic arrivals and departures of the users. In such a case, ensuring the stability of the network is the main goal. For that, we propose an optimization framework of the phase shifts, for which we derived a low-complexity solution. We then provide a theoretical analysis of the network stability and show that our framework stabilizes the network whenever it is possible. We also prove that a low complexity solution of our framework stabilizes a guaranteed fraction (higher than 78.5%) of the stability region. We provide also numerical results that corroborate the theoretical claims.
Energy Prediction using Federated Learning
Venue: arXiv cs.LG 2023
Year: 2023

In this work, we demonstrate the viability of using federated learning to successfully predict energy consumption as well as solar production for all households within a certain network using low-power and low-space consuming embedded devices. We also demonstrate our prediction performance improving over time without the need for sharing private consumer energy data. We simulate a system with four nodes using data for one year to show this.
MATT: Multimodal Attention Level Estimation for e-learning Platforms
Venue: arXiv cs.CV 2023
Year: 2023

This work presents a new multimodal system for remote attention level estimation based on multimodal face analysis. Our multimodal approach uses different parameters and signals obtained from the behavior and physiological processes that have been related to modeling cognitive load such as faces gestures (e.g., blink rate, facial actions units) and user actions (e.g., head pose, distance to the camera). The multimodal system uses the following modules based on Convolutional Neural Networks (CNNs): Eye blink detection, head pose estimation, facial landmark detection, and facial expression features. First, we individually evaluate the proposed modules in the task of estimating the student's attention level captured during online e-learning sessions. For that we trained binary classifiers (high or low attention) based on Support Vector Machines (SVM) for each module. Secondly, we find out to what extent multimodal score level fusion improves the attention level estimation. The mEBAL database is used in the experimental framework, a public multi-modal database for attention level estimation obtained in an e-learning environment that contains data from 38 users while conducting several e-learning tasks of variable difficulty (creating changes in student cognitive loads).
SPEC5G: A Dataset for 5G Cellular Network Protocol Analysis
Venue: arXiv cs.IR 2023
Year: 2023

5G is the 5th generation cellular network protocol. It is the state-of-the-art global wireless standard that enables an advanced kind of network designed to connect virtually everyone and everything with increased speed and reduced latency. Therefore, its development, analysis, and security are critical. However, all approaches to the 5G protocol development and security analysis, e.g., property extraction, protocol summarization, and semantic analysis of the protocol specifications and implementations are completely manual. To reduce such manual effort, in this paper, we curate SPEC5G the first-ever public 5G dataset for NLP research. The dataset contains 3,547,586 sentences with 134M words, from 13094 cellular network specifications and 13 online websites. By leveraging large-scale pre-trained language models that have achieved state-of-the-art results on NLP tasks, we use this dataset for security-related text classification and summarization. Security-related text classification can be used to extract relevant security-related properties for protocol testing. On the other hand, summarization can help developers and practitioners understand the high level of the protocol, which is itself a daunting task. Our results show the value of our 5G-centric dataset in 5G protocol analysis automation. We believe that SPEC5G will enable a new research direction into automatic analyses for the 5G cellular network protocol and numerous related downstream tasks. Our data and code are publicly available.
VeraSel: Verifiable Random Selection for Mixnets Construction
Venue: arXiv cs.CR 2023
Year: 2023

The security and performance of Mixnets depends on the trustworthiness of the Mixnodes in the network. The challenge is to limit the adversary's influence on which Mixnodes operate in the network. A trusted party (such as the Mixnet operator) may ensure this, however, it is a single point of failure in the event of corruption or coercion. Therefore, we study the problem of how to select a subset of Mixnodes in a distributed way for Mixnet construction. We present VeraSel, a scheme that enables Mixnodes to be chosen according to their weights in a distributed, unbiased, and verifiable fashion using Verifiable Random Functions (VRFs). It is shown that VeraSel enables any party to learn and verify which nodes has been selected based on the commitments and proofs generated by each Mixnode with VRF.
Applied Deep Learning to Identify and Localize Polyps from Endoscopic Images
Venue: arXiv cs.CV 2023
Year: 2023

Deep learning based neural networks have gained popularity for a variety of biomedical imaging applications. In the last few years several works have shown the use of these methods for colon cancer detection and the early results have been promising. These methods can potentially be utilized to assist doctor's and may help in identifying the number of lesions or abnormalities in a diagnosis session. From our literature survey we found out that there is a lack of publicly available labeled data. Thus, as part of this work, we have aimed at open sourcing a dataset which contains annotations of polyps and ulcers. This is the first dataset that's coming from India containing polyp and ulcer images. The dataset can be used for detection and classification tasks. We also evaluated our dataset with several popular deep learning object detection models that's trained on large publicly available datasets and found out empirically that the model trained on one dataset works well on our dataset that has data being captured in a different acquisition device.
GP-NAS-ensemble: a model for NAS Performance Prediction
Venue: arXiv cs.LG 2023
Year: 2023

It is of great significance to estimate the performance of a given model architecture without training in the application of Neural Architecture Search (NAS) as it may take a lot of time to evaluate the performance of an architecture. In this paper, a novel NAS framework called GP-NAS-ensemble is proposed to predict the performance of a neural network architecture with a small training dataset. We make several improvements on the GP-NAS model to make it share the advantage of ensemble learning methods. Our method ranks second in the CVPR2022 second lightweight NAS challenge performance prediction track.
Semantic-aware Contrastive Learning for Electroencephalography-to-Text Generation with Curriculum Learning
Venue: arXiv cs.HC 2023
Year: 2023

Electroencephalography-to-Text generation (EEG-to-Text), which aims to directly generate natural text from EEG signals has drawn increasing attention in recent years due to the enormous potential for Brain-computer interfaces (BCIs). However, the remarkable discrepancy between the subject-dependent EEG representation and the semantic-dependent text representation poses a great challenge to this task. To mitigate this challenge, we devise a Curriculum Semantic-aware Contrastive Learning strategy (C-SCL), which effectively re-calibrates the subject-dependent EEG representation to the semantic-dependent EEG representation, thus reducing the discrepancy. Specifically, our C-SCL pulls semantically similar EEG representations together while pushing apart dissimilar ones. Besides, in order to introduce more meaningful contrastive pairs, we carefully employ curriculum learning to not only craft meaningful contrastive pairs but also make the learning progressively. We conduct extensive experiments on the ZuCo benchmark and our method combined with diverse models and architectures shows stable improvements across three types of metrics while achieving the new state-of-the-art. Further investigation proves not only its superiority in both the single-subject and low-resource settings but also its robust generalizability in the zero-shot setting.
Congested Bandits: Optimal Routing via Short-term Resets
Venue: arXiv cs.LG 2023
Year: 2023

For traffic routing platforms, the choice of which route to recommend to a user depends on the congestion on these routes -- indeed, an individual's utility depends on the number of people using the recommended route at that instance. Motivated by this, we introduce the problem of Congested Bandits where each arm's reward is allowed to depend on the number of times it was played in the past $Δ$ timesteps. This dependence on past history of actions leads to a dynamical system where an algorithm's present choices also affect its future pay-offs, and requires an algorithm to plan for this. We study the congestion aware formulation in the multi-armed bandit (MAB) setup and in the contextual bandit setup with linear rewards. For the multi-armed setup, we propose a UCB style algorithm and show that its policy regret scales as $\tilde{O}(\sqrt{K ΔT})$. For the linear contextual bandit setup, our algorithm, based on an iterative least squares planner, achieves policy regret $\tilde{O}(\sqrt{dT} + Δ)$. From an experimental standpoint, we corroborate the no-regret properties of our algorithms via a simulation study.
Combined Use of Federated Learning and Image Encryption for Privacy-Preserving Image Classification with Vision Transformer
Venue: arXiv cs.CV 2023
Year: 2023

In recent years, privacy-preserving methods for deep learning have become an urgent problem. Accordingly, we propose the combined use of federated learning (FL) and encrypted images for privacy-preserving image classification under the use of the vision transformer (ViT). The proposed method allows us not only to train models over multiple participants without directly sharing their raw data but to also protect the privacy of test (query) images for the first time. In addition, it can also maintain the same accuracy as normally trained models. In an experiment, the proposed method was demonstrated to well work without any performance degradation on the CIFAR-10 and CIFAR-100 datasets.
Enabling Hard Constraints in Differentiable Neural Network and Accelerator Co-Exploration
Venue: arXiv cs.LG 2023
Year: 2023

Co-exploration of an optimal neural architecture and its hardware accelerator is an approach of rising interest which addresses the computational cost problem, especially in low-profile systems. The large co-exploration space is often handled by adopting the idea of differentiable neural architecture search. However, despite the superior search efficiency of the differentiable co-exploration, it faces a critical challenge of not being able to systematically satisfy hard constraints such as frame rate. To handle the hard constraint problem of differentiable co-exploration, we propose HDX, which searches for hard-constrained solutions without compromising the global design objectives. By manipulating the gradients in the interest of the given hard constraint, high-quality solutions satisfying the constraint can be obtained.
Modeling and Design of Longitudinal and Lateral Control System with a FeedForward Controller for a 4 Wheeled Robot
Venue: arXiv cs.RO 2023
Year: 2023

The work show in this paper progresses through a sequence of physics-based increasing fidelity models that are used to design the robot controllers that respect the limits of the robot capabilities, develop a reference simple controller applicable to a large subset of tracking conditions, which include mostly non-invasive or highly dynamic movements and define path geometry following the control problem and develop both a simple geometric control and a dynamic model predictive control approach. In this paper, we propose for a nonlinear model with disturbance effect, the mathematical modeling of the longitudinal and lateral movements using PID with a feed-forward controller. This study proposes a feedforward controller to eliminate the disturbance effect.