OpenRaiser/Intern-Atlas huggingface dataset:
Find Your ASMR: A Perceptual Retrieval Interface for Autonomous Sensory Meridian Response Videos
Venue: arXiv cs.HC 2022
Year: 2022

Autonomous sensory meridian response (ASMR) is a type of video contents designed to help people relax and feel comfortable. Users usually retrieve ASMR contents from various video websites using only keywords. However, it is challenging to examine satisfactory contents to reflect users' needs for ASMR videos using keywords or content-based retrieval. To solve this issue, we propose a perceptual video retrieval system for ASMR videos and provide a novel retrieval user interface that allows users to retrieve content according to watching purpose and anticipated expectations, such as excitement, calmness, stress and sadness. An ASMR video perception dataset is constructed with annotations on affective responses after watching the videos. To verify the proposed video retrieval system, a user study is conducted showing that users can retrieve satisfactory ASMR contents easily and efficiently compared to conventional keywords-based retrieval systems.
Fault-Aware Design and Training to Enhance DNNs Reliability with Zero-Overhead
Venue: arXiv cs.LG 2022
Year: 2022

Deep Neural Networks (DNNs) enable a wide series of technological advancements, ranging from clinical imaging, to predictive industrial maintenance and autonomous driving. However, recent findings indicate that transient hardware faults may corrupt the models prediction dramatically. For instance, the radiation-induced misprediction probability can be so high to impede a safe deployment of DNNs models at scale, urging the need for efficient and effective hardening solutions. In this work, we propose to tackle the reliability issue both at training and model design time. First, we show that vanilla models are highly affected by transient faults, that can induce a performances drop up to 37%. Hence, we provide three zero-overhead solutions, based on DNN re-design and re-train, that can improve DNNs reliability to transient faults up to one order of magnitude. We complement our work with extensive ablation studies to quantify the gain in performances of each hardening component.
Go Beyond Multiple Instance Neural Networks: Deep-learning Models based on Local Pattern Aggregation
Venue: arXiv cs.LG 2022
Year: 2022

Deep convolutional neural networks (CNNs) have brought breakthroughs in processing clinical electrocardiograms (ECGs), speaker-independent speech and complex images. However, typical CNNs require a fixed input size while it is common to process variable-size data in practical use. Recurrent networks such as long short-term memory (LSTM) are capable of eliminating the restriction, but suffer from high computational complexity. In this paper, we propose local pattern aggregation-based deep-learning models to effectively deal with both problems. The novel network structure, called LPANet, has cropping and aggregation operations embedded into it. With these new features, LPANet can reduce the difficulty of tuning model parameters and thus tend to improve generalization performance. To demonstrate the effectiveness, we applied it to the problem of premature ventricular contraction detection and the experimental results shows that our proposed method has certain advantages compared to classical network models, such as CNN and LSTM.
An adaptive admittance controller for collaborative drilling with a robot based on subtask classification via deep learning
Venue: arXiv cs.RO 2022
Year: 2022

In this paper, we propose a supervised learning approach based on an Artificial Neural Network (ANN) model for real-time classification of subtasks in a physical human-robot interaction (pHRI) task involving contact with a stiff environment. In this regard, we consider three subtasks for a given pHRI task: Idle, Driving, and Contact. Based on this classification, the parameters of an admittance controller that regulates the interaction between human and robot are adjusted adaptively in real time to make the robot more transparent to the operator (i.e. less resistant) during the Driving phase and more stable during the Contact phase. The Idle phase is primarily used to detect the initiation of task. Experimental results have shown that the ANN model can learn to detect the subtasks under different admittance controller conditions with an accuracy of 98% for 12 participants. Finally, we show that the admittance adaptation based on the proposed subtask classifier leads to 20% lower human effort (i.e. higher transparency) in the Driving phase and 25% lower oscillation amplitude (i.e. higher stability) during drilling in the Contact phase compared to an admittance controller with fixed parameters.
Perceptually Optimized Color Selection for Visualization
Venue: arXiv cs.CV 2022
Year: 2022

We propose an approach, called the Equilibrium Distribution Model (EDM), for automatically selecting colors with optimum perceptual contrast for scientific visualization. Given any number of features that need to be emphasized in a visualization task, our approach derives evenly distributed points in the CIELAB color space to assign colors to the features so that the minimum Euclidean Distance among the colors are optimized. Our approach can assign colors with high perceptual contrast even for very high numbers of features, where other color selection methods typically fail. We compare our approach with the widely used Harmonic color selection scheme and demonstrate that while the harmonic scheme can achieve reasonable color contrast for visualizing up to 20 different features, our Equilibrium scheme provides significantly better contrast and achieves perceptible contrast for visualizing even up to 100 unique features.
Performance Analysis of Self-Interference Cancellation in Full-Duplex Massive MIMO Systems: Subtraction versus Spatial Suppression
Venue: arXiv cs.IT 2022
Year: 2022

Massive multiple-input multiple-output (MIMO) and full-duplex (FD) are promising candidates for achieving the spectral efficiency to meet the needs of 5G communications. One essential key to realizing practical FD massive MIMO systems is how to effectively mitigate the self-interference (SI). Conventionally, however, the performance comparison of different SI methods by reflecting the actual channel characteristics was insufficient in the literature. Accordingly, this paper presents a performance analysis of SI cancellation (SIC) methods in FD massive MIMO systems. Analytical and numerical results confirm that, in an imperfect channel-estimation case, the ergodic rates performance of the spatial suppression in the uplink outperforms those of the SI subtraction, due to the correlation between the precoder and the estimation error of the SI channel. In addition, we discuss which method performs better under different given system constraints such as uplink and downlink sum rates, the total transmit power, and the power scaling law.
BadDet: Backdoor Attacks on Object Detection
Venue: arXiv cs.CV 2022
Year: 2022

Deep learning models have been deployed in numerous real-world applications such as autonomous driving and surveillance. However, these models are vulnerable in adversarial environments. Backdoor attack is emerging as a severe security threat which injects a backdoor trigger into a small portion of training data such that the trained model behaves normally on benign inputs but gives incorrect predictions when the specific trigger appears. While most research in backdoor attacks focuses on image classification, backdoor attacks on object detection have not been explored but are of equal importance. Object detection has been adopted as an important module in various security-sensitive applications such as autonomous driving. Therefore, backdoor attacks on object detection could pose severe threats to human lives and properties. We propose four kinds of backdoor attacks for object detection task: 1) Object Generation Attack: a trigger can falsely generate an object of the target class; 2) Regional Misclassification Attack: a trigger can change the prediction of a surrounding object to the target class; 3) Global Misclassification Attack: a single trigger can change the predictions of all objects in an image to the target class; and 4) Object Disappearance Attack: a trigger can make the detector fail to detect the object of the target class. We develop appropriate metrics to evaluate the four backdoor attacks on object detection. We perform experiments using two typical object detection models -- Faster-RCNN and YOLOv3 on different datasets. More crucially, we demonstrate that even fine-tuning on another benign dataset cannot remove the backdoor hidden in the object detection model. To defend against these backdoor attacks, we propose Detector Cleanse, an entropy-based run-time detection framework to identify poisoned testing samples for any deployed object detector.
All That's Happening behind the Scenes: Putting the Spotlight on Volunteer Moderator Labor in Reddit
Venue: arXiv cs.HC 2022
Year: 2022

Online volunteers are an uncompensated yet valuable labor force for many social platforms. For example, volunteer content moderators perform a vast amount of labor to maintain online communities. However, as social platforms like Reddit favor revenue generation and user engagement, moderators are under-supported to manage the expansion of online communities. To preserve these online communities, developers and researchers of social platforms must account for and support as much of this labor as possible. In this paper, we quantitatively characterize the publicly visible and invisible actions taken by moderators on Reddit, using a unique dataset of private moderator logs for 126 subreddits and over 900 moderators. Our analysis of this dataset reveals the heterogeneity of moderation work across both communities and moderators. Moreover, we find that analyzing only visible work - the dominant way that moderation work has been studied thus far - drastically underestimates the amount of human moderation labor on a subreddit. We discuss the implications of our results on content moderation research and social platforms.
Leg Shaping and Event-Driven Control of a Small-Scale, Low-DoF, Two-Mode Robot
Venue: arXiv cs.RO 2022
Year: 2022

Among small-scale mobile robots, multi-modal locomotion can help compensate for limited actuator capabilities. However, supporting multiple locomotion modes or gaits in small terrestrial robots typically requires complex designs with low locomotion efficiency. In this work, legged and rolling gaits are achieved by a 10~cm robot having just two degrees of freedom (DoF). This is acheived by leg shaping that facilitates whole body rolling and event-driven control that maintains motion using simple inertial sensor measurements. Speeds of approximately 0.4 and 2.2 body lengths per second are achieved in legged and rolling modes, respectively, with low cost of transport. The proposed design approach and control techniques may aid in design of further miniaturized robots reliant on transducers with small range-of-motion.
Formal Methods for Characterization and Analysis of Quality Specifications in Component-based Systems
Venue: arXiv cs.SE 2022
Year: 2022

Component-based design paradigm is of paramount importance due to prolific growth in the complexity of modern-day systems. Since the components are developed primarily by multi-party vendors and often assembled to realize the overall system, it is an onus of the designer to certify both the functional and non-functional requirements of such systems. Several of the earlier works concentrated on formally analyzing the behavioral correctness, safety, security, reliability and robustness of such compositional systems. However, the assurance for quality measures of such systems is also considered as an important parameter for their acceptance. Formalization of quality measures is still at an immature state and often dictated by the user satisfaction. This paper presents a novel compositional framework for reliable quality analysis of component-based systems from the formal quality specifications of its constituent components. The proposed framework enables elegant and generic computation methods for quality attributes of various component-based system structures. In addition to this, we provide a formal query-driven quality assessment and design exploration framework which enables the designer to explore various component structures and operating setups and finally converge into better acceptable systems. A detailed case-study is presented over a component-based system structure to show the efficacy and practicality of our proposed framework.
A General Multiple Data Augmentation Based Framework for Training Deep Neural Networks
Venue: arXiv cs.NE 2022
Year: 2022

Deep neural networks (DNNs) often rely on massive labelled data for training, which is inaccessible in many applications. Data augmentation (DA) tackles data scarcity by creating new labelled data from available ones. Different DA methods have different mechanisms and therefore using their generated labelled data for DNN training may help improving DNN's generalisation to different degrees. Combining multiple DA methods, namely multi-DA, for DNN training, provides a way to boost generalisation. Among existing multi-DA based DNN training methods, those relying on knowledge distillation (KD) have received great attention. They leverage knowledge transfer to utilise the labelled data sets created by multiple DA methods instead of directly combining them for training DNNs. However, existing KD-based methods can only utilise certain types of DA methods, incapable of utilising the advantages of arbitrary DA methods. We propose a general multi-DA based DNN training framework capable to use arbitrary DA methods. To train a DNN, our framework replicates a certain portion in the latter part of the DNN into multiple copies, leading to multiple DNNs with shared blocks in their former parts and independent blocks in their latter parts. Each of these DNNs is associated with a unique DA and a newly devised loss that allows comprehensively learning from the data generated by all DA methods and the outputs from all DNNs in an online and adaptive way. The overall loss, i.e., the sum of each DNN's loss, is used for training the DNN. Eventually, one of the DNNs with the best validation performance is chosen for inference. We implement the proposed framework by using three distinct DA methods and apply it for training representative DNNs. Experiments on the popular benchmarks of image classification demonstrate the superiority of our method to several existing single-DA and multi-DA based training methods.
Cervical Glandular Cell Detection from Whole Slide Image with Out-Of-Distribution Data
Venue: arXiv cs.CV 2022
Year: 2022

Cervical glandular cell (GC) detection is a key step in computer-aided diagnosis for cervical adenocarcinomas screening. It is challenging to accurately recognize GCs in cervical smears in which squamous cells are the major. Widely existing Out-Of-Distribution (OOD) data in the entire smear leads decreasing reliability of machine learning system for GC detection. Although, the State-Of-The-Art (SOTA) deep learning model can outperform pathologists in preselected regions of interest, the mass False Positive (FP) prediction with high probability is still unsolved when facing such gigapixel whole slide image. This paper proposed a novel PolarNet based on the morphological prior knowledge of GC trying to solve the FP problem via a self-attention mechanism in eight-neighbor. It estimates the polar orientation of nucleus of GC. As a plugin module, PolarNet can guide the deep feature and predicted confidence of general object detection models. In experiments, we discovered that general models based on four different frameworks can reject FP in small image set and increase the mean of average precision (mAP) by $\text{0.007}\sim\text{0.015}$ in average, where the highest exceeds the recent cervical cell detection model 0.037. By plugging PolarNet, the deployed C++ program improved by 8.8\% on accuracy of top-20 GC detection from external WSIs, while sacrificing 14.4 s of computational time. Code is available in https://github.com/Chrisa142857/PolarNet-GCdet
Superclass Adversarial Attack
Venue: arXiv cs.CV 2022
Year: 2022

Adversarial attacks have only focused on changing the predictions of the classifier, but their danger greatly depends on how the class is mistaken. For example, when an automatic driving system mistakes a Persian cat for a Siamese cat, it is hardly a problem. However, if it mistakes a cat for a 120km/h minimum speed sign, serious problems can arise. As a stepping stone to more threatening adversarial attacks, we consider the superclass adversarial attack, which causes misclassification of not only fine classes, but also superclasses. We conducted the first comprehensive analysis of superclass adversarial attacks (an existing and 19 new methods) in terms of accuracy, speed, and stability, and identified several strategies to achieve better performance. Although this study is aimed at superclass misclassification, the findings can be applied to other problem settings involving multiple classes, such as top-k and multi-label classification attacks.
Perceiving the Invisible: Proposal-Free Amodal Panoptic Segmentation
Venue: arXiv cs.CV 2022
Year: 2022

Amodal panoptic segmentation aims to connect the perception of the world to its cognitive understanding. It entails simultaneously predicting the semantic labels of visible scene regions and the entire shape of traffic participant instances, including regions that may be occluded. In this work, we formulate a proposal-free framework that tackles this task as a multi-label and multi-class problem by first assigning the amodal masks to different layers according to their relative occlusion order and then employing amodal instance regression on each layer independently while learning background semantics. We propose the \net architecture that incorporates a shared backbone and an asymmetrical dual-decoder consisting of several modules to facilitate within-scale and cross-scale feature aggregations, bilateral feature propagation between decoders, and integration of global instance-level and local pixel-level occlusion reasoning. Further, we propose the amodal mask refiner that resolves the ambiguity in complex occlusion scenarios by explicitly leveraging the embedding of unoccluded instance masks. Extensive evaluation on the BDD100K-APS and KITTI-360-APS datasets demonstrate that our approach set the new state-of-the-art on both benchmarks.
Glance to Count: Learning to Rank with Anchors for Weakly-supervised Crowd Counting
Venue: arXiv cs.CV 2022
Year: 2022

Crowd image is arguably one of the most laborious data to annotate. In this paper, we devote to reduce the massive demand of densely labeled crowd data, and propose a novel weakly-supervised setting, in which we leverage the binary ranking of two images with high-contrast crowd counts as training guidance. To enable training under this new setting, we convert the crowd count regression problem to a ranking potential prediction problem. In particular, we tailor a Siamese Ranking Network that predicts the potential scores of two images indicating the ordering of the counts. Hence, the ultimate goal is to assign appropriate potentials for all the crowd images to ensure their orderings obey the ranking labels. On the other hand, potentials reveal the relative crowd sizes but cannot yield an exact crowd count. We resolve this problem by introducing "anchors" during the inference stage. Concretely, anchors are a few images with count labels used for referencing the corresponding counts from potential scores by a simple linear mapping function. We conduct extensive experiments to study various combinations of supervision, and we show that the proposed method outperforms existing weakly-supervised methods without additional labeling effort by a large margin.
SFE-AI at SemEval-2022 Task 11: Low-Resource Named Entity Recognition using Large Pre-trained Language Models
Venue: arXiv cs.CL 2022
Year: 2022

Large scale pre-training models have been widely used in named entity recognition (NER) tasks. However, model ensemble through parameter averaging or voting can not give full play to the differentiation advantages of different models, especially in the open domain. This paper describes our NER system in the SemEval 2022 task11: MultiCoNER. We proposed an effective system to adaptively ensemble pre-trained language models by a Transformer layer. By assigning different weights to each model for different inputs, we adopted the Transformer layer to integrate the advantages of diverse models effectively. Experimental results show that our method achieves superior performances in Farsi and Dutch.
Evaluating Automated Driving Planner Robustness against Adversarial Influence
Venue: arXiv cs.CR 2022
Year: 2022

Evaluating the robustness of automated driving planners is a critical and challenging task. Although methodologies to evaluate vehicles are well established, they do not yet account for a reality in which vehicles with autonomous components share the road with adversarial agents. Our approach, based on probabilistic trust models, aims to help researchers assess the robustness of protections for machine learning-enabled planners against adversarial influence. In contrast with established practices that evaluate safety using the same evaluation dataset for all vehicles, we argue that adversarial evaluation fundamentally requires a process that seeks to defeat a specific protection. Hence, we propose that evaluations be based on estimating the difficulty for an adversary to determine conditions that effectively induce unsafe behavior. This type of inference requires precise statements about threats, protections, and aspects of planning decisions to be guarded. We demonstrate our approach by evaluating protections for planners relying on camera-based object detectors.
Extrinsic Calibration of Multiple Inertial Sensors from Arbitrary Trajectories
Venue: arXiv cs.RO 2022
Year: 2022

We present a method of extrinsic calibration for a system of multiple inertial measurement units (IMUs) that estimates the relative pose of each IMU on a rigid body using only measurements from the IMUs themselves, without the need to prescribe the trajectory. Our method is based on solving a nonlinear least-squares problem that penalizes inconsistency between measurements from pairs of IMUs. We validate our method with experiments both in simulation and in hardware. In particular, we show that it meets or exceeds the performance -- in terms of error, success rate, and computation time -- of an existing, state-of-the-art method that does not rely only on IMU measurements and instead requires the use of a camera and a fiducial marker. We also show that the performance of our method is largely insensitive to the choice of trajectory along which IMU measurements are collected.