OpenRaiser/Intern-Atlas huggingface dataset:
Learning from Mistakes: Using Mis-predictions as Harm Alerts in Language Pre-Training
Venue: arXiv cs.CL 2021
Year: 2021

Fitting complex patterns in the training data, such as reasoning and commonsense, is a key challenge for language pre-training. According to recent studies and our empirical observations, one possible reason is that some easy-to-fit patterns in the training data, such as frequently co-occurring word combinations, dominate and harm pre-training, making it hard for the model to fit more complex information. We argue that mis-predictions can help locate such dominating patterns that harm language understanding. When a mis-prediction occurs, there should be frequently co-occurring patterns with the mis-predicted word fitted by the model that lead to the mis-prediction. If we can add regularization to train the model to rely less on such dominating patterns when a mis-prediction occurs and focus more on the rest more subtle patterns, more information can be efficiently fitted at pre-training. Following this motivation, we propose a new language pre-training method, Mis-Predictions as Harm Alerts (MPA). In MPA, when a mis-prediction occurs during pre-training, we use its co-occurrence information to guide several heads of the self-attention modules. Some self-attention heads in the Transformer modules are optimized to assign lower attention weights to the words in the input sentence that frequently co-occur with the mis-prediction while assigning higher weights to the other words. By doing so, the Transformer model is trained to rely less on the dominating frequently co-occurring patterns with mis-predictions while focus more on the rest more complex information when mis-predictions occur. Our experiments show that MPA expedites the pre-training of BERT and ELECTRA and improves their performances on downstream tasks.
Affective visualization in Virtual Reality: An integrative review
Venue: arXiv cs.HC 2021
Year: 2021

A cluster of research in Affective Computing suggests that it is possible to infer some characteristics of users' affective states by analyzing their electrophysiological activity in real-time. However, it is not clear how to use the information extracted from electrophysiological signals to create visual representations of the affective states of Virtual Reality (VR) users. Visualization of users' affective states in VR can lead to biofeedback therapies for mental health care. Understanding how to visualize affective states in VR requires an interdisciplinary approach that integrates psychology, electrophysiology, and audio-visual design. Therefore, this review aims to integrate previous studies from these fields to understand how to develop virtual environments that can automatically create visual representations of users' affective states. The manuscript addresses this challenge in four sections: First, theories related to emotion and affect are summarized. Second, evidence suggesting that visual and sound cues tend to be associated with affective states are discussed. Third, some of the available methods for assessing affect are described. The fourth and final section contains five practical considerations for the development of virtual reality environments for affect visualization.
Copula-based synthetic data augmentation for machine-learning emulators
Venue: arXiv cs.LG 2021
Year: 2021

Can we improve machine-learning (ML) emulators with synthetic data? If data are scarce or expensive to source and a physical model is available, statistically generated data may be useful for augmenting training sets cheaply. Here we explore the use of copula-based models for generating synthetically augmented datasets in weather and climate by testing the method on a toy physical model of downwelling longwave radiation and corresponding neural network emulator. Results show that for copula-augmented datasets, predictions are improved by up to 62 % for the mean absolute error (from 1.17 to 0.44 W m$^{-2}$).
Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges
Venue: arXiv cs.CY 2021
Year: 2021

As the globally increasing population drives rapid urbanisation in various parts of the world, there is a great need to deliberate on the future of the cities worth living. In particular, as modern smart cities embrace more and more data-driven artificial intelligence services, it is worth remembering that technology can facilitate prosperity, wellbeing, urban livability, or social justice, but only when it has the right analog complements (such as well-thought out policies, mature institutions, responsible governance); and the ultimate objective of these smart cities is to facilitate and enhance human welfare and social flourishing. Researchers have shown that various technological business models and features can in fact contribute to social problems such as extremism, polarization, misinformation, and Internet addiction. In the light of these observations, addressing the philosophical and ethical questions involved in ensuring the security, safety, and interpretability of such AI algorithms that will form the technological bedrock of future cities assumes paramount importance. Globally there are calls for technology to be made more humane and human-centered. In this paper, we analyze and explore key challenges including security, robustness, interpretability, and ethical (data and algorithmic) challenges to a successful deployment of AI in human-centric applications, with a particular emphasis on the convergence of these concepts/challenges. We provide a detailed review of existing literature on these key challenges and analyze how one of these challenges may lead to others or help in solving other challenges. The paper also advises on the current limitations, pitfalls, and future directions of research in these domains, and how it can fill the current gaps and lead to better solutions. We believe such rigorous analysis will provide a baseline for future research in the domain.
Point Transformer
Venue: arXiv cs.CV 2021
Year: 2021

Self-attention networks have revolutionized natural language processing and are making impressive strides in image analysis tasks such as image classification and object detection. Inspired by this success, we investigate the application of self-attention networks to 3D point cloud processing. We design self-attention layers for point clouds and use these to construct self-attention networks for tasks such as semantic scene segmentation, object part segmentation, and object classification. Our Point Transformer design improves upon prior work across domains and tasks. For example, on the challenging S3DIS dataset for large-scale semantic scene segmentation, the Point Transformer attains an mIoU of 70.4% on Area 5, outperforming the strongest prior model by 3.3 absolute percentage points and crossing the 70% mIoU threshold for the first time.
Projected Distribution Loss for Image Enhancement
Venue: arXiv cs.CV 2021
Year: 2021

Features obtained from object recognition CNNs have been widely used for measuring perceptual similarities between images. Such differentiable metrics can be used as perceptual learning losses to train image enhancement models. However, the choice of the distance function between input and target features may have a consequential impact on the performance of the trained model. While using the norm of the difference between extracted features leads to limited hallucination of details, measuring the distance between distributions of features may generate more textures; yet also more unrealistic details and artifacts. In this paper, we demonstrate that aggregating 1D-Wasserstein distances between CNN activations is more reliable than the existing approaches, and it can significantly improve the perceptual performance of enhancement models. More explicitly, we show that in imaging applications such as denoising, super-resolution, demosaicing, deblurring and JPEG artifact removal, the proposed learning loss outperforms the current state-of-the-art on reference-based perceptual losses. This means that the proposed learning loss can be plugged into different imaging frameworks and produce perceptually realistic results.
FG-Net: Fast Large-Scale LiDAR Point Clouds Understanding Network Leveraging Correlated Feature Mining and Geometric-Aware Modelling
Venue: arXiv cs.CV 2021
Year: 2021

This work presents FG-Net, a general deep learning framework for large-scale point clouds understanding without voxelizations, which achieves accurate and real-time performance with a single NVIDIA GTX 1080 GPU. First, a novel noise and outlier filtering method is designed to facilitate subsequent high-level tasks. For effective understanding purpose, we propose a deep convolutional neural network leveraging correlated feature mining and deformable convolution based geometric-aware modelling, in which the local feature relationships and geometric patterns can be fully exploited. For the efficiency issue, we put forward an inverse density sampling operation and a feature pyramid based residual learning strategy to save the computational cost and memory consumption respectively. Extensive experiments on real-world challenging datasets demonstrated that our approaches outperform state-of-the-art approaches in terms of accuracy and efficiency. Moreover, weakly supervised transfer learning is also conducted to demonstrate the generalization capacity of our method.
Query-Competitive Sorting with Uncertainty
Venue: arXiv cs.DS 2021
Year: 2021

We study the problem of sorting under incomplete information, when queries are used to resolve uncertainties. Each of $n$ data items has an unknown value, which is known to lie in a given interval. We can pay a query cost to learn the actual value, and we may allow an error threshold in the sorting. The goal is to find a nearly-sorted permutation by performing a minimum-cost set of queries. We show that an offline optimum query set can be found in poly time, and that both oblivious and adaptive problems have simple competitive algorithms. The competitive ratio for the oblivious problem is $n$ for uniform query costs, and unbounded for arbitrary costs; for the adaptive problem, the ratio is 2. We present a unified adaptive strategy for uniform costs that yields the following improved results: (1) a 3/2-competitive randomized algorithm; (2) a 5/3-competitive deterministic algorithm if the dependency graph has no 2-components after some preprocessing, which has competitive ratio $3/2+\mathrm{O}(1/k)$ if the components obtained have size at least $k$; and (3) an exact algorithm for laminar families of intervals. The first two results have matching lower bounds, and we have a lower bound of 7/5 for large components. We also give a randomized adaptive algorithm with competitive ratio $1+\frac{4}{3\sqrt{3}}\approx 1.7698$ for arbitrary query costs, and we show that the 2-competitive deterministic adaptive algorithm can be generalized for queries returning intervals and for a more general vertex cover problem, by using the local ratio technique. Moreover, we prove that the advice complexity of the adaptive problem is $\lfloor n/2\rfloor$ if no error threshold is allowed, and $\lceil n/3\cdot\lg 3\rceil$ for the general case. Finally, we present some graph-theoretical results on co-threshold tolerance graphs, and we discuss uncertainty variants of some classical interval problems.
A Hierarchical Feature Constraint to Camouflage Medical Adversarial Attacks
Venue: arXiv cs.CV 2021
Year: 2021

Deep neural networks (DNNs) for medical images are extremely vulnerable to adversarial examples (AEs), which poses security concerns on clinical decision making. Luckily, medical AEs are also easy to detect in hierarchical feature space per our study herein. To better understand this phenomenon, we thoroughly investigate the intrinsic characteristic of medical AEs in feature space, providing both empirical evidence and theoretical explanations for the question: why are medical adversarial attacks easy to detect? We first perform a stress test to reveal the vulnerability of deep representations of medical images, in contrast to natural images. We then theoretically prove that typical adversarial attacks to binary disease diagnosis network manipulate the prediction by continuously optimizing the vulnerable representations in a fixed direction, resulting in outlier features that make medical AEs easy to detect. However, this vulnerability can also be exploited to hide the AEs in the feature space. We propose a novel hierarchical feature constraint (HFC) as an add-on to existing adversarial attacks, which encourages the hiding of the adversarial representation within the normal feature distribution. We evaluate the proposed method on two public medical image datasets, namely {Fundoscopy} and {Chest X-Ray}. Experimental results demonstrate the superiority of our adversarial attack method as it bypasses an array of state-of-the-art adversarial detectors more easily than competing attack methods, supporting that the great vulnerability of medical features allows an attacker more room to manipulate the adversarial representations.
Smartphone Sensing for the Well-being of Young Adults: A Review
Venue: arXiv cs.HC 2021
Year: 2021

Over the years, mobile phones have become versatile devices with a multitude of capabilities due to the plethora of embedded sensors that enable them to capture rich data unobtrusively. In a world where people are more conscious regarding their health and well-being, the pervasiveness of smartphones has enabled researchers to build apps that assist people to live healthier lifestyles, and to diagnose and monitor various health conditions. Motivated by the high smartphone coverage among young adults and the unique issues they face, in this review paper, we focus on studies that have used smartphone sensing for the well-being of young adults. We analyze existing work in the domain from two perspectives, namely Data Perspective and System Perspective. For both these perspectives, we propose taxonomies motivated from human science literature, which enable to identify important study areas. Furthermore, we emphasize the importance of diversity-awareness in smartphone sensing, and provide insights and future directions for researchers in ubiquitous and mobile computing, and especially to new researchers who want to understand the basics of smartphone sensing research targeting the well-being of young adults.
Automatic source localization and spectra generation from sparse beamforming maps
Venue: arXiv cs.SD 2021
Year: 2021

Beamforming is an imaging tool for the investigation of aeroacoustic phenomena and results in high dimensional data that is broken down to spectra by integrating spatial Regions Of Interest. This paper presents two methods that enable the automated identification of aeroacoustic sources in sparse beamforming maps and the extraction of their corresponding spectra to overcome the manual definition of Regions Of Interest. The methods are evaluated on two scaled airframe half-model wind-tunnel measurements and on a generic monopole source. The first relies on the spatial normal distribution of aeroacoustic broadband sources in sparse beamforming maps. The second uses hierarchical clustering methods. Both methods are robust to statistical noise and predict the existence, location, and spatial probability estimation for sources based on which Regions Of Interest are automatically determined.
AutoCaption: Image Captioning with Neural Architecture Search
Venue: arXiv cs.CV 2021
Year: 2021

Image captioning transforms complex visual information into abstract natural language for representation, which can help computers understanding the world quickly. However, due to the complexity of the real environment, it needs to identify key objects and realize their connections, and further generate natural language. The whole process involves a visual understanding module and a language generation module, which brings more challenges to the design of deep neural networks than other tasks. Neural Architecture Search (NAS) has shown its important role in a variety of image recognition tasks. Besides, RNN plays an essential role in the image captioning task. We introduce a AutoCaption method to better design the decoder module of the image captioning where we use the NAS to design the decoder module called AutoRNN automatically. We use the reinforcement learning method based on shared parameters for automatic design the AutoRNN efficiently. The search space of the AutoCaption includes connections between the layers and the operations in layers both, and it can make AutoRNN express more architectures. In particular, RNN is equivalent to a subset of our search space. Experiments on the MSCOCO datasets show that our AutoCaption model can achieve better performance than traditional hand-design methods.
Connectivity Maintenance for Multi-Robot Systems Under Motion and Sensing Uncertainties Using Distributed ADMM-based Trajectory Planning
Venue: arXiv cs.RO 2021
Year: 2021

Inter-robot communication enables multi-robot systems to coordinate and execute complex missions efficiently. Thus, maintaining connectivity of the communication network between robots is essential for many multi-robot systems. In this paper, we present a trajectory planner for connectivity maintenance of a multi-robot system. We first define a weighted undirected graph to represent the connectivity of the system. Unlike previous connectivity maintenance works, we explicitly account for robot motion and sensing uncertainties while formulating the graph edge weights. These uncertainties result in uncertain robot positions which directly affect the connectivity of the system. Next, the algebraic connectivity of the weighted undirected graph is maintained above a specified lower limit using a trajectory planner based on a distributed alternating direction method of multipliers (ADMM) framework. Here we derive an approximation for the Hessian matrices required within the ADMM optimization step to reduce the computational load. Finally, simulation results are presented to statistically validate the connectivity maintenance of our trajectory planner.
Efficient Verification of Optimized Code: Correct High-speed X25519
Venue: arXiv cs.CR 2021
Year: 2021

Code that is highly optimized poses a problem for program-level verification: programmers can employ various clever tricks that are non-trivial to reason about. For cryptography on low-power devices, it is nonetheless crucial that implementations be functionally correct, secure, and efficient. These are usually crafted in hand-optimized machine code that eschew conventional control flow as much as possible. We have formally verified such code: a library which implements elliptic curve cryptography on 8-bit AVR microcontrollers. The chosen implementation is the most efficient currently known for this microarchitecture. It consists of over 3000 lines of assembly instructions. Building on earlier work, we use the Why3 platform to model the code and prove verification conditions, using automated provers. We expect the approach to be re-usable and adaptable, and it allows for validation. Furthermore, an error in the original implementation was found and corrected, at the same time reducing its memory footprint. This shows that practical verification of cutting-edge code is not only possible, but can in fact add to its efficiency -- and is clearly necessary.
YouNiverse: Large-Scale Channel and Video Metadata from English-Speaking YouTube
Venue: arXiv cs.SI 2021
Year: 2021

YouTube plays a key role in entertaining and informing people around the globe. However, studying the platform is difficult due to the lack of randomly sampled data and of systematic ways to query the platform's colossal catalog. In this paper, we present YouNiverse, a large collection of channel and video metadata from English-language YouTube. YouNiverse comprises metadata from over 136k channels and 72.9M videos published between May 2005 and October 2019, as well as channel-level time-series data with weekly subscriber and view counts. Leveraging channel ranks from socialblade.com, an online service that provides information about YouTube, we are able to assess and enhance the representativeness of the sample of channels. Additionally, the dataset also contains a table specifying which videos a set of 449M anonymous users commented on. YouNiverse, publicly available at https://doi.org/10.5281/zenodo.4650046, will empower the community to do research with and about YouTube.
Humanoid Robot Pitch Axis Stabilization using Linear Quadratic Regulator with Fuzzy Logic and Capture Point
Venue: arXiv cs.RO 2021
Year: 2021

This paper aims for a controller that can stabilize a position-controlled humanoid robot when standing still or walking on synthetic grass even when subjected to external disturbances. Two types of controllers are designed and implemented: ankle strategy and stepping strategy. The robot's joints consist of position-controlled servos which can be complicated to model analytically due to nonlinearities and non-measurable parameters, hence the dynamic model of the humanoid robot is acquired using a non-recursive least squares system identification. This model is also used to design a Kalman Filter to estimate the system states from noisy inertial measurement unit (IMU) sensor and design a linear quadratic regulator (LQR) controller. To handle the nonlinearities, the LQR controller is extended with fuzzy logic algorithm that changes the LQR gain value based on angle and angular velocity membership functions. The proposed control system can maintain the humanoid robot's stability around the pitch axis when subject to pendulum disturbances or even restraining force from a spring balance.
Predicting the Objective and Priority of Issue Reports in Software Repositories
Venue: arXiv cs.SE 2021
Year: 2021

Developers collaboratively discuss, implement, use, and share software entities hosted on software repositories. Proper documentation plays an important role in successful software management and maintenance. Users exploit Issue Tracking Systems, a facility of software repositories, to keep track of issue reports, to manage the workload and processes, and finally, to document the highlight of their team's effort. An issue report is a rich source of collaboratively curated software knowledge, and can contain a reported problem, a request for new features, or merely a question about the software product. As the number of these issues increases, it becomes harder to manage them manually. GitHub provides labels for tagging issues, as a means of issue management. However, about half of the issues in GitHub's top 1000 repositories do not have any labels. We aim at automating the process of managing issue reports for software teams. We propose a two-stage approach to predict both the objective behind opening an issue and its priority level using feature engineering methods and state-of-the-art text classifiers. To the best of our knowledge, we are the first to fine-tune a Transformer for issue classification. We train and evaluate our models in both project-based and cross-project settings. The latter approach provides a generic prediction model applicable for any unseen software project or projects with little historical data. Our proposed approach can successfully predict the objective and priority level of issue reports with 82% and 75% accuracy, respectively. Moreover, we conducted human labeling and evaluation on unlabeled issues from six unseen GitHub projects to assess the performance of the cross-project model on new data. The model achieves 90% accuracy. We obtain 85% average Percent Agreement and 71% Randolph's free-marginal Kappa translating to substantial agreement among labelers.