OpenRaiser/Intern-Atlas huggingface dataset:
An Image-based Approach of Task-driven Driving Scene Categorization
Venue: arXiv cs.RO 2021
Year: 2021

Categorizing driving scenes via visual perception is a key technology for safe driving and the downstream tasks of autonomous vehicles. Traditional methods infer scene category by detecting scene-related objects or using a classifier that is trained on large datasets of fine-labeled scene images. Whereas at cluttered dynamic scenes such as campus or park, human activities are not strongly confined by rules, and the functional attributes of places are not strongly correlated with objects. So how to define, model and infer scene categories is crucial to make the technique really helpful in assisting a robot to pass through the scene. This paper proposes a method of task-driven driving scene categorization using weakly supervised data. Given a front-view video of a driving scene, a set of anchor points is marked by following the decision making of a human driver, where an anchor point is not a semantic label but an indicator meaning the semantic attribute of the scene is different from that of the previous one. A measure is learned to discriminate the scenes of different semantic attributes via contrastive learning, and a driving scene profiling and categorization method is developed based on that measure. Experiments are conducted on a front-view video that is recorded when a vehicle passed through the cluttered dynamic campus of Peking University. The scenes are categorized into straight road, turn road and alerting traffic. The results of semantic scene similarity learning and driving scene categorization are extensively studied, and positive result of scene categorization is 97.17 \% on the learning video and 85.44\% on the video of new scenes.
Vertex Fault-Tolerant Spanners for Weighted Points in Polygonal Domains
Venue: arXiv cs.CG 2021
Year: 2021

Given a set $S$ of $n$ points, a weight function $w$ to associate a non-negative weight to each point in $S$, a positive integer $k \ge 1$, and a real number $ε> 0$, we devise the following algorithms to compute a $k$-vertex fault-tolerant spanner network $G(S, E)$ for the metric space induced by the weighted points in $S$: (1) When the points in $S$ are located in a simple polygon, we present an algorithm to compute $G$ with multiplicative stretch $\sqrt{10}+ε$, and the number of edges in $G$ (size of $G$) is $O(k n (\lg{n})^2)$. (2) When the points in $S$ are located in the free space of a polygonal domain $\cal P$ with $h$ number of obstacles, we present an algorithm to compute $G$ with multiplicative stretch $6+ε$ and size $O(\sqrt{h} k n(\lg{n})^2)$. (3) When the points in $S$ are located on a polyhedral terrain, we devise an algorithm to compute $G$ with multiplicative stretch $6+ε$ and size $O(k n (\lg{n})^2)$.
Deep Convolutional Sparse Coding Network for Pansharpening with Guidance of Side Information
Venue: arXiv cs.CV 2021
Year: 2021

Pansharpening is a fundamental issue in remote sensing field. This paper proposes a side information partially guided convolutional sparse coding (SCSC) model for pansharpening. The key idea is to split the low resolution multispectral image into a panchromatic image related feature map and a panchromatic image irrelated feature map, where the former one is regularized by the side information from panchromatic images. With the principle of algorithm unrolling techniques, the proposed model is generalized as a deep neural network, called as SCSC pansharpening neural network (SCSC-PNN). Compared with 13 classic and state-of-the-art methods on three satellites, the numerical experiments show that SCSC-PNN is superior to others. The codes are available at https://github.com/xsxjtu/SCSC-PNN.
EBP-GEXIT Charts for M-ary AWGN Channel for Generalized LDPC and Turbo Codes
Venue: arXiv cs.IT 2021
Year: 2021

This work proposes a tractable estimation of the maximum a posteriori (MAP) threshold of various families of sparse-graph code ensembles, by using an approximation for the extended belief propagation generalized extrinsic information transfer (EBP-GEXIT) function, first proposed by Measson et al. We consider the transmission over non-binary complex-input additive white Gaussian noise channel and extend the existing results to obtain an expression for the GEXIT function. We estimate the MAP threshold by applying the Maxwell construction to the obtained approximate EBP-GEXIT charts for various families of low-density parity-check (LDPC), generalized LDPC, doubly generalized LDPC, and serially concatenated turbo codes (SC-TC). When codewords of SC-TC are modulated using Gray mapping, we also explore where the spatially-coupled belief propagation (BP) threshold is located with respect to the previously computed MAP threshold. Numerical results indicate that the BP threshold of the spatially-coupled SC-TC does saturate to the MAP threshold obtained via EBP-GEXIT chart.
COLA-Net: Collaborative Attention Network for Image Restoration
Venue: arXiv cs.CV 2021
Year: 2021

Local and non-local attention-based methods have been well studied in various image restoration tasks while leading to promising performance. However, most of the existing methods solely focus on one type of attention mechanism (local or non-local). Furthermore, by exploiting the self-similarity of natural images, existing pixel-wise non-local attention operations tend to give rise to deviations in the process of characterizing long-range dependence due to image degeneration. To overcome these problems, in this paper we propose a novel collaborative attention network (COLA-Net) for image restoration, as the first attempt to combine local and non-local attention mechanisms to restore image content in the areas with complex textures and with highly repetitive details respectively. In addition, an effective and robust patch-wise non-local attention model is developed to capture long-range feature correspondences through 3D patches. Extensive experiments on synthetic image denoising, real image denoising and compression artifact reduction tasks demonstrate that our proposed COLA-Net is able to achieve state-of-the-art performance in both peak signal-to-noise ratio and visual perception, while maintaining an attractive computational complexity. The source code is available on https://github.com/MC-E/COLA-Net.
Deep learning methods for screening patients' S-ICD implantation eligibility
Venue: arXiv cs.LG 2021
Year: 2021

Subcutaneous Implantable Cardioverter-Defibrillators (S-ICDs) are used for prevention of sudden cardiac death triggered by ventricular arrhythmias. T Wave Over Sensing (TWOS) is an inherent risk with S-ICDs which can lead to inappropriate shocks. A major predictor of TWOS is a high T:R ratio (the ratio between the amplitudes of the T and R waves). Currently patients' Electrocardiograms (ECGs) are screened over 10 seconds to measure the T:R ratio, determining the patients' eligibility for S-ICD implantation. Due to temporal variations in the T:R ratio, 10 seconds is not long enough to reliably determine the normal values of a patient's T:R ratio. In this paper, we develop a convolutional neural network (CNN) based model utilising phase space reconstruction matrices to predict T:R ratios from 10-second ECG segments without explicitly locating the R or T waves, thus avoiding the issue of TWOS. This tool can be used to automatically screen patients over a much longer period and provide an in-depth description of the behaviour of the T:R ratio over that period. The tool can also enable much more reliable and descriptive screenings to better assess patients' eligibility for S-ICD implantation.
Towards a Cognitive Compute Continuum: An Architecture for Ad-Hoc Self-Managed Swarms
Venue: arXiv cs.NI 2021
Year: 2021

In this paper we introduce our vision of a Cognitive Computing Continuum to address the changing IT service provisioning towards a distributed, opportunistic, self-managed collaboration between heterogeneous devices outside the traditional data center boundaries. The focal point of this continuum are cognitive devices, which have to make decisions autonomously using their on-board computation and storage capacity based on information sensed from their environment. Such devices are moving and cannot rely on fixed infrastructure elements, but instead realise on-the-fly networking and thus frequently join and leave temporal swarms. All this creates novel demands for the underlying architecture and resource management, which must bridge the gap from edge to cloud environments, while keeping the QoS parameters within required boundaries. The paper presents an initial architecture and a resource management framework for the implementation of this type of IT service provisioning.
Team Phoenix at WASSA 2021: Emotion Analysis on News Stories with Pre-Trained Language Models
Venue: arXiv cs.CL 2021
Year: 2021

Emotion is fundamental to humanity. The ability to perceive, understand and respond to social interactions in a human-like manner is one of the most desired capabilities in artificial agents, particularly in social-media bots. Over the past few years, computational understanding and detection of emotional aspects in language have been vital in advancing human-computer interaction. The WASSA Shared Task 2021 released a dataset of news-stories across two tracks, Track-1 for Empathy and Distress Prediction and Track-2 for Multi-Dimension Emotion prediction at the essay-level. We describe our system entry for the WASSA 2021 Shared Task (for both Track-1 and Track-2), where we leveraged the information from Pre-trained language models for Track-specific Tasks. Our proposed models achieved an Average Pearson Score of 0.417 and a Macro-F1 Score of 0.502 in Track 1 and Track 2, respectively. In the Shared Task leaderboard, we secured 4th rank in Track 1 and 2nd rank in Track 2.
Blindspots in Python and Java APIs Result in Vulnerable Code
Venue: arXiv cs.SE 2021
Year: 2021

Blindspots in APIs can cause software engineers to introduce vulnerabilities, but such blindspots are, unfortunately, common. We study the effect APIs with blindspots have on developers in two languages by replicating an 109-developer, 24-Java-API controlled experiment. Our replication applies to Python and involves 129 new developers and 22 new APIs. We find that using APIs with blindspots statistically significantly reduces the developers' ability to correctly reason about the APIs in both languages, but that the effect is more pronounced for Python. Interestingly, for Java, the effect increased with complexity of the code relying on the API, whereas for Python, the opposite was true. Whether the developers considered API uses to be more difficult, less clear, and less familiar did not have an effect on their ability to correctly reason about them. Developers with better long-term memory recall were more likely to correctly reason about APIs with blindspots, but short-term memory, processing speed, episodic memory, and memory span had no effect. Surprisingly, professional experience and expertice did not improve the developers' ability to reason about APIs with blindspots across both languages, with long-term professionals with many years of experience making mistakes as often as relative novices. Finally, personality traits did not significantly affect the Python developers' ability to reason about APIs with blindspots, but less extraverted and more open developers were better at reasoning about Java APIs with blindspots. Overall, our findings suggest that blindspots in APIs are a serious problem across languages, and that experience and education alone do not overcome that problem, suggesting that tools are needed to help developers recognize blindspots in APIs as they write code that uses those APIs.
Learning to Generate Music With Sentiment
Venue: arXiv cs.LG 2021
Year: 2021

Deep Learning models have shown very promising results in automatically composing polyphonic music pieces. However, it is very hard to control such models in order to guide the compositions towards a desired goal. We are interested in controlling a model to automatically generate music with a given sentiment. This paper presents a generative Deep Learning model that can be directed to compose music with a given sentiment. Besides music generation, the same model can be used for sentiment analysis of symbolic music. We evaluate the accuracy of the model in classifying sentiment of symbolic music using a new dataset of video game soundtracks. Results show that our model is able to obtain good prediction accuracy. A user study shows that human subjects agreed that the generated music has the intended sentiment, however negative pieces can be ambiguous.
Sim2Real 3D Object Classification using Spherical Kernel Point Convolution and a Deep Center Voting Scheme
Venue: arXiv cs.CV 2021
Year: 2021

While object semantic understanding is essential for most service robotic tasks, 3D object classification is still an open problem. Learning from artificial 3D models alleviates the cost of annotation necessary to approach this problem, but most methods still struggle with the differences existing between artificial and real 3D data. We conjecture that the cause of those issue is the fact that many methods learn directly from point coordinates, instead of the shape, as the former is hard to center and to scale under variable occlusions reliably. We introduce spherical kernel point convolutions that directly exploit the object surface, represented as a graph, and a voting scheme to limit the impact of poor segmentation on the classification results. Our proposed approach improves upon state-of-the-art methods by up to 36% when transferring from artificial objects to real objects.
Automatic Speaker Independent Dysarthric Speech Intelligibility Assessment System
Venue: arXiv cs.SD 2021
Year: 2021

Dysarthria is a condition which hampers the ability of an individual to control the muscles that play a major role in speech delivery. The loss of fine control over muscles that assist the movement of lips, vocal chords, tongue and diaphragm results in abnormal speech delivery. One can assess the severity level of dysarthria by analyzing the intelligibility of speech spoken by an individual. Continuous intelligibility assessment helps speech language pathologists not only study the impact of medication but also allows them to plan personalized therapy. It helps the clinicians immensely if the intelligibility assessment system is reliable, automatic, simple for (a) patients to undergo and (b) clinicians to interpret. Lack of availability of dysarthric data has resulted in development of speaker dependent automatic intelligibility assessment systems which requires patients to speak a large number of utterances. In this paper, we propose (a) a cost minimization procedure to select an optimal (small) number of utterances that need to be spoken by the dysarthric patient, (b) four different speaker independent intelligibility assessment systems which require the patient to speak a small number of words, and (c) the assessment score is close to the perceptual score that the Speech Language Pathologist (SLP) can relate to. The need for small number of utterances to be spoken by the patient and the score being relatable to the SLP benefits both the dysarthric patient and the clinician from usability perspective.
Equilibria in Auctions With Ad Types
Venue: arXiv cs.GT 2021
Year: 2021

This paper studies equilibrium quality of semi-separable position auctions (known as the Ad Types setting) with greedy or optimal allocation combined with generalized second-price (GSP) or Vickrey-Clarke-Groves (VCG) pricing. We make three contributions: first, we give upper and lower bounds on the Price of Anarchy (PoA) for auctions which use greedy allocation with GSP pricing, greedy allocations with VCG pricing, and optimal allocation with GSP pricing. Second, we give Bayes-Nash equilibrium characterizations for two-player, two-slot instances (for all auction formats) and show that there exists both a revenue hierarchy and revenue equivalence across some formats. Finally, we use no-regret learning algorithms and bidding data from a large online advertising platform and no-regret learning algorithms to evaluate the performance of the mechanisms under semi-realistic conditions. For welfare, we find that the optimal-to-realized welfare ratio (an empirical PoA analogue) is broadly better than our upper bounds on PoA; For revenue, we find that the hierarchy in practice may sometimes agree with simple theory, but generally appears sensitive to the underlying distribution of bidder valuations.
"This Browser is Lightning Fast": The Effects of Message Content on Perceived Performance
Venue: arXiv cs.HC 2021
Year: 2021

With technical performance being similar for various web browsers, improving user perceived performance is integral to optimizing browser quality. We investigated the importance of priming, which has a well-documented ability to affect people's beliefs, on users' perceptions of web browser performance. We studied 1495 participants who read either an article about performance improvements to Mozilla Firefox, an article about user interface updates to Firefox, or an article about self-driving cars, and then watched video clips of browser tasks. As the priming effect would suggest, we found that reading articles about Firefox increased participants' perceived performance of Firefox over the most widely used web browser, Google Chrome. In addition, we found that article content mattered, as the article about performance improvements led to higher performance ratings than the article about UI updates. Our findings demonstrate how perceived performance can be improved without making technical improvements and that designers and developers must consider a wider picture when trying to improve user attitudes about technology.
A Relational-learning Perspective to Multi-label Chest X-ray Classification
Venue: arXiv cs.AI 2021
Year: 2021

Multi-label classification of chest X-ray images is frequently performed using discriminative approaches, i.e. learning to map an image directly to its binary labels. Such approaches make it challenging to incorporate auxiliary information such as annotation uncertainty or a dependency among the labels. Building towards this, we propose a novel knowledge graph reformulation of multi-label classification, which not only readily increases predictive performance of an encoder but also serves as a general framework for introducing new domain knowledge. Specifically, we construct a multi-modal knowledge graph out of the chest X-ray images and its labels and pose multi-label classification as a link prediction problem. Incorporating auxiliary information can then simply be achieved by adding additional nodes and relations among them. When tested on a publicly-available radiograph dataset (CheXpert), our relational-reformulation using a naive knowledge graph outperforms the state-of-art by achieving an area-under-ROC curve of 83.5%, an improvement of "sim 1" over a purely discriminative approach.
Real-Time RGBD Odometry for Fused-State Navigation Systems
Venue: arXiv cs.RO 2021
Year: 2021

This article describes an algorithm that provides visual odometry estimates from sequential pairs of RGBD images. The key contribution of this article on RGBD odometry is that it provides both an odometry estimate and a covariance for the odometry parameters in real-time via a representative covariance matrix. Accurate, real-time parameter covariance is essential to effectively fuse odometry measurements into most navigation systems. To date, this topic has seen little treatment in research which limits the impact existing RGBD odometry approaches have for localization in these systems. Covariance estimates are obtained via a statistical perturbation approach motivated by real-world models of RGBD sensor measurement noise. Results discuss the accuracy of our RGBD odometry approach with respect to ground truth obtained from a motion capture system and characterizes the suitability of this approach for estimating the true RGBD odometry parameter uncertainty.
The AI Index 2021 Annual Report
Venue: arXiv cs.AI 2021
Year: 2021

Welcome to the fourth edition of the AI Index Report. This year we significantly expanded the amount of data available in the report, worked with a broader set of external organizations to calibrate our data, and deepened our connections with the Stanford Institute for Human-Centered Artificial Intelligence (HAI). The AI Index Report tracks, collates, distills, and visualizes data related to artificial intelligence. Its mission is to provide unbiased, rigorously vetted, and globally sourced data for policymakers, researchers, executives, journalists, and the general public to develop intuitions about the complex field of AI. The report aims to be the most credible and authoritative source for data and insights about AI in the world.
Hard Attention Control By Mutual Information Maximization
Venue: arXiv cs.AI 2021
Year: 2021

Biological agents have adopted the principle of attention to limit the rate of incoming information from the environment. One question that arises is if an artificial agent has access to only a limited view of its surroundings, how can it control its attention to effectively solve tasks? We propose an approach for learning how to control a hard attention window by maximizing the mutual information between the environment state and the attention location at each step. The agent employs an internal world model to make predictions about its state and focuses attention towards where the predictions may be wrong. Attention is trained jointly with a dynamic memory architecture that stores partial observations and keeps track of the unobserved state. We demonstrate that our approach is effective in predicting the full state from a sequence of partial observations. We also show that the agent's internal representation of the surroundings, a live mental map, can be used for control in two partially observable reinforcement learning tasks. Videos of the trained agent can be found at https://sites.google.com/view/hard-attention-control.
An Improved Two-Archive Evolutionary Algorithm for Constrained Multi-Objective Optimization
Venue: arXiv cs.NE 2021
Year: 2021

Constrained multi-objective optimization problems (CMOPs) are ubiquitous in real-world engineering optimization scenarios. A key issue in constrained multi-objective optimization is to strike a balance among convergence, diversity and feasibility. A recently proposed two-archive evolutionary algorithm for constrained multi-objective optimization (C-TAEA) has be shown as a latest algorithm. However, due to its simple implementation of the collaboration mechanism between its two co-evolving archives, C-TAEA is struggling when solving problems whose \textit{pseudo} Pareto-optimal front, which does not take constraints into consideration, dominates the \textit{feasible} Pareto-optimal front. In this paper, we propose an improved version C-TAEA, dubbed C-TAEA-II, featuring an improved update mechanism of two co-evolving archives and an adaptive mating selection mechanism to promote a better collaboration between co-evolving archives. Empirical results demonstrate the competitiveness of the proposed C-TAEA-II in comparison with five representative constrained evolutionary multi-objective optimization algorithms.