OpenRaiser/Intern-Atlas huggingface dataset:
Sound Event Localization and Detection using Squeeze-Excitation Residual CNNs
Venue: arXiv cs.SD 2021
Year: 2021

Sound Event Localization and Detection (SELD) is a problem related to the field of machine listening whose objective is to recognize individual sound events, detect their temporal activity, and estimate their spatial location. Thanks to the emergence of more hard-labeled audio datasets, deep learning techniques have become state-of-the-art solutions. The most common ones are those that implement a convolutional recurrent network (CRNN) having previously transformed the audio signal into multichannel 2D representation. The squeeze-excitation technique can be considered as a convolution enhancement that aims to learn spatial and channel feature maps independently rather than together as standard convolutions do. This is usually achieved by combining some global clustering operators, linear operators and a final calibration between the block input and its learned relationships. This work aims to improve the accuracy results of the baseline CRNN presented in DCASE 2020 Task 3 by adding residual squeeze-excitation (SE) blocks in the convolutional part of the CRNN. The followed procedure involves a grid search of the ratio parameter (used in the linear relationships) of the residual SE block, whereas the hyperparameters of the network remain the same as in the baseline. Experiments show that by simply introducing the residual SE blocks, the results obtained are able to improve the baseline considerably.
Fully Abstract and Robust Compilation and How to Reconcile the Two, Abstractly
Venue: arXiv cs.PL 2021
Year: 2021

The most prominent formal criterion for secure compilation is full abstraction, the preservation and reflection of contextual equivalence. Recent work introduced robust compilation, defined as the preservation of robust satisfaction of hyperproperties, i.e., their satisfaction against arbitrary attackers. In this paper, we initially set out to compare these two approaches to secure compilation. To that end, we provide an exact description of the hyperproperties that are robustly satisfied by programs compiled with a fully abstract compiler, and show that they can be meaningless or trivial. We then propose a novel criterion for secure compilation formulated in the framework of Mathematical Operational Semantics (MOS), guaranteeing both full abstraction and the preservation of robust satisfaction of hyperproperties in a more sensible manner.
A Loss Function for Generative Neural Networks Based on Watson's Perceptual Model
Venue: arXiv cs.LG 2021
Year: 2021

To train Variational Autoencoders (VAEs) to generate realistic imagery requires a loss function that reflects human perception of image similarity. We propose such a loss function based on Watson's perceptual model, which computes a weighted distance in frequency space and accounts for luminance and contrast masking. We extend the model to color images, increase its robustness to translation by using the Fourier Transform, remove artifacts due to splitting the image into blocks, and make it differentiable. In experiments, VAEs trained with the new loss function generated realistic, high-quality image samples. Compared to using the Euclidean distance and the Structural Similarity Index, the images were less blurry; compared to deep neural network based losses, the new approach required less computational resources and generated images with less artifacts.
Critic Regularized Regression
Venue: arXiv cs.LG 2021
Year: 2021

Offline reinforcement learning (RL), also known as batch RL, offers the prospect of policy optimization from large pre-recorded datasets without online environment interaction. It addresses challenges with regard to the cost of data collection and safety, both of which are particularly pertinent to real-world applications of RL. Unfortunately, most off-policy algorithms perform poorly when learning from a fixed dataset. In this paper, we propose a novel offline RL algorithm to learn policies from data using a form of critic-regularized regression (CRR). We find that CRR performs surprisingly well and scales to tasks with high-dimensional state and action spaces -- outperforming several state-of-the-art offline RL algorithms by a significant margin on a wide range of benchmark tasks.
Dominate or Delete: Decentralized Competing Bandits in Serial Dictatorship
Venue: arXiv cs.LG 2021
Year: 2021

Online learning in a two-sided matching market, with demand side agents continuously competing to be matched with supply side (arms), abstracts the complex interactions under partial information on matching platforms (e.g. UpWork, TaskRabbit). We study the decentralized serial dictatorship setting, a two-sided matching market where the demand side agents have unknown and heterogeneous valuation over the supply side (arms), while the arms have known uniform preference over the demand side (agents). We design the first decentralized algorithm -- UCB with Decentralized Dominant-arm Deletion (UCB-D3), for the agents, that does not require any knowledge of reward gaps or time horizon. UCB-D3 works in phases, where in each phase, agents delete \emph{dominated arms} -- the arms preferred by higher ranked agents, and play only from the non-dominated arms according to the UCB. At the end of the phase, agents broadcast in a decentralized fashion, their estimated preferred arms through {\em pure exploitation}. We prove both, a new regret lower bound for the decentralized serial dictatorship model, and that UCB-D3 is order optimal.
Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes
Venue: arXiv cs.CV 2021
Year: 2021

In this paper we propose a novel human-centered approach for detecting forgery in face images, using dynamic prototypes as a form of visual explanations. Currently, most state-of-the-art deepfake detections are based on black-box models that process videos frame-by-frame for inference, and few closely examine their temporal inconsistencies. However, the existence of such temporal artifacts within deepfake videos is key in detecting and explaining deepfakes to a supervising human. To this end, we propose Dynamic Prototype Network (DPNet) -- an interpretable and effective solution that utilizes dynamic representations (i.e., prototypes) to explain deepfake temporal artifacts. Extensive experimental results show that DPNet achieves competitive predictive performance, even on unseen testing datasets such as Google's DeepFakeDetection, DeeperForensics, and Celeb-DF, while providing easy referential explanations of deepfake dynamics. On top of DPNet's prototypical framework, we further formulate temporal logic specifications based on these dynamics to check our model's compliance to desired temporal behaviors, hence providing trustworthiness for such critical detection systems.
Robot Inner Attention Modeling for Task-Adaptive Teaming of Heterogeneous Multi Robots
Venue: arXiv cs.RO 2021
Year: 2021

Attracted by team scale and function diversity, a heterogeneous multi-robot system (HMRS), where multiple robots with different functions and numbers are coordinated to perform tasks, has been widely used for complex and large-scale scenarios, including disaster search and rescue, site surveillance, and social security. However, due to the variety of the task requirements, it is challenging to accurately compose a robot team with appropriate sizes and functions to dynamically satisfy task needs while limiting the robot resource cost to a low level. To solve this problem, in this paper, a novel adaptive cooperation method, inner attention (innerATT), is developed to flexibly team heterogeneous robots to execute tasks as task types and environment change. innerATT is designed by integrating a novel attention mechanism into a multi-agent actor-critic reinforcement learning architecture. With an attention mechanism, robot capability will be analyzed to flexibly form teams to meet task requirements. Scenarios with different task variety ("Single Task", "Double Task", and "Mixed Task") were designed. The effectiveness of the innerATT was validated by its accuracy in flexible cooperation.
Laplacian Regularized Few-Shot Learning
Venue: arXiv cs.LG 2021
Year: 2021

We propose a transductive Laplacian-regularized inference for few-shot tasks. Given any feature embedding learned from the base classes, we minimize a quadratic binary-assignment function containing two terms: (1) a unary term assigning query samples to the nearest class prototype, and (2) a pairwise Laplacian term encouraging nearby query samples to have consistent label assignments. Our transductive inference does not re-train the base model, and can be viewed as a graph clustering of the query set, subject to supervision constraints from the support set. We derive a computationally efficient bound optimizer of a relaxation of our function, which computes independent (parallel) updates for each query sample, while guaranteeing convergence. Following a simple cross-entropy training on the base classes, and without complex meta-learning strategies, we conducted comprehensive experiments over five few-shot learning benchmarks. Our LaplacianShot consistently outperforms state-of-the-art methods by significant margins across different models, settings, and data sets. Furthermore, our transductive inference is very fast, with computational times that are close to inductive inference, and can be used for large-scale few-shot tasks.
Reliable Categorical Variational Inference with Mixture of Discrete Normalizing Flows
Venue: arXiv cs.LG 2021
Year: 2021

Variational approximations are increasingly based on gradient-based optimization of expectations estimated by sampling. Handling discrete latent variables is then challenging because the sampling process is not differentiable. Continuous relaxations, such as the Gumbel-Softmax for categorical distribution, enable gradient-based optimization, but do not define a valid probability mass for discrete observations. In practice, selecting the amount of relaxation is difficult and one needs to optimize an objective that does not align with the desired one, causing problems especially with models having strong meaningful priors. We provide an alternative differentiable reparameterization for categorical distribution by composing it as a mixture of discrete normalizing flows. It defines a proper discrete distribution, allows directly optimizing the evidence lower bound, and is less sensitive to the hyperparameter controlling relaxation.
Random Access in Persistent Strings and Segment Selection
Venue: arXiv cs.DS 2021
Year: 2021

We consider compact representations of collections of similar strings that support random access queries. The collection of strings is given by a rooted tree where edges are labeled by an edit operation (inserting, deleting, or replacing a character) and a node represents the string obtained by applying the sequence of edit operations on the path from the root to the node. The goal is to compactly represent the entire collection while supporting fast random access to any part of a string in the collection. This problem captures natural scenarios such as representing the past history of an edited document or representing highly-repetitive collections. Given a tree with $n$ nodes, we show how to represent the corresponding collection in $O(n)$ space and $O(\log n/ \log \log n)$ query time. This improves the previous time-space trade-offs for the problem. Additionally, we show a lower bound proving that the query time is optimal for any solution using near-linear space. To achieve our bounds for random access in persistent strings we show how to reduce the problem to the following natural geometric selection problem on line segments. Consider a set of horizontal line segments in the plane. Given parameters $i$ and $j$, a segment selection query returns the $j$th smallest segment (the segment with the $j$th smallest $y$-coordinate) among the segments crossing the vertical line through $x$-coordinate $i$. The segment selection problem is to preprocess a set of horizontal line segments into a compact data structure that supports fast segment selection queries. We present a solution that uses $O(n)$ space and support segment selection queries in $O(\log n/ \log \log n)$ time, where $n$ is the number of segments. Furthermore, we prove that that this query time is also optimal for any solution using near-linear space.
Random Assignment Under Bi-Valued Utilities: Analyzing Hylland-Zeckhauser, Nash-Bargaining, and other Rules
Venue: arXiv cs.GT 2021
Year: 2021

The Hylland-Zeckhauser (HZ) rule is a well-known rule for random assignment of items. The complexity of the rule has received renewed interest recently with Vazirani and Yannakakis (2020) proposing a strongly polynomial-time algorithm for the rule under bi-valued utilities, and making several general insights. We study the rule under the case of agents having bi-valued utilities. We point out several characterizations of the HZ rule, drawing clearer relations with several well-known rules in the literature. As a consequence, we point out alternative strongly polynomial-time algorithms for the HZ solution. We also give reductions from computing the HZ solution to computing well-known solutions based on leximin or Nash social welfare. An interesting contrast is that the HZ rule is group-strategyproof whereas the unconstrained competitive equilibrium with equal incomes rule is not even strategyproof. We clarify which results change when moving from 1-0 utilities to the more general bi-valued utilities. Finally, we prove that the closely related Nash bargaining solution violates envy-freeness and strategyproofness even under 1-0 utilities.
Intrinsic Autoencoders for Joint Neural Rendering and Intrinsic Image Decomposition
Venue: arXiv cs.CV 2021
Year: 2021

Neural rendering techniques promise efficient photo-realistic image synthesis while at the same time providing rich control over scene parameters by learning the physical image formation process. While several supervised methods have been proposed for this task, acquiring a dataset of images with accurately aligned 3D models is very difficult. The main contribution of this work is to lift this restriction by training a neural rendering algorithm from unpaired data. More specifically, we propose an autoencoder for joint generation of realistic images from synthetic 3D models while simultaneously decomposing real images into their intrinsic shape and appearance properties. In contrast to a traditional graphics pipeline, our approach does not require to specify all scene properties, such as material parameters and lighting by hand. Instead, we learn photo-realistic deferred rendering from a small set of 3D models and a larger set of unaligned real images, both of which are easy to acquire in practice. Simultaneously, we obtain accurate intrinsic decompositions of real images while not requiring paired ground truth. Our experiments confirm that a joint treatment of rendering and decomposition is indeed beneficial and that our approach outperforms state-of-the-art image-to-image translation baselines both qualitatively and quantitatively.
An Investigation of Traffic Density Changes inside Wuhan during the COVID-19 Epidemic with GF-2 Time-Series Images
Venue: arXiv cs.CV 2021
Year: 2021

In order to mitigate the spread of COVID-19, Wuhan was the first city to implement strict lockdown policy in 2020. Even though numerous researches have discussed the travel restriction between cities and provinces, few studies focus on the effect of transportation control inside the city due to the lack of the measurement and available data in Wuhan. Since the public transports have been shut down in the beginning of city lockdown, the change of traffic density is a good indicator to reflect the intracity population flow. Therefore, in this paper, we collected time-series high-resolution remote sensing images with the resolution of 1m acquired before, during and after Wuhan lockdown by GF-2 satellite. Vehicles on the road were extracted and counted for the statistics of traffic density to reflect the changes of human transmissions in the whole period of Wuhan lockdown. Open Street Map was used to obtain observation road surfaces, and a vehicle detection method combing morphology filter and deep learning was utilized to extract vehicles with the accuracy of 62.56%. According to the experimental results, the traffic density of Wuhan dropped with the percentage higher than 80%, and even higher than 90% on main roads during city lockdown; after lockdown lift, the traffic density recovered to the normal rate. Traffic density distributions also show the obvious reduction and increase throughout the whole study area. The significant reduction and recovery of traffic density indicates that the lockdown policy in Wuhan show effectiveness in controlling human transmission inside the city, and the city returned to normal after lockdown lift.
Adversarial Learning for Debiasing Knowledge Graph Embeddings
Venue: arXiv cs.LG 2021
Year: 2021

Knowledge Graphs (KG) are gaining increasing attention in both academia and industry. Despite their diverse benefits, recent research have identified social and cultural biases embedded in the representations learned from KGs. Such biases can have detrimental consequences on different population and minority groups as applications of KG begin to intersect and interact with social spheres. This paper aims at identifying and mitigating such biases in Knowledge Graph (KG) embeddings. As a first step, we explore popularity bias -- the relationship between node popularity and link prediction accuracy. In case of node2vec graph embeddings, we find that prediction accuracy of the embedding is negatively correlated with the degree of the node. However, in case of knowledge-graph embeddings (KGE), we observe an opposite trend. As a second step, we explore gender bias in KGE, and a careful examination of popular KGE algorithms suggest that sensitive attribute like the gender of a person can be predicted from the embedding. This implies that such biases in popular KGs is captured by the structural properties of the embedding. As a preliminary solution to debiasing KGs, we introduce a novel framework to filter out the sensitive attribute information from the KG embeddings, which we call FAN (Filtering Adversarial Network). We also suggest the applicability of FAN for debiasing other network embeddings which could be explored in future work.
A Survey on Recent Progress in the Theory of Evolutionary Algorithms for Discrete Optimization
Venue: arXiv cs.NE 2021
Year: 2021

The theory of evolutionary computation for discrete search spaces has made significant progress in the last ten years. This survey summarizes some of the most important recent results in this research area. It discusses fine-grained models of runtime analysis of evolutionary algorithms, highlights recent theoretical insights on parameter tuning and parameter control, and summarizes the latest advances for stochastic and dynamic problems. We regard how evolutionary algorithms optimize submodular functions and we give an overview over the large body of recent results on estimation of distribution algorithms. Finally, we present the state of the art of drift analysis, one of the most powerful analysis technique developed in this field.
A data science approach to drug safety: Semantic and visual mining of adverse drug events from clinical trials of pain treatments
Venue: arXiv cs.CY 2021
Year: 2021

Clinical trials are the basis of Evidence-Based Medicine. Trial results are reviewed by experts and consensus panels for producing meta-analyses and clinical practice guidelines. However, reviewing these results is a long and tedious task, hence the meta-analyses and guidelines are not updated each time a new trial is published. Moreover, the independence of experts may be difficult to appraise. On the contrary, in many other domains, including medical risk analysis, the advent of data science, big data and visual analytics allowed moving from expert-based to fact-based knowledge. Since 12 years, many trial results are publicly available online in trial registries. Nevertheless, data science methods have not yet been applied widely to trial data. In this paper, we present a platform for analyzing the safety events reported during clinical trials and published in trial registries. This platform is based on an ontological model including 582 trials on pain treatments, and uses semantic web technologies for querying this dataset at various levels of granularity. It also relies on a 26-dimensional flower glyph for the visualization of the Adverse Drug Events (ADE) rates in 13 categories and 2 levels of seriousness. We illustrate the interest of this platform through several use cases and we were able to find back conclusions that were initially found during meta-analyses. The platform was presented to four experts in drug safety, and is publicly available online, with the ontology of pain treatment ADE.
Learning Post-Hoc Causal Explanations for Recommendation
Venue: arXiv cs.IR 2021
Year: 2021

State-of-the-art recommender systems have the ability to generate high-quality recommendations, but usually cannot provide intuitive explanations to humans due to the usage of black-box prediction models. The lack of transparency has highlighted the critical importance of improving the explainability of recommender systems. In this paper, we propose to extract causal rules from the user interaction history as post-hoc explanations for the black-box sequential recommendation mechanisms, whilst maintain the predictive accuracy of the recommendation model. Our approach firstly achieves counterfactual examples with the aid of a perturbation model, and then extracts personalized causal relationships for the recommendation model through a causal rule mining algorithm. Experiments are conducted on several state-of-the-art sequential recommendation models and real-world datasets to verify the performance of our model on generating causal explanations. Meanwhile, We evaluate the discovered causal explanations in terms of quality and fidelity, which show that compared with conventional association rules, causal rules can provide personalized and more effective explanations for the behavior of black-box recommendation models.
Data Movement Is All You Need: A Case Study on Optimizing Transformers
Venue: arXiv cs.LG 2021
Year: 2021

Transformers are one of the most important machine learning workloads today. Training one is a very compute-intensive task, often taking days or weeks, and significant attention has been given to optimizing transformers. Despite this, existing implementations do not efficiently utilize GPUs. We find that data movement is the key bottleneck when training. Due to Amdahl's Law and massive improvements in compute performance, training has now become memory-bound. Further, existing frameworks use suboptimal data layouts. Using these insights, we present a recipe for globally optimizing data movement in transformers. We reduce data movement by up to 22.91% and overall achieve a 1.30x performance improvement over state-of-the-art frameworks when training a BERT encoder layer and 1.19x for the entire BERT. Our approach is applicable more broadly to optimizing deep neural networks, and offers insight into how to tackle emerging performance bottlenecks.