OpenRaiser/Intern-Atlas huggingface dataset:
Counterfactual fairness: removing direct effects through regularization
Venue: arXiv cs.AI 2020
Year: 2020

Building machine learning models that are fair with respect to an unprivileged group is a topical problem. Modern fairness-aware algorithms often ignore causal effects and enforce fairness through modifications applicable to only a subset of machine learning models. In this work, we propose a new definition of fairness that incorporates causality through the Controlled Direct Effect (CDE). We develop regularizations to tackle classical fairness measures and present a causal regularization that satisfies our new fairness definition by removing the impact of unprivileged group variables on the model outcomes as measured by the CDE. These regularizations are applicable to any model trained using by iteratively minimizing a loss through differentiation. We demonstrate our approaches using both gradient boosting and logistic regression on: a synthetic dataset, the UCI Adult (Census) Dataset, and a real-world credit-risk dataset. Our results were found to mitigate unfairness from the predictions with small reductions in model performance.
What BERT Sees: Cross-Modal Transfer for Visual Question Generation
Venue: arXiv cs.CL 2020
Year: 2020

Pre-trained language models have recently contributed to significant advances in NLP tasks. Recently, multi-modal versions of BERT have been developed, using heavy pre-training relying on vast corpora of aligned textual and image data, primarily applied to classification tasks such as VQA. In this paper, we are interested in evaluating the visual capabilities of BERT out-of-the-box, by avoiding pre-training made on supplementary data. We choose to study Visual Question Generation, a task of great interest for grounded dialog, that enables to study the impact of each modality (as input can be visual and/or textual). Moreover, the generation aspect of the task requires an adaptation since BERT is primarily designed as an encoder. We introduce BERT-gen, a BERT-based architecture for text generation, able to leverage on either mono- or multi- modal representations. The results reported under different configurations indicate an innate capacity for BERT-gen to adapt to multi-modal data and text generation, even with few data available, avoiding expensive pre-training. The proposed model obtains substantial improvements over the state-of-the-art on two established VQG datasets.
Routing in Unit Disk Graphs without Dynamic Headers
Venue: arXiv cs.CG 2020
Year: 2020

Let $V\subset\mathbb{R}^2$ be a set of $n$ sites in the plane. The unit disk graph $DG(V)$ of $V$ is the graph with vertex set $V$ in which two sites $v$ and $w$ are adjacent if and only if their Euclidean distance is at most $1$. We develop a compact routing scheme for $DG(V)$. The routing scheme preprocesses $DG(V)$ by assigning a label $l(v)$ to every site $v$ in $V$. After that, for any two sites $s$ and $t$, the scheme must be able to route a packet from $s$ to $t$ as follows: given the label of a current vertex $r$ (initially, $r=s$) and the label of the target vertex $t$, the scheme determines a neighbor $r'$ of $r$. Then, the packet is forwarded to $r'$, and the process continues until the packet reaches its desired target $t$. The resulting path between the source $s$ and the target $t$ is called the routing path of $s$ and $t$. The stretch of the routing scheme is the maximum ratio of the total Euclidean length of the routing path and of the shortest path in $DG(V)$, between any two sites $s, t \in V$. We show that for any given $\varepsilon>0$, we can construct a routing scheme for $DG(V)$ with diameter $D$ that achieves stretch $1+\varepsilon$ and label size $O(\log D\log^3n/\log\log n)$ (the constant in the $O$-Notation depends on $\varepsilon$). In the past, several routing schemes for unit disk graphs have been proposed. Our scheme is the first one to achieve poly-logarithmic label size and arbitrarily small stretch without storing any additional information in the packet.
Small-Footprint Open-Vocabulary Keyword Spotting with Quantized LSTM Networks
Venue: arXiv cs.CL 2020
Year: 2020

We explore a keyword-based spoken language understanding system, in which the intent of the user can directly be derived from the detection of a sequence of keywords in the query. In this paper, we focus on an open-vocabulary keyword spotting method, allowing the user to define their own keywords without having to retrain the whole model. We describe the different design choices leading to a fast and small-footprint system, able to run on tiny devices, for any arbitrary set of user-defined keywords, without training data specific to those keywords. The model, based on a quantized long short-term memory (LSTM) neural network, trained with connectionist temporal classification (CTC), weighs less than 500KB. Our approach takes advantage of some properties of the predictions of CTC-trained networks to calibrate the confidence scores and implement a fast detection algorithm. The proposed system outperforms a standard keyword-filler model approach.
Learning Machines from Simulation to Real World
Venue: arXiv cs.RO 2020
Year: 2020

Learning Machines is developing a flexible, cross-industry, advanced analytics platform, targeted during stealth-stage at a limited number of specific vertical applications. In this paper, we aim to integrate a general machine system to learn a variant of tasks from simulation to real world. In such a machine system, it involves real-time robot vision, sensor fusion, and learning algorithms (reinforcement learning). To this end, we demonstrate the general machine system on three fundamental tasks including obstacle avoidance, foraging, and predator-prey robot. The proposed solutions are implemented on Robobo robots with mobile device (smartphone with camera) as interface and built-in infrared (IR) sensors. The agent is trained in a virtual environment. In order to assess its performance, the learned agent is tested in the virtual environment and reproduce the same results in a real environment. The results show that the reinforcement learning algorithm can be reliably used for a variety of tasks in unknown environments.
Toward dynamical crowd control to prevent hazardous situations
Venue: arXiv cs.HC 2020
Year: 2020

It is common for large crowds to gather to attend games, exhibitions, political rallies, and other events. Thus, careful designs and operational plans are made to ensure the safe, secure, and efficient movement of people in these crowded environments. However, the congestion created by large crowds has resulted in hazardous incidents across the world. Developments in information technology can provide new means to disseminate public information, thus changing human behavior in situations of danger and duress. In this paper, we propose a crowd control and evacuation guidance management system using digital promotional signage to demonstrate the effects of crowd control via simulations.
Cross-layer Feature Pyramid Network for Salient Object Detection
Venue: arXiv cs.CV 2020
Year: 2020

Feature pyramid network (FPN) based models, which fuse the semantics and salient details in a progressive manner, have been proven highly effective in salient object detection. However, it is observed that these models often generate saliency maps with incomplete object structures or unclear object boundaries, due to the \emph{indirect} information propagation among distant layers that makes such fusion structure less effective. In this work, we propose a novel Cross-layer Feature Pyramid Network (CFPN), in which direct cross-layer communication is enabled to improve the progressive fusion in salient object detection. Specifically, the proposed network first aggregates multi-scale features from different layers into feature maps that have access to both the high- and low-level information. Then, it distributes the aggregated features to all the involved layers to gain access to richer context. In this way, the distributed features per layer own both semantics and salient details from all other layers simultaneously, and suffer reduced loss of important information. Extensive experimental results over six widely used salient object detection benchmarks and with three popular backbones clearly demonstrate that CFPN can accurately locate fairly complete salient regions and effectively segment the object boundaries.
A more abstractive summarization model
Venue: arXiv cs.CL 2020
Year: 2020

Pointer-generator network is an extremely popular method of text summarization. More recent works in this domain still build on top of the baseline pointer generator by augmenting a content selection phase, or by decomposing the decoder into a contextual network and a language model. However, all such models that are based on the pointer-generator base architecture cannot generate novel words in the summary and mostly copy words from the source text. In our work, we first thoroughly investigate why the pointer-generator network is unable to generate novel words, and then address that by adding an Out-of-vocabulary (OOV) penalty. This enables us to improve the amount of novelty/abstraction significantly. We use normalized n-gram novelty scores as a metric for determining the level of abstraction. Moreover, we also report rouge scores of our model since most summarization models are evaluated with R-1, R-2, R-L scores.
AutoFoley: Artificial Synthesis of Synchronized Sound Tracks for Silent Videos with Deep Learning
Venue: arXiv cs.SD 2020
Year: 2020

In movie productions, the Foley Artist is responsible for creating an overlay soundtrack that helps the movie come alive for the audience. This requires the artist to first identify the sounds that will enhance the experience for the listener thereby reinforcing the Directors's intention for a given scene. In this paper, we present AutoFoley, a fully-automated deep learning tool that can be used to synthesize a representative audio track for videos. AutoFoley can be used in the applications where there is either no corresponding audio file associated with the video or in cases where there is a need to identify critical scenarios and provide a synthesized, reinforced soundtrack. An important performance criterion of the synthesized soundtrack is to be time-synchronized with the input video, which provides for a realistic and believable portrayal of the synthesized sound. Unlike existing sound prediction and generation architectures, our algorithm is capable of precise recognition of actions as well as inter-frame relations in fast moving video clips by incorporating an interpolation technique and Temporal Relationship Networks (TRN). We employ a robust multi-scale Recurrent Neural Network (RNN) associated with a Convolutional Neural Network (CNN) for a better understanding of the intricate input-to-output associations over time. To evaluate AutoFoley, we create and introduce a large scale audio-video dataset containing a variety of sounds frequently used as Foley effects in movies. Our experiments show that the synthesized sounds are realistically portrayed with accurate temporal synchronization of the associated visual inputs. Human qualitative testing of AutoFoley show over 73% of the test subjects considered the generated soundtrack as original, which is a noteworthy improvement in cross-modal research in sound synthesis.
Semantic Relatedness for Keyword Disambiguation: Exploiting Different Embeddings
Venue: arXiv cs.CL 2020
Year: 2020

Understanding the meaning of words is crucial for many tasks that involve human-machine interaction. This has been tackled by research in Word Sense Disambiguation (WSD) in the Natural Language Processing (NLP) field. Recently, WSD and many other NLP tasks have taken advantage of embeddings-based representation of words, sentences, and documents. However, when it comes to WSD, most embeddings models suffer from ambiguity as they do not capture the different possible meanings of the words. Even when they do, the list of possible meanings for a word (sense inventory) has to be known in advance at training time to be included in the embeddings space. Unfortunately, there are situations in which such a sense inventory is not known in advance (e.g., an ontology selected at run-time), or it evolves with time and its status diverges from the one at training time. This hampers the use of embeddings models for WSD. Furthermore, traditional WSD techniques do not perform well in situations in which the available linguistic information is very scarce, such as the case of keyword-based queries. In this paper, we propose an approach to keyword disambiguation which grounds on a semantic relatedness between words and senses provided by an external inventory (ontology) that is not known at training time. Building on previous works, we present a semantic relatedness measure that uses word embeddings, and explore different disambiguation algorithms to also exploit both word and sentence representations. Experimental results show that this approach achieves results comparable with the state of the art when applied for WSD, without training for a particular domain.
Interactive, Effort-Aware Library Version Harmonization
Venue: arXiv cs.SE 2020
Year: 2020

As a mixed result of intensive dependency on third-party libraries, flexible mechanism to declare dependencies, and increased number of modules in a project, multiple versions of the same third-party library are directly depended in different modules of a project. Such library version inconsistencies can increase dependency maintenance cost, or even lead to dependency conflicts when modules are inter-dependent. Although automated build tools (e.g., Maven's enforcer plugin) provide partial support to detect library version inconsistencies, they do not provide any support to harmonize inconsistent library versions. We first conduct a survey with 131 Java developers from GitHub to retrieve first-hand information about the root causes, detection methods, reasons for fixing or not fixing, fixing strategies, fixing efforts, and tool expectations on library version inconsistencies. Then, based on the insights from our survey, we propose LibHarmo, an interactive, effort-aware library version harmonization technique, to detect library version inconsistencies, interactively suggest a harmonized version with the least harmonization efforts based on library API usage analysis, and refactor build configuration files. LibHarmo is currently developed for Java Maven projects. Our experimental study on 443 highly-starred Java Maven projects from GitHub indicates that i) LibHarmo identifies 621 library version inconsistencies covering 152 (34.3%) of projects, and ii) the average harmonization efforts are that 1 and 12 library API calls are affected, respectively due to the deleted and changed library APIs in the harmonized version. 5 library version inconsistencies have been confirmed, and 1 of them has been already harmonized by developers.
Optimal Gradient Quantization Condition for Communication-Efficient Distributed Training
Venue: arXiv cs.LG 2020
Year: 2020

The communication of gradients is costly for training deep neural networks with multiple devices in computer vision applications. In particular, the growing size of deep learning models leads to higher communication overheads that defy the ideal linear training speedup regarding the number of devices. Gradient quantization is one of the common methods to reduce communication costs. However, it can lead to quantization error in the training and result in model performance degradation. In this work, we deduce the optimal condition of both the binary and multi-level gradient quantization for \textbf{ANY} gradient distribution. Based on the optimal condition, we develop two novel quantization schemes: biased BinGrad and unbiased ORQ for binary and multi-level gradient quantization respectively, which dynamically determine the optimal quantization levels. Extensive experimental results on CIFAR and ImageNet datasets with several popular convolutional neural networks show the superiority of our proposed methods.
Who is the Centre of the Movie Universe? Using Python and NetworkX to Analyse the Social Network of Movie Stars
Venue: arXiv cs.SI 2020
Year: 2020

This paper provides the technical details of an article originally published in The Conversation in February 2020. The purpose is to use centrality measures to analyse the social network of movie stars and thereby identify the most "important" actors in the movie business. The analysis is presented in a step-by-step, tutorial-like fashion and makes use of the Python programming language together with the NetworkX library. It reveals that the most central actors in the network are those with lengthy acting careers, such as Christopher Lee, Nassar, Sukumari, Michael Caine, Om Puri, Jackie Chan, and Robert De Niro. We also present similar results for the movie releases of each decade. These indicate that the most central actors since the turn of the millennium include people like Angelina Jolie, Brahmanandam, Samuel L. Jackson, Nassar, and Ben Kingsley.
PASCAL: Timing SCA Resistant Design and Verification Flow
Venue: arXiv cs.CR 2020
Year: 2020

A large number of crypto accelerators are being deployed with the widespread adoption of IoT. It is vitally important that these accelerators and other security hardware IPs are provably secure. Security is an extra functional requirement and hence many security verification tools are not mature. We propose an approach/flow-PASCAL-that works on RTL designs and discovers potential Timing Side-Channel Attack(SCA) vulnerabilities in them. Based on information flow analysis, this is able to identify Timing Disparate Security Paths that could lead to information leakage. This flow also (automatically) eliminates the information leakage caused by the timing channel. The insertion of a lightweight Compensator Block as balancing or compliance FSM removes the timing channel with minimum modifications to the design with no impact on the clock cycle time or combinational delay of the critical path in the circuit.
End-to-End Entity Linking and Disambiguation leveraging Word and Knowledge Graph Embeddings
Venue: arXiv cs.CL 2020
Year: 2020

Entity linking - connecting entity mentions in a natural language utterance to knowledge graph (KG) entities is a crucial step for question answering over KGs. It is often based on measuring the string similarity between the entity label and its mention in the question. The relation referred to in the question can help to disambiguate between entities with the same label. This can be misleading if an incorrect relation has been identified in the relation linking step. However, an incorrect relation may still be semantically similar to the relation in which the correct entity forms a triple within the KG; which could be captured by the similarity of their KG embeddings. Based on this idea, we propose the first end-to-end neural network approach that employs KG as well as word embeddings to perform joint relation and entity classification of simple questions while implicitly performing entity disambiguation with the help of a novel gating mechanism. An empirical evaluation shows that the proposed approach achieves a performance comparable to state-of-the-art entity linking while requiring less post-processing.
Topologically sensitive metaheuristics
Venue: arXiv cs.AI 2020
Year: 2020

This paper proposes topologically sensitive metaheuristics, and describes conceptual design of topologically sensitive Variable Neighborhood Search method (TVNS) and topologically sensitive Electromagnetism Metaheuristic (TEM).
Voronoi-based similarity distances between arbitrary crystal lattices
Venue: arXiv cs.CG 2020
Year: 2020

This paper develops a new continuous approach to a similarity between periodic lattices of ideal crystals. Quantifying a similarity between crystal structures is needed to substantially speed up the Crystal Structure Prediction, because the prediction of many target properties of crystal structures is computationally slow and is essentially repeated for many nearly identical simulated structures. The proposed distances between arbitrary periodic lattices of crystal structures are invariant under all rigid motions, satisfy the metric axioms and continuity under atomic perturbations. The above properties make these distances ideal tools for clustering and visualizing large datasets of crystal structures. All the conclusions are rigorously proved and justified by experiments on real and simulated crystal structures reported in the Nature 2017 paper "Functional materials discovery using energy-structure-function maps".
Dynamic Set Cover: Improved Amortized and Worst-Case Update Time
Venue: arXiv cs.DS 2020
Year: 2020

In the dynamic minimum set cover problem, a challenge is to minimize the update time while guaranteeing close to the optimal $\min(O(\log n), f)$ approximation factor. (Throughout, $m$, $n$, $f$, and $C$ are parameters denoting the maximum number of sets, number of elements, frequency, and the cost range.) In the high-frequency range, when $f=Ω(\log n)$, this was achieved by a deterministic $O(\log n)$-approximation algorithm with $O(f \log n)$ amortized update time [Gupta et al. STOC'17]. In the low-frequency range, the line of work by Gupta et al. [STOC'17], Abboud et al. [STOC'19], and Bhattacharya et al. [ICALP'15, IPCO'17, FOCS'19] led to a deterministic $(1+ε)f$-approximation algorithm with $O(f \log (Cn)/ε^2)$ amortized update time. In this paper we improve the latter update time and provide the first bounds that subsume (and sometimes improve) the state-of-the-art dynamic vertex cover algorithms. We obtain: 1. $(1+ε)f$-approximation ratio in $O(f\log^2 (Cn)/ε^3)$ worst-case update time: No non-trivial worst-case update time was previously known for dynamic set cover. Our bound subsumes and improves by a logarithmic factor the $O(\log^3 n/\text{poly}(ε))$ worst-case update time for unweighted dynamic vertex cover (i.e., when $f=2$ and $C=1$) by Bhattacharya et al. [SODA'17]. 2. $(1+ε)f$-approximation ratio in $O\left((f^2/ε^3)+(f/ε^2) \log C\right)$ amortized update time: This result improves the previous $O(f \log (Cn)/ε^2)$ update time bound for most values of $f$ in the low-frequency range, i.e. whenever $f=o(\log n)$. It is the first that is independent of $m$ and $n$. It subsumes the constant amortized update time of Bhattacharya and Kulkarni [SODA'19] for unweighted dynamic vertex cover (i.e., when $f = 2$ and $C = 1$).