OpenRaiser/Intern-Atlas huggingface dataset:
Context-guided Triple Matching for Multiple Choice Question Answering
Venue: arXiv cs.CL 2021
Year: 2021

The task of multiple choice question answering (MCQA) refers to identifying a suitable answer from multiple candidates, by estimating the matching score among the triple of the passage, question and answer. Despite the general research interest in this regard, existing methods decouple the process into several pair-wise or dual matching steps, that limited the ability of assessing cases with multiple evidence sentences. To alleviate this issue, this paper introduces a novel Context-guided Triple Matching algorithm, which is achieved by integrating a Triple Matching (TM) module and a Contrastive Regularization (CR). The former is designed to enumerate one component from the triple as the background context, and estimate its semantic matching with the other two. Additionally, the contrastive term is further proposed to capture the dissimilarity between the correct answer and distractive ones. We validate the proposed algorithm on several benchmarking MCQA datasets, which exhibits competitive performances against state-of-the-arts.
Attention Gate in Traffic Forecasting
Venue: arXiv cs.CV 2021
Year: 2021

Because of increased urban complexity and growing populations, more and more challenges about predicting city-wide mobility behavior are being organized. Traffic Map Movie Forecasting Challenge 2020 is secondly held in the competition track of the Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS). Similar to Traffic4Cast 2019, the task is to predict traffic flow volume, average speed in major directions on the geographical area of three big cities: Berlin, Istanbul, and Moscow. In this paper, we apply the attention mechanism on U-Net based model, especially we add an attention gate on the skip-connection between contraction path and expansion path. An attention gates filter features from the contraction path before combining with features on the expansion path, it enables our model to reduce the effect of non-traffic region features and focus more on crucial region features. In addition to the competition data, we also propose two extra features which often affect traffic flow, that are time and weekdays. We experiment with our model on the competition dataset and reproduce the winner solution in the same environment. Overall, our model archives better performance than recent methods.
Using neural networks to solve the 2D Poisson equation for electric field computation in plasma fluid simulations
Venue: arXiv cs.LG 2021
Year: 2021

The Poisson equation is critical to get a self-consistent solution in plasma fluid simulations used for Hall effect thrusters and streamer discharges, since the Poisson solution appears as a source term of the unsteady nonlinear flow equations. As a first step, solving the 2D Poisson equation with zero Dirichlet boundary conditions using a deep neural network is investigated using multiple-scale architectures, defined in terms of number of branches, depth and receptive field. One key objective is to better understand how neural networks learn the Poisson solutions and provide guidelines to achieve optimal network configurations, especially when coupled to the time-varying Euler equations with plasma source terms. Here, the Receptive Field is found critical to correctly capture large topological structures of the field. The investigation of multiple architectures, losses, and hyperparameters provides an optimal network to solve accurately the steady Poisson problem. The performance of the optimal neural network solver, called PlasmaNet, is then monitored on meshes with increasing number of nodes, and compared with classical parallel linear solvers. Next, PlasmaNet is coupled with an unsteady Euler plasma fluid equations solver in the context of the electron plasma oscillation test case. In this time-evolving problem, a physical loss is necessary to produce a stable simulation. PlasmaNet is finally tested on a more complex case of discharge propagation involving chemistry and advection. The guidelines established in previous sections are applied to build the CNN to solve the same Poisson equation in cylindrical coordinates with different boundary conditions. Results reveal good CNN predictions and pave the way to new computational strategies using modern GPU-based hardware to predict unsteady problems involving a Poisson equation.
Inferring Facing Direction from Voice Signals
Venue: arXiv cs.SD 2021
Year: 2021

Consider a home or office where multiple devices are running voice assistants (e.g., TVs, lights, ovens, refrigerators, etc.). A human user turns to a particular device and gives a voice command, such as ``Alexa, can you ...''. This paper focuses on the problem of detecting which device the user was facing, and therefore, enabling only that device to respond to the command. Our core intuition emerges from the fact that human voice exhibits a directional radiation pattern, and the orientation of this pattern should influence the signal received at each device. Unfortunately, indoor multipath, unknown user location, and unknown voice signals pose as critical hurdles. Through a new algorithm that estimates the line-of-sight (LoS) power from a given signal, and combined with beamforming and triangulation, we design a functional solution called CoDIR. Results from $500+$ configurations, across $5$ rooms and $9$ different users, are encouraging. While improvements are necessary, we believe this is an important step forward in a challenging but urgent problem space.
GANiry: Bald-to-Hairy Translation Using CycleGAN
Venue: arXiv cs.CV 2021
Year: 2021

This work presents our computer vision course project called bald men-to-hairy men translation using CycleGAN. On top of CycleGAN architecture, we utilize perceptual loss in order to achieve more realistic results. We also integrate conditional constrains to obtain different stylized and colored hairs on bald men. We conducted extensive experiments and present qualitative results in this paper. Our code and models are available at https://github.com/fidansamet/GANiry.
FQuAD2.0: French Question Answering and knowing that you know nothing
Venue: arXiv cs.CL 2021
Year: 2021

Question Answering, including Reading Comprehension, is one of the NLP research areas that has seen significant scientific breakthroughs over the past few years, thanks to the concomitant advances in Language Modeling. Most of these breakthroughs, however, are centered on the English language. In 2020, as a first strong initiative to bridge the gap to the French language, Illuin Technology introduced FQuAD1.1, a French Native Reading Comprehension dataset composed of 60,000+ questions and answers samples extracted from Wikipedia articles. Nonetheless, Question Answering models trained on this dataset have a major drawback: they are not able to predict when a given question has no answer in the paragraph of interest, therefore making unreliable predictions in various industrial use-cases. In the present work, we introduce FQuAD2.0, which extends FQuAD with 17,000+ unanswerable questions, annotated adversarially, in order to be similar to answerable ones. This new dataset, comprising a total of almost 80,000 questions, makes it possible to train French Question Answering models with the ability of distinguishing unanswerable questions from answerable ones. We benchmark several models with this dataset: our best model, a fine-tuned CamemBERT-large, achieves a F1 score of 82.3% on this classification task, and a F1 score of 83% on the Reading Comprehension task.
Compact Redistricting Plans Have Many Spanning Trees
Venue: arXiv cs.DM 2021
Year: 2021

In the design and analysis of political redistricting maps, it is often useful to be able to sample from the space of all partitions of the graph of census blocks into connected subgraphs of equal population. There are influential Markov chain Monte Carlo methods for doing so that are based on sampling and splitting random spanning trees. Empirical evidence suggests that the distributions such algorithms sample from place higher weight on more "compact" redistricting plans, which is a practically useful and desirable property. In this paper, we confirm these observations analytically, establishing an inverse exponential relationship between the total length of the boundaries separating districts and the probability that such a map will be sampled. This result provides theoretical underpinnings for algorithms that are already making a significant real-world impact.
Interactive Dynamic Walking: Learning Gait Switching Policies with Generalization Guarantees
Venue: arXiv cs.RO 2021
Year: 2021

In this paper, we consider the problem of adapting a dynamically walking bipedal robot to follow a leading co-worker while engaging in tasks that require physical interaction. Our approach relies on switching among a family of Dynamic Movement Primitives (DMPs) as governed by a supervisor. We train the supervisor to orchestrate the switching among the DMPs in order to adapt to the leader's intentions, which are only implicitly available in the form of interaction forces. The primary contribution of our approach is its ability to furnish certificates of generalization to novel leader intentions for the trained supervisor. This is achieved by leveraging the Probably Approximately Correct (PAC)-Bayes bounds from generalization theory. We demonstrate the efficacy of our approach by training a neural-network supervisor to adapt the gait of a dynamically walking biped to a leading collaborator whose intended trajectory is not known explicitly.
An Open Problem on the Bentness of Mesnager's Functions
Venue: arXiv cs.IT 2021
Year: 2021

Let $n=2m$. In the present paper, we study the binomial Boolean functions of the form $$f_{a,b}(x) = \mathrm{Tr}_1^{n}(a x^{2^m-1 }) +\mathrm{Tr}_1^{2}(bx^{\frac{2^n-1}{3} }), $$ where $m$ is an even positive integer, $a\in \mathbb{F}_{2^n}^*$ and $b\in \mathbb{F}_4^*$. We show that $ f_{a,b}$ is a bent function if the Kloosterman sum $$K_{m}\left(a^{2^m+1}\right)=1+ \sum_{x\in \mathbb{F}_{2^m}^*} (-1)^{\mathrm{Tr}_1^{m}(a^{2^m+1} x+ \frac{1}{x})}$$ equals $4$, thus settling an open problem of Mesnager. The proof employs tools including computing Walsh coefficients of Boolean functions via multiplicative characters, divisibility properties of Gauss sums, and graph theory.
Not Only Domain Randomization: Universal Policy with Embedding System Identification
Venue: arXiv cs.RO 2021
Year: 2021

Domain randomization (DR) cannot provide optimal policies for adapting the learning agent to the dynamics of the environment, although it can generalize sub-optimal policies to work in a transferred domain. In this paper, we present Universal Policy with Embedding System Identification (UPESI) as an implicit system identification (SI) approach with universal policies (UPs), as a learning-based control method to execute optimal actions adaptively in environments with various dynamic properties. Previous approaches of SI for adaptive policies either conduct explicit SI, which is testified to be an ill-posed problem, or suffer from low efficiency without leveraging the simulation oracle. We propose to conduct SI in the embedding space of system dynamics by leveraging a learned forward dynamics model, and use Bayesian optimization for the SI process given transition data in a new environment. The identified embeddings are applied as additional input to the UP to enable its dynamics adaptability. Experiments demonstrate the advantageous performances of our proposed UP with embedding SI over standard DR and conventional SI approaches on both low-dimensional and high-dimensional simulation tasks.
Runtime Safety Assurance for Learning-enabled Control of Autonomous Driving Vehicles
Venue: arXiv cs.RO 2021
Year: 2021

Providing safety guarantees for Autonomous Vehicle (AV) systems with machine-learning-based controllers remains a challenging issue. In this work, we propose Simplex-Drive, a framework that can achieve runtime safety assurance for machine-learning enabled controllers of AVs. The proposed Simplex-Drive consists of an unverified Deep Reinforcement Learning (DRL)-based advanced controller (AC) that achieves desirable performance in complex scenarios, a Velocity-Obstacle (VO) based baseline safe controller (BC) with provably safety guarantees, and a verified mode management unit that monitors the operation status and switches the control authority between AC and BC based on safety-related conditions. We provide a formal correctness proof of Simplex-Drive and conduct a lane-changing case study in dense traffic scenarios. The simulation experiment results demonstrate that Simplex-Drive can always ensure operation safety without sacrificing control performance, even if the DRL policy may lead to deviations from the safe status.
Lithium-ion Battery State of Health Estimation based on Cycle Synchronization using Dynamic Time Warping
Venue: arXiv cs.LG 2021
Year: 2021

The state of health (SOH) estimation plays an essential role in battery-powered applications to avoid unexpected breakdowns due to battery capacity fading. However, few studies have paid attention to the problem of uneven length of degrading cycles, simply employing manual operation or leaving to the automatic processing mechanism of advanced machine learning models, like long short-term memory (LSTM). As a result, this causes information loss and caps the full capability of the data-driven SOH estimation models. To address this challenge, this paper proposes an innovative cycle synchronization way to change the existing coordinate system using dynamic time warping, not only enabling the equal length inputs of the estimation model but also preserving all information. By exploiting the time information of the time series, the proposed method embeds the time index and the original measurements into a novel indicator to reflect the battery degradation status, which could have the same length over cycles. Adopting the LSTM as the basic estimation model, the cycle synchronization-based SOH model could significantly improve the prediction accuracy by more than 30% compared to the traditional LSTM.
When in Doubt: Improving Classification Performance with Alternating Normalization
Venue: arXiv cs.LG 2021
Year: 2021

We introduce Classification with Alternating Normalization (CAN), a non-parametric post-processing step for classification. CAN improves classification accuracy for challenging examples by re-adjusting their predicted class probability distribution using the predicted class distributions of high-confidence validation examples. CAN is easily applicable to any probabilistic classifier, with minimal computation overhead. We analyze the properties of CAN using simulated experiments, and empirically demonstrate its effectiveness across a diverse set of classification tasks.
On the Geometry of Stable Steiner Tree Instances
Venue: arXiv cs.DS 2021
Year: 2021

In this note we consider the Steiner tree problem under Bilu-Linial stability. We give strong geometric structural properties that need to be satisfied by stable instances. We then make use of, and strengthen, these geometric properties to show that $1.562$-stable instances of Euclidean Steiner trees are polynomial-time solvable. We also provide a connection between certain approximation algorithms and Bilu-Linial stability for Steiner trees.
Self-Improving Voronoi Construction for a Hidden Mixture of Product Distributions
Venue: arXiv cs.CG 2021
Year: 2021

We propose a self-improving algorithm for computing Voronoi diagrams under a given convex distance function with constant description complexity. The $n$ input points are drawn from a hidden mixture of product distributions; we are only given an upper bound $m = o(\sqrt{n})$ on the number of distributions in the mixture, and the property that for each distribution, an input instance is drawn from it with a probability of $Ω(1/n)$. For any $\varepsilon \in (0,1)$, after spending $O\bigl(mn\log^{O(1)} (mn) + m^{\varepsilon} n^{1+\varepsilon}\log(mn)\bigr)$ time in a training phase, our algorithm achieves an $O\bigl(\frac{1}{\varepsilon}n\log m + \frac{1}{\varepsilon}n2^{O(\log^* n)} + \frac{1}{\varepsilon}H\bigr)$ expected running time with probability at least $1 - O(1/n)$, where $H$ is the entropy of the distribution of the Voronoi diagram output. The expectation is taken over the input distribution and the randomized decisions of the algorithm. For the Euclidean metric, the expected running time improves to $O\bigl(\frac{1}{\varepsilon}n\log m + \frac{1}{\varepsilon}H\bigr)$.
Exploring Teacher-Student Learning Approach for Multi-lingual Speech-to-Intent Classification
Venue: arXiv cs.CL 2021
Year: 2021

End-to-end speech-to-intent classification has shown its advantage in harvesting information from both text and speech. In this paper, we study a technique to develop such an end-to-end system that supports multiple languages. To overcome the scarcity of multi-lingual speech corpus, we exploit knowledge from a pre-trained multi-lingual natural language processing model. Multi-lingual bidirectional encoder representations from transformers (mBERT) models are trained on multiple languages and hence expected to perform well in the multi-lingual scenario. In this work, we employ a teacher-student learning approach to sufficiently extract information from an mBERT model to train a multi-lingual speech model. In particular, we use synthesized speech generated from an English-Mandarin text corpus for analysis and training of a multi-lingual intent classification model. We also demonstrate that the teacher-student learning approach obtains an improved performance (91.02%) over the traditional end-to-end (89.40%) intent classification approach in a practical multi-lingual scenario.
A Strong Baseline for the VIPriors Data-Efficient Image Classification Challenge
Venue: arXiv cs.CV 2021
Year: 2021

Learning from limited amounts of data is the hallmark of intelligence, requiring strong generalization and abstraction skills. In a machine learning context, data-efficient methods are of high practical importance since data collection and annotation are prohibitively expensive in many domains. Thus, coordinated efforts to foster progress in this area emerged recently, e.g., in the form of dedicated workshops and competitions. Besides a common benchmark, measuring progress requires strong baselines. We present such a strong baseline for data-efficient image classification on the VIPriors challenge dataset, which is a sub-sampled version of ImageNet-1k with 100 images per class. We do not use any methods tailored to data-efficient classification but only standard models and techniques as well as common competition tricks and thorough hyper-parameter tuning. Our baseline achieves 69.7% accuracy on the VIPriors image classification dataset and outperforms 50% of submissions to the VIPriors 2021 challenge.
Information Elevation Network for Fast Online Action Detection
Venue: arXiv cs.CV 2021
Year: 2021

Online action detection (OAD) is a task that receives video segments within a streaming video as inputs and identifies ongoing actions within them. It is important to retain past information associated with a current action. However, long short-term memory (LSTM), a popular recurrent unit for modeling temporal information from videos, accumulates past information from the previous hidden and cell states and the extracted visual features at each timestep without considering the relationships between the past and current information. Consequently, the forget gate of the original LSTM can lose the accumulated information relevant to the current action because it determines which information to forget without considering the current action. We introduce a novel information elevation unit (IEU) that lifts up and accumulate the past information relevant to the current action in order to model the past information that is especially relevant to the current action. To the best of our knowledge, our IEN is the first attempt that considers the computational overhead for the practical use of OAD. Through ablation studies, we design an efficient and effective OAD network using IEUs, called an information elevation network (IEN). Our IEN uses visual features extracted by a fast action recognition network taking only RGB frames because extracting optical flows requires heavy computation overhead. On two OAD benchmark datasets, THUMOS-14 and TVSeries, our IEN outperforms state-of-the-art OAD methods using only RGB frames. Furthermore, on the THUMOS-14 dataset, our IEN outperforms the state-of-the-art OAD methods using two-stream features based on RGB frames and optical flows.
Adopting Automated Bug Assignment in Practice -- A Registered Report of an Industrial Case Study
Venue: arXiv cs.SE 2021
Year: 2021

[Background/Context] The continuous inflow of bug reports is a considerable challenge in large development projects. Inspired by contemporary work on mining software repositories, we designed a prototype bug assignment solution based on machine learning in 2011-2016. The prototype evolved into an internal Ericsson product, TRR, in 2017-2018. TRR's first bug assignment without human intervention happened in 2019. [Objective/Aim] Our exploratory study will evaluate the adoption of TRR within its industrial context at Ericsson. We seek to understand 1) how TRR performs in the field, 2) what value TRR provides to Ericsson, and 3) how TRR has influenced the ways of working. Secondly, we will provide lessons learned related to productization of a research prototype within a company. [Method] We design an industrial case study combining interviews with TRR developers and users with analysis of data extracted from the bug tracking system at Ericsson. Furthermore, we will analyze sprint planning meetings recorded during the productization. Our data analysis will include thematic analysis, descriptive statistics, and Bayesian causal analysis.
On Joint Detection and Decoding in Short-Packet Communications
Venue: arXiv cs.IT 2021
Year: 2021

We consider a communication problem in which the receiver must first detect the presence of an information packet and, if detected, decode the message carried within it. We present general nonasymptotic upper and lower bounds on the maximum coding rate that depend on the blocklength, the probability of false alarm, the probability of misdetection, and the packet error probability. The bounds, which are expressed in terms of binary-hypothesis-testing performance metrics, generalize finite-blocklength bounds derived previously for the scenario when a genie informs the receiver whether a packet is present. The bounds apply to detection performed either jointly with decoding on the entire data packet, or separately on a dedicated preamble. The results presented in this paper can be used to determine the blocklength values at which the performance of a communication system is limited by its ability to perform packet detection satisfactorily, and to assess the difference in performance between preamble-based detection, and joint detection and decoding. Numerical results pertaining to the binary-input AWGN channel are provided.
Anomaly Detection for High-Dimensional Data Using Large Deviations Principle
Venue: arXiv cs.LG 2021
Year: 2021

Most current anomaly detection methods suffer from the curse of dimensionality when dealing with high-dimensional data. We propose an anomaly detection algorithm that can scale to high-dimensional data using concepts from the theory of large deviations. The proposed Large Deviations Anomaly Detection (LAD) algorithm is shown to outperform state of art anomaly detection methods on a variety of large and high-dimensional benchmark data sets. Exploiting the ability of the algorithm to scale to high-dimensional data, we propose an online anomaly detection method to identify anomalies in a collection of multivariate time series. We demonstrate the applicability of the online algorithm in identifying counties in the United States with anomalous trends in terms of COVID-19 related cases and deaths. Several of the identified anomalous counties correlate with counties with documented poor response to the COVID pandemic.