OpenRaiser/Intern-Atlas huggingface dataset:
LP-based algorithms for multistage minimization problems
Venue: arXiv cs.DS 2019
Year: 2019

We consider a multistage framework introduced recently where, given a time horizon t=1,2,...,T, the input is a sequence of instances of a (static) combinatorial optimization problem I_1,I_2,...,I_T, (one for each time step), and the goal is to find a sequence of solutions S_1,S_2,...,S_T (one for each time step) reaching a tradeoff between the quality of the solutions in each time step and the stability/similarity of the solutions in consecutive time steps. For several polynomial-time solvable problems, such as Minimum Cost Perfect Matching, the multistage variant becomes hard to approximate (even for two time steps for Minimum Cost Perfect Matching). In this paper, we study the multistage variants of some important discrete minimization problems (including Minimum Cut, Vertex Cover, Set Cover, Prize-Collecting Steiner Tree, Prize-Collecting Traveling Salesman). We focus on the natural question of whether linear-programming-based methods may help in developing good approximation algorithms in this framework. We first show that Min Cut remains polytime solvable in its multistage variant, and Vertex Cover remains 2-approximable, as particular case of a more general statement which easily follows from the work of (Hochbaum, EJOR 2002) on monotone and IP2 problems. Then, we tackle other problems and for this we introduce a new two-threshold rounding scheme, tailored for multistage problems. As a first application, we show that this rounding scheme gives a 2$f$-approximation algorithm for the multistage variant of the f-Set Cover problem, where each element belongs to at most f sets. More interestingly, we are able to use our rounding scheme in order to propose a 3.53-approximation algorithm for the multistage variant of the Prize-Collecting Steiner Tree problem, and a 3.034-approximation algorithm for the multistage variant of the Prize-Collecting Traveling Salesman problem.
Decentralized Connectivity Control in Quadcopters: a Field Study of Communication Performance
Venue: arXiv cs.RO 2019
Year: 2019

Redundancy and parallelism make decentralized multi-robot systems appealing solutions for the exploration of extreme environments. However, effective cooperation often requires team-wide connectivity and a carefully designed communication strategy. Several recently proposed decentralized connectivity maintenance approaches exploit elegant algebraic results drawn from spectral graph theory. Yet, these proposals are rarely taken beyond simulations or laboratory implementations. In this work, we present two major contributions: (i) we describe the full-stack implementation---from hardware to software---of a decentralized control law for robust connectivity maintenance; and (ii) we assess, in the field, our setup's ability to correctly exchange all the necessary information required to maintain connectivity in a team of quadcopters.
LEAP: A Lightweight Encryption and Authentication Protocol for In-Vehicle Communications
Venue: arXiv cs.CR 2019
Year: 2019

The Controller Area Network (CAN) is considered as the de-facto standard for the in-vehicle communications due to its real-time performance and high reliability. Unfortunately, the lack of security protection on the CAN bus gives attackers the opportunity to remotely compromise a vehicle. In this paper, we propose a Lightweight Encryption and Authentication Protocol (LEAP) with low cost and high efficiency to address the security issue of the CAN bus. LEAP exploits the security-enhanced stream cipher primitive to provide encryption and authentication for the CAN messages. Compared with the state-of-the-art Message Authentication Code (MAC) based approaches, LEAP requires less memory, is 8X faster, and thwarts the most recently proposed attacks.
CochleaNet: A Robust Language-independent Audio-Visual Model for Speech Enhancement
Venue: arXiv cs.SD 2019
Year: 2019

Noisy situations cause huge problems for suffers of hearing loss as hearing aids often make the signal more audible but do not always restore the intelligibility. In noisy settings, humans routinely exploit the audio-visual (AV) nature of the speech to selectively suppress the background noise and to focus on the target speaker. In this paper, we present a causal, language, noise and speaker independent AV deep neural network (DNN) architecture for speech enhancement (SE). The model exploits the noisy acoustic cues and noise robust visual cues to focus on the desired speaker and improve the speech intelligibility. To evaluate the proposed SE framework a first of its kind AV binaural speech corpus, called ASPIRE, is recorded in real noisy environments including cafeteria and restaurant. We demonstrate superior performance of our approach in terms of objective measures and subjective listening tests over the state-of-the-art SE approaches as well as recent DNN based SE models. In addition, our work challenges a popular belief that a scarcity of multi-language large vocabulary AV corpus and wide variety of noises is a major bottleneck to build a robust language, speaker and noise independent SE systems. We show that a model trained on synthetic mixture of Grid corpus (with 33 speakers and a small English vocabulary) and ChiME 3 Noises (consisting of only bus, pedestrian, cafeteria, and street noises) generalise well not only on large vocabulary corpora but also on completely unrelated languages (such as Mandarin), wide variety of speakers and noises.
Go Wider: An Efficient Neural Network for Point Cloud Analysis via Group Convolutions
Venue: arXiv cs.CV 2019
Year: 2019

In order to achieve better performance for point cloud analysis, many researchers apply deeper neural networks using stacked Multi-Layer-Perceptron (MLP) convolutions over irregular point cloud. However, applying dense MLP convolutions over large amount of points (e.g. autonomous driving application) leads to inefficiency in memory and computation. To achieve high performance but less complexity, we propose a deep-wide neural network, called ShufflePointNet, to exploit fine-grained local features and reduce redundancy in parallel using group convolution and channel shuffle operation. Unlike conventional operation that directly applies MLPs on high-dimensional features of point cloud, our model goes wider by splitting features into groups in advance, and each group with certain smaller depth is only responsible for respective MLP operation, which can reduce complexity and allows to encode more useful information. Meanwhile, we connect communication between groups by shuffling groups in feature channel to capture fine-grained features. We claim that, multi-branch method for wider neural networks is also beneficial to feature extraction for point cloud. We present extensive experiments for shape classification task on ModelNet40 dataset and semantic segmentation task on large scale datasets ShapeNet part, S3DIS and KITTI. We further perform ablation study and compare our model to other state-of-the-art algorithms in terms of complexity and accuracy.
Subpacketization in Coded Caching with Demand Privacy
Venue: arXiv cs.IT 2019
Year: 2019

Coded caching is a technique where we utilize multi-casting opportunities to reduce rate in cached networks. One limitation of coded caching schemes is that they reveal the demands of all users to their peers. In this work, we consider coded caching schemes that assure privacy for user demands with a particular focus on reducing subpacketization. For the 2-user, 2-file case, we present a new linear demand-private scheme with the lowest possible subpacketization. This is done by presenting the scheme explicitly and proving impossibility results under lower subpacketization. Additionally, when only partial privacy is required, we show that subpacketization can be significantly reduced when there are a large number of files.
Learning Dense Representations for Entity Retrieval
Venue: arXiv cs.CL 2019
Year: 2019

We show that it is feasible to perform entity linking by training a dual encoder (two-tower) model that encodes mentions and entities in the same dense vector space, where candidate entities are retrieved by approximate nearest neighbor search. Unlike prior work, this setup does not rely on an alias table followed by a re-ranker, and is thus the first fully learned entity retrieval model. We show that our dual encoder, trained using only anchor-text links in Wikipedia, outperforms discrete alias table and BM25 baselines, and is competitive with the best comparable results on the standard TACKBP-2010 dataset. In addition, it can retrieve candidates extremely fast, and generalizes well to a new dataset derived from Wikinews. On the modeling side, we demonstrate the dramatic value of an unsupervised negative mining algorithm for this task.
HealthGuard: A Machine Learning-Based Security Framework for Smart Healthcare Systems
Venue: arXiv cs.CR 2019
Year: 2019

The integration of Internet-of-Things and pervasive computing in medical devices have made the modern healthcare system "smart". Today, the function of the healthcare system is not limited to treat the patients only. With the help of implantable medical devices and wearables, Smart Healthcare System (SHS) can continuously monitor different vital signs of a patient and automatically detect and prevent critical medical conditions. However, these increasing functionalities of SHS raise several security concerns and attackers can exploit the SHS in numerous ways: they can impede normal function of the SHS, inject false data to change vital signs, and tamper a medical device to change the outcome of a medical emergency. In this paper, we propose HealthGuard, a novel machine learning-based security framework to detect malicious activities in a SHS. HealthGuard observes the vital signs of different connected devices of a SHS and correlates the vitals to understand the changes in body functions of the patient to distinguish benign and malicious activities. HealthGuard utilizes four different machine learning-based detection techniques (Artificial Neural Network, Decision Tree, Random Forest, k-Nearest Neighbor) to detect malicious activities in a SHS. We trained HealthGuard with data collected for eight different smart medical devices for twelve benign events including seven normal user activities and five disease-affected events. Furthermore, we evaluated the performance of HealthGuard against three different malicious threats. Our extensive evaluation shows that HealthGuard is an effective security framework for SHS with an accuracy of 91% and an F-1 score of 90%.
On some cryptographic properties of Boolean functions and their second-order derivatives
Venue: arXiv cs.CR 2019
Year: 2019

In this paper some cryptographic properties of Boolean functions, including weight, balancedness and nonlinearity, are studied, particularly focusing on splitting functions and cubic Boolean functions. Moreover, we present some quantities derived from the behaviour of second-order derivatives which allow us to determine whether a quadratic or cubic function is APN.
Multi-stage Pretraining for Abstractive Summarization
Venue: arXiv cs.CL 2019
Year: 2019

Neural models for abstractive summarization tend to achieve the best performance in the presence of highly specialized, summarization specific modeling add-ons such as pointer-generator, coverage-modeling, and inferencetime heuristics. We show here that pretraining can complement such modeling advancements to yield improved results in both short-form and long-form abstractive summarization using two key concepts: full-network initialization and multi-stage pretraining. Our method allows the model to transitively benefit from multiple pretraining tasks, from generic language tasks to a specialized summarization task to an even more specialized one such as bullet-based summarization. Using this approach, we demonstrate improvements of 1.05 ROUGE-L points on the Gigaword benchmark and 1.78 ROUGE-L points on the CNN/DailyMail benchmark, compared to a randomly-initialized baseline.
Acceptable Planning: Influencing Individual Behavior to Reduce Transportation Energy Expenditure of a City
Venue: arXiv cs.AI 2019
Year: 2019

Our research aims at developing intelligent systems to reduce the transportation-related energy expenditure of a large city by influencing individual behavior. We introduce COPTER - an intelligent travel assistant that evaluates multi-modal travel alternatives to find a plan that is acceptable to a person given their context and preferences. We propose a formulation for acceptable planning that brings together ideas from AI, machine learning, and economics. This formulation has been incorporated in COPTER that produces acceptable plans in real-time. We adopt a novel empirical evaluation framework that combines human decision data with a high fidelity multi-modal transportation simulation to demonstrate a 4\% energy reduction and 20\% delay reduction in a realistic deployment scenario in Los Angeles, California, USA.
Why Does Hierarchy (Sometimes) Work So Well in Reinforcement Learning?
Venue: arXiv cs.LG 2019
Year: 2019

Hierarchical reinforcement learning has demonstrated significant success at solving difficult reinforcement learning (RL) tasks. Previous works have motivated the use of hierarchy by appealing to a number of intuitive benefits, including learning over temporally extended transitions, exploring over temporally extended periods, and training and exploring in a more semantically meaningful action space, among others. However, in fully observed, Markovian settings, it is not immediately clear why hierarchical RL should provide benefits over standard "shallow" RL architectures. In this work, we isolate and evaluate the claimed benefits of hierarchical RL on a suite of tasks encompassing locomotion, navigation, and manipulation. Surprisingly, we find that most of the observed benefits of hierarchy can be attributed to improved exploration, as opposed to easier policy learning or imposed hierarchical structures. Given this insight, we present exploration techniques inspired by hierarchy that achieve performance competitive with hierarchical RL while at the same time being much simpler to use and implement.
Non-monotonic Logical Reasoning Guiding Deep Learning for Explainable Visual Question Answering
Venue: arXiv cs.AI 2019
Year: 2019

State of the art algorithms for many pattern recognition problems rely on deep network models. Training these models requires a large labeled dataset and considerable computational resources. Also, it is difficult to understand the working of these learned models, limiting their use in some critical applications. Towards addressing these limitations, our architecture draws inspiration from research in cognitive systems, and integrates the principles of commonsense logical reasoning, inductive learning, and deep learning. In the context of answering explanatory questions about scenes and the underlying classification problems, the architecture uses deep networks for extracting features from images and for generating answers to queries. Between these deep networks, it embeds components for non-monotonic logical reasoning with incomplete commonsense domain knowledge, and for decision tree induction. It also incrementally learns and reasons with previously unknown constraints governing the domain's states. We evaluated the architecture in the context of datasets of simulated and real-world images, and a simulated robot computing, executing, and providing explanatory descriptions of plans. Experimental results indicate that in comparison with an ``end to end'' architecture of deep networks, our architecture provides better accuracy on classification problems when the training dataset is small, comparable accuracy with larger datasets, and more accurate answers to explanatory questions. Furthermore, incremental acquisition of previously unknown constraints improves the ability to answer explanatory questions, and extending non-monotonic logical reasoning to support planning and diagnostics improves the reliability and efficiency of computing and executing plans on a simulated robot.
The Field-of-View Constraint of Markers for Mobile Robot with Pan-Tilt Camera
Venue: arXiv cs.RO 2019
Year: 2019

In the field of navigation and visual servo, it is common to calculate relative pose by feature points on markers, so keeping markers in camera's view is an important problem. In this paper, we propose a novel approach to calculate field-of-view (FOV) constraint of markers for camera. Our method can make the camera maintain the visibility of all feature points during the motion of mobile robot. According to the angular aperture of camera, the mobile robot can obtain the FOV constraint region where the camera cannot keep all feature points in an image. Based on the FOV constraint region, the mobile robot can be guided to move from the initial position to destination. Finally simulations and experiments are conducted based on a mobile robot equipped with a pan-tilt camera, which validates the effectiveness of the method to obtain the FOV constraints.
Situating Sentence Embedders with Nearest Neighbor Overlap
Venue: arXiv cs.CL 2019
Year: 2019

As distributed approaches to natural language semantics have developed and diversified, embedders for linguistic units larger than words have come to play an increasingly important role. To date, such embedders have been evaluated using benchmark tasks (e.g., GLUE) and linguistic probes. We propose a comparative approach, nearest neighbor overlap (N2O), that quantifies similarity between embedders in a task-agnostic manner. N2O requires only a collection of examples and is simple to understand: two embedders are more similar if, for the same set of inputs, there is greater overlap between the inputs' nearest neighbors. Though applicable to embedders of texts of any size, we focus on sentence embedders and use N2O to show the effects of different design choices and architectures.
FEED: Feature-level Ensemble for Knowledge Distillation
Venue: arXiv cs.CV 2019
Year: 2019

Knowledge Distillation (KD) aims to transfer knowledge in a teacher-student framework, by providing the predictions of the teacher network to the student network in the training stage to help the student network generalize better. It can use either a teacher with high capacity or {an} ensemble of multiple teachers. However, the latter is not convenient when one wants to use feature-map-based distillation methods. For a solution, this paper proposes a versatile and powerful training algorithm named FEature-level Ensemble for knowledge Distillation (FEED), which aims to transfer the ensemble knowledge using multiple teacher networks. We introduce a couple of training algorithms that transfer ensemble knowledge to the student at the feature map level. Among the feature-map-based distillation methods, using several non-linear transformations in parallel for transferring the knowledge of the multiple teacher{s} helps the student find more generalized solutions. We name this method as parallel FEED, andexperimental results on CIFAR-100 and ImageNet show that our method has clear performance enhancements, without introducing any additional parameters or computations at test time. We also show the experimental results of sequentially feeding teacher's information to the student, hence the name sequential FEED, and discuss the lessons obtained. Additionally, the empirical results on measuring the reconstruction errors at the feature map give hints for the enhancements.
Jointly Learning to Detect Emotions and Predict Facebook Reactions
Venue: arXiv cs.LG 2019
Year: 2019

The growing ubiquity of Social Media data offers an attractive perspective for improving the quality of machine learning-based models in several fields, ranging from Computer Vision to Natural Language Processing. In this paper we focus on Facebook posts paired with reactions of multiple users, and we investigate their relationships with classes of emotions that are typically considered in the task of emotion detection. We are inspired by the idea of introducing a connection between reactions and emotions by means of First-Order Logic formulas, and we propose an end-to-end neural model that is able to jointly learn to detect emotions and predict Facebook reactions in a multi-task environment, where the logic formulas are converted into polynomial constraints. Our model is trained using a large collection of unsupervised texts together with data labeled with emotion classes and Facebook posts that include reactions. An extended experimental analysis that leverages a large collection of Facebook posts shows that the tasks of emotion classification and reaction prediction can both benefit from their interaction.
WATTNet: Learning to Trade FX via Hierarchical Spatio-Temporal Representation of Highly Multivariate Time Series
Venue: arXiv cs.LG 2019
Year: 2019

Finance is a particularly challenging application area for deep learning models due to low noise-to-signal ratio, non-stationarity, and partial observability. Non-deliverable-forwards (NDF), a derivatives contract used in foreign exchange (FX) trading, presents additional difficulty in the form of long-term planning required for an effective selection of start and end date of the contract. In this work, we focus on tackling the problem of NDF tenor selection by leveraging high-dimensional sequential data consisting of spot rates, technical indicators and expert tenor patterns. To this end, we construct a dataset from the Depository Trust & Clearing Corporation (DTCC) NDF data that includes a comprehensive list of NDF volumes and daily spot rates for 64 FX pairs. We introduce WaveATTentionNet (WATTNet), a novel temporal convolution (TCN) model for spatio-temporal modeling of highly multivariate time series, and validate it across NDF markets with varying degrees of dissimilarity between the training and test periods in terms of volatility and general market regimes. The proposed method achieves a significant positive return on investment (ROI) in all NDF markets under analysis, outperforming recurrent and classical baselines by a wide margin. Finally, we propose two orthogonal interpretability approaches to verify noise stability and detect the driving factors of the learned tenor selection strategy.
Enhancing Traffic Scene Predictions with Generative Adversarial Networks
Venue: arXiv cs.CV 2019
Year: 2019

We present a new two-stage pipeline for predicting frames of traffic scenes where relevant objects can still reliably be detected. Using a recent video prediction network, we first generate a sequence of future frames based on past frames. A second network then enhances these frames in order to make them appear more realistic. This ensures the quality of the predicted frames to be sufficient to enable accurate detection of objects, which is especially important for autonomously driving cars. To verify this two-stage approach, we conducted experiments on the Cityscapes dataset. For enhancing, we trained two image-to-image translation methods based on generative adversarial networks, one for blind motion deblurring and one for image super-resolution. All resulting predictions were quantitatively evaluated using both traditional metrics and a state-of-the-art object detection network showing that the enhanced frames appear qualitatively improved. While the traditional image comparison metrics, i.e., MSE, PSNR, and SSIM, failed to confirm this visual impression, the object detection evaluation resembles it well. The best performing prediction-enhancement pipeline is able to increase the average precision values for detecting cars by about 9% for each prediction step, compared to the non-enhanced predictions.