OpenRaiser/Intern-Atlas huggingface dataset:
Multi-objective Consensus Clustering Framework for Flight Search Recommendation
Venue: arXiv cs.IR 2020
Year: 2020

In the travel industry, online customers book their travel itinerary according to several features, like cost and duration of the travel or the quality of amenities. To provide personalized recommendations for travel searches, an appropriate segmentation of customers is required. Clustering ensemble approaches were developed to overcome well-known problems of classical clustering approaches, that each rely on a different theoretical model and can thus identify in the data space only clusters corresponding to this model. Clustering ensemble approaches combine multiple clustering results, each from a different algorithmic configuration, for generating more robust consensus clusters corresponding to agreements between initial clusters. We present a new clustering ensemble multi-objective optimization-based framework developed for analyzing Amadeus customer search data and improve personalized recommendations. This framework optimizes diversity in the clustering ensemble search space and automatically determines an appropriate number of clusters without requiring user's input. Experimental results compare the efficiency of this approach with other existing approaches on Amadeus customer search data in terms of internal (Adjusted Rand Index) and external (Amadeus business metric) validations.
Complex Markov Logic Networks: Expressivity and Liftability
Venue: arXiv cs.AI 2020
Year: 2020

We study expressivity of Markov logic networks (MLNs). We introduce complex MLNs, which use complex-valued weights, and we show that, unlike standard MLNs with real-valued weights, complex MLNs are fully expressive. We then observe that discrete Fourier transform can be computed using weighted first order model counting (WFOMC) with complex weights and use this observation to design an algorithm for computing relational marginal polytopes which needs substantially less calls to a WFOMC oracle than a recent algorithm.
Imagining Data-Objects for Reflective Self-Tracking
Venue: arXiv cs.HC 2020
Year: 2020

While self-tracking data is typically captured real-time in a lived experience, the data is often stored in a manner detached from the context where it belongs. Research has shown that there is a potential to enhance people's lived experiences with data-objects (artifacts representing contextually relevant data), for individual and collective reflections through a physical portrayal of data. This paper expands that research by studying how to design contextually relevant data-objects based on people's needs. We conducted a participatory research project with five households using object theater as a core method to encourage participants to speculate upon combinations of meaningful objects and personal data archives. In this paper, we detail three aspects that seem relevant for designing data-objects: social sharing, contextual ambiguity and interaction with the body. We show how an experience-centric view on data-objects can contribute with the contextual, social and bodily interplay between people, data and objects.
FSinR: an exhaustive package for feature selection
Venue: arXiv cs.LG 2020
Year: 2020

Feature Selection (FS) is a key task in Machine Learning. It consists in selecting a number of relevant variables for the model construction or data analysis. We present the R package, FSinR, which implements a variety of widely known filter and wrapper methods, as well as search algorithms. Thus, the package provides the possibility to perform the feature selection process, which consists in the combination of a guided search on the subsets of features with the filter or wrapper methods that return an evaluation measure of those subsets. In this article, we also present some examples on the usage of the package and a comparison with other packages available in R that contain methods for feature selection.
Improving BERT Fine-Tuning via Self-Ensemble and Self-Distillation
Venue: arXiv cs.CL 2020
Year: 2020

Fine-tuning pre-trained language models like BERT has become an effective way in NLP and yields state-of-the-art results on many downstream tasks. Recent studies on adapting BERT to new tasks mainly focus on modifying the model structure, re-designing the pre-train tasks, and leveraging external data and knowledge. The fine-tuning strategy itself has yet to be fully explored. In this paper, we improve the fine-tuning of BERT with two effective mechanisms: self-ensemble and self-distillation. The experiments on text classification and natural language inference tasks show our proposed methods can significantly improve the adaption of BERT without any external data or knowledge.
A Model-Based Derivative-Free Approach to Black-Box Adversarial Examples: BOBYQA
Venue: arXiv cs.LG 2020
Year: 2020

We demonstrate that model-based derivative free optimisation algorithms can generate adversarial targeted misclassification of deep networks using fewer network queries than non-model-based methods. Specifically, we consider the black-box setting, and show that the number of networks queries is less impacted by making the task more challenging either through reducing the allowed $\ell^{\infty}$ perturbation energy or training the network with defences against adversarial misclassification. We illustrate this by contrasting the BOBYQA algorithm with the state-of-the-art model-free adversarial targeted misclassification approaches based on genetic, combinatorial, and direct-search algorithms. We observe that for high $\ell^{\infty}$ energy perturbations on networks, the aforementioned simpler model-free methods require the fewest queries. In contrast, the proposed BOBYQA based method achieves state-of-the-art results when the perturbation energy decreases, or if the network is trained against adversarial perturbations.
Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition
Venue: arXiv cs.CL 2020
Year: 2020

Existing research on fairness evaluation of document classification models mainly uses synthetic monolingual data without ground truth for author demographic attributes. In this work, we assemble and publish a multilingual Twitter corpus for the task of hate speech detection with inferred four author demographic factors: age, country, gender and race/ethnicity. The corpus covers five languages: English, Italian, Polish, Portuguese and Spanish. We evaluate the inferred demographic labels with a crowdsourcing platform, Figure Eight. To examine factors that can cause biases, we take an empirical analysis of demographic predictability on the English corpus. We measure the performance of four popular document classifiers and evaluate the fairness and bias of the baseline classifiers on the author-level demographic attributes.
BLAST: Bridging Length/time scales via Atomistic Simulation Toolkit
Venue: arXiv cs.CE 2020
Year: 2020

The ever-increasing power of supercomputers coupled with highly scalable simulation codes have made molecular dynamics an indispensable tool in applications ranging from predictive modeling of materials to computational design and discovery of new materials for a broad range of applications. Multi-fidelity scale bridging between the various flavors of molecular dynamics i.e. ab-initio, classical and coarse-grained models has remained a long-standing challenge. Here, we introduce our framework BLAST (Bridging Length/time scales via Atomistic Simulation Toolkit) that leverages machine learning principles to address this challenge. BLAST is a multi-fidelity scale bridging framework that provide users with the capabilities to train and develop their own classical atomistic and coarse-grained interatomic potentials (force fields) for molecular simulations. BLAST is designed to address several long-standing problems in the molecular simulations community, such as unintended misuse of existing force fields due to knowledge gap between developers and users, bottlenecks in traditional force field development approaches, and other issues relating to the accuracy, efficiency, and transferability of force fields. Here, we discuss several important aspects in force field development and highlight features in BLAST that enable its functionalities and ease of use.
From Chess and Atari to StarCraft and Beyond: How Game AI is Driving the World of AI
Venue: arXiv cs.AI 2020
Year: 2020

This paper reviews the field of Game AI, which not only deals with creating agents that can play a certain game, but also with areas as diverse as creating game content automatically, game analytics, or player modelling. While Game AI was for a long time not very well recognized by the larger scientific community, it has established itself as a research area for developing and testing the most advanced forms of AI algorithms and articles covering advances in mastering video games such as StarCraft 2 and Quake III appear in the most prestigious journals. Because of the growth of the field, a single review cannot cover it completely. Therefore, we put a focus on important recent developments, including that advances in Game AI are starting to be extended to areas outside of games, such as robotics or the synthesis of chemicals. In this article, we review the algorithms and methods that have paved the way for these breakthroughs, report on the other important areas of Game AI research, and also point out exciting directions for the future of Game AI.
Distributed Training of Deep Neural Network Acoustic Models for Automatic Speech Recognition
Venue: arXiv cs.DC 2020
Year: 2020

The past decade has witnessed great progress in Automatic Speech Recognition (ASR) due to advances in deep learning. The improvements in performance can be attributed to both improved models and large-scale training data. Key to training such models is the employment of efficient distributed learning techniques. In this article, we provide an overview of distributed training techniques for deep neural network acoustic models for ASR. Starting with the fundamentals of data parallel stochastic gradient descent (SGD) and ASR acoustic modeling, we will investigate various distributed training strategies and their realizations in high performance computing (HPC) environments with an emphasis on striking the balance between communication and computation. Experiments are carried out on a popular public benchmark to study the convergence, speedup and recognition performance of the investigated strategies.
Efficient exploration of zero-sum stochastic games
Venue: arXiv cs.AI 2020
Year: 2020

We investigate the increasingly important and common game-solving setting where we do not have an explicit description of the game but only oracle access to it through gameplay, such as in financial or military simulations and computer games. During a limited-duration learning phase, the algorithm can control the actions of both players in order to try to learn the game and how to play it well. After that, the algorithm has to produce a strategy that has low exploitability. Our motivation is to quickly learn strategies that have low exploitability in situations where evaluating the payoffs of a queried strategy profile is costly. For the stochastic game setting, we propose using the distribution of state-action value functions induced by a belief distribution over possible environments. We compare the performance of various exploration strategies for this task, including generalizations of Thompson sampling and Bayes-UCB to this new setting. These two consistently outperform other strategies.
A Double Q-Learning Approach for Navigation of Aerial Vehicles with Connectivity Constraint
Venue: arXiv cs.AI 2020
Year: 2020

This paper studies the trajectory optimization problem for an aerial vehicle with the mission of flying between a pair of given initial and final locations. The objective is to minimize the travel time of the aerial vehicle ensuring that the communication connectivity constraint required for the safe operation of the aerial vehicle is satisfied. We consider two different criteria for the connectivity constraint of the aerial vehicle which leads to two different scenarios. In the first scenario, we assume that the maximum continuous time duration that the aerial vehicle is out of the coverage of the ground base stations (GBSs) is limited to a given threshold. In the second scenario, however, we assume that the total time periods that the aerial vehicle is not covered by the GBSs is restricted. Based on these two constraints, we formulate two trajectory optimization problems. To solve these non-convex problems, we use an approach based on the double Q-learning method which is a model-free reinforcement learning technique and unlike the existing algorithms does not need perfect knowledge of the environment. Moreover, in contrast to the well-known Q-learning technique, our double Q-learning algorithm does not suffer from the over-estimation issue. Simulation results show that although our algorithm does not require prior information of the environment, it works well and shows near optimal performance.
On Reinforcement Learning for Turn-based Zero-sum Markov Games
Venue: arXiv cs.LG 2020
Year: 2020

We consider the problem of finding Nash equilibrium for two-player turn-based zero-sum games. Inspired by the AlphaGo Zero (AGZ) algorithm, we develop a Reinforcement Learning based approach. Specifically, we propose Explore-Improve-Supervise (EIS) method that combines "exploration", "policy improvement"' and "supervised learning" to find the value function and policy associated with Nash equilibrium. We identify sufficient conditions for convergence and correctness for such an approach. For a concrete instance of EIS where random policy is used for "exploration", Monte-Carlo Tree Search is used for "policy improvement" and Nearest Neighbors is used for "supervised learning", we establish that this method finds an $\varepsilon$-approximate value function of Nash equilibrium in $\widetilde{O}(\varepsilon^{-(d+4)})$ steps when the underlying state-space of the game is continuous and $d$-dimensional. This is nearly optimal as we establish a lower bound of $\widetildeΩ(\varepsilon^{-(d+2)})$ for any policy.
Non-Volatile Memory Array Based Quantization- and Noise-Resilient LSTM Neural Networks
Venue: arXiv cs.NE 2020
Year: 2020

In cloud and edge computing models, it is important that compute devices at the edge be as power efficient as possible. Long short-term memory (LSTM) neural networks have been widely used for natural language processing, time series prediction and many other sequential data tasks. Thus, for these applications there is increasing need for low-power accelerators for LSTM model inference at the edge. In order to reduce power dissipation due to data transfers within inference devices, there has been significant interest in accelerating vector-matrix multiplication (VMM) operations using non-volatile memory (NVM) weight arrays. In NVM array-based hardware, reduced bit-widths also significantly increases the power efficiency. In this paper, we focus on the application of quantization-aware training algorithm to LSTM models, and the benefits these models bring in terms of resilience against both quantization error and analog device noise. We have shown that only 4-bit NVM weights and 4-bit ADC/DACs are needed to produce equivalent LSTM network performance as floating-point baseline. Reasonable levels of ADC quantization noise and weight noise can be naturally tolerated within our NVMbased quantized LSTM network. Benchmark analysis of our proposed LSTM accelerator for inference has shown at least 2.4x better computing efficiency and 40x higher area efficiency than traditional digital approaches (GPU, FPGA, and ASIC). Some other novel approaches based on NVM promise to deliver higher computing efficiency (up to 4.7x) but require larger arrays with potential higher error rates.
RMB-DPOP: Refining MB-DPOP by Reducing Redundant Inferences
Venue: arXiv cs.MA 2020
Year: 2020

MB-DPOP is an important complete algorithm for solving Distributed Constraint Optimization Problems (DCOPs) by exploiting a cycle-cut idea to implement memory-bounded inference. However, each cluster root in the algorithm is responsible for enumerating all the instantiations of its cycle-cut nodes, which would cause redundant inferences when its branches do not have the same cycle-cut nodes. Additionally, a large number of cycle-cut nodes and the iterative nature of MB-DPOP further exacerbate the pathology. As a result, MB-DPOP could suffer from huge coordination overheads and cannot scale up well. Therefore, we present RMB-DPOP which incorporates several novel mechanisms to reduce redundant inferences and improve the scalability of MB-DPOP. First, using the independence among the cycle-cut nodes in different branches, we distribute the enumeration of instantiations into different branches whereby the number of nonconcurrent instantiations reduces significantly and each branch can perform memory bounded inference asynchronously. Then, taking the topology into the consideration, we propose an iterative allocation mechanism to choose the cycle-cut nodes that cover a maximum of active nodes in a cluster and break ties according to their relative positions in a pseudo-tree. Finally, a caching mechanism is proposed to further reduce unnecessary inferences when the historical results are compatible with the current instantiations. We theoretically show that with the same number of cycle-cut nodes RMB-DPOP requires as many messages as MB-DPOP in the worst case and the experimental results show our superiorities over the state-of-the-art.
A Note on Empty Balanced Tetrahedra in Two colored Point sets in $\mathbb{R}^3$
Venue: arXiv cs.CG 2020
Year: 2020

Let $S$ be a set of $n$ red and $n$ blue points in general position in $\mathbb{R}^3$. Let $τ$ be a tetrahedra with vertices on $S$. We say that $τ$ is \emph{empty} if it does not contain any point of $S$ in its interior. We say that $τ$ is \emph{balanced} if it contains two blue vertices and two red vertices. In this paper we show that $S$ spans $Ω(n^{5/2})$ empty balanced tetrahedra.
Enhancing Physical Layer Security of Random Caching in Large-Scale Multi-Antenna Heterogeneous Wireless Networks
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we propose a novel secure random caching scheme for large-scale multi-antenna heterogeneous wireless networks, where the base stations (BSs) deliver randomly cached confidential contents to the legitimate users in the presence of passive eavesdroppers as well as active jammers. In order to safeguard the content delivery, we consider that the BSs transmits the artificial noise together with the useful signals. By using tools from stochastic geometry, we first analyze the average reliable transmission probability (RTP) and the average confidential transmission probability (CTP), which take both the impact of the eavesdroppers and the impact of the jammers into consideration. We further provide tight upper and lower bounds on the average RTP. These analytical results enable us to obtain rich insights into the behaviors of the average RTP and the average CTP with respect to key system parameters. Moreover, we optimize the caching distribution of the files to maximize the average RTP of the system, while satisfying the constraints on the caching size and the average CTP. Through numerical results, we show that our proposed secure random caching scheme can effectively boost the secrecy performance of the system compared to the existing solutions.
Learning Beam Codebooks with Neural Networks: Towards Environment-Aware mmWave MIMO
Venue: arXiv cs.IT 2020
Year: 2020

Scaling the number of antennas up is a key characteristic of current and future wireless communication systems. The hardware cost and power consumption, however, motivate large-scale MIMO systems, especially at millimeter wave (mmWave) bands, to rely on analog-only or hybrid analog/digital transceiver architectures. With these architectures, mmWave base stations normally use pre-defined beamforming codebooks for both initial access and data transmissions. Current beam codebooks, however, generally adopt single-lobe narrow beams and scan the entire angular space. This leads to high beam training overhead and loss in the achievable beamforming gains. In this paper, we propose a new machine learning framework for learning beamforming codebooks in hardware-constrained large-scale MIMO systems. More specifically, we develop a neural network architecture that accounts for the hardware constraints and learns beam codebooks that adapt to the surrounding environment and the user locations. Simulation results highlight the capability of the proposed solution in learning multi-lobe beams and reducing the codebook size, which leads to noticeable gains compared to classical codebook design approaches.
Gödel's Sentence Is An Adversarial Example But Unsolvable
Venue: arXiv cs.LG 2020
Year: 2020

In recent years, different types of adversarial examples from different fields have emerged endlessly, including purely natural ones without perturbations. A variety of defenses are proposed and then broken quickly. Two fundamental questions need to be asked: What's the reason for the existence of adversarial examples and are adversarial examples unsolvable? In this paper, we will show the reason for the existence of adversarial examples is there are non-isomorphic natural explanations that can all explain data set. Specifically, for two natural explanations of being true and provable, Gödel's sentence is an adversarial example but ineliminable. It can't be solved by the re-accumulation of data set or the re-improvement of learning algorithm. Finally, from the perspective of computability, we will prove the incomputability for adversarial examples, which are unrecognizable.
CAKE: An Efficient Group Key Management for Dynamic Groups
Venue: arXiv cs.CR 2020
Year: 2020

With rapid increase of mobile computing and wireless network linkage, the information exchange between connected systems and within groups increases heavily. Exchanging confidential information within groups via unsecured communication channels is a high security threat. In order to prevent third parties from accessing this data, it is essential to encrypt it. For this purpose, the group participants need a common group key to enable encrypted broadcast messages. But efficient key management of secured group communication is a challenging task, if participants rely on low performance hardware and small bandwidth. For coordination and distribution, we present the modular group key management procedure CAKE that is centrally organized and meets strict security requirements. The lightweight G-IKEv2 protocol in combination with the key exchange concept of CAKE leads to an efficiently integrated solution. The hybrid approach combines the advantages of the existing protocols with the objective to reduce the computation and communication effort. It is shown that the procedure is more suitable for changing MANET groups than the existing ones. Moreover, the exchanged group key can be used for any services which provides a wide range of applications.