OpenRaiser/Intern-Atlas huggingface dataset:
TANDA: Transfer and Adapt Pre-Trained Transformer Models for Answer Sentence Selection
Venue: arXiv cs.CL 2019
Year: 2019

We propose TANDA, an effective technique for fine-tuning pre-trained Transformer models for natural language tasks. Specifically, we first transfer a pre-trained model into a model for a general task by fine-tuning it with a large and high-quality dataset. We then perform a second fine-tuning step to adapt the transferred model to the target domain. We demonstrate the benefits of our approach for answer sentence selection, which is a well-known inference task in Question Answering. We built a large scale dataset to enable the transfer step, exploiting the Natural Questions dataset. Our approach establishes the state of the art on two well-known benchmarks, WikiQA and TREC-QA, achieving MAP scores of 92% and 94.3%, respectively, which largely outperform the previous highest scores of 83.4% and 87.5%, obtained in very recent work. We empirically show that TANDA generates more stable and robust models reducing the effort required for selecting optimal hyper-parameters. Additionally, we show that the transfer step of TANDA makes the adaptation step more robust to noise. This enables a more effective use of noisy datasets for fine-tuning. Finally, we also confirm the positive impact of TANDA in an industrial setting, using domain specific datasets subject to different types of noise.
GraCT: A Grammar-based Compressed Index for Trajectory Data
Venue: arXiv cs.DS 2019
Year: 2019

We introduce a compressed data structure for the storage of free trajectories of moving objects (such as ships and planes) that efficiently supports various spatio-temporal queries. Our structure, dubbed GraCT, stores the absolute positions of all the objects at regular time intervals (snapshots) using a $k^2$-tree, which is a space- and time-efficient version of a region quadtree. Positions between snapshots are represented as logs of relative movements and compressed using Re-Pair, a grammar-based compressor. The nonterminals of this grammar are enhanced with MBR information to enable fast queries. The GraCT structure of a dataset occupies less than the raw data compressed with a powerful traditional compressor such as p7zip. Further, instead of requiring full decompression to access the data like a traditional compressor, GraCT supports direct access to object trajectories or to their position at specific time instants, as well as spatial range and nearest-neighbor queries on time instants and/or time intervals. Compared to traditional methods for storing and indexing spatio-temporal data, GraCT requires two orders of magnitude less space, and is competitive in query times. In particular, thanks to its compressed representation, the GraCT structure may reside in main memory in situations where any classical uncompressed index must resort to disk, thereby being one or two orders of magnitude faster.
Dv2v: A Dynamic Variable-to-Variable Compressor
Venue: arXiv cs.DS 2019
Year: 2019

We present Dv2v, a new dynamic (one-pass) variable-to-variable compressor. Variable-to-variable compression aims at using a modeler that gathers variable-length input symbols and a variable-length statistical coder that assigns shorter codewords to the more frequent symbols. In Dv2v, we process the input text word-wise to gather variable-length symbols that can be either terminals (new words) or non-terminals, subsequences of words seen before in the input text. Those input symbols are set in a vocabulary that is kept sorted by frequency. Therefore, those symbols can be easily encoded with dense codes. Our Dv2v permits real-time transmission of data, i.e. compression/transmission can begin as soon as data become available. Our experiments show that Dv2v is able to overcome the compression ratios of the v2vDC, the state-of-the-art semi-static variable-to-variable compressor, and to almost reach p7zip values. It also draws a competitive performance at both compression and decompression.
Learning Preferences and Demands in Visual Recommendation
Venue: arXiv cs.IR 2019
Year: 2019

Visual information is an important factor in recommender systems, in which users' selections consist of two components: \emph{preferences} and \emph{demands}. Some studies has been done for modeling users' preferences in visual recommendation. However, conventional methods models items in a common visual feature space, which may fail in capturing \emph{styles} of items. We propose a DeepStyle method for learning style features of items. DeepStyle eliminates the categorical information of items, which is dominant in the original visual feature space, based on a Convolutional Neural Networks (CNN) architecture. For modeling users' demands on different categories of items, the problem can be formulated as recommendation with contextual and sequential information. To solve this problem, we propose a Context-Aware Gated Recurrent Unit (CA-GRU) method, which can capture sequential and contextual information simultaneously. Furthermore, the aggregation of prediction on preferences and demands, i.e., prediction generated by DeepStyle and CA-GRU, can model users' selection behaviors more completely. Experiments conducted on real-world datasets illustrates the effectiveness of our proposed methods in visual recommendation.
Learning The Best Expert Efficiently
Venue: arXiv cs.LG 2019
Year: 2019

We consider online learning problems where the aim is to achieve regret which is efficient in the sense that it is the same order as the lowest regret amongst K experts. This is a substantially stronger requirement that achieving $O(\sqrt{n})$ or $O(\log n)$ regret with respect to the best expert and standard algorithms are insufficient, even in easy cases where the regrets of the available actions are very different from one another. We show that a particular lazy form of the online subgradient algorithm can be used to achieve minimal regret in a number of "easy" regimes while retaining an $O(\sqrt{n})$ worst-case regret guarantee. We also show that for certain classes of problem minimal regret strategies exist for some of the remaining "hard" regimes.
On the costs and profit of software defect prediction
Venue: arXiv cs.SE 2019
Year: 2019

Defect prediction can be a powerful tool to guide the use of quality assurance resources. However, while lots of research covered methods for defect prediction as well as methodological aspects of defect prediction research, the actual cost saving potential of defect prediction is still unclear. Within this article, we close this research gap and formulate a cost model for software defect prediction. We derive mathematically provable boundary conditions that must be fulfilled by defect prediction models such that there is a positive profit when the defect prediction model is used. Our cost model includes aspects like the costs for quality assurance, the costs of post-release defects, the possibility that quality assurance fails to reveal predicted defects, and the relationship between software artifacts and defects. We initialize the cost model using different assumptions, perform experiments to show trends of the behavior of costs on real projects. Our results show that the unrealistic assumption that defects only affect a single software artifact, which is a standard practice in the defect prediction literature, leads to inaccurate cost estimations. Moreover, the results indicate that thresholds for machine learning metrics are also not suited to define success criteria for software defect prediction.
Security Issues in WBANs
Venue: arXiv cs.NI 2019
Year: 2019

Wireless Body Area Network (WBAN) refers to short-range, wireless communications near or inside a human body. WBAN is emerging solution to cater the needs of local and remote health care related facility. Medical and non-medical applications have been revolutionarily under consideration for providing a healthy and gratify service to the humanity. Being very critical in communication from body it faces a lot of challenges which are to be tackled for the safety of life and benefit of the user. There is variety of challenges faced by WBANs. WBAN is favorite playground for attackers due to its usability in various applications. This article provides systematic overview of main challenges in WBANs in security perspectives.
Active Learning for Black-Box Adversarial Attacks in EEG-Based Brain-Computer Interfaces
Venue: arXiv cs.LG 2019
Year: 2019

Deep learning has made significant breakthroughs in many fields, including electroencephalogram (EEG) based brain-computer interfaces (BCIs). However, deep learning models are vulnerable to adversarial attacks, in which deliberately designed small perturbations are added to the benign input samples to fool the deep learning model and degrade its performance. This paper considers transferability-based black-box attacks, where the attacker trains a substitute model to approximate the target model, and then generates adversarial examples from the substitute model to attack the target model. Learning a good substitute model is critical to the success of these attacks, but it requires a large number of queries to the target model. We propose a novel framework which uses query synthesis based active learning to improve the query efficiency in training the substitute model. Experiments on three convolutional neural network (CNN) classifiers and three EEG datasets demonstrated that our method can improve the attack success rate with the same number of queries, or, in other words, our method requires fewer queries to achieve a desired attack performance. To our knowledge, this is the first work that integrates active learning and adversarial attacks for EEG-based BCIs.
To Populate is To Regulate
Venue: arXiv cs.LG 2019
Year: 2019

We examine the effects of instantiating Lewis signaling games within a population of speaker and listener agents with the aim of producing a set of general and robust representations of unstructured pixel data. Preliminary experiments suggest that the set of representations associated with languages generated within a population outperform those generated between a single speaker-listener pair on this objective, making a case for the adoption of population-based approaches in emergent communication studies. Furthermore, post-hoc analysis reveals that population-based learning induces a number of novel factors to the conventional emergent communication setup, inviting a wide range of future research questions regarding communication dynamics and the flow of information within them.
Beyond Social Fragmentation: Coexistence of Cultural Diversity and Structural Connectivity Is Possible with Social Constituent Diversity
Venue: arXiv cs.SI 2019
Year: 2019

Social fragmentation caused by widening differences among constituents has recently become a highly relevant issue to our modern society. Theoretical models of social fragmentation using the adaptive network framework have been proposed and studied in earlier literature, which are known to either converge to a homogeneous, well-connected network or fragment into many disconnected sub-networks with distinct states. Here we introduced the diversities of behavioral attributes among social constituents and studied their effects on social network evolution. We investigated, using a networked agent-based simulation model, how the resulting network states and topologies would be affected when individual constituents' cultural tolerance, cultural state change rate, and edge weight change rate were systematically diversified. The results showed that the diversity of cultural tolerance had the most direct effect to keep the cultural diversity within the society high and simultaneously reduce the average shortest path length of the social network, which was not previously reported in the earlier literature. Diversities of other behavioral attributes also had effects on final states of the social network, with some nonlinear interactions. Our results suggest that having a broad distribution of cultural tolerance levels within society can help promote the coexistence of cultural diversity and structural connectivity.
Simplifying Random Forests: On the Trade-off between Interpretability and Accuracy
Venue: arXiv cs.LG 2019
Year: 2019

We analyze the trade-off between model complexity and accuracy for random forests by breaking the trees up into individual classification rules and selecting a subset of them. We show experimentally that already a few rules are sufficient to achieve an acceptable accuracy close to that of the original model. Moreover, our results indicate that in many cases, this can lead to simpler models that clearly outperform the original ones.
Draw This Object: A Study of Debugging Representations
Venue: arXiv cs.SE 2019
Year: 2019

Domain-specific debugging visualizations try to provide a view of a runtime object tailored to a specific domain and highlighting its important properties. The research in this area has focused mainly on the technical aspects of the creation of such views so far. However, we still lack answers to questions such as what properties of objects are considered important for these visualizations, whether all objects have an appropriate domain-specific view, or what clues could help us to construct these views fully automatically. In this paper, we describe an exploratory study where the participants were asked to inspect runtime states of objects displayed in a traditional debugger and draw ideal domain-specific views of these objects on paper. We describe interesting observations and findings obtained during this study and a preliminary taxonomy of these visualizations.
A Routing and Link Scheduling Strategy for Smart Grid NAN Communications
Venue: arXiv cs.NI 2019
Year: 2019

As large scale deployment of smart devices in the power grid continues, research efforts need to increasingly focus on efficient communication of generated information. This paper describes a strategy for static routing and scheduling of messages in a multi-hop wireless Smart Grid Neighborhood Area Network (NAN) with multiple source nodes and a common set of destinations or gateways. The problem is formulated as a Mixed Integer Linear Program (MILP) and solved using commercial optimization solver CPLEX. Feasibility of the scheme is demonstrated using different network models, constraints, message injection rates, and initial conditions. It is shown that the proposed approach can be used to generate an optimal link schedule for collecting user-generated bids in a transactive energy market in the least possible time. It is also shown that the methodology is applicable to multiple destination nodes and that their location affects message delivery time.
On the Capacity of Channels with Deletions and States
Venue: arXiv cs.IT 2019
Year: 2019

We consider the class of channels formed from the concatenation of a deletion channel and a finite-state channel. For this class of channels, we show that the operational capacity is equal to the stationary capacity, which can be approached by a sequence of Markov processes with increasing Markovian orders. As a by-product, we show that the polar coding scheme constructed by Tal, Pfister, Fazeli and Vardy achieves the capacity of the deletion channel.
A Computing Kernel for Network Binarization on PyTorch
Venue: arXiv cs.LG 2019
Year: 2019

Deep Neural Networks have now achieved state-of-the-art results in a wide range of tasks including image classification, object detection and so on. However, they are both computation consuming and memory intensive, making them difficult to deploy on low-power devices. Network binarization is one of the existing effective techniques for model compression and acceleration, but there is no computing kernel yet to support it on PyTorch. In this paper we developed a computing kernel supporting 1-bit xnor and bitcount computation on PyTorch. Experimental results show that our kernel could accelerate the inference of the binarized neural network by 3 times in GPU and by 4.5 times in CPU compared with the control group.
Supervised Initialization of LSTM Networks for Fundamental Frequency Detection in Noisy Speech Signals
Venue: arXiv cs.SD 2019
Year: 2019

Fundamental frequency is one of the most important parameters of human speech, of importance for the classification of accent, gender, speaking styles, speaker identification, age, among others. The proper detection of this parameter remains as an important challenge for severely degraded signals. In previous references for detecting fundamental frequency in noisy speech using deep learning, the networks, such as Long Short-term Memory (LSTM) has been initialized with random weights, and then trained following a back-propagation through time algorithm. In this work, a proposal for a more efficient initialization, based on a supervised training using an Auto-associative network, is presented. This initialization is a better starting point for the detection of fundamental frequency in noisy speech. The advantages of this initialization are noticeable using objective measures for the accuracy of the detection and for the training of the networks, under the presence of additive white noise at different signal-to-noise levels.
ContamiNet: Detecting Contamination in Municipal Solid Waste
Venue: arXiv cs.CV 2019
Year: 2019

Leveraging over 30,000 images each with up to 89 labels collected by Recology---an integrated resource recovery company with both residential and commercial trash, recycling and composting services---the authors develop ContamiNet, a convolutional neural network, to identify contaminating material in residential recycling and compost bins. When training the model on a subset of labels that meet a minimum frequency threshold, ContamiNet preforms almost as well human experts in detecting contamination (0.86 versus 0.88 AUC). Recology is actively piloting ContamiNet in their daily municipal solid waste (MSW) collection to identify contaminants in recycling and compost bins to subsequently inform and educate customers about best sorting practices.
Building the National Radio Recordings Database: A Big Data Approach to Documenting Audio Heritage
Venue: arXiv cs.DL 2019
Year: 2019

This paper traces strategies used by the Radio Preservation Task Force of the Library of Congress's National Recording Preservation Board to develop a publicly searchable database documenting extant radio materials held by collecting institutions throughout the country. Having aggregated metadata on 2,500 unique collections to date, the project has encountered a series of logistical challenges that are not only technical in nature but also institutional and social, raising critical issues involving organizational structure, political representation, and the ethics of data access. As the project continues to expand and evolve, lessons from its early development offer valuable reminders of the human judgment, hidden labor, and interpersonal relations required for successful big data work.
Robust Design of Deep Neural Networks against Adversarial Attacks based on Lyapunov Theory
Venue: arXiv cs.LG 2019
Year: 2019

Deep neural networks (DNNs) are vulnerable to subtle adversarial perturbations applied to the input. These adversarial perturbations, though imperceptible, can easily mislead the DNN. In this work, we take a control theoretic approach to the problem of robustness in DNNs. We treat each individual layer of the DNN as a nonlinear dynamical system and use Lyapunov theory to prove stability and robustness locally. We then proceed to prove stability and robustness globally for the entire DNN. We develop empirically tight bounds on the response of the output layer, or any hidden layer, to adversarial perturbations added to the input, or the input of hidden layers. Recent works have proposed spectral norm regularization as a solution for improving robustness against l2 adversarial attacks. Our results give new insights into how spectral norm regularization can mitigate the adversarial effects. Finally, we evaluate the power of our approach on a variety of data sets and network architectures and against some of the well-known adversarial attacks.
LAC-Nav: Collision-Free Mutiagent Navigation Based on The Local Action Cells
Venue: arXiv cs.MA 2019
Year: 2019

Collision avoidance is one of the most primary requirement in the decentralized multiagent navigations: while the agents are moving towards their own targets, attentions should be paid to avoid the collisions with the others. In this paper, we introduce the concept of local action cell, which provides for each agent a set of velocities that are safe to perform. Based on the realtime updated local action cells, we propose the LAC-Nav approach to navigate the agent with the properly selected velocity; and furthermore, we coupled the local action cell with an adaptive learning framework, in which the effect of selections are evaluated and used as the references for making decisions in the following updates. Through the experiments for three commonly considered scenarios, we demonstrated the efficiency of the proposed approaches, with the comparison to several widely studied strategies.
Norm-Explicit Quantization: Improving Vector Quantization for Maximum Inner Product Search
Venue: arXiv cs.IR 2019
Year: 2019

Vector quantization (VQ) techniques are widely used in similarity search for data compression, fast metric computation and etc. Originally designed for Euclidean distance, existing VQ techniques (e.g., PQ, AQ) explicitly or implicitly minimize the quantization error. In this paper, we present a new angle to analyze the quantization error, which decomposes the quantization error into norm error and direction error. We show that quantization errors in norm have much higher influence on inner products than quantization errors in direction, and small quantization error does not necessarily lead to good performance in maximum inner product search (MIPS). Based on this observation, we propose norm-explicit quantization (NEQ) --- a general paradigm that improves existing VQ techniques for MIPS. NEQ quantizes the norms of items in a dataset explicitly to reduce errors in norm, which is crucial for MIPS. For the direction vectors, NEQ can simply reuse an existing VQ technique to quantize them without modification. We conducted extensive experiments on a variety of datasets and parameter configurations. The experimental results show that NEQ improves the performance of various VQ techniques for MIPS, including PQ, OPQ, RQ and AQ.
Random Projections of Mel-Spectrograms as Low-Level Features for Automatic Music Genre Classification
Venue: arXiv cs.SD 2019
Year: 2019

In this work, we analyse the random projections of Mel-spectrograms as low-level features for music genre classification. This approach was compared to handcrafted features, features learned using an auto-encoder and features obtained from a transfer learning setting. Tests in five different well-known, publicly available datasets show that random projections leads to results comparable to learned features and outperforms features obtained via transfer learning in a shallow learning scenario. Random projections do not require using extensive specialist knowledge and, simultaneously, requires less computational power for training than other projection-based low-level features. Therefore, they can be are a viable choice for usage in shallow learning content-based music genre classification.