OpenRaiser/Intern-Atlas huggingface dataset:
Machine Learning for Reliable mmWave Systems: Blockage Prediction and Proactive Handoff
Venue: arXiv cs.IT 2018
Year: 2018

The sensitivity of millimeter wave (mmWave) signals to blockages is a fundamental challenge for mobile mmWave communication systems. The sudden blockage of the line-of-sight (LOS) link between the base station and the mobile user normally leads to disconnecting the communication session, which highly impacts the system reliability. Further, reconnecting the user to another LOS base station incurs high beam training overhead and critical latency problems. In this paper, we leverage machine learning tools and propose a novel solution for these reliability and latency challenges in mmWave MIMO systems. In the developed solution, the base stations learn how to predict that a certain link will experience blockage in the next few time frames using their past observations of adopted beamforming vectors. This allows the serving base station to proactively hand-over the user to another base station with a highly probable LOS link. Simulation results show that the developed deep learning based strategy successfully predicts blockage/hand-off in close to 95% of the times. This reduces the probability of communication session disconnection, which ensures high reliability and low latency in mobile mmWave systems.
Abnormality Detection inside Blood Vessels with Mobile Nanomachines
Venue: arXiv cs.IT 2018
Year: 2018

Motivated by the numerous healthcare applications of molecular communication within Internet of Bio-Nano Things (IoBNT), this work addresses the problem of abnormality detection in a blood vessel using multiple biological embedded computing devices called cooperative biological nanomachines (CNs), and a common receiver called the fusion center (FC). Due to blood flow inside a vessel, each CN and the FC are assumed to be mobile. In this work, each of the CNs perform abnormality detection with certain probabilities of detection and false alarm by counting the number of molecules received from a source, e.g., infected tissue. These CNs subsequently report their local decisions to a FC over a diffusion-advection blood flow channel using different types of molecules in the presence of inter-symbol interference, multi-source interference, and counting errors. Due to limited computational capability at the FC, OR and AND logic based fusion rules are employed to make the final decision after obtaining each local decision based on the optimal likelihood ratio test. For the aforementioned system, probabilities of detection and false alarm at the FC are derived for OR and AND fusion rules. Finally, simulation results are presented to validate the derived analytical results, which provide important insights.
Latent Semantic Analysis Approach for Document Summarization Based on Word Embeddings
Venue: arXiv cs.CL 2018
Year: 2018

Since the amount of information on the internet is growing rapidly, it is not easy for a user to find relevant information for his/her query. To tackle this issue, much attention has been paid to Automatic Document Summarization. The key point in any successful document summarizer is a good document representation. The traditional approaches based on word overlapping mostly fail to produce that kind of representation. Word embedding, distributed representation of words, has shown an excellent performance that allows words to match on semantic level. Naively concatenating word embeddings makes the common word dominant which in turn diminish the representation quality. In this paper, we employ word embeddings to improve the weighting schemes for calculating the input matrix of Latent Semantic Analysis method. Two embedding-based weighting schemes are proposed and then combined to calculate the values of this matrix. The new weighting schemes are modified versions of the augment weight and the entropy frequency. The new schemes combine the strength of the traditional weighting schemes and word embedding. The proposed approach is experimentally evaluated on three well-known English datasets, DUC 2002, DUC 2004 and Multilingual 2015 Single-document Summarization for English. The proposed model performs comprehensively better compared to the state-of-the-art methods, by at least 1% ROUGE points, leading to a conclusion that it provides a better document representation and a better document summary as a result.
Densely Connected CNNs for Bird Audio Detection
Venue: arXiv cs.SD 2018
Year: 2018

Detecting bird sounds in audio recordings automatically, if accurate enough, is expected to be of great help to the research community working in bio- and ecoacoustics, interested in monitoring biodiversity based on audio field recordings. To estimate how accurate the state-of-the-art machine learning approaches are, the Bird Audio Detection challenge involving large audio datasets was recently organized. In this paper, experiments using several types of convolutional neural networks (i.e. standard CNNs, residual nets and densely connected nets) are reported in the framework of this challenge. DenseNets were the preferred solution since they were the best performing and most compact models, leading to a 88.22% area under the receiver operator curve score on the test set of the challenge. Performance gains were obtained thank to data augmentation through time and frequency shifting, model parameter averaging during training and ensemble methods using the geometric mean. On the contrary, the attempts to enlarge the training dataset with samples of the test set with automatic predictions used as pseudo-groundtruth labels consistently degraded performance.
BALSON: Bayesian Least Squares Optimization with Nonnegative L1-Norm Constraint
Venue: arXiv cs.LG 2018
Year: 2018

A Bayesian approach termed BAyesian Least Squares Optimization with Nonnegative L1-norm constraint (BALSON) is proposed. The error distribution of data fitting is described by Gaussian likelihood. The parameter distribution is assumed to be a Dirichlet distribution. With the Bayes rule, searching for the optimal parameters is equivalent to finding the mode of the posterior distribution. In order to explicitly characterize the nonnegative L1-norm constraint of the parameters, we further approximate the true posterior distribution by a Dirichlet distribution. We estimate the statistics of the approximating Dirichlet posterior distribution by sampling methods. Four sampling methods have been introduced. With the estimated posterior distributions, the original parameters can be effectively reconstructed in polynomial fitting problems, and the BALSON framework is found to perform better than conventional methods.
Distillation Techniques for Pseudo-rehearsal Based Incremental Learning
Venue: arXiv cs.CV 2018
Year: 2018

The ability to learn from incrementally arriving data is essential for any life-long learning system. However, standard deep neural networks forget the knowledge about the old tasks, a phenomenon called catastrophic forgetting, when trained on incrementally arriving data. We discuss the biases in current Generative Adversarial Networks (GAN) based approaches that learn the classifier by knowledge distillation from previously trained classifiers. These biases cause the trained classifier to perform poorly. We propose an approach to remove these biases by distilling knowledge from the classifier of AC-GAN. Experiments on MNIST and CIFAR10 show that this method is comparable to current state of the art rehearsal based approaches. The code for this paper is available at https://bit.ly/incremental-learning
Deeply Supervised Rotation Equivariant Network for Lesion Segmentation in Dermoscopy Images
Venue: arXiv cs.CV 2018
Year: 2018

Automatic lesion segmentation in dermoscopy images is an essential step for computer-aided diagnosis of melanoma. The dermoscopy images exhibits rotational and reflectional symmetry, however, this geometric property has not been encoded in the state-of-the-art convolutional neural networks based skin lesion segmentation methods. In this paper, we present a deeply supervised rotation equivariant network for skin lesion segmentation by extending the recent group rotation equivariant network~\cite{cohen2016group}. Specifically, we propose the G-upsampling and G-projection operations to adapt the rotation equivariant classification network for our skin lesion segmentation problem. To further increase the performance, we integrate the deep supervision scheme into our proposed rotation equivariant segmentation architecture. The whole framework is equivariant to input transformations, including rotation and reflection, which improves the network efficiency and thus contributes to the segmentation performance. We extensively evaluate our method on the ISIC 2017 skin lesion challenge dataset. The experimental results show that our rotation equivariant networks consistently excel the regular counterparts with the same model complexity under different experimental settings. Our best model achieves 77.23\%(JA) on the test dataset, outperforming the state-of-the-art challenging methods and further demonstrating the effectiveness of our proposed deeply supervised rotation equivariant segmentation network. Our best model also outperforms the state-of-the-art challenging methods, which further demonstrate the effectiveness of our proposed deeply supervised rotation equivariant segmentation network.
Social network aided plagiarism detection: Social network aided plagiarism detection
Venue: arXiv cs.SI 2018
Year: 2018

The prevalence of different kinds of electronic devices and the volume of content on the Web have increased the amount of plagiarism, which is considered an unethical act. If we want to be efficient in the detection and prevention of these acts, we have to improve today's methods of discovering plagiarism. The paper presents a research study where a framework for the improved detection of plagiarism is proposed. The framework focuses on the integration of social network information, information from the Web, and an advanced semantically enriched visualization of information about authors and documents that enables the exploration of obtained data by seeking of advanced patterns of plagiarism. To support the proposed framework, a special software tool was also developed. The statistical evaluation confirmed that the employment of social network analysis and advanced visualization techniques led to improvements in the confirmation and investigation stages of the plagiarism detection process, thereby enhancing the overall efficiency of the plagiarism detection process.
QDDS: A Novel Quantum Swarm Algorithm Inspired by a Double Dirac Delta Potential
Venue: arXiv cs.MA 2018
Year: 2018

In this paper a novel Quantum Double Delta Swarm (QDDS) algorithm modeled after the mechanism of convergence to the center of attractive potential field generated within a single well in a double Dirac delta well setup has been put forward and the preliminaries discussed. Theoretical foundations and experimental illustrations have been incorporated to provide a first basis for further development, specifically in refinement of solutions and applicability to problems in high dimensional spaces. Simulations are carried out over varying dimensionality on four benchmark functions, viz. Rosenbrock, Rastrigrin, Griewank and Sphere as well as the multidimensional Finite Impulse Response (FIR) Filter design problem with different population sizes. Test results illustrate the algorithm yields superior results to some related reports in the literature while reinforcing the need of substantial future work to deliver near-optimal results consistently, especially if dimensionality scales up.
Automated labeling of bugs and tickets using attention-based mechanisms in recurrent neural networks
Venue: arXiv cs.IR 2018
Year: 2018

We explore solutions for automated labeling of content in bug trackers and customer support systems. In order to do that, we classify content in terms of several criteria, such as priority or product area. In the first part of the paper, we provide an overview of existing methods used for text classification. These methods fall into two categories - the ones that rely on neural networks and the ones that don't. We evaluate results of several solutions of both kinds. In the second part of the paper we present our own recurrent neural network solution based on hierarchical attention paradigm. It consists of several Hierarchical Attention network blocks with varying Gated Recurrent Unit cell sizes and a complementary shallow network that goes alongside. Lastly, we evaluate above-mentioned methods when predicting fields from two datasets - Arch Linux bug tracker and Chromium bug tracker. Our contributions include a comprehensive benchmark between a variety of methods on relevant datasets; a novel solution that outperforms previous generation methods; and two new datasets that are made public for further research.
Vulnerability Analysis of Chest X-Ray Image Classification Against Adversarial Attacks
Venue: arXiv cs.CV 2018
Year: 2018

Recently, there have been several successful deep learning approaches for automatically classifying chest X-ray images into different disease categories. However, there is not yet a comprehensive vulnerability analysis of these models against the so-called adversarial perturbations/attacks, which makes deep models more trustful in clinical practices. In this paper, we extensively analyzed the performance of two state-of-the-art classification deep networks on chest X-ray images. These two networks were attacked by three different categories (ten methods in total) of adversarial methods (both white- and black-box), namely gradient-based, score-based, and decision-based attacks. Furthermore, we modified the pooling operations in the two classification networks to measure their sensitivities against different attacks, on the specific task of chest X-ray classification.
A Neural Network Lattice Decoding Algorithm
Venue: arXiv cs.IT 2018
Year: 2018

Neural network decoding algorithms are recently introduced by Nachmani et al. to decode high-density parity-check (HDPC) codes. In contrast with iterative decoding algorithms such as sum-product or min-sum algorithms in which the weight of each edge is set to $1$, in the neural network decoding algorithms, the weight of every edge depends on its impact in the transmitted codeword. In this paper, we provide a novel \emph{feed-forward neural network lattice decoding algorithm} suitable to decode lattices constructed based on Construction A, whose underlying codes have HDPC matrices. We first establish the concept of feed-forward neural network for HDPC codes and improve their decoding algorithms compared to Nachmani et al. We then apply our proposed decoder for a Construction A lattice with HDPC underlying code, for which the well-known iterative decoding algorithms show poor performances. The main advantage of our proposed algorithm is that instead of assigning and training weights for all edges, which turns out to be time-consuming especially for high-density parity-check matrices, we concentrate on edges which are present in most of $4$-cycles and removing them gives a girth-$6$ Tanner graph. This approach, by slight modifications using updated LLRs instead of initial ones, simultaneously accelerates the training process and improves the error performance of our proposed decoding algorithm.
Step-by-step Erasion, One-by-one Collection: A Weakly Supervised Temporal Action Detector
Venue: arXiv cs.CV 2018
Year: 2018

Weakly supervised temporal action detection is a Herculean task in understanding untrimmed videos, since no supervisory signal except the video-level category label is available on training data. Under the supervision of category labels, weakly supervised detectors are usually built upon classifiers. However, there is an inherent contradiction between classifier and detector; i.e., a classifier in pursuit of high classification performance prefers top-level discriminative video clips that are extremely fragmentary, whereas a detector is obliged to discover the whole action instance without missing any relevant snippet. To reconcile this contradiction, we train a detector by driving a series of classifiers to find new actionness clips progressively, via step-by-step erasion from a complete video. During the test phase, all we need to do is to collect detection results from the one-by-one trained classifiers at various erasing steps. To assist in the collection process, a fully connected conditional random field is established to refine the temporal localization outputs. We evaluate our approach on two prevailing datasets, THUMOS'14 and ActivityNet. The experiments show that our detector advances state-of-the-art weakly supervised temporal action detection results, and even compares with quite a few strongly supervised methods.
Toward Demand-Aware Networking: A Theory for Self-Adjusting Networks
Venue: arXiv cs.NI 2018
Year: 2018

The physical topology is emerging as the next frontier in an ongoing effort to render communication networks more flexible. While first empirical results indicate that these flexibilities can be exploited to reconfigure and optimize the network toward the workload it serves and, e.g., providing the same bandwidth at lower infrastructure cost, only little is known today about the fundamental algorithmic problems underlying the design of reconfigurable networks. This paper initiates the study of the theory of demand-aware, self-adjusting networks. Our main position is that self-adjusting networks should be seen through the lense of self-adjusting datastructures. Accordingly, we present a taxonomy classifying the different algorithmic models of demand-oblivious, fixed demand-aware, and reconfigurable demand-aware networks, introduce a formal model, and identify objectives and evaluation metrics. We also demonstrate, by examples, the inherent advantage of demand-aware networks over state-of-the-art demand-oblivious, fixed networks (such as expanders).
RACK: Automatic API Recommendation using Crowdsourced Knowledge
Venue: arXiv cs.SE 2018
Year: 2018

Traditional code search engines often do not perform well with natural language queries since they mostly apply keyword matching. These engines thus need carefully designed queries containing information about programming APIs for code search. Unfortunately, existing studies suggest that preparing an effective code search query is both challenging and time consuming for the developers. In this paper, we propose a novel API recommendation technique--RACK that recommends a list of relevant APIs for a natural language query for code search by exploiting keyword-API associations from the crowdsourced knowledge of Stack Overflow. We first motivate our technique using an exploratory study with 11 core Java packages and 344K Java posts from Stack Overflow. Experiments using 150 code search queries randomly chosen from three Java tutorial sites show that our technique recommends correct API classes within the top 10 results for about 79% of the queries which is highly promising. Comparison with two variants of the state-of-the-art technique also shows that RACK outperforms both of them not only in Top-K accuracy but also in mean average precision and mean recall by a large margin.
Jointly learning relevant subgraph patterns and nonlinear models of their indicators
Venue: arXiv cs.LG 2018
Year: 2018

Classification and regression in which the inputs are graphs of arbitrary size and shape have been paid attention in various fields such as computational chemistry and bioinformatics. Subgraph indicators are often used as the most fundamental features, but the number of possible subgraph patterns are intractably large due to the combinatorial explosion. We propose a novel efficient algorithm to jointly learn relevant subgraph patterns and nonlinear models of their indicators. Previous methods for such joint learning of subgraph features and models are based on search for single best subgraph features with specific pruning and boosting procedures of adding their indicators one by one, which result in linear models of subgraph indicators. In contrast, the proposed approach is based on directly learning regression trees for graph inputs using a newly derived bound of the total sum of squares for data partitions by a given subgraph feature, and thus can learn nonlinear models through standard gradient boosting. An illustrative example we call the Graph-XOR problem to consider nonlinearity, numerical experiments with real datasets, and scalability comparisons to naive approaches using explicit pattern enumeration are also presented.
Decreasing the size of the Restricted Boltzmann machine
Venue: arXiv cs.LG 2018
Year: 2018

We propose a method to decrease the number of hidden units of the restricted Boltzmann machine while avoiding decrease of the performance measured by the Kullback-Leibler divergence. Then, we demonstrate our algorithm by using numerical simulations.
XNOR Neural Engine: a Hardware Accelerator IP for 21.6 fJ/op Binary Neural Network Inference
Venue: arXiv cs.NE 2018
Year: 2018

Binary Neural Networks (BNNs) are promising to deliver accuracy comparable to conventional deep neural networks at a fraction of the cost in terms of memory and energy. In this paper, we introduce the XNOR Neural Engine (XNE), a fully digital configurable hardware accelerator IP for BNNs, integrated within a microcontroller unit (MCU) equipped with an autonomous I/O subsystem and hybrid SRAM / standard cell memory. The XNE is able to fully compute convolutional and dense layers in autonomy or in cooperation with the core in the MCU to realize more complex behaviors. We show post-synthesis results in 65nm and 22nm technology for the XNE IP and post-layout results in 22nm for the full MCU indicating that this system can drop the energy cost per binary operation to 21.6fJ per operation at 0.4V, and at the same time is flexible and performant enough to execute state-of-the-art BNN topologies such as ResNet-34 in less than 2.2mJ per frame at 8.9 fps.
Position-aware Self-attention with Relative Positional Encodings for Slot Filling
Venue: arXiv cs.CL 2018
Year: 2018

This paper describes how to apply self-attention with relative positional encodings to the task of relation extraction. We propose to use the self-attention encoder layer together with an additional position-aware attention layer that takes into account positions of the query and the object in the sentence. The self-attention encoder also uses a custom implementation of relative positional encodings which allow each word in the sentence to take into account its left and right context. The evaluation of the model is done on the TACRED dataset. The proposed model relies only on attention (no recurrent or convolutional layers are used), while improving performance w.r.t. the previous state of the art.
Efficient Characterization of Hidden Processor Memory Hierarchies
Venue: arXiv cs.DC 2018
Year: 2018

A processor's memory hierarchy has a major impact on the performance of running code. However, computing platforms, where the actual hardware characteristics are hidden from both the end user and the tools that mediate execution, such as a compiler, a JIT and a runtime system, are used more and more, for example, performing large scale computation in cloud and cluster. Even worse, in such environments, a single computation may use a collection of processors with dissimilar characteristics. Ignorance of the performance-critical parameters of the underlying system makes it difficult to improve performance by optimizing the code or adjusting runtime-system behaviors; it also makes application performance harder to understand. To address this problem, we have developed a suite of portable tools that can efficiently derive many of the parameters of processor memory hierarchies, such as levels, effective capacity and latency of caches and TLBs, in a matter of seconds. The tools use a series of carefully considered experiments to produce and analyze cache response curves automatically. The tools are inexpensive enough to be used in a variety of contexts that may include install time, compile time or runtime adaption, or performance understanding tools.