OpenRaiser/Intern-Atlas huggingface dataset:
NE-Table: A Neural key-value table for Named Entities
Venue: arXiv cs.CL 2019
Year: 2019

Many Natural Language Processing (NLP) tasks depend on using Named Entities (NEs) that are contained in texts and in external knowledge sources. While this is easy for humans, the present neural methods that rely on learned word embeddings may not perform well for these NLP tasks, especially in the presence of Out-Of-Vocabulary (OOV) or rare NEs. In this paper, we propose a solution for this problem, and present empirical evaluations on: a) a structured Question-Answering task, b) three related Goal-Oriented dialog tasks, and c) a Reading-Comprehension task, which show that the proposed method can be effective in dealing with both in-vocabulary and OOV NEs. We create extended versions of dialog bAbI tasks 1,2 and 4 and OOV versions of the CBT test set available at - https://github.com/IBM/ne-table-datasets.
Applying Faster R-CNN for Object Detection on Malaria Images
Venue: arXiv cs.CV 2019
Year: 2019

Deep learning based models have had great success in object detection, but the state of the art models have not yet been widely applied to biological image data. We apply for the first time an object detection model previously used on natural images to identify cells and recognize their stages in brightfield microscopy images of malaria-infected blood. Many micro-organisms like malaria parasites are still studied by expert manual inspection and hand counting. This type of object detection task is challenging due to factors like variations in cell shape, density, and color, and uncertainty of some cell classes. In addition, annotated data useful for training is scarce, and the class distribution is inherently highly imbalanced due to the dominance of uninfected red blood cells. We use Faster Region-based Convolutional Neural Network (Faster R-CNN), one of the top performing object detection models in recent years, pre-trained on ImageNet but fine tuned with our data, and compare it to a baseline, which is based on a traditional approach consisting of cell segmentation, extraction of several single-cell features, and classification using random forests. To conduct our initial study, we collect and label a dataset of 1300 fields of view consisting of around 100,000 individual cells. We demonstrate that Faster R-CNN outperforms our baseline and put the results in context of human performance.
A Novel Geometric Approach for Outlier Recognition in High Dimension
Venue: arXiv cs.CG 2019
Year: 2019

Outlier recognition is a fundamental problem in data analysis and has attracted a great deal of attention in the past decades. However, most existing methods still suffer from several issues such as high time and space complexities or unstable performances for different datasets. In this paper, we provide a novel algorithm for outlier recognition in high dimension based on the elegant geometric technique ``core-set". The algorithm needs only linear time and space complexities and achieves a solid theoretical quality guarantee. Another advantage over the existing methods is that our algorithm can be naturally extended to handle multi-class inliers. Our experimental results show that our algorithm outperforms existing algorithms on both random and benchmark datasets.
Learning for Video Compression
Venue: arXiv cs.MM 2019
Year: 2019

One key challenge to learning-based video compression is that motion predictive coding, a very effective tool for video compression, can hardly be trained into a neural network. In this paper we propose the concept of PixelMotionCNN (PMCNN) which includes motion extension and hybrid prediction networks. PMCNN can model spatiotemporal coherence to effectively perform predictive coding inside the learning network. On the basis of PMCNN, we further explore a learning-based framework for video compression with additional components of iterative analysis/synthesis, binarization, etc. Experimental results demonstrate the effectiveness of the proposed scheme. Although entropy coding and complex configurations are not employed in this paper, we still demonstrate superior performance compared with MPEG-2 and achieve comparable results with H.264 codec. The proposed learning-based scheme provides a possible new direction to further improve compression efficiency and functionalities of future video coding.
Quantum Algorithm for Dynamic Programming Approach for DAGs. Applications for Zhegalkin Polynomial Evaluation and Some Problems on DAGs
Venue: arXiv cs.DS 2019
Year: 2019

In this paper, we present a quantum algorithm for dynamic programming approach for problems on directed acyclic graphs (DAGs). The running time of the algorithm is $O(\sqrt{\hat{n}m}\log \hat{n})$, and the running time of the best known deterministic algorithm is $O(n+m)$, where $n$ is the number of vertices, $\hat{n}$ is the number of vertices with at least one outgoing edge; $m$ is the number of edges. We show that we can solve problems that use OR, AND, NAND, MAX and MIN functions as the main transition steps. The approach is useful for a couple of problems. One of them is computing a Boolean formula that is represented by Zhegalkin polynomial, a Boolean circuit with shared input and non-constant depth evaluating. Another two are the single source longest paths search for weighted DAGs and the diameter search problem for unweighted DAGs.
Deep Learning Coordinated Beamforming for Highly-Mobile Millimeter Wave Systems
Venue: arXiv cs.IT 2019
Year: 2019

Supporting high mobility in millimeter wave (mmWave) systems enables a wide range of important applications such as vehicular communications and wireless virtual/augmented reality. Realizing this in practice, though, requires overcoming several challenges. First, the use of narrow beams and the sensitivity of mmWave signals to blockage greatly impact the coverage and reliability of highly-mobile links. Second, highly-mobile users in dense mmWave deployments need to frequently hand-off between base stations (BSs), which is associated with critical control and latency overhead. Further, identifying the optimal beamforming vectors in large antenna array mmWave systems requires considerable training overhead, which significantly affects the efficiency of these mobile systems. In this paper, a novel integrated machine learning and coordinated beamforming solution is developed to overcome these challenges and enable highly-mobile mmWave applications. In the proposed solution, a number of distributed yet coordinating BSs simultaneously serve a mobile user. This user ideally needs to transmit only one uplink training pilot sequence that will be jointly received at the coordinating BSs using omni or quasi-omni beam patterns. These received signals draw a defining signature not only for the user location, but also for its interaction with the surrounding environment. The developed solution then leverages a deep learning model that learns how to use these signatures to predict the beamforming vectors at the BSs. This renders a comprehensive solution that supports highly-mobile mmWave applications with reliable coverage, low latency, and negligible training overhead. Simulation results show that the proposed deep-learning coordinated beamforming strategy approaches the achievable rate of the genie-aided solution that knows the optimal beamforming vectors with no training overhead.
Dense Adaptive Cascade Forest: A Self Adaptive Deep Ensemble for Classification Problems
Venue: arXiv cs.LG 2019
Year: 2019

Recent researches have shown that deep forest ensemble achieves a considerable increase in classification accuracy compared with the general ensemble learning methods, especially when the training set is small. In this paper, we take advantage of deep forest ensemble and introduce the Dense Adaptive Cascade Forest (daForest). Our model has a better performance than the original Cascade Forest with three major features: first, we apply SAMME.R boosting algorithm to improve the performance of the model. It guarantees the improvement as the number of layers increases. Second, our model connects each layer to the subsequent ones in a feed-forward fashion, which enhances the capability of the model to resist performance degeneration. Third, we add a hyper-parameters optimization layer before the first classification layer, making our model spend less time to set up and find the optimal hyper-parameters. Experimental results show that daForest performs significantly well, and in some cases, even outperforms neural networks and achieves state-of-the-art results.
Analytic Analysis of Narrowband IoT Coverage Enhancement Approaches
Venue: arXiv cs.NI 2019
Year: 2019

The introduction of Narrowband Internet of Things (NB-IoT) as a cellular IoT technology aims to support massive Machine-Type Communications applications. These applications are characterized by massive connections from a large number of low-complexity and low-power devices. One of the goals of NB-IoT is to improve coverage extension beyond existing cellular technologies. In order to do that, NB-IoT introduces transmission repetitions and different bandwidth allocation configurations in uplink. These new transmission approaches yield many transmission options in uplink. In this paper, we propose analytical expressions that describe the influence of these new approaches in the transmission. Our analysis is based on the Shannon theorem. The transmission is studied in terms of the required Signal to Noise Ratio, bandwidth utilization, and energy per transmitted bit. Additionally, we propose an uplink link adaptation algorithm that contemplates these new transmission approaches. The conducted evaluation summarizes the influence of these approaches. Furthermore, we present the resulting uplink link adaptation from our proposed algorithm sweeping the device's coverage.
Competitive Training of Mixtures of Independent Deep Generative Models
Venue: arXiv cs.LG 2019
Year: 2019

A common assumption in causal modeling posits that the data is generated by a set of independent mechanisms, and algorithms should aim to recover this structure. Standard unsupervised learning, however, is often concerned with training a single model to capture the overall distribution or aspects thereof. Inspired by clustering approaches, we consider mixtures of implicit generative models that ``disentangle'' the independent generative mechanisms underlying the data. Relying on an additional set of discriminators, we propose a competitive training procedure in which the models only need to capture the portion of the data distribution from which they can produce realistic samples. As a by-product, each model is simpler and faster to train. We empirically show that our approach splits the training distribution in a sensible way and increases the quality of the generated samples.
Multiobjective Optimization Differential Evolution Enhanced with Principle Component Analysis for Constrained Optimization
Venue: arXiv cs.NE 2019
Year: 2019

Multiobjective evolutionary algorithms (MOEAs) have been successfully applied to a number of constrained optimization problems. Many of them adopt mutation and crossover operators from differential evolution. However, these operators do not explicitly utilise features of fitness landscapes. To improve the performance of algorithms, this paper aims at designing a search operator adapting to fitness landscapes. Through an observation, we find that principle component analysis (PCA) can be used to characterise fitness landscapes. Based on this finding, a new search operator, called PCA-projection, is proposed. In order to verify the effectiveness of PCA-projection, we design two algorithms enhanced with PCA-projection for solving constrained optimization problems, called PMODE and HECO-PDE, respectively. Experiments have been conducted on the IEEE CEC 2017 competition benchmark suite in constrained optimisation. PMODE and HECO-PDE are compared with the algorithms from the IEEE CEC 2018 competition and another recent MOEA for constrained optimisation. Experimental results show that an algorithm enhanced with PCA-projection performs better than its corresponding opponent without this operator. Furthermore, HECO-PDE is ranked first on all dimensions according to the competition rules. This study reveals that decomposition-based MOEAs, such as HECO-PDE, are competitive with best single-objective and multiobjective evolutionary algorithms for constrained optimisation, but MOEAs based on non-dominance, such as PMODE, may not.
Estimating Learnability in the Sublinear Data Regime
Venue: arXiv cs.LG 2019
Year: 2019

We consider the problem of estimating how well a model class is capable of fitting a distribution of labeled data. We show that it is often possible to accurately estimate this "learnability" even when given an amount of data that is too small to reliably learn any accurate model. Our first result applies to the setting where the data is drawn from a $d$-dimensional distribution with isotropic covariance (or known covariance), and the label of each datapoint is an arbitrary noisy function of the datapoint. In this setting, we show that with $O(\sqrt{d})$ samples, one can accurately estimate the fraction of the variance of the label that can be explained via the best linear function of the data. In contrast to this sublinear sample size, finding an approximation of the best-fit linear function requires on the order of $d$ samples. Our sublinear sample results and approach also extend to the non-isotropic setting, where the data distribution has an (unknown) arbitrary covariance matrix: we show that, if the label $y$ of point $x$ is a linear function with independent noise, $y = \langle x , β\rangle + noise$ with $\|β\|$ bounded, the variance of the noise can be estimated to error $ε$ with $O(d^{1-1/\log{1/ε}})$ if the covariance matrix has bounded condition number, or $O(d^{1-\sqrtε})$ if there are no bounds on the condition number. We also establish that these sample complexities are optimal, to constant factors. Finally, we extend these techniques to the setting of binary classification, where we obtain analogous sample complexities for the problem of estimating the prediction error of the best linear classifier, in a natural model of binary labeled data. We demonstrate the practical viability of our approaches on several real and synthetic datasets.
Characterizing the Usage, Evolution and Impact of Java Annotations in Practice
Venue: arXiv cs.SE 2019
Year: 2019

Annotations have been formally introduced into Java since Java 5. Since then, annotations have been widely used by the Java community for different purposes, such as compiler guidance and runtime processing. Despite the ever-growing use, there is still limited empirical knowledge about the actual usage of annotations in practice, the changes made to annotations during software evolution, and the potential impact of annotations on code quality. To fill this gap, we perform the first large-scale empirical study about Java annotations on 1,094 notable open-source projects hosted on GitHub. Our study systematically investigates annotation usage, annotation evolution, and annotation impact, and generates 10 novel and important findings. We also present the implications of our findings, which shed light for developers, researchers, tool builders, and language or library designers in order to improve all facets of Java annotation engineering.
Transfer Learning of Artist Group Factors to Musical Genre Classification
Venue: arXiv cs.LG 2019
Year: 2019

The automated recognition of music genres from audio information is a challenging problem, as genre labels are subjective and noisy. Artist labels are less subjective and less noisy, while certain artists may relate more strongly to certain genres. At the same time, at prediction time, it is not guaranteed that artist labels are available for a given audio segment. Therefore, in this work, we propose to apply the transfer learning framework, learning artist-related information which will be used at inference time for genre classification. We consider different types of artist-related information, expressed through artist group factors, which will allow for more efficient learning and stronger robustness to potential label noise. Furthermore, we investigate how to achieve the highest validation accuracy on the given FMA dataset, by experimenting with various kinds of transfer methods, including single-task transfer, multi-task transfer and finally multi-task learning.
Generalized Random Gilbert-Varshamov Codes
Venue: arXiv cs.IT 2019
Year: 2019

We introduce a random coding technique for transmission over discrete memoryless channels, reminiscent of the basic construction attaining the Gilbert-Varshamov bound for codes in Hamming spaces. The code construction is based on drawing codewords recursively from a fixed type class, in such a way that a newly generated codeword must be at a certain minimum distance from all previously chosen codewords, according to some generic distance function. We derive an achievable error exponent for this construction, and prove its tightness with respect to the ensemble average. We show that the exponent recovers the Csiszár and K{ö}rner exponent as a special case, which is known to be at least as high as both the random-coding and expurgated exponents, and we establish the optimality of certain choices of the distance function. In addition, for additive distances and decoding metrics, we present an equivalent dual expression, along with a generalization to infinite alphabets via cost-constrained random coding.
Non-Asymptotic Achievable Rates for Gaussian Energy-Harvesting Channels: Best-Effort and Save-and-Transmit
Venue: arXiv cs.IT 2019
Year: 2019

An additive white Gaussian noise energy-harvesting channel with an infinite-sized battery is considered. The energy arrival process is modeled as a sequence of independent and identically distributed random variables. The channel capacity $\frac{1}{2}\log(1+P)$ is achievable by the so-called best-effort and save-and-transmit schemes where $P$ denotes the battery recharge rate. This paper analyzes the save-and-transmit scheme whose transmit power is strictly less than $P$ and the best-effort scheme as a special case of save-and-transmit without a saving phase. In the finite blocklength regime, we obtain new non-asymptotic achievable rates for these schemes that approach the capacity with gaps vanishing at rates proportional to $1/\sqrt{n}$ and $\sqrt{(\log n)/n}$ respectively where~$n$ denotes the blocklength. The proof technique involves analyzing the escape probability of a Markov process. When $P$ is sufficiently large, we show that allowing the transmit power to back off from $P$ can improve the performance for save-and-transmit. The results are extended to a block energy arrival model where the length of each energy block $L$ grows sublinearly in $n$. We show that the save-and-transmit and best-effort schemes achieve coding rates that approach the capacity with gaps vanishing at rates proportional to $\sqrt{L/n}$ and $\sqrt{\max\{\log n, L\}/n}$ respectively.
Graph Neural Networks for Learning Robot Team Coordination
Venue: arXiv cs.RO 2019
Year: 2019

This paper shows how Graph Neural Networks can be used for learning distributed coordination mechanisms in connected teams of robots. We capture the relational aspect of robot coordination by modeling the robot team as a graph, where each robot is a node, and edges represent communication links. During training, robots learn how to pass messages and update internal states, so that a target behavior is reached. As a proxy for more complex problems, this short paper considers the problem where each robot must locally estimate the algebraic connectivity of the team's network topology.
Standalone and Non-Standalone Beam Management for 3GPP NR at mmWaves
Venue: arXiv cs.NI 2019
Year: 2019

The next generation of cellular networks will exploit mmWave frequencies to dramatically increase the network capacity. The communication at such high frequencies, however, requires directionality to compensate the increase in propagation loss. Users and base stations need to align their beams during both initial access and data transmissions, to ensure the maximum gain is reached. The accuracy of the beam selection, and the delay in updating the beam pair or performing initial access, impact the end-to-end performance and the quality of service. In this paper we will present the beam management procedures that 3GPP has included in the NR specifications, focusing on the different operations that can be performed in Standalone (SA) and in Non-Standalone (NSA) deployments. We will also provide a performance comparison among different schemes, along with design insights on the most important parameters related to beam management frameworks.
A Local Stochastic Algorithm for Separation in Heterogeneous Self-Organizing Particle Systems
Venue: arXiv cs.DC 2019
Year: 2019

We present and rigorously analyze the behavior of a distributed, stochastic algorithm for separation and integration in self-organizing particle systems, an abstraction of programmable matter. Such systems are composed of individual computational particles with limited memory, strictly local communication abilities, and modest computational power. We consider heterogeneous particle systems of two different colors and prove that these systems can collectively separate into different color classes or integrate, indifferent to color. We accomplish both behaviors with the same fully distributed, local, stochastic algorithm. Achieving separation or integration depends only on a single global parameter determining whether particles prefer to be next to other particles of the same color or not; this parameter is meant to represent external, environmental influences on the particle system. The algorithm is a generalization of a previous distributed, stochastic algorithm for compression (PODC '16), which can be viewed as a special case of separation where all particles have the same color. It is significantly more challenging to prove that the desired behavior is achieved in the heterogeneous setting, however, even in the bichromatic case we focus on. This requires combining several new techniques, including the cluster expansion from statistical physics, a new variant of the bridging argument of Miracle, Pascoe and Randall (RANDOM '11), the high-temperature expansion of the Ising model, and careful probabilistic arguments.
Direction-aware Spatial Context Features for Shadow Detection and Removal
Venue: arXiv cs.CV 2019
Year: 2019

Shadow detection and shadow removal are fundamental and challenging tasks, requiring an understanding of the global image semantics. This paper presents a novel deep neural network design for shadow detection and removal by analyzing the spatial image context in a direction-aware manner. To achieve this, we first formulate the direction-aware attention mechanism in a spatial recurrent neural network (RNN) by introducing attention weights when aggregating spatial context features in the RNN. By learning these weights through training, we can recover direction-aware spatial context (DSC) for detecting and removing shadows. This design is developed into the DSC module and embedded in a convolutional neural network (CNN) to learn the DSC features at different levels. Moreover, we design a weighted cross entropy loss to make effective the training for shadow detection and further adopt the network for shadow removal by using a Euclidean loss function and formulating a color transfer function to address the color and luminosity inconsistencies in the training pairs. We employed two shadow detection benchmark datasets and two shadow removal benchmark datasets, and performed various experiments to evaluate our method. Experimental results show that our method performs favorably against the state-of-the-art methods for both shadow detection and shadow removal.