OpenRaiser/Intern-Atlas huggingface dataset:
Beyond Weight Tying: Learning Joint Input-Output Embeddings for Neural Machine Translation
Venue: arXiv cs.CL 2018
Year: 2018

Tying the weights of the target word embeddings with the target word classifiers of neural machine translation models leads to faster training and often to better translation quality. Given the success of this parameter sharing, we investigate other forms of sharing in between no sharing and hard equality of parameters. In particular, we propose a structure-aware output layer which captures the semantic structure of the output space of words within a joint input-output embedding. The model is a generalized form of weight tying which shares parameters but allows learning a more flexible relationship with input word embeddings and allows the effective capacity of the output layer to be controlled. In addition, the model shares weights across output classifiers and translation contexts which allows it to better leverage prior knowledge about them. Our evaluation on English-to-Finnish and English-to-German datasets shows the effectiveness of the method against strong encoder-decoder baselines trained with or without weight tying.
Total Recall, Language Processing, and Software Engineering
Venue: arXiv cs.SE 2018
Year: 2018

A broad class of software engineering problems can be generalized as the "total recall problem". This short paper claims that identifying and exploring total recall language processing problems in software engineering is an important task with wide applicability. To make that case, we show that by applying and adapting the state of the art active learning and text mining, solutions of the total recall problem, can help solve two important software engineering tasks: (a) supporting large literature reviews and (b) identifying software security vulnerabilities. Furthermore, we conjecture that (c) test case prioritization and (d) static warning identification can also be categorized as the total recall problem. The widespread applicability of "total recall" to software engineering suggests that there exists some underlying framework that encompasses not just natural language processing, but a wide range of important software engineering tasks.
A Multi-Timescale Data-Driven Approach to Enhance Distribution System Observability
Venue: arXiv cs.SY 2018
Year: 2018

This paper presents a novel data-driven method that determines the daily consumption patterns of customers without smart meters (SMs) to enhance the observability of distribution systems. Using the proposed method, the daily consumption of unobserved customers is extracted from their monthly billing data based on three machine learning models: first, a spectral clustering (SC) algorithm is used to infer the typical daily load profiles of customers with SMs. Each typical daily load behavior represents a distinct class of customer behavior. In the second module, a multi-timescale learning (MTSL) model is trained to estimate the hourly consumption using monthly energy data for the customers of each class. The third stage leverages a recursive Bayesian learning (RBL) method and branch current state estimation (BCSE) residuals to estimate the daily load profiles of unobserved customers without SMs. The proposed data-driven method has been tested and verified using real utility data.
On The Capacity of Gaussian MIMO Channels Under The Joint Power Constraints
Venue: arXiv cs.IT 2018
Year: 2018

The capacity and optimal signaling over a fixed Gaussian MIMO channel are considered under the joint total and per-antenna power constraints (TPC and PAC). While the general case remains an open problem, a closed-form full-rank solution is obtained along with its sufficient and necessary conditions. The conditions for each constraint to be inactive are established. The high and low-SNR regimes are studied. Isotropic signaling is shown to be optimal in the former case while rank-1 signaling (beamforming) is not necessarily optimal in the latter case. Unusual properties of optimal covariance under the joint constraints are pointed out.
Collective fast delivery by energy-efficient agents
Venue: arXiv cs.DS 2018
Year: 2018

We consider k mobile agents initially located at distinct nodes of an undirected graph (on n nodes, with edge lengths) that have to deliver a single item from a given source node s to a given target node t. The agents can move along the edges of the graph, starting at time 0 with respect to the following: Each agent i has a weight w_i that defines the rate of energy consumption while travelling a distance in the graph, and a velocity v_i with which it can move. We are interested in schedules (operating the k agents) that result in a small delivery time T (time when the package arrives at t), and small total energy consumption E. Concretely, we ask for a schedule that: either (i) Minimizes T, (ii) Minimizes lexicographically (T,E) (prioritizing fast delivery), or (iii) Minimizes epsilon*T + (1-epsilon)*E, for a given epsilon, 0<epsilon<1. We show that (i) is solvable in polynomial time, and show that (ii) is polynomial-time solvable for uniform velocities and solvable in time O(n + k log k) for arbitrary velocities on paths, but in general is NP-hard even on planar graphs. As a corollary of our hardness result, (iii) is NP-hard, too. We show that there is a 3-approximation algorithm for (iii) using a single agent.
Eliminating Boundaries in Cloud Storage with Anna
Venue: arXiv cs.DB 2018
Year: 2018

In this paper, we describe how we extended a distributed key-value store called Anna into an elastic, multi-tier service for the cloud. In its extended form, Anna is designed to overcome the narrow cost-performance limitations typical of current cloud storage systems. We describe three key aspects of Anna's new design: multi-master selective replication of hot keys, a vertical tiering of storage layers with different cost-performance tradeoffs, and horizontal elasticity of each tier to add and remove nodes in response to load dynamics. Anna's policy engine uses these mechanisms to balance service-level objectives around cost, latency and fault tolerance. Experimental results explore the behavior of Anna's mechanisms and policy, exhibiting orders of magnitude efficiency improvements over both commodity cloud KVS services and research systems.
LIUM-CVC Submissions for WMT18 Multimodal Translation Task
Venue: arXiv cs.CL 2018
Year: 2018

This paper describes the multimodal Neural Machine Translation systems developed by LIUM and CVC for WMT18 Shared Task on Multimodal Translation. This year we propose several modifications to our previous multimodal attention architecture in order to better integrate convolutional features and refine them using encoder-side information. Our final constrained submissions ranked first for English-French and second for English-German language pairs among the constrained submissions according to the automatic evaluation metric METEOR.
Hypergraph Modeling and Visualisation of Complex Co-occurence Networks
Venue: arXiv cs.SI 2018
Year: 2018

Finding inherent or processed links within a dataset allows to discover potential knowledge. The main contribution of this article is to define a global framework that enables optimal knowledge discovery by visually rendering co-occurences (i.e. groups of linked data instances attached to a metadata reference) - either inherently present or processed - from a dataset as facets. Hypergraphs are well suited for modeling co-occurences since they support multi-adicity whereas graphs only support pairwise relationships. This article introduces an efficient navigation between different facets of an information space based on hypergraph modelisation and visualisation.
Linear regression analysis of template aging in iris biometrics
Venue: arXiv cs.CV 2018
Year: 2018

The aim of this work is to determine how vulnerable different iris coding methods are in relation to biometric template aging phenomenon. This is considered to be particularly important when the time lapse between gallery and probe samples extends significantly, to more than a few years. Our experiments employ iris aging analysis conducted using three different iris recognition algorithms and a database of 583 samples from 58 irises collected up to nine years apart. To determine the degradation rates of similarity scores with extending time lapse and also in relation to multiple image quality and geometrical factors of sample images, a linear regression analysis was performed. 29 regression models have been tested with both the time parameter and geometrical factors being statistically significant in every model. Quality measures that showed statistically significant influence on the predicted variable were, depending on the method, image sharpness and local contrast or their mutual relations. To our best knowledge, this is the first paper describing aging analysis using multiple regression models with data covering such a wide time period. Results presented suggest that template aging effect occurs in iris biometrics to a statistically significant extent. Image quality and geometrical factors may contribute to the degradation of similarity score. However, the estimate of time parameter showed statistical significance and similar value in each of the tested models. This reveals that the aging phenomenon may as well be unrelated to quality and geometrical measures of the image.
Iris Recognition with a Database of Iris Images Obtained in Visible Light Using Smartphone Camera
Venue: arXiv cs.CV 2018
Year: 2018

This paper delivers a new database of iris images collected in visible light using a mobile phone's camera and presents results of experiments involving existing commercial and open-source iris recognition methods, namely: IriCore, VeriEye, MIRLIN and OSIRIS. Several important observations are made. First, we manage to show that after simple preprocessing, such images offer good visibility of iris texture even in heavily-pigmented irides. Second, for all four methods, the enrollment stage is not much affected by the fact that different type of data is used as input. This translates to zero or close-to-zero Failure To Enroll, i.e., cases when templates could not be extracted from the samples. Third, we achieved good matching accuracy, with correct genuine match rate exceeding 94.5% for all four methods, while simultaneously being able to maintain zero false match rate in every case. Correct genuine match rate of over 99.5% was achieved using one of the commercial methods, showing that such images can be used with the existing biometric solutions with minimum additional effort required. Finally, the experiments revealed that incorrect image segmentation is the most prevalent cause of recognition accuracy decrease. To our best knowledge, this is the first database of iris images captured using a mobile device, in which image quality exceeds this of a near-infrared illuminated iris images, as defined in ISO/IEC 19794-6 and 29794-6 documents. This database will be publicly available to all researchers.
ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
Venue: arXiv cs.CV 2018
Year: 2018

The Super-Resolution Generative Adversarial Network (SRGAN) is a seminal work that is capable of generating realistic textures during single image super-resolution. However, the hallucinated details are often accompanied with unpleasant artifacts. To further enhance the visual quality, we thoroughly study three key components of SRGAN - network architecture, adversarial loss and perceptual loss, and improve each of them to derive an Enhanced SRGAN (ESRGAN). In particular, we introduce the Residual-in-Residual Dense Block (RRDB) without batch normalization as the basic network building unit. Moreover, we borrow the idea from relativistic GAN to let the discriminator predict relative realness instead of the absolute value. Finally, we improve the perceptual loss by using the features before activation, which could provide stronger supervision for brightness consistency and texture recovery. Benefiting from these improvements, the proposed ESRGAN achieves consistently better visual quality with more realistic and natural textures than SRGAN and won the first place in the PIRM2018-SR Challenge. The code is available at https://github.com/xinntao/ESRGAN .
Activity Recognition on a Large Scale in Short Videos - Moments in Time Dataset
Venue: arXiv cs.CV 2018
Year: 2018

Moments capture a huge part of our lives. Accurate recognition of these moments is challenging due to the diverse and complex interpretation of the moments. Action recognition refers to the act of classifying the desired action/activity present in a given video. In this work, we perform experiments on Moments in Time dataset to recognize accurately activities occurring in 3 second clips. We use state of the art techniques for visual, auditory and spatio temporal localization and develop method to accurately classify the activity in the Moments in Time dataset. Our novel approach of using Visual Based Textual features and fusion techniques performs well providing an overall 89.23 % Top - 5 accuracy on the 20 classes - a significant improvement over the Baseline TRN model.
A Contextual-bandit-based Approach for Informed Decision-making in Clinical Trials
Venue: arXiv cs.AI 2018
Year: 2018

Clinical trials involving multiple treatments utilize randomization of the treatment assignments to enable the evaluation of treatment efficacies in an unbiased manner. Such evaluation is performed in post hoc studies that usually use supervised-learning methods that rely on large amounts of data collected in a randomized fashion. That approach often proves to be suboptimal in that some participants may suffer and even die as a result of having not received the most appropriate treatments during the trial. Reinforcement-learning methods improve the situation by making it possible to learn the treatment efficacies dynamically during the course of the trial, and to adapt treatment assignments accordingly. Recent efforts using \textit{multi-arm bandits}, a type of reinforcement-learning methods, have focused on maximizing clinical outcomes for a population that was assumed to be homogeneous. However, those approaches have failed to account for the variability among participants that is becoming increasingly evident as a result of recent clinical-trial-based studies. We present a contextual-bandit-based online treatment optimization algorithm that, in choosing treatments for new participants in the study, takes into account not only the maximization of the clinical outcomes but also the patient characteristics. We evaluated our algorithm using a real clinical trial dataset from the International Stroke Trial. The results of our retrospective analysis indicate that the proposed approach performs significantly better than either a random assignment of treatments (the current gold standard) or a multi-arm-bandit-based approach, providing substantial gains in the percentage of participants who are assigned the most suitable treatments. The contextual-bandit and multi-arm bandit approaches provide 72.63% and 64.34% gains, respectively, compared to a random assignment.
Exploring the Limits of Complexity: A Survey of Empirical Studies on Graph Visualisation
Venue: arXiv cs.HC 2018
Year: 2018

For decades, researchers in information visualisation and graph drawing have focused on developing techniques for the layout and display of very large and complex networks. Experiments involving human participants have also explored the readability of different styles of layout and representations for such networks. In both bodies of literature, networks are frequently referred to as being 'large' or 'complex', yet these terms are relative. From a human-centred, experiment point-of-view, what constitutes 'large' (for example) depends on several factors, such as data complexity, visual complexity, and the technology used. In this paper, we survey the literature on human-centred experiments to understand how, in practice, different features and characteristics of node-link diagrams affect visual complexity.
Momentum Model-based Minimal Parameter Identification of a Space Robot
Venue: arXiv cs.RO 2018
Year: 2018

Accurate information of inertial parameters is critical to motion planning and control of space robots. Before the launch, only a rudimentary estimate of the inertial parameters is available from experiments and computer-aided design (CAD) models. After the launch, on-orbit operations substantially alter the value of inertial parameters. In this work, we propose a new momentum model-based method for identifying the minimal parameters of a space robot while on orbit. Minimal parameters are combinations of the inertial parameters of the links and uniquely define the momentum and dynamic models. Consequently, they are sufficient for motion planning and control of both the satellite and robotic arms mounted on it. The key to the proposed framework is the unique formulation of momentum model in the linear form of minimal parameters. Further, to estimate the minimal parameters, we propose a novel joint trajectory planning and optimization technique based on direction combinations of joints' velocity. The efficacy of the identification framework is demonstrated on a 12 degrees-of-freedom, spatial, dual-arm space robot. The methodology is developed for tree-type space robots, requires just the pose and twist data, and scalable with increasing number of joints.
Learning to Navigate Autonomously in Outdoor Environments : MAVNet
Venue: arXiv cs.RO 2018
Year: 2018

In the modern era of automation and robotics, autonomous vehicles are currently the focus of academic and industrial research. With the ever increasing number of unmanned aerial vehicles getting involved in activities in the civilian and commercial domain, there is an increased need for autonomy in these systems too. Due to guidelines set by the governments regarding the operation ceiling of civil drones, road-tracking based navigation is garnering interest . In an attempt to achieve the above mentioned tasks, we propose an imitation learning based, data-driven solution to UAV autonomy for navigating through city streets by learning to fly by imitating an expert pilot. Derived from the classic image classification algorithms, our classifier has been constructed in the form of a fast 39-layered Inception model, that evaluates the presence of roads using the tomographic reconstructions of the input frames. Based on the Inception-v3 architecture, our system performs better in terms of processing complexity and accuracy than many existing models for imitation learning. The data used for training the system has been captured from the drone, by flying it in and around urban and semi-urban streets, by experts having at least 6-8 years of flying experience. Permissions were taken from required authorities who made sure that minimal risk (to pedestrians) is involved in the data collection process. With the extensive amount of drone data that we collected, we have been able to navigate successfully through roads without crashing or overshooting, with an accuracy of 98.44%. The computational efficiency of MAVNet enables the drone to fly at high speeds of upto 6m/sec. We present the same results in this research and compare them with other state-of-the-art methods of vision and learning based navigation.
Modeling Topical Coherence in Discourse without Supervision
Venue: arXiv cs.CL 2018
Year: 2018

Coherence of text is an important attribute to be measured for both manually and automatically generated discourse; but well-defined quantitative metrics for it are still elusive. In this paper, we present a metric for scoring topical coherence of an input paragraph on a real-valued scale by analyzing its underlying topical structure. We first extract all possible topics that the sentences of a paragraph of text are related to. Coherence of this text is then measured by computing: (a) the degree of uncertainty of the topics with respect to the paragraph, and (b) the relatedness between these topics. All components of our modular framework rely only on unlabeled data and WordNet, thus making it completely unsupervised, which is an important feature for general-purpose usage of any metric. Experiments are conducted on two datasets - a publicly available dataset for essay grading (representing human discourse), and a synthetic dataset constructed by mixing content from multiple paragraphs covering diverse topics. Our evaluation shows that the measured coherence scores are positively correlated with the ground truth for both the datasets. Further validation to our coherence scores is provided by conducting human evaluation on the synthetic data, showing a significant agreement of 79.3%
GB-KMV: An Augmented KMV Sketch for Approximate Containment Similarity Search
Venue: arXiv cs.IR 2018
Year: 2018

In this paper, we study the problem of approximate containment similarity search. Given two records Q and X, the containment similarity between Q and X with respect to Q is |Q intersect X|/ |Q|. Given a query record Q and a set of records S, the containment similarity search finds a set of records from S whose containment similarity regarding Q are not less than the given threshold. This problem has many important applications in commercial and scientific fields such as record matching and domain search. Existing solution relies on the asymmetric LSH method by transforming the containment similarity to well-studied Jaccard similarity. In this paper, we use a different framework by transforming the containment similarity to set intersection. We propose a novel augmented KMV sketch technique, namely GB-KMV, which is data-dependent and can achieve a good trade-off between the sketch size and the accuracy. We provide a set of theoretical analysis to underpin the proposed augmented KMV sketch technique, and show that it outperforms the state-of-the-art technique LSH-E in terms of estimation accuracy under practical assumption. Our comprehensive experiments on real-life datasets verify that GB-KMV is superior to LSH-E in terms of the space-accuracy trade-off, time-accuracy trade-off, and the sketch construction time. For instance, with similar estimation accuracy (F-1 score), GB-KMV is over 100 times faster than LSH-E on some real-life dataset.
Image Segmentation with Pseudo-marginal MCMC Sampling and Nonparametric Shape Priors
Venue: arXiv cs.CV 2018
Year: 2018

In this paper, we propose an efficient pseudo-marginal Markov chain Monte Carlo (MCMC) sampling approach to draw samples from posterior shape distributions for image segmentation. The computation time of the proposed approach is independent from the size of the training set used to learn the shape prior distribution nonparametrically. Therefore, it scales well for very large data sets. Our approach is able to characterize the posterior probability density in the space of shapes through its samples, and to return multiple solutions, potentially from different modes of a multimodal probability density, which would be encountered, e.g., in segmenting objects from multiple shape classes. Experimental results demonstrate the potential of the proposed approach.