OpenRaiser/Intern-Atlas huggingface dataset:
A Communication-Efficient Multi-Agent Actor-Critic Algorithm for Distributed Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

This paper considers a distributed reinforcement learning problem in which a network of multiple agents aim to cooperatively maximize the globally averaged return through communication with only local neighbors. A randomized communication-efficient multi-agent actor-critic algorithm is proposed for possibly unidirectional communication relationships depicted by a directed graph. It is shown that the algorithm can solve the problem for strongly connected graphs by allowing each agent to transmit only two scalar-valued variables at one time.
Deep Learning for Fine-Grained Image Analysis: A Survey
Venue: arXiv cs.CV 2019
Year: 2019

Computer vision (CV) is the process of using machines to understand and analyze imagery, which is an integral branch of artificial intelligence. Among various research areas of CV, fine-grained image analysis (FGIA) is a longstanding and fundamental problem, and has become ubiquitous in diverse real-world applications. The task of FGIA targets analyzing visual objects from subordinate categories, \eg, species of birds or models of cars. The small inter-class variations and the large intra-class variations caused by the fine-grained nature makes it a challenging problem. During the booming of deep learning, recent years have witnessed remarkable progress of FGIA using deep learning techniques. In this paper, we aim to give a survey on recent advances of deep learning based FGIA techniques in a systematic way. Specifically, we organize the existing studies of FGIA techniques into three major categories: fine-grained image recognition, fine-grained image retrieval and fine-grained image generation. In addition, we also cover some other important issues of FGIA, such as publicly available benchmark datasets and its related domain specific applications. Finally, we conclude this survey by highlighting several directions and open problems which need be further explored by the community in the future.
AMD Severity Prediction And Explainability Using Image Registration And Deep Embedded Clustering
Venue: arXiv cs.CV 2019
Year: 2019

We propose a method to predict severity of age related macular degeneration (AMD) from input optical coherence tomography (OCT) images. Although there is no standard clinical severity scale for AMD, we leverage deep learning (DL) based image registration and clustering methods to identify diseased cases and predict their severity. Experiments demonstrate our approach's disease classification performance matches state of the art methods. The predicted disease severity performs well on previously unseen data. Registration output provides better explainability than class activation maps regarding label and severity decisions
Resource Allocation for Secure IRS-assisted Multiuser MISO Systems
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, we study resource allocation design for secure communication in intelligent reflecting surface (IRS)-assisted multiuser multiple-input single-output (MISO) communication systems. To enhance physical layer security, artificial noise (AN) is transmitted from the base station (BS) to deliberately impair the channel of an eavesdropper. In particular, we jointly optimize the phase shift matrix at the IRS and the beamforming vectors and AN covariance matrix at the BS for maximization of the system sum secrecy rate. To handle the resulting non-convex optimization problem, we develop an efficient suboptimal algorithm based on alternating optimization, successive convex approximation, semidefinite relaxation, and manifold optimization. Our simulation results reveal that the proposed scheme substantially improves the system sum secrecy rate compared to two baseline schemes.
ANETAC: Arabic Named Entity Transliteration and Classification Dataset
Venue: arXiv cs.CL 2019
Year: 2019

In this paper, we make freely accessible ANETAC our English-Arabic named entity transliteration and classification dataset that we built from freely available parallel translation corpora. The dataset contains 79,924 instances, each instance is a triplet (e, a, c), where e is the English named entity, a is its Arabic transliteration and c is its class that can be either a Person, a Location, or an Organization. The ANETAC dataset is mainly aimed for the researchers that are working on Arabic named entity transliteration, but it can also be used for named entity classification purposes.
Diachronic Embedding for Temporal Knowledge Graph Completion
Venue: arXiv cs.LG 2019
Year: 2019

Knowledge graphs (KGs) typically contain temporal facts indicating relationships among entities at different times. Due to their incompleteness, several approaches have been proposed to infer new facts for a KG based on the existing ones-a problem known as KG completion. KG embedding approaches have proved effective for KG completion, however, they have been developed mostly for static KGs. Developing temporal KG embedding models is an increasingly important problem. In this paper, we build novel models for temporal KG completion through equipping static models with a diachronic entity embedding function which provides the characteristics of entities at any point in time. This is in contrast to the existing temporal KG embedding approaches where only static entity features are provided. The proposed embedding function is model-agnostic and can be potentially combined with any static model. We prove that combining it with SimplE, a recent model for static KG embedding, results in a fully expressive model for temporal KG completion. Our experiments indicate the superiority of our proposal compared to existing baselines.
TEAGS: Time-aware Text Embedding Approach to Generate Subgraphs
Venue: arXiv cs.IR 2019
Year: 2019

Contagions (e.g. virus, gossip) spread over the nodes in propagation graphs. We can use the temporal and textual data of the nodes to compute the edge weights and then generate subgraphs with highly relevant nodes. This is beneficial to many applications. Yet, challenges abound. First, the propagation pattern between each pair of nodes may change by time. Second, not always the same contagion propagates. Hence, the state-of-the-art text mining approaches including topic-modeling cannot effectively compute the edge weights. Third, since the propagation is affected by time, the word-word co-occurrence patterns may differ in various temporal dimensions, that can decrease the effectiveness of word embedding approaches. We argue that multi-aspect temporal dimensions (hour, day, etc) should be considered to better calculate the correlation weights between the nodes. In this work, we devise a novel framework that on the one hand, integrates a neural network based time-aware word embedding component to construct the word vectors through multiple temporal facets, and on the other hand, uses a temporal generative model to compute the weights. Subsequently, we propose a Max-Heap Graph cutting algorithm to generate subgraphs. We validate our model through comprehensive experiments on real-world datasets. The results show that our model can retrieve the subgraphs more effective than other rivals and the temporal dynamics should be noticed both in word embedding and propagation processes.
Search-Based Serving Architecture of Embeddings-Based Recommendations
Venue: arXiv cs.IR 2019
Year: 2019

Over the past 10 years, many recommendation techniques have been based on embedding users and items in latent vector spaces, where the inner product of a (user,item) pair of vectors represents the predicted affinity of the user to the item. A wealth of literature has focused on the various modeling approaches that result in embeddings, and has compared their quality metrics, learning complexity, etc. However, much less attention has been devoted to the issues surrounding productization of an embeddings-based high throughput, low latency recommender system. In particular, how the system might keep up with the changing embeddings as new models are learnt. This paper describes a reference architecture of a high-throughput, large scale recommendation service which leverages a search engine as its runtime core. We describe how the search index and the query builder adapt to changes in the embeddings, which often happen at a different cadence than index builds. We provide solutions for both id-based and feature-based embeddings, as well as for batch indexing and incremental indexing setups. The described system is at the core of a Web content discovery service that serves tens of billions recommendations per day in response to billions of user requests.
Copula & Marginal Flows: Disentangling the Marginal from its Joint
Venue: arXiv cs.LG 2019
Year: 2019

Deep generative networks such as GANs and normalizing flows flourish in the context of high-dimensional tasks such as image generation. However, so far exact modeling or extrapolation of distributional properties such as the tail asymptotics generated by a generative network is not available. In this paper, we address this issue for the first time in the deep learning literature by making two novel contributions. First, we derive upper bounds for the tails that can be expressed by a generative network and demonstrate Lp-space related properties. There we show specifically that in various situations an optimal generative network does not exist. Second, we introduce and propose copula and marginal generative flows (CM flows) which allow for an exact modeling of the tail and any prior assumption on the CDF up to an approximation of the uniform distribution. Our numerical results support the use of CM flows.
Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale
Venue: arXiv cs.LG 2019
Year: 2019

Probabilistic programming languages (PPLs) are receiving widespread attention for performing Bayesian inference in complex generative models. However, applications to science remain limited because of the impracticability of rewriting complex scientific simulators in a PPL, the computational cost of inference, and the lack of scalable implementations. To address these, we present a novel PPL framework that couples directly to existing scientific simulators through a cross-platform probabilistic execution protocol and provides Markov chain Monte Carlo (MCMC) and deep-learning-based inference compilation (IC) engines for tractable inference. To guide IC inference, we perform distributed training of a dynamic 3DCNN--LSTM architecture with a PyTorch-MPI-based framework on 1,024 32-core CPU nodes of the Cori supercomputer with a global minibatch size of 128k: achieving a performance of 450 Tflop/s through enhancements to PyTorch. We demonstrate a Large Hadron Collider (LHC) use-case with the C++ Sherpa simulator and achieve the largest-scale posterior inference in a Turing-complete PPL.
Distill-2MD-MTL: Data Distillation based on Multi-Dataset Multi-Domain Multi-Task Frame Work to Solve Face Related Tasksks, Multi Task Learning, Semi-Supervised Learning
Venue: arXiv cs.CV 2019
Year: 2019

We propose a new semi-supervised learning method on face-related tasks based on Multi-Task Learning (MTL) and data distillation. The proposed method exploits multiple datasets with different labels for different-but-related tasks such as simultaneous age, gender, race, facial expression estimation. Specifically, when there are only a few well-labeled data for a specific task among the multiple related ones, we exploit the labels of other related tasks in different domains. Our approach is composed of (1) a new MTL method which can deal with weakly labeled datasets and perform several tasks simultaneously, and (2) an MTL-based data distillation framework which enables network generalization for the training and test data from different domains. Experiments show that the proposed multi-task system performs each task better than the baseline single task. It is also demonstrated that using different domain datasets along with the main dataset can enhance network generalization and overcome the domain differences between datasets. Also, comparing data distillation both on the baseline and MTL framework, the latter shows more accurate predictions on unlabeled data from different domains. Furthermore, by proposing a new learning-rate optimization method, our proposed network is able to dynamically tune its learning rate.
Segway DRIVE Benchmark: Place Recognition and SLAM Data Collected by A Fleet of Delivery Robots
Venue: arXiv cs.RO 2019
Year: 2019

Visual place recognition and simultaneous localization and mapping (SLAM) have recently begun to be used in real-world autonomous navigation tasks like food delivery. Existing datasets for SLAM research are often not representative of in situ operations, leaving a gap between academic research and real-world deployment. In response, this paper presents the Segway DRIVE benchmark, a novel and challenging dataset suite collected by a fleet of Segway delivery robots. Each robot is equipped with a global-shutter fisheye camera, a consumer-grade IMU synced to the camera on chip, two low-cost wheel encoders, and a removable high-precision lidar for generating reference solutions. As they routinely carry out tasks in office buildings and shopping malls while collecting data, the dataset spanning a year is characterized by planar motions, moving pedestrians in scenes, and changing environment and lighting. Such factors typically pose severe challenges and may lead to failures for SLAM algorithms. Moreover, several metrics are proposed to evaluate metric place recognition algorithms. With these metrics, sample SLAM and metric place recognition methods were evaluated on this benchmark. The first release of our benchmark has hundreds of sequences, covering more than 50 km of indoor floors. More data will be added as the robot fleet continues to operate in real life. The benchmark is available at http://drive.segwayrobotics.com/#/dataset/download.
Quantifying Transparency of Machine Learning Systems through Analysis of Contributions
Venue: arXiv cs.LG 2019
Year: 2019

Increased adoption and deployment of machine learning (ML) models into business, healthcare and other organisational processes, will result in a growing disconnect between the engineers and researchers who developed the models and the model's users and other stakeholders, such as regulators or auditors. This disconnect is inevitable, as models begin to be used over a number of years or are shared among third parties through user communities or via commercial marketplaces, and it will become increasingly difficult for users to maintain ongoing insight into the suitability of the parties who created the model, or the data that was used to train it. This could become problematic, particularly where regulations change and once-acceptable standards become outdated, or where data sources are discredited, perhaps judged to be biased or corrupted, either deliberately or unwittingly. In this paper we present a method for arriving at a quantifiable metric capable of ranking the transparency of the process pipelines used to generate ML models and other data assets, such that users, auditors and other stakeholders can gain confidence that they will be able to validate and trust the data sources and human contributors in the systems that they rely on for their business operations. The methodology for calculating the transparency metric, and the type of criteria that could be used to make judgements on the visibility of contributions to systems are explained and illustrated through an example scenario.
Physics Informed Extreme Learning Machine (PIELM) -- A rapid method for the numerical solution of partial differential equations
Venue: arXiv cs.LG 2019
Year: 2019

There has been rapid progress recently on the application of deep networks to the solution of partial differential equations, collectively labelled as Physics Informed Neural Networks (PINNs). In this paper, we develop Physics Informed Extreme Learning Machine (PIELM), a rapid version of PINNs which can be applied to stationary and time dependent linear partial differential equations. We demonstrate that PIELM matches or exceeds the accuracy of PINNs on a range of problems. We also discuss the limitations of neural network based approaches, including our PIELM, in the solution of PDEs on large domains and suggest an extension, a distributed version of our algorithm -{}- DPIELM. We show that DPIELM produces excellent results comparable to conventional numerical techniques in the solution of time-dependent problems. Collectively, this work contributes towards making the use of neural networks in the solution of partial differential equations in complex domains as a competitive alternative to conventional discretization techniques.
Publication modalities 'article in press' and 'open access' in relation to journal average citation
Venue: arXiv cs.DL 2019
Year: 2019

There has been a generalization in the use of two publication practices by scientific journals during the past decade: 1. 'article in press' or early view, which allows access to the accepted paper before its formal publication in an issue; 2. 'open access', which allows readers to obtain it freely and free of charge. This paper studies the influence of both publication modalities on the average impact of the journal and its evolution over time. It tries to identify the separate effect of access on citation into two major parts: early view and selection effect, managing to provide some evidence of the positive effect of both. Scopus is used as the database and CiteScore as the measure of journal impact. The prevalence of both publication modalities is quantified. Differences in the average impact factor of group of journals, according to their publication modalities, are tested. The evolution over time of the citation influence, from 2011 to 2016, is also analysed. Finally, a linear regression to explain the correlation of these publication practices with the CiteScore in 2016, in a ceteris paribus context, is estimated. Our main findings show evidence of a positive correlation between average journal impact and advancing the publication of accepted articles, moreover this correlation increases over time. The open access modality, in a ceteris paribus context, also correlates positively with average journal impact.
Secure Network Coding in the Setting in Which a Non-Source Node May Generate Random Keys
Venue: arXiv cs.IT 2019
Year: 2019

It is common in the study of secure multicast network coding in the presence of an eavesdropper that has access to $z$ network links, to assume that the source node is the only node that generates random keys. In this setting, the secure multicast rate is well understood. Computing the secure multicast rate, or even the secure unicast rate, in the more general setting in which all network nodes may generate (independent) random keys is known to be as difficult as computing the (non-secure) capacity of multiple-unicast network coding instances --- a well known open problem. This work treats an intermediate model of secure unicast in which only one node can generate random keys, however that node need not be the source node. The secure communication rate for this setting is characterized again with an eavesdropper that has access to $z$ network links.
Enabling Ultra Reliable Wireless Communications for Factory Automation with Distributed MIMO
Venue: arXiv cs.IT 2019
Year: 2019

Factory automation is one of the most challenging use cases for 5G-and-beyond mobile networks due to strict latency, availability and reliability constraints. In this work, an indoor factory scenario is considered, and distributed multiple-input multiple-output (MIMO) schemes are investigated in order to enable reliable communication to the actuators (ACs) active in the factory. Different levels of coordination among the access points serving the ACs and several beamforming schemes are considered and analyzed. To enforce system reliability, a max-min power allocation (MPA) algorithm is proposed, aimed at improving the signal to interference plus noise ratio (SINR) of the ACs with the worst channel conditions. Extensive system simulations are performed in a realistic scenario, which includes a new path-loss model based on recent measurements in factory scenarios, and, also, the presence of non-Gaussian impulsive noise. Numerical results show that distributed MIMO schemes with zero-forcing (ZF) beamforming and MPA have the potential of providing SINR gains in the order of tens of dB with respect to a centralized MIMO deployment, as well as that the impulsive noise can strongly degrade the system performance and thus requires specific detection and mitigation techniques.
Linking Art through Human Poses
Venue: arXiv cs.CV 2019
Year: 2019

We address the discovery of composition transfer in artworks based on their visual content. Automated analysis of large art collections, which are growing as a result of art digitization among museums and galleries, is an important tool for art history and assists cultural heritage preservation. Modern image retrieval systems offer good performance on visually similar artworks, but fail in the cases of more abstract composition transfer. The proposed approach links artworks through a pose similarity of human figures depicted in images. Human figures are the subject of a large fraction of visual art from middle ages to modernity and their distinctive poses were often a source of inspiration among artists. The method consists of two steps -- fast pose matching and robust spatial verification. We experimentally show that explicit human pose matching is superior to standard content-based image retrieval methods on a manually annotated art composition transfer dataset.
xRAC: Execution and Access Control for Restricted Application Containers on Managed Hosts
Venue: arXiv cs.NI 2019
Year: 2019

We propose xRAC to permit users to run special applications on managed hosts and to grant them access to protected network resources. We use restricted application containers (RACs) for that purpose. A RAC is a virtualization container with only a selected set of applications. Authentication verifies the RAC user's identity and the integrity of the RAC image. If the user is permitted to use the RAC on a managed host, launching the RAC is authorized and access to protected network resources may be given, e.g., to internal networks, servers, or the Internet. xRAC simplifies traffic control as the traffic of a RAC has a unique IPv6 address so that it can be easily identified in the network. The architecture of xRAC reuses standard technologies, protocols, and infrastructure. Those are the Docker virtualization platform and 802.1X including EAP-over-UDP and RADIUS. Thus, xRAC improves network security without modifying core parts of applications, hosts, and infrastructure. In this paper, we review the technological background of xRAC, explain its architecture, discuss selected use cases, and investigate on the performance. To demonstrate the feasibility of xRAC, we implement it based on standard components with only a few modifications. Finally, we validate xRAC through experiments.