OpenRaiser/Intern-Atlas huggingface dataset:
Blockchain-based Bidirectional Updates on Fine-grained Medical Data
Venue: arXiv cs.SE 2019
Year: 2019

Electronic medical data sharing between stakeholders, such as patients, doctors, and researchers, can promote more effective medical treatment collaboratively. These sensitive and private data should only be accessed by authorized users. Given a total medical data, users may care about parts of them and other unrelated information might interfere with the user interested data search and increase the risk of exposure. Besides accessing these data, users may want to update them and propagate to other sharing peers so that all peers keep identical data after each update. To satisfy these requirements, in this paper we propose a medical data sharing architecture that addresses the permission control using smart contracts on the blockchain and splits data into fined grained pieces shared with different peers then synchronize full data and these pieces with bidirectional transformations. Medical data reside on each userś local database and permission related data are stored on smart contracts. Only all peers have gained the newest shared data after updates can they start to do next operations on it, which are enforced by smart contracts. Blockchain based immutable shared ledge enables users to trace data updates history. This paper can provide a new perspective to view full medical data as different slices to be shared with various peers but consistency after updates between them are still promised, which can protect the privacy and improve data search efficiency.
Design Automation for Efficient Deep Learning Computing
Venue: arXiv cs.LG 2019
Year: 2019

Efficient deep learning computing requires algorithm and hardware co-design to enable specialization: we usually need to change the algorithm to reduce memory footprint and improve energy efficiency. However, the extra degree of freedom from the algorithm makes the design space much larger: it's not only about designing the hardware but also about how to tweak the algorithm to best fit the hardware. Human engineers can hardly exhaust the design space by heuristics. It's labor consuming and sub-optimal. We propose design automation techniques for efficient neural networks. We investigate automatically designing specialized fast models, auto channel pruning, and auto mixed-precision quantization. We demonstrate such learning-based, automated design achieves superior performance and efficiency than rule-based human design. Moreover, we shorten the design cycle by 200x than previous work, so that we can afford to design specialized neural network models for different hardware platforms.
Better Automatic Evaluation of Open-Domain Dialogue Systems with Contextualized Embeddings
Venue: arXiv cs.CL 2019
Year: 2019

Despite advances in open-domain dialogue systems, automatic evaluation of such systems is still a challenging problem. Traditional reference-based metrics such as BLEU are ineffective because there could be many valid responses for a given context that share no common words with reference responses. A recent work proposed Referenced metric and Unreferenced metric Blended Evaluation Routine (RUBER) to combine a learning-based metric, which predicts relatedness between a generated response and a given query, with reference-based metric; it showed high correlation with human judgments. In this paper, we explore using contextualized word embeddings to compute more accurate relatedness scores, thus better evaluation metrics. Experiments show that our evaluation metrics outperform RUBER, which is trained on static embeddings.
Facilitating Bayesian Continual Learning by Natural Gradients and Stein Gradients
Venue: arXiv cs.LG 2019
Year: 2019

Continual learning aims to enable machine learning models to learn a general solution space for past and future tasks in a sequential manner. Conventional models tend to forget the knowledge of previous tasks while learning a new task, a phenomenon known as catastrophic forgetting. When using Bayesian models in continual learning, knowledge from previous tasks can be retained in two ways: 1). posterior distributions over the parameters, containing the knowledge gained from inference in previous tasks, which then serve as the priors for the following task; 2). coresets, containing knowledge of data distributions of previous tasks. Here, we show that Bayesian continual learning can be facilitated in terms of these two means through the use of natural gradients and Stein gradients respectively.
Mapping Hearthstone Deck Spaces through MAP-Elites with Sliding Boundaries
Venue: arXiv cs.NE 2019
Year: 2019

Quality diversity (QD) algorithms such as MAP-Elites have emerged as a powerful alternative to traditional single-objective optimization methods. They were initially applied to evolutionary robotics problems such as locomotion and maze navigation, but have yet to see widespread application. We argue that these algorithms are perfectly suited to the rich domain of video games, which contains many relevant problems with a multitude of successful strategies and often also multiple dimensions along which solutions can vary. This paper introduces a novel modification of the MAP-Elites algorithm called MAP-Elites with Sliding Boundaries (MESB) and applies it to the design and rebalancing of Hearthstone, a popular collectible card game chosen for its number of multidimensional behavior features relevant to particular styles of play. To avoid overpopulating cells with conflated behaviors, MESB slides the boundaries of cells based on the distribution of evolved individuals. Experiments in this paper demonstrate the performance of MESB in Hearthstone. Results suggest MESB finds diverse ways of playing the game well along the selected behavioral dimensions. Further analysis of the evolved strategies reveals common patterns that recur across behavioral dimensions and explores how MESB can help rebalance the game.
Organizing Network Management Logic with Circular Economy Principles
Venue: arXiv cs.NI 2019
Year: 2019

The traditional cycle of industrial products has been linear since its inception. Raw resources are acquired, processed, distributed, used and ultimately disposed of. This linearity has led to a dangerously low efficiency degree in resource use, and has brought forth serious concerns for the viability of our natural ecosystem. Circular economy is introducing a circular workflow for the lifetime of products. It generalizes the disposal phase, reconnecting it to manufacturing, distribution and end-use, thus limiting true deposition to the environment. This process has not been extended so far to software. Nonetheless, the development of software follows the same phases, and also entails the use-and waste-of considerable resources. This include human effort, as well as human and infrastructure sustenance products such as food, traveling and energy. This paper introduces circular economy principles to the software development, and particularly to network management logic and security. It employs a recently proposed concept-the Socket Store-which is an online store distributing end-user network logic in modular form. The Store modules act as mediators between the end-user network logic and the network resources. It is shown that the Socket Store can implement all circular economy principles to the software life-cycle, with considerable gains in resource waste.
On the Challenges and KPIs for Benchmarking Open-Source NFV MANO Systems: OSM vs ONAP
Venue: arXiv cs.NI 2019
Year: 2019

NFV management and orchestration (MANO) systems are being developed to meet the agile and flexible management requirements of virtualized network services in the 5G era and beyond. In this regard, ETSI ISG NFV has specified a standard NFV MANO system that is being used as a reference by MANO system vendors as well as open-source MANO projects. However, in the absence of MANO specific KPIs, it is difficult for users to make an informed decision on the choice of the MANO system better suited to meet their needs. Given the absence of any formal MANO specific KPIs on the basis of which a performance of a MANO system can be quantified, benchmarked and compared, users are left with simply comparing the claimed feature set. It is thus the motivation of this paper to highlight the challenges of testing and validating MANO systems in general, and propose MANO specific KPIs. Based on the proposed KPIs, we analyze and compare the performance of the two most popular open-source MANO projects, namely ONAP and OSM, using a complex open-source vCPE VNF and identify the features/performance gaps. In addition, we also provide a sketch of a test-jig that has been designed for benchmarking MANO systems.
A CNN-RNN Architecture for Multi-Label Weather Recognition
Venue: arXiv cs.CV 2019
Year: 2019

Weather Recognition plays an important role in our daily lives and many computer vision applications. However, recognizing the weather conditions from a single image remains challenging and has not been studied thoroughly. Generally, most previous works treat weather recognition as a single-label classification task, namely, determining whether an image belongs to a specific weather class or not. This treatment is not always appropriate, since more than one weather conditions may appear simultaneously in a single image. To address this problem, we make the first attempt to view weather recognition as a multi-label classification task, i.e., assigning an image more than one labels according to the displayed weather conditions. Specifically, a CNN-RNN based multi-label classification approach is proposed in this paper. The convolutional neural network (CNN) is extended with a channel-wise attention model to extract the most correlated visual features. The Recurrent Neural Network (RNN) further processes the features and excavates the dependencies among weather classes. Finally, the weather labels are predicted step by step. Besides, we construct two datasets for the weather recognition task and explore the relationships among different weather conditions. Experimental results demonstrate the superiority and effectiveness of the proposed approach. The new constructed datasets will be available at https://github.com/wzgwzg/Multi-Label-Weather-Recognition.
Bayesian Gaussian mixture model for robotic policy imitation
Venue: arXiv cs.RO 2019
Year: 2019

A common approach to learn robotic skills is to imitate a demonstrated policy. Due to the compounding of small errors and perturbations, this approach may let the robot leave the states in which the demonstrations were provided. This requires the consideration of additional strategies to guarantee that the robot will behave appropriately when facing unknown states. We propose to use a Bayesian method to quantify the action uncertainty at each state. The proposed Bayesian method is simple to set up, computationally efficient, and can adapt to a wide range of problems. Our approach exploits the estimated uncertainty to fuse the imitation policy with additional policies. It is validated on a Panda robot with the imitation of three manipulation tasks in the continuous domain using different control input/state pairs.
Reoptimization of Path Vertex Cover Problem
Venue: arXiv cs.DS 2019
Year: 2019

Most optimization problems are notoriously hard. Considerable efforts must be spent in obtaining an optimal solution to certain instances that we encounter in the real world scenarios. Often it turns out that input instances get modified locally in some small ways due to changes in the application world. The natural question here is, given an optimal solution for an old instance $I_O$, can we construct an optimal solution for the new instance $I_N$, where $I_N$ is the instance $I_O$ with some local modifications. Reoptimization of NP-hard optimization problem precisely addresses this concern. It turns out that for some reoptimization versions of the NP-hard problems, we may only hope to obtain an approximate solution to a new instance. In this paper, we specifically address the reoptimization of path vertex cover problem. The objective in $k$-$path$ vertex cover problem is to compute a minimum subset $S$ of the vertices in a graph $G$ such that after removal of $S$ from $G$ there is no path with $k$ vertices in the graph. We show that when a constant number of vertices are inserted, reoptimizing unweighted $k$-$path$ vertex cover problem admits a PTAS. For weighted $3$-$path$ vertex cover problem, we show that when a constant number of vertices are inserted, the reoptimization algorithm achieves an approximation factor of $1.5$, hence an improvement from known $2$-approximation algorithm for the optimization version. We provide reoptimization algorithm for weighted $k$-$path$ vertex cover problem $(k \geq 4)$ on bounded degree graphs, which is also an NP-hard problem. Given a $ρ$-approximation algorithm for $k$-$path$ vertex cover problem on bounded degree graphs, we show that it can be reoptimized within an approximation factor of $(2-\frac{1}ρ)$ under constant number of vertex insertions.
Data Cleaning for Accurate, Fair, and Robust Models: A Big Data - AI Integration Approach
Venue: arXiv cs.DB 2019
Year: 2019

The wide use of machine learning is fundamentally changing the software development paradigm (a.k.a. Software 2.0) where data becomes a first-class citizen, on par with code. As machine learning is used in sensitive applications, it becomes imperative that the trained model is accurate, fair, and robust to attacks. While many techniques have been proposed to improve the model training process (in-processing approach) or the trained model itself (post-processing), we argue that the most effective method is to clean the root cause of error: the data the model is trained on (pre-processing). Historically, there are at least three research communities that have been separately studying this problem: data management, machine learning (model fairness), and security. Although a significant amount of research has been done by each community, ultimately the same datasets must be preprocessed, and there is little understanding how the techniques relate to each other and can possibly be integrated. We contend that it is time to extend the notion of data cleaning for modern machine learning needs. We identify dependencies among the data preprocessing techniques and propose MLClean, a unified data cleaning framework that integrates the techniques and helps train accurate and fair models. This work is part of a broader trend of Big data -- Artificial Intelligence (AI) integration.
CED: Color Event Camera Dataset
Venue: arXiv cs.CV 2019
Year: 2019

Event cameras are novel, bio-inspired visual sensors, whose pixels output asynchronous and independent timestamped spikes at local intensity changes, called 'events'. Event cameras offer advantages over conventional frame-based cameras in terms of latency, high dynamic range (HDR) and temporal resolution. Until recently, event cameras have been limited to outputting events in the intensity channel, however, recent advances have resulted in the development of color event cameras, such as the Color-DAVIS346. In this work, we present and release the first Color Event Camera Dataset (CED), containing 50 minutes of footage with both color frames and events. CED features a wide variety of indoor and outdoor scenes, which we hope will help drive forward event-based vision research. We also present an extension of the event camera simulator ESIM that enables simulation of color events. Finally, we present an evaluation of three state-of-the-art image reconstruction methods that can be used to convert the Color-DAVIS346 into a continuous-time, HDR, color video camera to visualise the event stream, and for use in downstream vision applications.
Latent Variable Session-Based Recommendation
Venue: arXiv cs.IR 2019
Year: 2019

Session based recommendation provides an attractive alternative to the traditional feature engineering approach to recommendation. Feature engineering approaches require hand tuned features of the users history to be created to produce a context vector. In contrast a session based approach is able to dynamically model the users state as they act. We present a probabilistic framework for session based recommendation. A latent variable for the user state is updated as the user views more items and we learn more about their interests. The latent variable model is conceptually simple and elegant; yet requires sophisticated computational technique to approximate the integral over the latent variable. We provide computational solutions using both the re-parameterization trick and also using the Bouchard bound for the softmax function, we further explore employing a variational auto-encoder and a variational Expectation-Maximization algorithm for tightening the variational bound. The model performs well against a number of baselines. The intuitive nature of the model allows an elegant formulation combining correlations between items and their popularity and that sheds light on other popular recommendation methods. An attractive feature of the latent variable approach is that, as the user continues to act, the posterior on the user's state tightens reflecting the recommender system's increased knowledge about that user.
Deep Packet Inspection in FPGAs via Approximate Nondeterministic Automata
Venue: arXiv cs.FL 2019
Year: 2019

Deep packet inspection via regular expression (RE) matching is a crucial task of network intrusion detection systems (IDSes), which secure Internet connection against attacks and suspicious network traffic. Monitoring high-speed computer networks (100 Gbps and faster) in a single-box solution demands that the RE matching, traditionally based on finite automata (FAs), is accelerated in hardware. In this paper, we describe a novel FPGA architecture for RE matching that is able to process network traffic beyond 100 Gbps. The key idea is to reduce the required FPGA resources by leveraging approximate nondeterministic FAs (NFAs). The NFAs are compiled into a multi-stage architecture starting with the least precise stage with a high throughput and ending with the most precise stage with a low throughput. To obtain the reduced NFAs, we propose new approximate reduction techniques that take into account the profile of the network traffic. Our experiments showed that using our approach, we were able to perform matching of large sets of REs from SNORT, a popular IDS, on unprecedented network speeds.
Simultaneous regression and feature learning for facial landmarking
Venue: arXiv cs.CV 2019
Year: 2019

Face alignment (or facial landmarking) is an important task in many face-related applications, ranging from registration, tracking and animation to higher-level classification problems such as face, expression or attribute recognition. While several solutions have been presented in the literature for this task so far, reliably locating salient facial features across a wide range of posses still remains challenging. To address this issue, we propose in this paper a novel method for automatic facial landmark localization in 3D face data designed specifically to address appearance variability caused by significant pose variations. Our method builds on recent cascaded-regression-based methods to facial landmarking and uses a gating mechanism to incorporate multiple linear cascaded regression models each trained for a limited range of poses into a single powerful landmarking model capable of processing arbitrary posed input data. We develop two distinct approaches around the proposed gating mechanism: i) the first uses a gated multiple ridge descent (GRID) mechanism in conjunction with established (hand-crafted) HOG features for face alignment and achieves state-of-the-art landmarking performance across a wide range of facial poses, ii) the second simultaneously learns multiple-descent directions as well as binary features (SMUF) that are optimal for the alignment tasks and in addition to competitive landmarking results also ensures extremely rapid processing. We evaluate both approaches in rigorous experiments on several popular datasets of 3D face images, i.e., the FRGCv2 and Bosphorus 3D Face datasets and image collections F and G from the University of Notre Dame. The results of our evaluation show that both approaches are competitive in comparison to the state-of-the-art, while exhibiting considerable robustness to pose variations.
The iterative convolution-thresholding method (ICTM) for image segmentation
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we propose a novel iterative convolution-thresholding method (ICTM) that is applicable to a range of variational models for image segmentation. A variational model usually minimizes an energy functional consisting of a fidelity term and a regularization term. In the ICTM, the interface between two different segment domains is implicitly represented by their characteristic functions. The fidelity term is then usually written as a linear functional of the characteristic functions and the regularized term is approximated by a functional of characteristic functions in terms of heat kernel convolution. This allows us to design an iterative convolution-thresholding method to minimize the approximate energy. The method is simple, efficient and enjoys the energy-decaying property. Numerical experiments show that the method is easy to implement, robust and applicable to various image segmentation models.
Plug-in, Trainable Gate for Streamlining Arbitrary Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Architecture optimization, which is a technique for finding an efficient neural network that meets certain requirements, generally reduces to a set of multiple-choice selection problems among alternative sub-structures or parameters. The discrete nature of the selection problem, however, makes this optimization difficult. To tackle this problem we introduce a novel concept of a trainable gate function. The trainable gate function, which confers a differentiable property to discretevalued variables, allows us to directly optimize loss functions that include non-differentiable discrete values such as 0-1 selection. The proposed trainable gate can be applied to pruning. Pruning can be carried out simply by appending the proposed trainable gate functions to each intermediate output tensor followed by fine-tuning the overall model, using any gradient-based training methods. So the proposed method can jointly optimize the selection of the pruned channels while fine-tuning the weights of the pruned model at the same time. Our experimental results demonstrate that the proposed method efficiently optimizes arbitrary neural networks in various tasks such as image classification, style transfer, optical flow estimation, and neural machine translation.
Evolving Neural Networks in Reinforcement Learning by means of UMDAc
Venue: arXiv cs.NE 2019
Year: 2019

Neural networks are gaining popularity in the reinforcement learning field due to the vast number of successfully solved complex benchmark problems. In fact, artificial intelligence algorithms are, in some cases, able to overcome human professionals. Usually, neural networks have more than a couple of hidden layers, and thus, they involve a large quantity of parameters that need to be optimized. Commonly, numeric approaches are used to optimize the inner parameters of neural networks, such as the stochastic gradient descent. However, these techniques tend to be computationally very expensive, and for some tasks, where effectiveness is crucial, high computational costs are not acceptable. Along these research lines, in this paper we propose to optimize the parameters of neural networks by means of estimation of distribution algorithms. More precisely, the univariate marginal distribution algorithm is used for evolving neural networks in various reinforcement learning tasks. For the sake of validating our idea, we run the proposed algorithm on four OpenAI Gym benchmark problems. In addition, the obtained results were compared with a standard genetic algorithm. Revealing, that optimizing with UMDAc provides better results than the genetic algorithm in most of the cases.