OpenRaiser/Intern-Atlas huggingface dataset:
Convolutional Analysis Operator Learning: Dependence on Training Data
Venue: arXiv cs.LG 2019
Year: 2019

Convolutional analysis operator learning (CAOL) enables the unsupervised training of (hierarchical) convolutional sparsifying operators or autoencoders from large datasets. One can use many training images for CAOL, but a precise understanding of the impact of doing so has remained an open question. This paper presents a series of results that lend insight into the impact of dataset size on the filter update in CAOL. The first result is a general deterministic bound on errors in the estimated filters, and is followed by a bound on the expected errors as the number of training samples increases. The second result provides a high probability analogue. The bounds depend on properties of the training data, and we investigate their empirical values with real data. Taken together, these results provide evidence for the potential benefit of using more training data in CAOL.
L2 Observers for a Class of Nonlinear Systems with Unknown Inputs
Venue: arXiv cs.SY 2019
Year: 2019

We consider the problem of estimating the state and unknown input for a large class of nonlinear systems subject to unknown exogenous inputs. The exogenous inputs themselves are modeled as being generated by a nonlinear system subject to unknown inputs. The nonlinearities considered in this work are characterized by multiplier matrices that include many commonly encountered nonlinearities. We obtain a linear matrix inequality (LMI), that, if feasible, provides the gains for an observer which results in certified L2 performance of the error dynamics associated with the observer. We also present conditions which guarantee that the L2 norm of the error can be made arbitrarily small and investigate conditions for feasibility of the proposed LMIs.
Bottom-up strategy for data retrieval and data entry over front-end application Software
Venue: arXiv cs.SE 2019
Year: 2019

Some people implement pattern and best practices without analyzing its efficiency on their projects. Consequently, our goal in this article is to convince software developers that it is worth to make an earnest effort to evaluate the use of best practices and software patterns. For such purpose, in this study we took a concrete case system for geographical locations inputs through user interfaces. Then, we performed a comparative study on a traditional method against our approach, named reverse logistic to retrieve results, by measuring the time that a user spends to perform actions when entering data into a system. Surprisingly, we had a decrease of 59% in the amount of time spent in comparison to the time spent on the traditional method. This result lays a foundation for feeding data from the typical final step and search based on string matching algorithms, speeding up the interaction between people and computer response
Capsule Neural Networks for Graph Classification using Explicit Tensorial Graph Representations
Venue: arXiv cs.LG 2019
Year: 2019

Graph classification is a significant problem in many scientific domains. It addresses tasks such as the classification of proteins and chemical compounds into categories according to their functions, or chemical and structural properties. In a supervised setting, this problem can be framed as learning the structure, features and relationships between features within a set of labelled graphs and being able to correctly predict the labels or categories of unseen graphs. A significant difficulty in this task arises when attempting to apply established classification algorithms due to the requirement for fixed size matrix or tensor representations of the graphs which may vary greatly in their numbers of nodes and edges. Building on prior work combining explicit tensor representations with a standard image-based classifier, we propose a model to perform graph classification by extracting fixed size tensorial information from each graph in a given set, and using a Capsule Network to perform classification. The graphs we consider here are undirected and with categorical features on the nodes. Using standard benchmarking chemical and protein datasets, we demonstrate that our graph Capsule Network classification model using an explicit tensorial representation of the graphs is competitive with current state of the art graph kernels and graph neural network models despite only limited hyper-parameter searching.
On optimal multiplexing of an ensemble of discrete-time constrained control systems on matrix Lie groups
Venue: arXiv cs.SY 2019
Year: 2019

We study a constrained optimal control problem for an ensemble of control systems. Each sub-system (or plant) evolves on a matrix Lie group, and must satisfy given state and control action constraints pointwise in time. In addition, certain multiplexing requirement is imposed: the controller must be shared between the plants in the sense that at any time instant the control signal may be sent to only one plant. We provide first-order necessary conditions for optimality in the form of suitable Pontryagin maximum principle in this problem. Detailed numerical experiments are presented for a system of two satellites performing energy optimal maneuvers under the preceding family of constraints.
Optimizing Space of Parallel Processes
Venue: arXiv cs.PL 2019
Year: 2019

This paper is a contribution to exploring and analyzing space-improvements in concurrent programming languages, in particular in the functional process-calculus CHF. Space-improvements are defined as a generalization of the corresponding notion in deterministic pure functional languages. The main part of the paper is the O(n*log n) algorithm SpOptN for offline space optimization of several parallel independent processes. Applications of this algorithm are: (i) affirmation of space improving transformations for particular classes of program transformations; (ii) support of an interpreter-based method for refuting space-improvements; and (iii) as a stand-alone offline-optimizer for space (or similar resources) of parallel processes.
Optimal Channel Estimation for Hybrid Energy Beamforming under Phase Shifter Impairments
Venue: arXiv cs.IT 2019
Year: 2019

Smart multiantenna wireless power transmission can enable perpetual operation of energy harvesting (EH) nodes in the internet-of-things. Moreover, to overcome the increased hardware cost and space constraints associated with having large antenna arrays at the radio frequency (RF) energy source, the hybrid energy beamforming (EBF) architecture with single RF chain can be adopted. Using the recently proposed hybrid EBF architecture modeling the practical analog phase shifter impairments (API), we derive the optimal least-squares estimator for the energy source to EH user channel. Next, the average harvested power at the user is derived while considering the nonlinear RF EH model and a tight analytical approximation for it is also presented by exploring the practical limits on the API. Using these developments, the jointly global optimal transmit power and time allocation for channel estimation (CE) and EBF phases, that maximizes the average energy stored at the EH user is derived in closed form. Numerical results validate the proposed analysis and present nontrivial design insights on the impact of API and CE errors on the achievable EBF performance. It is shown that the optimized hybrid EBF protocol with joint resource allocation yields an average performance improvement of $37\%$ over benchmark fixed allocation scheme.
How Do Non-Ideal UAV Antennas Affect Air-to-Ground Communications?
Venue: arXiv cs.IT 2019
Year: 2019

Analysis of the performance of Unmanned Aerial Vehicle (UAV)-enabled communications systems often relies upon idealized antenna characteristic, where the side-lobe gain of UAVs' antenna is ignored. In practice, however, side-lobe cause inevitable interference to the ground users. We investigate the impact of UAVs' antenna side-lobe on the performance of UAV-enabled communication. Our analysis shows that even for a very small antenna's side-lobe gain, the ground receiver can experience substantial interference. We further show that a rather large exclusion zone is required to ensure a sufficient level of protection for the ground receiver. Nevertheless, in a multiple-antenna setting for the ground users, even when such a large exclusion zone was in place, UAVs' antenna side-lobe creates a high level of correlation among the interference signals received across receive antennas. Such a correlation limits the system ability to exploit channel diversity in a multiple-antenna setting for improving capacity. We then quantify the impact of UAVs' antenna side-lobes on the overall system performance by deriving the corresponding loss of the achieved capacity in various communications environments. We provide a new quantitative insight on the cost of adopting non-ideal UAV antenna on the overall capacity. Our analysis also shows that the capacity loss can be confined by careful selection of system parameters.
Adversarial Neural Network Inversion via Auxiliary Knowledge Alignment
Venue: arXiv cs.CR 2019
Year: 2019

The rise of deep learning technique has raised new privacy concerns about the training data and test data. In this work, we investigate the model inversion problem in the adversarial settings, where the adversary aims at inferring information about the target model's training data and test data from the model's prediction values. We develop a solution to train a second neural network that acts as the inverse of the target model to perform the inversion. The inversion model can be trained with black-box accesses to the target model. We propose two main techniques towards training the inversion model in the adversarial settings. First, we leverage the adversary's background knowledge to compose an auxiliary set to train the inversion model, which does not require access to the original training data. Second, we design a truncation-based technique to align the inversion model to enable effective inversion of the target model from partial predictions that the adversary obtains on victim user's data. We systematically evaluate our inversion approach in various machine learning tasks and model architectures on multiple image datasets. Our experimental results show that even with no full knowledge about the target model's training data, and with only partial prediction values, our inversion approach is still able to perform accurate inversion of the target model, and outperform previous approaches.
Learning Hash Function through Codewords
Venue: arXiv cs.LG 2019
Year: 2019

In this paper, we propose a novel hash learning approach that has the following main distinguishing features, when compared to past frameworks. First, the codewords are utilized in the Hamming space as ancillary techniques to accomplish its hash learning task. These codewords, which are inferred from the data, attempt to capture grouping aspects of the data's hash codes. Furthermore, the proposed framework is capable of addressing supervised, unsupervised and, even, semi-supervised hash learning scenarios. Additionally, the framework adopts a regularization term over the codewords, which automatically chooses the codewords for the problem. To efficiently solve the problem, one Block Coordinate Descent algorithm is showcased in the paper. We also show that one step of the algorithms can be casted into several Support Vector Machine problems which enables our algorithms to utilize efficient software package. For the regularization term, a closed form solution of the proximal operator is provided in the paper. A series of comparative experiments focused on content-based image retrieval highlights its performance advantages.
Scalable Hyperbolic Recommender Systems
Venue: arXiv cs.IR 2019
Year: 2019

We present a large scale hyperbolic recommender system. We discuss why hyperbolic geometry is a more suitable underlying geometry for many recommendation systems and cover the fundamental milestones and insights that we have gained from its development. In doing so, we demonstrate the viability of hyperbolic geometry for recommender systems, showing that they significantly outperform Euclidean models on datasets with the properties of complex networks. Key to the success of our approach are the novel choice of underlying hyperbolic model and the use of the Einstein midpoint to define an asymmetric recommender system in hyperbolic space. These choices allow us to scale to millions of users and hundreds of thousands of items.
An SDN-Based Transmission Protocol with In-Path Packet Caching and Retransmission
Venue: arXiv cs.NI 2019
Year: 2019

In this paper, a comprehensive software-defined networking (SDN) based transmission protocol (SDTP) is presented for fifth generation (5G) communication networks, where an SDN controller gathers network state information from the physical network to improve data transmission efficiency between end hosts, with in-path packet retransmission. In the SDTP, we first develop a new two-way handshake mechanism for connection establishment between a pair of end host. With the aid of SDN control module, signaling exchanges for establishing E2E connections are migrated to the control plane to improve resource utilization in the data plane. A new SDTP packet header format is designed to support efficient data transmission with in-path packet caching and packet retransmission. Based on the new data packet format, a novel in-path receiver-based packet loss detection and caching-based packet retransmission scheme is proposed to achieve in-path fast recovery of lost packets. Extensive simulation results are presented to validate the effectiveness of the proposed protocol in terms of low connection establishment delay and low end-to-end packet transmission delay.
Designing for Participation and Change in Digital Institutions
Venue: arXiv cs.SI 2019
Year: 2019

Whether we recognize it or not, the Internet is rife with exciting and original institutional forms that are transforming social organization on and offline. Issues of governance in these Internet platforms and other digital institutions have posed a challenge for software engineers, many of whom have little exposure to the relevant history or theory of institutional design. Here, we offer one useful framework with an aim to stimulate dialogue between computer scientists and political scientists. The dominant guiding practices for the design of digital institutions to date in human-computer interaction, computer-supported cooperative work, and the tech industry at large have been an incentive-focused behavioral engineering paradigm, a collection of atheoretical approaches such as A/B-testing, and incremental issue-driven software engineering. One institutional analysis framework that has been useful in the design of traditional institutions is the body of resource governance literature known as the "Ostrom Workshop". A key finding of this literature that has yet to be broadly incorporated in the design of many digital institutions is the importance of including participatory change process mechanisms in what is called a "constitutional layer" of institutional design---in other words, defining rules that allow and facilitate diverse stakeholder participation in the ongoing process of institutional design change. We explore to what extent this consideration is met or could be better met in three varied cases of digital institutions: cryptocurrencies, cannabis informatics, and amateur Minecraft server governance. Examining such highly varied cases allows us to demonstrate the broad relevance of constitutional layers in many different types of digital institutions.
AliGraph: A Comprehensive Graph Neural Network Platform
Venue: arXiv cs.DC 2019
Year: 2019

An increasing number of machine learning tasks require dealing with large graph datasets, which capture rich and complex relationship among potentially billions of elements. Graph Neural Network (GNN) becomes an effective way to address the graph learning problem by converting the graph data into a low dimensional space while keeping both the structural and property information to the maximum extent and constructing a neural network for training and referencing. However, it is challenging to provide an efficient graph storage and computation capabilities to facilitate GNN training and enable development of new GNN algorithms. In this paper, we present a comprehensive graph neural network system, namely AliGraph, which consists of distributed graph storage, optimized sampling operators and runtime to efficiently support not only existing popular GNNs but also a series of in-house developed ones for different scenarios. The system is currently deployed at Alibaba to support a variety of business scenarios, including product recommendation and personalized search at Alibaba's E-Commerce platform. By conducting extensive experiments on a real-world dataset with 492.90 million vertices, 6.82 billion edges and rich attributes, AliGraph performs an order of magnitude faster in terms of graph building (5 minutes vs hours reported from the state-of-the-art PowerGraph platform). At training, AliGraph runs 40%-50% faster with the novel caching strategy and demonstrates around 12 times speed up with the improved runtime. In addition, our in-house developed GNN models all showcase their statistically significant superiorities in terms of both effectiveness and efficiency (e.g., 4.12%-17.19% lift by F1 scores).
Augmenting Neural Machine Translation with Knowledge Graphs
Venue: arXiv cs.CL 2019
Year: 2019

While neural networks have been used extensively to make substantial progress in the machine translation task, they are known for being heavily dependent on the availability of large amounts of training data. Recent efforts have tried to alleviate the data sparsity problem by augmenting the training data using different strategies, such as back-translation. Along with the data scarcity, the out-of-vocabulary words, mostly entities and terminological expressions, pose a difficult challenge to Neural Machine Translation systems. In this paper, we hypothesize that knowledge graphs enhance the semantic feature extraction of neural models, thus optimizing the translation of entities and terminological expressions in texts and consequently leading to a better translation quality. We hence investigate two different strategies for incorporating knowledge graphs into neural models without modifying the neural network architectures. We also examine the effectiveness of our augmentation method to recurrent and non-recurrent (self-attentional) neural architectures. Our knowledge graph augmented neural translation model, dubbed KG-NMT, achieves significant and consistent improvements of +3 BLEU, METEOR and chrF3 on average on the newstest datasets between 2014 and 2018 for WMT English-German translation task.
Transfer Learning for Non-Intrusive Load Monitoring
Venue: arXiv cs.LG 2019
Year: 2019

Non-intrusive load monitoring (NILM) is a technique to recover source appliances from only the recorded mains in a household. NILM is unidentifiable and thus a challenge problem because the inferred power value of an appliance given only the mains could not be unique. To mitigate the unidentifiable problem, various methods incorporating domain knowledge into NILM have been proposed and shown effective experimentally. Recently, among these methods, deep neural networks are shown performing best. Arguably, the recently proposed sequence-to-point (seq2point) learning is promising for NILM. However, the results were only carried out on the same data domain. It is not clear if the method could be generalised or transferred to different domains, e.g., the test data were drawn from a different country comparing to the training data. We address this issue in the paper, and two transfer learning schemes are proposed, i.e., appliance transfer learning (ATL) and cross-domain transfer learning (CTL). For ATL, our results show that the latent features learnt by a `complex' appliance, e.g., washing machine, can be transferred to a `simple' appliance, e.g., kettle. For CTL, our conclusion is that the seq2point learning is transferable. Precisely, when the training and test data are in a similar domain, seq2point learning can be directly applied to the test data without fine tuning; when the training and test data are in different domains, seq2point learning needs fine tuning before applying to the test data. Interestingly, we show that only the fully connected layers need fine tuning for transfer learning. Source code can be found at https://github.com/MingjunZhong/transferNILM.
Evidence Sentence Extraction for Machine Reading Comprehension
Venue: arXiv cs.CL 2019
Year: 2019

Remarkable success has been achieved in the last few years on some limited machine reading comprehension (MRC) tasks. However, it is still difficult to interpret the predictions of existing MRC models. In this paper, we focus on extracting evidence sentences that can explain or support the answers of multiple-choice MRC tasks, where the majority of answer options cannot be directly extracted from reference documents. Due to the lack of ground truth evidence sentence labels in most cases, we apply distant supervision to generate imperfect labels and then use them to train an evidence sentence extractor. To denoise the noisy labels, we apply a recently proposed deep probabilistic logic learning framework to incorporate both sentence-level and cross-sentence linguistic indicators for indirect supervision. We feed the extracted evidence sentences into existing MRC models and evaluate the end-to-end performance on three challenging multiple-choice MRC datasets: MultiRC, RACE, and DREAM, achieving comparable or better performance than the same models that take as input the full reference document. To the best of our knowledge, this is the first work extracting evidence sentences for multiple-choice MRC.
Sustaining Research Software: an SC18 Panel
Venue: arXiv cs.SE 2019
Year: 2019

Many science advances have been possible thanks to the use of research software, which has become essential to advancing virtually every Science, Technology, Engineering and Mathematics (STEM) discipline and many non-STEM disciplines including social sciences and humanities. And while much of it is made available under open source licenses, work is needed to develop, support, and sustain it, as underlying systems and software as well as user needs evolve. In addition, the changing landscape of high-performance computing (HPC) platforms, where performance and scaling advances are ever more reliant on software and algorithm improvements as we hit hardware scaling barriers, is causing renewed tension between sustainability of software and its performance. We must do more to highlight the trade-off between performance and sustainability, and to emphasize the need for sustainability given the fact that complex software stacks don't survive without frequent maintenance; made more difficult as a generation of developers of established and heavily-used research software retire. Several HPC forums are doing this, and it has become an active area of funding as well. In response, the authors organized and ran a panel at the SC18 conference. The objectives of the panel were to highlight the importance of sustainability, to illuminate the tension between pure performance and sustainability, and to steer SC community discussion toward understanding and addressing this issue and this tension. The outcome of the discussions, as presented in this paper, can inform choices of advance compute and data infrastructures to positively impact future research software and future research.