OpenRaiser/Intern-Atlas huggingface dataset:
Regularizing linear inverse problems with convolutional neural networks
Venue: arXiv cs.LG 2019
Year: 2019

Deep convolutional neural networks trained on large datsets have emerged as an intriguing alternative for compressing images and solving inverse problems such as denoising and compressive sensing. However, it has only recently been realized that even without training, convolutional networks can function as concise image models, and thus regularize inverse problems. In this paper, we provide further evidence for this finding by studying variations of convolutional neural networks that map few weight parameters to an image. The networks we consider only consist of convolutional operations, with either fixed or parameterized filters followed by ReLU non-linearities. We demonstrate that with both fixed and parameterized convolutional filters those networks enable representing images with few coefficients. What is more, the underparameterization enables regularization of inverse problems, in particular recovering an image from few observations. We show that, similar to standard compressive sensing guarantees, on the order of the number of model parameters many measurements suffice for recovering an image from compressive measurements. Finally, we demonstrate that signal recovery with a un-trained convolutional network outperforms standard l1 and total variation minimization for magnetic resonance imaging (MRI).
Intrinsic Motivation Driven Intuitive Physics Learning using Deep Reinforcement Learning with Intrinsic Reward Normalization
Venue: arXiv cs.LG 2019
Year: 2019

At an early age, human infants are able to learn and build a model of the world very quickly by constantly observing and interacting with objects around them. One of the most fundamental intuitions human infants acquire is intuitive physics. Human infants learn and develop these models, which later serve as prior knowledge for further learning. Inspired by such behaviors exhibited by human infants, we introduce a graphical physics network integrated with deep reinforcement learning. Specifically, we introduce an intrinsic reward normalization method that allows our agent to efficiently choose actions that can improve its intuitive physics model the most. Using a 3D physics engine, we show that our graphical physics network is able to infer object's positions and velocities very effectively, and our deep reinforcement learning network encourages an agent to improve its model by making it continuously interact with objects only using intrinsic motivation. We experiment our model in both stationary and non-stationary state problems and show benefits of our approach in terms of the number of different actions the agent performs and the accuracy of agent's intuition model. Videos are at https://www.youtube.com/watch?v=pDbByp91r3M&t=2s
TrustSAS: A Trustworthy Spectrum Access System for the 3.5 GHz CBRS Band
Venue: arXiv cs.NI 2019
Year: 2019

As part of its ongoing efforts to meet the increased spectrum demand, the Federal Communications Commission (FCC) has recently opened up 150 MHz in the 3.5 GHz band for shared wireless broadband use. Access and operations in this band, aka Citizens Broadband Radio Service (CBRS), will be managed by a dynamic spectrum access system (SAS) to enable seamless spectrum sharing between secondary users (SUs) and incumbent users. Despite its benefits, SAS's design requirements, as set by FCC, present privacy risks to SUs, merely because SUs are required to share sensitive operational information (e.g., location, identity, spectrum usage) with SAS to be able to learn about spectrum availability in their vicinity. In this paper, we propose TrustSAS , a trustworthy framework for SAS that synergizes state-of-the-art cryptographic techniques with blockchain technology in an innovative way to address these privacy issues while complying with FCC's regulatory design requirements. We analyze the security of our framework and evaluate its performance through analysis, simulation and experimentation. We show that TrustSAS can offer high security guarantees with reasonable overhead, making it an ideal solution for addressing SUs' privacy issues in an operational SAS environment.
Weakly-supervised Knowledge Graph Alignment with Adversarial Learning
Venue: arXiv cs.LG 2019
Year: 2019

This paper studies aligning knowledge graphs from different sources or languages. Most existing methods train supervised methods for the alignment, which usually require a large number of aligned knowledge triplets. However, such a large number of aligned knowledge triplets may not be available or are expensive to obtain in many domains. Therefore, in this paper we propose to study aligning knowledge graphs in fully-unsupervised or weakly-supervised fashion, i.e., without or with only a few aligned triplets. We propose an unsupervised framework to align the entity and relation embddings of different knowledge graphs with an adversarial learning framework. Moreover, a regularization term which maximizes the mutual information between the embeddings of different knowledge graphs is used to mitigate the problem of mode collapse when learning the alignment functions. Such a framework can be further seamlessly integrated with existing supervised methods by utilizing a limited number of aligned triples as guidance. Experimental results on multiple datasets prove the effectiveness of our proposed approach in both the unsupervised and the weakly-supervised settings.
Applying a Pre-trained Language Model to Spanish Twitter Humor Prediction
Venue: arXiv cs.CL 2019
Year: 2019

Our entry into the HAHA 2019 Challenge placed $3^{rd}$ in the classification task and $2^{nd}$ in the regression task. We describe our system and innovations, as well as comparing our results to a Naive Bayes baseline. A large Twitter based corpus allowed us to train a language model from scratch focused on Spanish and transfer that knowledge to our competition model. To overcome the inherent errors in some labels we reduce our class confidence with label smoothing in the loss function. All the code for our project is included in a GitHub repository for easy reference and to enable replication by others.
Complexity of planar signed graph homomorphisms to cycles
Venue: arXiv cs.DS 2019
Year: 2019

We study homomorphism problems of signed graphs. A signed graph is an undirected graph where each edge is given a sign, positive or negative. An important concept for signed graphs is the operation of switching at a vertex, which is to change the sign of each incident edge. A homomorphism of a graph is a vertex-mapping that preserves the adjacencies; in the case of signed graphs, we also preserve the edge-signs. Special homomorphisms of signed graphs, called s-homomorphisms, have been studied. In an s-homomorphism, we allow, before the mapping, to perform any number of switchings on the source signed graph. This concept has been extensively studied, and a full complexity classification (polynomial or NP-complete) for s-homomorphism to a fixed target signed graph has recently been obtained. Such a dichotomy is not known when we restrict the input graph to be planar (not even for non-signed graph homomorphisms). We show that deciding whether a (non-signed) planar graph admits a homomorphism to the square $C_t^2$ of a cycle with $t\ge 6$, or to the circular clique $K_{4t/(2t-1)}$ with $t\ge2$, are NP-complete problems. We use these results to show that deciding whether a planar signed graph admits an s-homomorphism to an unbalanced even cycle is NP-complete. (A cycle is unbalanced if it has an odd number of negative edges). We deduce a complete complexity dichotomy for the planar s-homomorphism problem with any signed cycle as a target. We also study further restrictions involving the maximum degree and the girth of the input signed graph. We prove that planar s-homomorphism problems to signed cycles remain NP-complete even for inputs of maximum degree~$3$ (except for the case of unbalanced $4$-cycles, for which we show this for maximum degree~$4$). We also show that for a given integer $g$, the problem for signed bipartite planar inputs of girth $g$ is either trivial or NP-complete.
Deep Learning based Wireless Resource Allocation with Application to Vehicular Networks
Venue: arXiv cs.IT 2019
Year: 2019

It has been a long-held belief that judicious resource allocation is critical to mitigating interference, improving network efficiency, and ultimately optimizing wireless communication performance. The traditional wisdom is to explicitly formulate resource allocation as an optimization problem and then exploit mathematical programming to solve the problem to a certain level of optimality. Nonetheless, as wireless networks become increasingly diverse and complex, e.g., in the high-mobility vehicular networks, the current design methodologies face significant challenges and thus call for rethinking of the traditional design philosophy. Meanwhile, deep learning, with many success stories in various disciplines, represents a promising alternative due to its remarkable power to leverage data for problem solving. In this paper, we discuss the key motivations and roadblocks of using deep learning for wireless resource allocation with application to vehicular networks. We review major recent studies that mobilize the deep learning philosophy in wireless resource allocation and achieve impressive results. We first discuss deep learning assisted optimization for resource allocation. We then highlight the deep reinforcement learning approach to address resource allocation problems that are difficult to handle in the traditional optimization framework. We also identify some research directions that deserve further investigation.
Spacetime Graph Optimization for Video Object Segmentation
Venue: arXiv cs.CV 2019
Year: 2019

We address the challenging task of foreground object discovery and segmentation in video. We introduce an efficient solution, suitable for both unsupervised and supervised scenarios, based on a spacetime graph representation of the video sequence. We ensure a fine grained representation with one-to-one correspondences between graph nodes and video pixels. We formulate the task as a spectral clustering problem by exploiting the spatio-temporal consistency between the scene elements in terms of motion and appearance. Graph nodes that belong to the main object of interest should form a strong cluster, as they are linked through long range optical flow chains and have similar motion and appearance features along those chains. On one hand, the optimization problem aims to maximize the segmentation clustering score based on the motion structure through space and time. On the other hand, the segmentation should be consistent with respect to node features. Our approach leads to a graph formulation in which the segmentation solution becomes the principal eigenvector of a novel Feature-Motion matrix. While the actual matrix is not computed explicitly, the proposed algorithm efficiently computes, in a few iteration steps, the principal eigenvector that captures the segmentation of the main object in the video. The proposed algorithm, GO-VOS, produces a global optimum solution and, consequently, it does not depend on initialization. In practice, GO-VOS achieves state of the art results on three challenging datasets used in current literature: DAVIS, SegTrack and YouTube-Objects.
Facial Makeup Transfer Combining Illumination Transfer
Venue: arXiv cs.CV 2019
Year: 2019

To meet the women appearance needs, we present a novel virtual experience approach of facial makeup transfer, developed into windows platform application software. The makeup effects could present on the user's input image in real time, with an only single reference image. The input image and reference image are divided into three layers by facial feature points landmarked: facial structure layer, facial color layer, and facial detail layer. Except for the above layers are processed by different algorithms to generate output image, we also add illumination transfer, so that the illumination effect of the reference image is automatically transferred to the input image. Our approach has the following three advantages: (1) Black or dark and white facial makeup could be effectively transferred by introducing illumination transfer; (2) Efficiently transfer facial makeup within seconds compared to those methods based on deep learning frameworks; (3) Reference images with the air-bangs could transfer makeup perfectly.
The Price of Interpretability
Venue: arXiv cs.LG 2019
Year: 2019

When quantitative models are used to support decision-making on complex and important topics, understanding a model's ``reasoning'' can increase trust in its predictions, expose hidden biases, or reduce vulnerability to adversarial attacks. However, the concept of interpretability remains loosely defined and application-specific. In this paper, we introduce a mathematical framework in which machine learning models are constructed in a sequence of interpretable steps. We show that for a variety of models, a natural choice of interpretable steps recovers standard interpretability proxies (e.g., sparsity in linear models). We then generalize these proxies to yield a parametrized family of consistent measures of model interpretability. This formal definition allows us to quantify the ``price'' of interpretability, i.e., the tradeoff with predictive accuracy. We demonstrate practical algorithms to apply our framework on real and synthetic datasets.
parboiled2: a macro-based approach for effective generators of parsing expressions grammars in Scala
Venue: arXiv cs.PL 2019
Year: 2019

In today's computerized world, parsing is ubiquitous. Developers parse logs, queries to databases and websites, programming and natural languages. When Java ecosystem maturity, concise syntax, and runtime speed matters, developers choose parboiled2 that generates grammars for parsing expression grammars (PEG). The following open source libraries have chosen parboiled2 for parsing facilities: - akka-http is the Streaming-first HTTP server/module of Lightbend Akka - Sangria is a Scala GraphQL implementation - http4s is a minimal, idiomatic Scala interface for HTTP - cornichon is Scala DSL for testing HTTP JSON API - scala-uri is a simple Scala library for building and parsing URIs The library uses a wide range of Scala facilities to provide required functionality. We also discuss the extensions to PEGs. In particular, we show the implementation of an internal Scala DSL that features intuitive syntax and semantics. We demonstrate how parboiled2 extensively uses Scala typing to verify DSL integrity. We also show the connections to inner structures of parboiled2, which can give the developer a better understanding of how to compose more effective grammars. Finally, we expose how a grammar is expanded with Scala Macros to an effective runtime code.
Capacity and Algorithms for a Cognitive Network with Primary-Secondary User Cooperation
Venue: arXiv cs.IT 2019
Year: 2019

In this work, we examine cognitive radio networks, where secondary users may act as relays for messages sent by the primary user, hence offering performance improvement of primary transmissions, while at the same time obtaining more transmission opportunities for their own data. In particular, assuming the broadcast packet erasure model with feedback, we investigate the capacity of the fundamental cooperative cognitive radio network which consists of one primary and one secondary transmitter-receiver pairs. The primary transmitter is the owner of the channel and as such, we intend to keep its operations simple and to avoid increasing its storage requirements. Specifically, the primary transmitter does not receive data sent by the secondary transmitter and does not perform any coding operations. On the other hand, the secondary transmitter can overhear primary transmissions and is allowed to perform any coding operations. We develop an outer bound to the capacity of the fundamental cooperative cognitive radio network under consideration. Then, we propose a coding-scheduling algorithm suitable for this type of networks, which involves only XOR network coding operations. The complexity of the scheduling decisions of the proposed algorithm depends on the channel statistical parameters and three cases, depending on the relations between channel erasure probabilities, are distinguished. For the first two cases the rate region of the proposed algorithm coincides with the developed capacity outer bound, hence the algorithm is capacity achieving. For the third case, the rate region of the proposed algorithm is not identical to the outer bound; however, numerical results show that it is fairly close to the derived outer bound for a wide range of the statistical parameters of the system.
Bounded Model Checking of Max-Plus Linear Systems via Predicate Abstractions
Venue: arXiv cs.LO 2019
Year: 2019

This paper introduces the abstraction of max-plus linear (MPL) systems via predicates. Predicates are automatically selected from system matrix, as well as from the specifications under consideration. We focus on verifying time-difference specifications, which encompass the relation between successive events in MPL systems. We implement a bounded model checking (BMC) procedure over a predicate abstraction of the given MPL system, to verify the satisfaction of time-difference specifications. Our predicate abstractions are experimentally shown to improve on existing MPL abstractions algorithms. Furthermore, with focus on the BMC algorithm, we can provide an explicit upper bound on the completeness threshold by means of the transient and the cyclicity of the underlying MPL system.
Catfish Effect Between Internal and External Attackers:Being Semi-honest is Helpful
Venue: arXiv cs.CR 2019
Year: 2019

The consensus protocol named proof of work (PoW) is widely applied by cryptocurrencies like Bitcoin. Although security of a PoW cryptocurrency is always the top priority, it is threatened by mining attacks like selfish mining. Researchers have proposed many mining attack models with one attacker, and optimized the attacker's strategy. During these mining attacks, an attacker pursues a higher relative revenue (RR) by wasting a large amount of computational power of the honest miners at the cost of a small amount of computational power of himself. In this paper, we propose a mining attack model with two phases: the original system and the multi-attacker system. It is the first model to provide both theoretical and quantitative analysis of mining attacks with two attackers. We explain how the original system turns into the multi-attacker system by introducing two attackers: the internal attacker and the external attacker. If both attackers take the attacking strategy selfish mining, the RR of the internal attacker in multi-attacker system will drop by up to 31.9% compared with his RR in original system. The external attacker will overestimate his RR by up to 44.6% in multiattacker system. Unexpected competitions, auctions between attackers and overestimation of attackers' influence factor are three main causes of both attackers' dropping RR. We propose a mining strategy named Partial Initiative Release (PIR) which is a semi-honest mining strategy in multi-attacker system. In some specific situations, PIR allows the attacker to get more block reward by launching an attack in multi-attacker system.
Neural Networks on Groups
Venue: arXiv cs.NE 2019
Year: 2019

Although neural networks traditionally are typically used to approximate functions defined over $\mathbb{R}^n$, the successes of graph neural networks, point-cloud neural networks, and manifold deep learning among other methods have demonstrated the clear value of leveraging neural networks to approximate functions defined over more general spaces. The theory of neural networks has not kept up however,and the relevant theoretical results (when they exist at all) have been proven on a case-by-case basis without a general theory or connection to classical work. The process of deriving new theoretical backing for each new type of network has become a bottleneck to understanding and validating new approaches. In this paper we extend the definition of neural networks to general topological groups and prove that neural networks with a single hidden layer and a bounded non-constant activation function can approximate any $\mathcal{L}^p$ function defined over any locally compact Abelian group. This framework and universal approximation theorem encompass all of the aforementioned contexts. We also derive important corollaries and extensions with minor modification, including the case for approximating continuous functions on a compact subset, neural networks with ReLU activation functions on a linearly bi-ordered group, and neural networks with affine transformations on a vector space. Our work obtains as special cases the recent theorems of Qi et al. [2017], Sennai et al. [2019], Keriven and Peyre [2019], and Maron et al. [2019]
A K-means-based Multi-subpopulation Particle Swarm Optimization for Neural Network Ensemble
Venue: arXiv cs.NE 2019
Year: 2019

This paper presents a k-means-based multi-subpopulation particle swarm optimization, denoted as KMPSO, for training the neural network ensemble. In the proposed KMPSO, particles are dynamically partitioned into clusters via the k-means clustering algorithm at every iteration, and each of the resulting clusters is responsible for training a component neural network. The performance of the KMPSO has been evaluated on several benchmark problems. Our results show that the proposed method can effectively control the trade-off between the diversity and accuracy in the ensemble, thus achieving competitive results in comparison with related algorithms.
The Advent of Technological Singularity: a Formal Metric
Venue: arXiv cs.CY 2019
Year: 2019

The Technological Singularity; that is, the possibility of achieving a General Artificial Intelligence (AGI) that surpasses human intelligence, is one of the vital paradigms of today's humanity. However, until now only opinions about its possibility and/or achievement were issued, therefore, in this work, a metric is presented, for the first time, to objectively measure the actual state in which the advent of technological singularity is found.
Artificial Intelligence Governance and Ethics: Global Perspectives
Venue: arXiv cs.CY 2019
Year: 2019

Artificial intelligence (AI) is a technology which is increasingly being utilised in society and the economy worldwide, and its implementation is planned to become more prevalent in coming years. AI is increasingly being embedded in our lives, supplementing our pervasive use of digital technologies. But this is being accompanied by disquiet over problematic and dangerous implementations of AI, or indeed, even AI itself deciding to do dangerous and problematic actions, especially in fields such as the military, medicine and criminal justice. These developments have led to concerns about whether and how AI systems adhere, and will adhere to ethical standards. These concerns have stimulated a global conversation on AI ethics, and have resulted in various actors from different countries and sectors issuing ethics and governance initiatives and guidelines for AI. Such developments form the basis for our research in this report, combining our international and interdisciplinary expertise to give an insight into what is happening in Australia, China, Europe, India and the US.
Proof compression and NP versus PSPACE. Part 2
Venue: arXiv cs.LO 2019
Year: 2019

We upgrade [1] to a complete proof of the conjecture NP = PSPACE. [1]: L. Gordeev, E. H. Haeusler, Proof Compression and NP Versus PSPACE, Studia Logica (107) (1): 55-83 (2019)