OpenRaiser/Intern-Atlas huggingface dataset:
Adversarial Deformation Regularization for Training Image Registration Neural Networks
Venue: arXiv cs.LG 2018
Year: 2018

We describe an adversarial learning approach to constrain convolutional neural network training for image registration, replacing heuristic smoothness measures of displacement fields often used in these tasks. Using minimally-invasive prostate cancer intervention as an example application, we demonstrate the feasibility of utilizing biomechanical simulations to regularize a weakly-supervised anatomical-label-driven registration network for aligning pre-procedural magnetic resonance (MR) and 3D intra-procedural transrectal ultrasound (TRUS) images. A discriminator network is optimized to distinguish the registration-predicted displacement fields from the motion data simulated by finite element analysis. During training, the registration network simultaneously aims to maximize similarity between anatomical labels that drives image alignment and to minimize an adversarial generator loss that measures divergence between the predicted- and simulated deformation. The end-to-end trained network enables efficient and fully-automated registration that only requires an MR and TRUS image pair as input, without anatomical labels or simulated data during inference. 108 pairs of labelled MR and TRUS images from 76 prostate cancer patients and 71,500 nonlinear finite-element simulations from 143 different patients were used for this study. We show that, with only gland segmentation as training labels, the proposed method can help predict physically plausible deformation without any other smoothness penalty. Based on cross-validation experiments using 834 pairs of independent validation landmarks, the proposed adversarial-regularized registration achieved a target registration error of 6.3 mm that is significantly lower than those from several other regularization methods.
Toward Super-Polynomial Size Lower Bounds for Depth-Two Threshold Circuits
Venue: arXiv cs.CC 2018
Year: 2018

Proving super-polynomial size lower bounds for $\textsf{TC}^0$, the class of constant-depth, polynomial-size circuits of Majority gates, is a notorious open problem in complexity theory. A major frontier is to prove that $\textsf{NEXP}$ does not have poly-size $\textsf{THR} \circ \textsf{THR}$ circuit (depth-two circuits with linear threshold gates). In recent years, R.~Williams proposed a program to prove circuit lower bounds via improved algorithms. In this paper, following Williams' framework, we show that the above frontier question can be resolved by devising slightly faster algorithms for several fundamental problems: 1. Shaving Logs for $\textsf{$\ell_2$-Furthest-Pair}$. An $n^2 \textrm{poly}(d) / \log^{ω(1)} n$ time algorithm for $\textsf{$\ell_2$-Furthest-Pair}$ in $\mathbb{R}^d$ for polylogarithmic $d$ implies $\textsf{NEXP}$ has no polynomial size $\textsf{THR} \circ \textsf{THR}$ circuits. The same holds for Hopcroft's problem, $\textsf{Bichrom.-$\ell_2$-Closest-Pair}$ and Integer $\textsf{Max-IP}$. 2. Shaving Logs for Approximate $\textsf{Bichrom.-$\ell_2$-Closest-Pair}$. An $n^2 \textrm(d) / \log^{ω(1)} n$ time algorithm for $(1+1/\log^{ω(1)} n)$-approximation to $\textsf{Bichrom.-$\ell_2$-Closest-Pair}$ or $\textsf{Bichrom.-$\ell_1$-Closest-Pair}$ for polylogarithmic $d$ implies $\textsf{NEXP}$ has no polynomial size $\textsf{SYM}\circ\textsf{THR}$ circuits. 3. Shaving Logs for Modest Dimension Boolean $\textsf{Max-IP}$. An $n^2 / \log^{ω(1)} n$ time algorithm for Bichromatic Maximum Inner Product with vector dimension $d = n^ε$ for any small constant $ε$ would imply $\textsf{NEXP}$ has no polynomial size $\textsf{THR} \circ \textsf{THR}$ circuits. Note there is an $n^2\textrm{polylog}(n)$ time algorithm via fast rectangle matrix multiplication. Our results build on two structure lemmas for threshold circuits.
Learning Simplicial Complexes from Persistence Diagrams
Venue: arXiv cs.CG 2018
Year: 2018

Topological Data Analysis (TDA) studies the shape of data. A common topological descriptor is the persistence diagram, which encodes topological features in a topological space at different scales. Turner, Mukeherjee, and Boyer showed that one can reconstruct a simplicial complex embedded in R^3 using persistence diagrams generated from all possible height filtrations (an uncountably infinite number of directions). In this paper, we present an algorithm for reconstructing plane graphs K=(V,E) in R^2 , i.e., a planar graph with vertices in general position and a straight-line embedding, from a quadratic number height filtrations and their respective persistence diagrams.
Multiple-Lithography-Compliant Verification for Standard Cell Library Development Flow
Venue: arXiv cs.OH 2018
Year: 2018

Starting from 22-nm, a standard cell must be designed to be full lithography-compliant, which includes Design Rule Check, Design-for-Manufacturability and Double-Patterning compliant. It has become a great challenge for physical layout designers to provide a full lithography-compliant standard cell layout that is optimized for area, power, timing, signal integrity, and yield. This challenge is further exacerbated with abutted single- and multiple-height standard cells. At present, different foundries and library vendors have different approaches for full lithography-compliant library preparation and validation. To the best of our knowledge, there is no single tool integrates all types of lithography-compliant check in standard cell libraries validation flow. In this work, we will demonstrate multiple lithography-compliant verification for standard cell library development flow. Validation flow and detailed algorithm implementation will be explained to assist engineers to achieve full lithography-compliant standard cell libraries. An area-efficient standard cell placement methodology will also be discussed to validate the issues arises from standard cell abutment.
Ranking and Repulsing Supermartingales for Reachability in Probabilistic Programs
Venue: arXiv cs.PL 2018
Year: 2018

Computing reachability probabilities is a fundamental problem in the analysis of probabilistic programs. This paper aims at a comprehensive and comparative account on various martingale-based methods for over- and under-approximating reachability probabilities. Based on the existing works that stretch across different communities (formal verification, control theory, etc.), we offer a unifying account. In particular, we emphasize the role of order-theoretic fixed points---a classic topic in computer science---in the analysis of probabilistic programs. This leads us to two new martingale-based techniques, too. We give rigorous proofs for their soundness and completeness. We also make an experimental comparison using our implementation of template-based synthesis algorithms for those martingales.
Universality of Deep Convolutional Neural Networks
Venue: arXiv cs.LG 2018
Year: 2018

Deep learning has been widely applied and brought breakthroughs in speech recognition, computer vision, and many other domains. The involved deep neural network architectures and computational issues have been well studied in machine learning. But there lacks a theoretical foundation for understanding the approximation or generalization ability of deep learning methods generated by the network architectures such as deep convolutional neural networks having convolutional structures. Here we show that a deep convolutional neural network (CNN) is universal, meaning that it can be used to approximate any continuous function to an arbitrary accuracy when the depth of the neural network is large enough. This answers an open question in learning theory. Our quantitative estimate, given tightly in terms of the number of free parameters to be computed, verifies the efficiency of deep CNNs in dealing with large dimensional data. Our study also demonstrates the role of convolutions in deep CNNs.
Keep and Learn: Continual Learning by Constraining the Latent Space for Knowledge Preservation in Neural Networks
Venue: arXiv cs.CV 2018
Year: 2018

Data is one of the most important factors in machine learning. However, even if we have high-quality data, there is a situation in which access to the data is restricted. For example, access to the medical data from outside is strictly limited due to the privacy issues. In this case, we have to learn a model sequentially only with the data accessible in the corresponding stage. In this work, we propose a new method for preserving learned knowledge by modeling the high-level feature space and the output space to be mutually informative, and constraining feature vectors to lie in the modeled space during training. The proposed method is easy to implement as it can be applied by simply adding a reconstruction loss to an objective function. We evaluate the proposed method on CIFAR-10/100 and a chest X-ray dataset, and show benefits in terms of knowledge preservation compared to previous approaches.
Local Rule-Based Explanations of Black Box Decision Systems
Venue: arXiv cs.AI 2018
Year: 2018

The recent years have witnessed the rise of accurate but obscure decision systems which hide the logic of their internal decision processes to the users. The lack of explanations for the decisions of black box systems is a key ethical issue, and a limitation to the adoption of machine learning components in socially sensitive and safety-critical contexts. %Therefore, we need explanations that reveals the reasons why a predictor takes a certain decision. In this paper we focus on the problem of black box outcome explanation, i.e., explaining the reasons of the decision taken on a specific instance. We propose LORE, an agnostic method able to provide interpretable and faithful explanations. LORE first leans a local interpretable predictor on a synthetic neighborhood generated by a genetic algorithm. Then it derives from the logic of the local interpretable predictor a meaningful explanation consisting of: a decision rule, which explains the reasons of the decision; and a set of counterfactual rules, suggesting the changes in the instance's features that lead to a different outcome. Wide experiments show that LORE outperforms existing methods and baselines both in the quality of explanations and in the accuracy in mimicking the black box.
High Probability Frequency Moment Sketches
Venue: arXiv cs.DS 2018
Year: 2018

We consider the problem of sketching the $p$-th frequency moment of a vector, $p>2$, with multiplicative error at most $1\pm ε$ and \emph{with high confidence} $1-δ$. Despite the long sequence of work on this problem, tight bounds on this quantity are only known for constant $δ$. While one can obtain an upper bound with error probability $δ$ by repeating a sketching algorithm with constant error probability $O(\log(1/δ))$ times in parallel, and taking the median of the outputs, we show this is a suboptimal algorithm! Namely, we show optimal upper and lower bounds of $Θ(n^{1-2/p} \log(1/δ) + n^{1-2/p} \log^{2/p} (1/δ) \log n)$ on the sketching dimension, for any constant approximation. Our result should be contrasted with results for estimating frequency moments for $1 \leq p \leq 2$, for which we show the optimal algorithm for general $δ$ is obtained by repeating the optimal algorithm for constant error probability $O(\log(1/δ))$ times and taking the median output. We also obtain a matching lower bound for this problem, up to constant factors.
Evolutionary Algorithms and Submodular Functions: Benefits of Heavy-Tailed Mutations
Venue: arXiv cs.DS 2018
Year: 2018

A core feature of evolutionary algorithms is their mutation operator. Recently, much attention has been devoted to the study of mutation operators with dynamic and non-uniform mutation rates. Following up on this line of work, we propose a new mutation operator and analyze its performance on the (1+1) Evolutionary Algorithm (EA). Our analyses show that this mutation operator competes with pre-existing ones, when used by the (1+1) EA on classes of problems for which results on the other mutation operators are available. We show that the (1+1) EA using our mutation operator finds a (1/3)-approximation ratio on any non-negative submodular function in polynomial time. We also consider the problem of maximizing a symmetric submodular function under a single matroid constraint and show that the (1+1) EA using our operator finds a (1/3)-approximation within polynomial time. This performance matches that of combinatorial local search algorithms specifically designed to solve these problems and outperforms them with constant probability. Finally, we evaluate the performance of the (1+1)EA using our operator experimentally by considering two applications: (a) the maximum directed cut problem on real-world graphs of different origins, and with up to 6.6 million vertices and 56 million edges and (b) the symmetric mutual information problem using a four month period air pollution data set. In comparison with uniform mutation and a recently proposed dynamic scheme our operator comes out on top on these instances.
Two types of permutation polynomials with special forms
Venue: arXiv cs.IT 2018
Year: 2018

Let $q$ be a power of a prime and $\mathbb{F}_q$ be a finite field with $q$ elements. In this paper, we propose four families of infinite classes of permutation trinomials having the form $cx-x^s + x^{qs}$ over $\mathbb{F}_{q^2}$, and investigate the relationship between this type of permutation polynomials with that of the form $(x^q-x+δ)^s+cx$. Based on this relation, many classes of permutation trinomials having the form $(x^q-x+δ)^s+cx$ without restriction on $δ$ over $\mathbb{F}_{q^2}$ are derived from known permutation trinomials having the form $cx-x^s + x^{qs}$.
Deeply learning molecular structure-property relationships using attention- and gate-augmented graph convolutional network
Venue: arXiv cs.LG 2018
Year: 2018

Molecular structure-property relationships are key to molecular engineering for materials and drug discovery. The rise of deep learning offers a new viable solution to elucidate the structure-property relationships directly from chemical data. Here we show that the performance of graph convolutional networks (GCNs) for the prediction of molecular properties can be improved by incorporating attention and gate mechanisms. The attention mechanism enables a GCN to identify atoms in different environments. The gated skip-connection further improves the GCN by updating feature maps at an appropriate rate. We demonstrate that the resulting attention- and gate-augmented GCN could extract better structural features related to a target molecular property such as solubility, polarity, synthetic accessibility and photovoltaic efficiency compared to the vanilla GCN. More interestingly, it identified two distinct parts of molecules as essential structural features for high photovoltaic efficiency, and each of them coincided with the areas of donor and acceptor orbitals for charge-transfer excitations, respectively. As a result, the new model could accurately predict molecular properties and place molecules with similar properties close to each other in a well-trained latent space, which is critical for successful molecular engineering.
The Coming Era of AlphaHacking? A Survey of Automatic Software Vulnerability Detection, Exploitation and Patching Techniques
Venue: arXiv cs.CR 2018
Year: 2018

With the success of the Cyber Grand Challenge (CGC) sponsored by DARPA, the topic of Autonomous Cyber Reasoning System (CRS) has recently attracted extensive attention from both industry and academia. Utilizing automated system to detect, exploit and patch software vulnerabilities seems so attractive because of its scalability and cost-efficiency compared with the human expert based solution. In this paper, we give an extensive survey of former representative works related to the underlying technologies of a CRS, including vulnerability detection, exploitation and patching. As an important supplement, we then review several pioneer studies that explore the potential of machine learning technologies in this field, and point out that the future development of Autonomous CRS is inseparable from machine learning.
Typed Embedding of a Relational Language in OCaml
Venue: arXiv cs.PL 2018
Year: 2018

We present an implementation of the relational programming language miniKanren as a set of combinators and syntax extensions for OCaml. The key feature of our approach is polymorphic unification, which can be used to unify data structures of arbitrary types. In addition we provide a useful generic programming pattern to systematically develop relational specifications in a typed manner, and address the problem of integration of relational subsystems into functional applications.
Evolutionary Algorithms
Venue: arXiv cs.NE 2018
Year: 2018

Evolutionary algorithms (EAs) are population-based metaheuristics, originally inspired by aspects of natural evolution. Modern varieties incorporate a broad mixture of search mechanisms, and tend to blend inspiration from nature with pragmatic engineering concerns; however, all EAs essentially operate by maintaining a population of potential solutions and in some way artificially 'evolving' that population over time. Particularly well-known categories of EAs include genetic algorithms (GAs), Genetic Programming (GP), and Evolution Strategies (ES). EAs have proven very successful in practical applications, particularly those requiring solutions to combinatorial problems. EAs are highly flexible and can be configured to address any optimization task, without the requirements for reformulation and/or simplification that would be needed for other techniques. However, this flexibility goes hand in hand with a cost: the tailoring of an EA's configuration and parameters, so as to provide robust performance for a given class of tasks, is often a complex and time-consuming process. This tailoring process is one of the many ongoing research areas associated with EAs.
Memory Augmented Self-Play
Venue: arXiv cs.LG 2018
Year: 2018

Self-play is an unsupervised training procedure which enables the reinforcement learning agents to explore the environment without requiring any external rewards. We augment the self-play setting by providing an external memory where the agent can store experience from the previous tasks. This enables the agent to come up with more diverse self-play tasks resulting in faster exploration of the environment. The agent pretrained in the memory augmented self-play setting easily outperforms the agent pretrained in no-memory self-play setting.
Potential and Pitfalls of Multi-Armed Bandits for Decentralized Spatial Reuse in WLANs
Venue: arXiv cs.NI 2018
Year: 2018

Spatial Reuse (SR) has recently gained attention to maximize the performance of IEEE 802.11 Wireless Local Area Networks (WLANs). Decentralized mechanisms are expected to be key in the development of SR solutions for next-generation WLANs, since many deployments are characterized by being uncoordinated by nature. However, the potential of decentralized mechanisms is limited by the significant lack of knowledge with respect to the overall wireless environment. To shed some light on this subject, we show the main considerations and possibilities of applying online learning to address the SR problem in uncoordinated WLANs. In particular, we provide a solution based on Multi-Armed Bandits (MABs) whereby independent WLANs dynamically adjust their frequency channel, transmit power and sensitivity threshold. To that purpose, we provide two different strategies, which refer to selfish and environment-aware learning. While the former stands for pure individual behavior, the second one considers the performance experienced by surrounding networks, thus taking into account the impact of individual actions on the environment. Through these two strategies we delve into practical issues of applying MABs in wireless networks, such as convergence guarantees or adversarial effects. Our simulation results illustrate the potential of the proposed solutions for enabling SR in future WLANs. We show that substantial improvements on network performance can be achieved regarding throughput and fairness.
Learning From Less Data: Diversified Subset Selection and Active Learning in Image Classification Tasks
Venue: arXiv cs.CV 2018
Year: 2018

Supervised machine learning based state-of-the-art computer vision techniques are in general data hungry and pose the challenges of not having adequate computing resources and of high costs involved in human labeling efforts. Training data subset selection and active learning techniques have been proposed as possible solutions to these challenges respectively. A special class of subset selection functions naturally model notions of diversity, coverage and representation and they can be used to eliminate redundancy and thus lend themselves well for training data subset selection. They can also help improve the efficiency of active learning in further reducing human labeling efforts by selecting a subset of the examples obtained using the conventional uncertainty sampling based techniques. In this work we empirically demonstrate the effectiveness of two diversity models, namely the Facility-Location and Disparity-Min models for training-data subset selection and reducing labeling effort. We do this for a variety of computer vision tasks including Gender Recognition, Scene Recognition and Object Recognition. Our results show that subset selection done in the right way can add 2-3% in accuracy on existing baselines, particularly in the case of less training data. This allows the training of complex machine learning models (like Convolutional Neural Networks) with much less training data while incurring minimal performance loss.
CapsNet comparative performance evaluation for image classification
Venue: arXiv cs.CV 2018
Year: 2018

Image classification has become one of the main tasks in the field of computer vision technologies. In this context, a recent algorithm called CapsNet that implements an approach based on activity vectors and dynamic routing between capsules may overcome some of the limitations of the current state of the art artificial neural networks (ANN) classifiers, such as convolutional neural networks (CNN). In this paper, we evaluated the performance of the CapsNet algorithm in comparison with three well-known classifiers (Fisher-faces, LeNet, and ResNet). We tested the classification accuracy on four datasets with a different number of instances and classes, including images of faces, traffic signs, and everyday objects. The evaluation results show that even for simple architectures, training the CapsNet algorithm requires significant computational resources and its classification performance falls below the average accuracy values of the other three classifiers. However, we argue that CapsNet seems to be a promising new technique for image classification, and further experiments using more robust computation resources and re-fined CapsNet architectures may produce better outcomes.
Unsupervised Alignment of Embeddings with Wasserstein Procrustes
Venue: arXiv cs.LG 2018
Year: 2018

We consider the task of aligning two sets of points in high dimension, which has many applications in natural language processing and computer vision. As an example, it was recently shown that it is possible to infer a bilingual lexicon, without supervised data, by aligning word embeddings trained on monolingual data. These recent advances are based on adversarial training to learn the mapping between the two embeddings. In this paper, we propose to use an alternative formulation, based on the joint estimation of an orthogonal matrix and a permutation matrix. While this problem is not convex, we propose to initialize our optimization algorithm by using a convex relaxation, traditionally considered for the graph isomorphism problem. We propose a stochastic algorithm to minimize our cost function on large scale problems. Finally, we evaluate our method on the problem of unsupervised word translation, by aligning word embeddings trained on monolingual data. On this task, our method obtains state of the art results, while requiring less computational resources than competing approaches.