OpenRaiser/Intern-Atlas huggingface dataset:
Iterative Inversion of Deformation Vector Fields with Feedback Control
Venue: arXiv cs.CV 2018
Year: 2018

Purpose: Often, the inverse deformation vector field (DVF) is needed together with the corresponding forward DVF in 4D reconstruction and dose calculation, adaptive radiation therapy, and simultaneous deformable registration. This study aims at improving both accuracy and efficiency of iterative algorithms for DVF inversion, and advancing our understanding of divergence and latency conditions. Method: We introduce a framework of fixed-point iteration algorithms with active feedback control for DVF inversion. Based on rigorous convergence analysis, we design control mechanisms for modulating the inverse consistency (IC) residual of the current iterate, to be used as feedback into the next iterate. The control is designed adaptively to the input DVF with the objective to enlarge the convergence area and expedite convergence. Three particular settings of feedback control are introduced: constant value over the domain throughout the iteration; alternating values between iteration steps; and spatially variant values. We also introduce three spectral measures of the displacement Jacobian for characterizing a DVF. These measures reveal the critical role of what we term the non-translational displacement component (NTDC) of the DVF. We carry out inversion experiments with an analytical DVF pair, and with DVFs associated with thoracic CT images of 6 patients at end of expiration and end of inspiration. Results: NTDC-adaptive iterations are shown to attain a larger convergence region at a faster pace compared to previous non-adaptive DVF inversion iteration algorithms. By our numerical experiments, alternating control yields smaller IC residuals and inversion errors than constant control. Spatially variant control renders smaller residuals and errors by at least an order of magnitude, compared to other schemes, in no more than 10 steps. Inversion results also show remarkable quantitative agreement with analysis-based predictions. Conclusion: Our analysis captures properties of DVF data associated with clinical CT images, and provides new understanding of iterative DVF inversion algorithms with a simple residual feedback control. Adaptive control is necessary and highly effective in the presence of non-small NTDCs. The adaptive iterations or the spectral measures, or both, may potentially be incorporated into deformable image registration methods.
A Review of 40 Years of Cognitive Architecture Research: Core Cognitive Abilities and Practical Applications
Venue: arXiv cs.AI 2018
Year: 2018

In this paper we present a broad overview of the last 40 years of research on cognitive architectures. Although the number of existing architectures is nearing several hundred, most of the existing surveys do not reflect this growth and focus on a handful of well-established architectures. Thus, in this survey we wanted to shift the focus towards a more inclusive and high-level overview of the research on cognitive architectures. Our final set of 84 architectures includes 49 that are still actively developed, and borrow from a diverse set of disciplines, spanning areas from psychoanalysis to neuroscience. To keep the length of this paper within reasonable limits we discuss only the core cognitive abilities, such as perception, attention mechanisms, action selection, memory, learning and reasoning. In order to assess the breadth of practical applications of cognitive architectures we gathered information on over 900 practical projects implemented using the cognitive architectures in our list. We use various visualization techniques to highlight overall trends in the development of the field. In addition to summarizing the current state-of-the-art in the cognitive architecture research, this survey describes a variety of methods and ideas that have been tried and their relative success in modeling human cognitive abilities, as well as which aspects of cognitive behavior need more research with respect to their mechanistic counterparts and thus can further inform how cognitive science might progress.
Joint Target Detection and Tracking in Multipath Environment: A Variational Bayesian Approach
Venue: arXiv cs.CV 2018
Year: 2018

We consider multitarget detection and tracking problem for a class of multipath detection system where one target may generate multiple measurements via multiple propagation paths, and the association relationship among targets, measurements and propagation paths is unknown. In order to effectively utilize multipath measurements from one target to improve detection and tracking performance, a tracker has to handle high-dimensional estimation of latent variables including target active/dormant meta-state, target kinematic state, and multipath data association. Based on variational Bayesian inference, we propose a novel joint detection and tracking algorithm that incorporates multipath data association, target detection and target state estimation in a unified Bayesian framework. The posterior probabilities of these latent variables are derived in a closed-form iterative manner, which is effective for reducing the performance deterioration caused by the coupling between estimation errors and identification errors. Loopy belief propagation is exploited to approximately calculate the probability of multipath data association, saving the computational cost significantly. Simulation results of over-the-horizon radar multitarget tracking show that the proposed algorithm outperforms multihypothesis multipath track fusion and multi-detection (hypothesis-oriented) multiple hypothesis tracker, especially under low signal-to-noise ratio circumstance.
Preserving Randomness for Adaptive Algorithms
Venue: arXiv cs.CC 2018
Year: 2018

Suppose $\mathsf{Est}$ is a randomized estimation algorithm that uses $n$ random bits and outputs values in $\mathbb{R}^d$. We show how to execute $\mathsf{Est}$ on $k$ adaptively chosen inputs using only $n + O(k \log(d + 1))$ random bits instead of the trivial $nk$ (at the cost of mild increases in the error and failure probability). Our algorithm combines a variant of the INW pseudorandom generator (STOC '94) with a new scheme for shifting and rounding the outputs of $\mathsf{Est}$. We prove that modifying the outputs of $\mathsf{Est}$ is necessary in this setting, and furthermore, our algorithm's randomness complexity is near-optimal in the case $d \leq O(1)$. As an application, we give a randomness-efficient version of the Goldreich-Levin algorithm; our algorithm finds all Fourier coefficients with absolute value at least $θ$ of a function $F: \{0, 1\}^n \to \{-1, 1\}$ using $O(n \log n) \cdot \text{poly}(1/θ)$ queries to $F$ and $O(n)$ random bits (independent of $θ$), improving previous work by Bshouty et al. (JCSS '04).
Combating Reinforcement Learning's Sisyphean Curse with Intrinsic Fear
Venue: arXiv cs.LG 2018
Year: 2018

Many practical environments contain catastrophic states that an optimal agent would visit infrequently or never. Even on toy problems, Deep Reinforcement Learning (DRL) agents tend to periodically revisit these states upon forgetting their existence under a new policy. We introduce intrinsic fear (IF), a learned reward shaping that guards DRL agents against periodic catastrophes. IF agents possess a fear model trained to predict the probability of imminent catastrophe. This score is then used to penalize the Q-learning objective. Our theoretical analysis bounds the reduction in average return due to learning on the perturbed objective. We also prove robustness to classification errors. As a bonus, IF models tend to learn faster, owing to reward shaping. Experiments demonstrate that intrinsic-fear DQNs solve otherwise pathological environments and improve on several Atari games.
Almost universal codes for MIMO wiretap channels
Venue: arXiv cs.IT 2018
Year: 2018

Despite several works on secrecy coding for fading and MIMO wiretap channels from an error probability perspective, the construction of information-theoretically secure codes over such channels remains an open problem. In this paper, we consider a fading wiretap channel model where the transmitter has only partial statistical channel state information. Our channel model includes static channels, i.i.d. block fading channels, and ergodic stationary fading with fast decay of large deviations for the eavesdropper's channel. We extend the flatness factor criterion from the Gaussian wiretap channel to fading and MIMO wiretap channels, and establish a simple design criterion where the normalized product distance / minimum determinant of the lattice and its dual should be maximized simultaneously. Moreover, we propose concrete lattice codes satisfying this design criterion, which are built from algebraic number fields with constant root discriminant in the single-antenna case, and from division algebras centered at such number fields in the multiple-antenna case. The proposed lattice codes achieve strong secrecy and semantic security for all rates $R<C_b-C_e-κ$, where $C_b$ and $C_e$ are Bob and Eve's channel capacities respectively, and $κ$ is an explicit constant gap. Furthermore, these codes are almost universal in the sense that a fixed code is good for secrecy for a wide range of fading models. Finally, we consider a compound wiretap model with a more restricted uncertainty set, and show that rates $R<\bar{C}_b-\bar{C}_e-κ$ are achievable, where $\bar{C}_b$ is a lower bound for Bob's capacity and $\bar{C}_e$ is an upper bound for Eve's capacity for all the channels in the set.
A Differentiable Physics Engine for Deep Learning in Robotics
Venue: arXiv cs.NE 2018
Year: 2018

An important field in robotics is the optimization of controllers. Currently, robots are often treated as a black box in this optimization process, which is the reason why derivative-free optimization methods such as evolutionary algorithms or reinforcement learning are omnipresent. When gradient-based methods are used, models are kept small or rely on finite difference approximations for the Jacobian. This method quickly grows expensive with increasing numbers of parameters, such as found in deep learning. We propose the implementation of a modern physics engine, which can differentiate control parameters. This engine is implemented for both CPU and GPU. Firstly, this paper shows how such an engine speeds up the optimization process, even for small problems. Furthermore, it explains why this is an alternative approach to deep Q-learning, for using deep learning in robotics. Finally, we argue that this is a big step for deep learning in robotics, as it opens up new possibilities to optimize robots, both in hardware and software.
On High-Resolution Adaptive Sampling of Deterministic Signals
Venue: arXiv cs.IT 2018
Year: 2018

In this work we study the topic of high-resolution adaptive sampling of a given deterministic signal and establish a connection with classic approaches to high-rate quantization. Specifically, we formulate solutions for the task of optimal high-resolution sampling, counterparts of well-known results for high-rate quantization. Our results reveal that the optimal high-resolution sampling structure is determined by the density of the signal-gradient energy, just as the probability-density-function defines the optimal high-rate quantization form. This paper has three main contributions: the first is establishing a fundamental paradigm bridging the topics of sampling and quantization. The second is a theoretical analysis of nonuniform sampling relevant to the emerging field of high-resolution signal processing. The third is a new practical approach to nonuniform sampling of one-dimensional signals that enables reconstruction based only on the sampling time-points and the signal extrema locations and values. Experiments for signal sampling and coding showed that our method outperforms an optimized tree-structured sampling technique.
Finite Satisfiability of the Two-Variable Guarded Fragment with Transitive Guards and Related Variants
Venue: arXiv cs.LO 2018
Year: 2018

We consider extensions of the two-variable guarded fragment, GF2, where distinguished binary predicates that occur only in guards are required to be interpreted in a special way (as transitive relations, equivalence relations, pre-orders or partial orders). We prove that the only fragment that retains the finite (exponential) model property is GF2 with equivalence guards without equality. For remaining fragments we show that the size of a minimal finite model is at most doubly exponential. To obtain the result we invent a strategy of building finite models that are formed from a number of multidimensional grids placed over a cylindrical surface. The construction yields a 2NExpTime-upper bound on the complexity of the finite satisfiability problem for these fragments. We improve the bounds and obtain optimal ones for all the fragments considered, in particular NExpTime for GF2 with equivalence guards, and 2ExpTime for GF2 with transitive guards. To obtain our results we essentially use some results from integer programming.
Deeply supervised salient object detection with short connections
Venue: arXiv cs.CV 2018
Year: 2018

Recent progress on saliency detection is substantial, benefiting mostly from the explosive development of Convolutional Neural Networks (CNNs). Semantic segmentation and saliency detection algorithms developed lately have been mostly based on Fully Convolutional Neural Networks (FCNs). There is still a large room for improvement over the generic FCN models that do not explicitly deal with the scale-space problem. Holistically-Nested Edge Detector (HED) provides a skip-layer structure with deep supervision for edge and boundary detection, but the performance gain of HED on salience detection is not obvious. In this paper, we propose a new method for saliency detection by introducing short connections to the skip-layer structures within the HED architecture. Our framework provides rich multi-scale feature maps at each layer, a property that is critically needed to perform segment detection. Our method produces state-of-the-art results on 5 widely tested salient object detection benchmarks, with advantages in terms of efficiency (0.15 seconds per image), effectiveness, and simplicity over the existing algorithms.
LOTS about Attacking Deep Features
Venue: arXiv cs.CV 2018
Year: 2018

Deep neural networks provide state-of-the-art performance on various tasks and are, therefore, widely used in real world applications. DNNs are becoming frequently utilized in biometrics for extracting deep features, which can be used in recognition systems for enrolling and recognizing new individuals. It was revealed that deep neural networks suffer from a fundamental problem, namely, they can unexpectedly misclassify examples formed by slightly perturbing correctly recognized inputs. Various approaches have been developed for generating these so-called adversarial examples, but they aim at attacking end-to-end networks. For biometrics, it is natural to ask whether systems using deep features are immune to or, at least, more resilient to attacks than end-to-end networks. In this paper, we introduce a general technique called the layerwise origin-target synthesis (LOTS) that can be efficiently used to form adversarial examples that mimic the deep features of the target. We analyze and compare the adversarial robustness of the end-to-end VGG Face network with systems that use Euclidean or cosine distance between gallery templates and extracted deep features. We demonstrate that iterative LOTS is very effective and show that systems utilizing deep features are easier to attack than the end-to-end network.
On MDS Negacyclic LCD Codes
Venue: arXiv cs.IT 2018
Year: 2018

Linear codes with complementary duals (LCD) have a great deal of significance amongst linear codes. Maximum distance separable (MDS) codes are also an important class of linear codes since they achieve the greatest error correcting and detecting capabilities for fixed length and dimension. The construction of linear codes that are both LCD and MDS is a hard task in coding theory. In this paper, we study the constructions of LCD codes that are MDS from negacyclic codes over finite fields of odd prime power $q$ elements. We construct four families of MDS negacyclic LCD codes of length $n|\frac{q-1}{2}$, $n|\frac{q+1}{2}$ and a family of negacyclic LCD codes of length $n=q-1$. Furthermore, we obtain five families of $q^{2}$-ary Hermitian MDS negacyclic LCD codes of length $n|\left( q-1\right)$ and four families of Hermitian negacyclic LCD codes of length $n=q^{2}+1.$ For both Euclidean and Hermitian cases the dimensions of these codes are determined and for some classes the minimum distances are settled. For the other cases, by studying $q$ and $q^{2}$-cyclotomic classes we give lower bounds on the minimum distance.
The second generalized Hamming weight of some evaluation codes arising from a projective torus
Venue: arXiv cs.IT 2018
Year: 2018

In this paper we find the second generalized Hamming weight of some evaluation codes arising from a projective torus, and it allows to compute the second generalized Hamming weight of the codes parameterized by the edges of any complete bipartite graph. Also, at the beginning, we obtain some results about the generalized Hamming weights of some evaluation codes arising from a complete intersection when the minimum distance is known and they are non--degenerate codes. Finally we give an example where we use these results to determine the complete weight hierarchy of some codes.
Prototypical Recurrent Unit
Venue: arXiv cs.LG 2018
Year: 2018

Despite the great successes of deep learning, the effectiveness of deep neural networks has not been understood at any theoretical depth. This work is motivated by the thrust of developing a deeper understanding of recurrent neural networks, particularly LSTM/GRU-like networks. As the highly complex structure of the recurrent unit in LSTM and GRU networks makes them difficult to analyze, our methodology in this research theme is to construct an alternative recurrent unit that is as simple as possible and yet also captures the key components of LSTM/GRU recurrent units. Such a unit can then be used for the study of recurrent networks and its structural simplicity may allow easier analysis. Towards that goal, we take a system-theoretic perspective to design a new recurrent unit, which we call the prototypical recurrent unit (PRU). Not only having minimal complexity, PRU is demonstrated experimentally to have comparable performance to GRU and LSTM unit. This establishes PRU networks as a prototype for future study of LSTM/GRU-like recurrent networks. This paper also studies the memorization abilities of LSTM, GRU and PRU networks, motivated by the folk belief that such networks possess long-term memory. For this purpose, we design a simple and controllable task, called ``memorization problem'', where the networks are trained to memorize certain targeted information. We show that the memorization performance of all three networks depends on the amount of targeted information, the amount of ``interfering" information, and the state space dimension of the recurrent unit. Experiments are also performed for another controllable task, the adding problem, and similar conclusions are obtained.
Faster Population Counts Using AVX2 Instructions
Venue: arXiv cs.DS 2018
Year: 2018

Counting the number of ones in a binary stream is a common operation in database, information-retrieval, cryptographic and machine-learning applications. Most processors have dedicated instructions to count the number of ones in a word (e.g., popcnt on x64 processors). Maybe surprisingly, we show that a vectorized approach using SIMD instructions can be twice as fast as using the dedicated instructions on recent Intel processors. The benefits can be even greater for applications such as similarity measures (e.g., the Jaccard index) that require additional Boolean operations. Our approach has been adopted by LLVM: it is used by its popular C compiler (clang).
EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces
Venue: arXiv cs.LG 2018
Year: 2018

Brain computer interfaces (BCI) enable direct communication with a computer, using neural activity as the control signal. This neural signal is generally chosen from a variety of well-studied electroencephalogram (EEG) signals. For a given BCI paradigm, feature extractors and classifiers are tailored to the distinct characteristics of its expected EEG control signal, limiting its application to that specific signal. Convolutional Neural Networks (CNNs), which have been used in computer vision and speech recognition, have successfully been applied to EEG-based BCIs; however, they have mainly been applied to single BCI paradigms and thus it remains unclear how these architectures generalize to other paradigms. Here, we ask if we can design a single CNN architecture to accurately classify EEG signals from different BCI paradigms, while simultaneously being as compact as possible. In this work we introduce EEGNet, a compact convolutional network for EEG-based BCIs. We introduce the use of depthwise and separable convolutions to construct an EEG-specific model which encapsulates well-known EEG feature extraction concepts for BCI. We compare EEGNet to current state-of-the-art approaches across four BCI paradigms: P300 visual-evoked potentials, error-related negativity responses (ERN), movement-related cortical potentials (MRCP), and sensory motor rhythms (SMR). We show that EEGNet generalizes across paradigms better than the reference algorithms when only limited training data is available. We demonstrate three different approaches to visualize the contents of a trained EEGNet model to enable interpretation of the learned features. Our results suggest that EEGNet is robust enough to learn a wide variety of interpretable features over a range of BCI tasks, suggesting that the observed performances were not due to artifact or noise sources in the data.
Learning Shape Abstractions by Assembling Volumetric Primitives
Venue: arXiv cs.CV 2018
Year: 2018

We present a learning framework for abstracting complex shapes by learning to assemble objects using 3D volumetric primitives. In addition to generating simple and geometrically interpretable explanations of 3D objects, our framework also allows us to automatically discover and exploit consistent structure in the data. We demonstrate that using our method allows predicting shape representations which can be leveraged for obtaining a consistent parsing across the instances of a shape collection and constructing an interpretable shape similarity measure. We also examine applications for image-based prediction as well as shape manipulation.
A Calculus of Cyber-Physical Systems
Venue: arXiv cs.LO 2018
Year: 2018

We propose a hybrid process calculus for modelling and reasoning on cyber-physical systems (CPS{s}). The dynamics of the calculus is expressed in terms of a labelled transition system in the SOS style of Plotkin. This is used to define a bisimulation-based behavioural semantics which support compositional reasonings. Finally, we prove run-time properties and system equalities for a non-trivial case study.
Globular: an online proof assistant for higher-dimensional rewriting
Venue: arXiv cs.LO 2018
Year: 2018

This article introduces Globular, an online proof assistant for the formalization and verification of proofs in higher-dimensional category theory. The tool produces graphical visualizations of higher-dimensional proofs, assists in their construction with a point-and- click interface, and performs type checking to prevent incorrect rewrites. Hosted on the web, it has a low barrier to use, and allows hyperlinking of formalized proofs directly from research papers. It allows the formalization of proofs from logic, topology and algebra which are not formalizable by other methods, and we give several examples.