OpenRaiser/Intern-Atlas huggingface dataset:
MoE-SPNet: A Mixture-of-Experts Scene Parsing Network
Venue: arXiv cs.CV 2018
Year: 2018

Scene parsing is an indispensable component in understanding the semantics within a scene. Traditional methods rely on handcrafted local features and probabilistic graphical models to incorporate local and global cues. Recently, methods based on fully convolutional neural networks have achieved new records on scene parsing. An important strategy common to these methods is the aggregation of hierarchical features yielded by a deep convolutional neural network. However, typical algorithms usually aggregate hierarchical convolutional features via concatenation or linear combination, which cannot sufficiently exploit the diversities of contextual information in multi-scale features and the spatial inhomogeneity of a scene. In this paper, we propose a mixture-of-experts scene parsing network (MoE-SPNet) that incorporates a convolutional mixture-of-experts layer to assess the importance of features from different levels and at different spatial locations. In addition, we propose a variant of mixture-of-experts called the adaptive hierarchical feature aggregation (AHFA) mechanism which can be incorporated into existing scene parsing networks that use skip-connections to fuse features layer-wisely. In the proposed networks, different levels of features at each spatial location are adaptively re-weighted according to the local structure and surrounding contextual information before aggregation. We demonstrate the effectiveness of the proposed methods on two scene parsing datasets including PASCAL VOC 2012 and SceneParse150 based on two kinds of baseline models FCN-8s and DeepLab-ASPP.
Creating Tailored and Adaptive Network Services with the Open Orchestration C-RAN Framework
Venue: arXiv cs.NI 2018
Year: 2018

Next generation wireless communications networks will leverage software-defined radio and networking technologies, combined with cloud and fog computing. A pool of resources can then be dynamically allocated to create personalized network services (NSs). The enabling technologies are abstraction, virtualization and consolidation of resources, automatization of processes, and programmatic provisioning and orchestration. ETSI's network functions virtualization (NFV) management and orchestration (MANO) framework provides the architecture and specifications of the management layers. We introduce OOCRAN, an open-source software framework and testbed that extends existing NFV management solutions by incorporating the radio communications layers. This paper presents OOCRAN and illustrates how it monitors and manages the pool of resources for creating tailored NSs. OOCRAN can automate NS reconfiguration, but also facilitates user control. We demonstrate the dynamic deployment of cellular NSs and discuss the challenges of dynamically creating and managing tailored NSs on shared infrastructure.
Effect of Hyper-Parameter Optimization on the Deep Learning Model Proposed for Distributed Attack Detection in Internet of Things Environment
Venue: arXiv cs.LG 2018
Year: 2018

This paper studies the effect of various hyper-parameters and their selection for the best performance of the deep learning model proposed in [1] for distributed attack detection in the Internet of Things (IoT). The findings show that there are three hyper-parameters that have more influence on the best performance achieved by the model. As a consequence, this study shows that the model's accuracy as reported in the paper is not achievable, based on the best selections of parameters, which is also supported by another recent publication [2].
Cancer Metastasis Detection With Neural Conditional Random Field
Venue: arXiv cs.CV 2018
Year: 2018

Breast cancer diagnosis often requires accurate detection of metastasis in lymph nodes through Whole-slide Images (WSIs). Recent advances in deep convolutional neural networks (CNNs) have shown significant successes in medical image analysis and particularly in computational histopathology. Because of the outrageous large size of WSIs, most of the methods divide one slide into lots of small image patches and perform classification on each patch independently. However, neighboring patches often share spatial correlations, and ignoring these spatial correlations may result in inconsistent predictions. In this paper, we propose a neural conditional random field (NCRF) deep learning framework to detect cancer metastasis in WSIs. NCRF considers the spatial correlations between neighboring patches through a fully connected CRF which is directly incorporated on top of a CNN feature extractor. The whole deep network can be trained end-to-end with standard back-propagation algorithm with minor computational overhead from the CRF component. The CNN feature extractor can also benefit from considering spatial correlations via the CRF component. Compared to the baseline method without considering spatial correlations, we show that the proposed NCRF framework obtains probability maps of patch predictions with better visual quality. We also demonstrate that our method outperforms the baseline in cancer metastasis detection on the Camelyon16 dataset and achieves an average FROC score of 0.8096 on the test set. NCRF is open sourced at https://github.com/baidu-research/NCRF.
The Polylog-Time Hierarchy Captured by Restricted Second-Order Logic
Venue: arXiv cs.LO 2018
Year: 2018

Let $\mathrm{SO}^{\mathit{plog}}$ denote the restriction of second-order logic, where second-order quantification ranges over relations of size at most poly-logarithmic in the size of the structure. In this article we investigate the problem, which Turing machine complexity class is captured by Boolean queries over ordered relational structures that can be expressed in this logic. For this we define a hierarchy of fragments $Σ^{\mathit{plog}}_m$ (and $Π^{\mathit{plog}}_m$) defined by formulae with alternating blocks of existential and universal second-order quantifiers in quantifier-prenex normal form. We first show that the existential fragment $Σ^{\mathit{plog}}_1$ captures NPolyLogTime, i.e. the class of Boolean queries that can be accepted by a non-deterministic Turing machine with random access to the input in time $O((\log n)^k)$ for some $k \ge 0$. Using alternating Turing machines with random access input allows us to characterise also the fragments $Σ^{\mathit{plog}}_m$ (and $Π^{\mathit{plog}}_m$) as those Boolean queries with at most $m$ alternating blocks of second-order quantifiers that are accepted by an alternating Turing machine. Consequently, $\mathrm{SO}^{\mathit{plog}}$ captures the whole poly-logarithmic time hierarchy. We demonstrate the relevance of this logic and complexity class by several problems in database theory.
Computer-aided proofs for multiparty computation with active security
Venue: arXiv cs.LO 2018
Year: 2018

Secure multi-party computation (MPC) is a general cryptographic technique that allows distrusting parties to compute a function of their individual inputs, while only revealing the output of the function. It has found applications in areas such as auctioning, email filtering, and secure teleconference. Given its importance, it is crucial that the protocols are specified and implemented correctly. In the programming language community it has become good practice to use computer proof assistants to verify correctness proofs. In the field of cryptography, EasyCrypt is the state of the art proof assistant. It provides an embedded language for probabilistic programming, together with a specialized logic, embedded into an ambient general purpose higher-order logic. It allows us to conveniently express cryptographic properties. EasyCrypt has been used successfully on many applications, including public-key encryption, signatures, garbled circuits and differential privacy. Here we show for the first time that it can also be used to prove security of MPC against a malicious adversary. We formalize additive and replicated secret sharing schemes and apply them to Maurer's MPC protocol for secure addition and multiplication. Our method extends to general polynomial functions. We follow the insights from EasyCrypt that security proofs can be often be reduced to proofs about program equivalence, a topic that is well understood in the verification of programming languages. In particular, we show that in the passive case the non-interference-based definition is equivalent to a standard game-based security definition. For the active case we provide a new NI definition, which we call input independence.
ASIC Implementation of Time-Domain Digital Backpropagation with Deep-Learned Chromatic Dispersion Filters
Venue: arXiv cs.IT 2018
Year: 2018

We consider time-domain digital backpropagation with chromatic dispersion filters jointly optimized and quantized using machine-learning techniques. Compared to the baseline implementations, we show improved BER performance and >40% power dissipation reductions in 28-nm CMOS.
PaMpeR: Proof Method Recommendation System for Isabelle/HOL
Venue: arXiv cs.LO 2018
Year: 2018

Deciding which sub-tool to use for a given proof state requires expertise specific to each ITP. To mitigate this problem, we present PaMpeR, a Proof Method Recommendation system for Isabelle/HOL. Given a proof state, PaMpeR recommends proof methods to discharge the proof goal and provides qualitative explanations as to why it suggests these methods. PaMpeR generates these recommendations based on existing hand-written proof corpora, thus transferring experienced users' expertise to new users. Our evaluation shows that PaMpeR correctly predicts experienced users' proof methods invocation especially when it comes to special purpose proof methods.
Emergent Open-Endedness from Contagion of the Fittest
Venue: arXiv cs.SI 2018
Year: 2018

In this paper, we study emergent irreducible information in populations of randomly generated computable systems that are networked and follow a "Susceptible-Infected-Susceptible" contagion model of imitation of the fittest neighbor. We show that there is a lower bound for the stationary prevalence (or average density of "infected" nodes) that triggers an unlimited increase of the expected local emergent algorithmic complexity (or information) of a node as the population size grows. We call this phenomenon expected (local) emergent open-endedness. In addition, we show that static networks with a power-law degree distribution following the Barabási-Albert model satisfy this lower bound and, thus, display expected (local) emergent open-endedness.
Forest Packing: Fast, Parallel Decision Forests
Venue: arXiv cs.PF 2018
Year: 2018

Machine learning has an emerging critical role in high-performance computing to modulate simulations, extract knowledge from massive data, and replace numerical models with efficient approximations. Decision forests are a critical tool because they provide insight into model operation that is critical to interpreting learned results. While decision forests are trivially parallelizable, the traversals of tree data structures incur many random memory accesses and are very slow. We present memory packing techniques that reorganize learned forests to minimize cache misses during classification. The resulting layout is hierarchical. At low levels, we pack the nodes of multiple trees into contiguous memory blocks so that each memory access fetches data for multiple trees. At higher levels, we use leaf cardinality to identify the most popular paths through a tree and collocate those paths in cache lines. We extend this layout with out-of-order execution and cache-line prefetching to increase memory throughput. Together, these optimizations increase the performance of classification in ensembles by a factor of four over an optimized C++ implementation and a actor of 50 over a popular R language implementation.
Rate-Memory Trade-Off for Caching and Delivery of Correlated Sources
Venue: arXiv cs.IT 2018
Year: 2018

This paper studies the fundamental limits of content delivery in a cache-aided broadcast network for correlated content generated by a discrete memoryless source with arbitrary joint distribution. Each receiver is equipped with a cache of equal capacity, and the requested files are delivered over a shared error-free broadcast link. A class of achievable correlation-aware schemes based on a two-step source coding approach is proposed. Library files are first compressed, and then cached and delivered using a combination of correlation-unaware multiple-request cache-aided coded multicast schemes. The first step uses Gray-Wyner source coding to represent the library via private descriptions and descriptions that are common to more than one file. The second step then becomes a multiple-request caching problem, where the demand structure is dictated by the configuration of the compressed library, and it is interesting in its own right. The performance of the proposed two-step scheme is evaluated by comparing its achievable rate with a lower bound on the optimal peak and average rate-memory tradeoffs in a two-file multiple-receiver network, and in a three-file two-receiver network. Specifically, in a network with two files and two receivers, the achievable rate matches the lower bound for a significant memory regime and it is within half of the conditional entropy of files for all other memory values. In the three-file two-receiver network, the two-step strategy achieves the lower bound for large cache capacities, and it is within half of the joint entropy of two of the sources conditioned on the third one for all other cache sizes.
Movement-efficient Sensor Deployment in Wireless Sensor Networks with Limited Communication Range
Venue: arXiv cs.DC 2018
Year: 2018

We study a mobile wireless sensor network (MWSN) consisting of multiple mobile sensors or robots. Three key factors in MWSNs, sensing quality, energy consumption, and connectivity, have attracted plenty of attention, but the interaction of these factors is not well studied. To take all the three factors into consideration, we model the sensor deployment problem as a constrained source coding problem. %, which can be applied to different coverage tasks, such as area coverage, target coverage, and barrier coverage. Our goal is to find an optimal sensor deployment (or relocation) to optimize the sensing quality with a limited communication range and a specific network lifetime constraint. We derive necessary conditions for the optimal sensor deployment in both homogeneous and heterogeneous MWSNs. According to our derivation, some sensors are idle in the optimal deployment of heterogeneous MWSNs. Using these necessary conditions, we design both centralized and distributed algorithms to provide a flexible and explicit trade-off between sensing uncertainty and network lifetime. The proposed algorithms are successfully extended to more applications, such as area coverage and target coverage, via properly selected density functions. Simulation results show that our algorithms outperform the existing relocation algorithms.
Faster SGD training by minibatch persistency
Venue: arXiv cs.LG 2018
Year: 2018

It is well known that, for most datasets, the use of large-size minibatches for Stochastic Gradient Descent (SGD) typically leads to slow convergence and poor generalization. On the other hand, large minibatches are of great practical interest as they allow for a better exploitation of modern GPUs. Previous literature on the subject concentrated on how to adjust the main SGD parameters (in particular, the learning rate) when using large minibatches. In this work we introduce an additional feature, that we call minibatch persistency, that consists in reusing the same minibatch for K consecutive SGD iterations. The computational conjecture here is that a large minibatch contains a significant sample of the training set, so one can afford to slightly overfitting it without worsening generalization too much. The approach is intended to speedup SGD convergence, and also has the advantage of reducing the overhead related to data loading on the internal GPU memory. We present computational results on CIFAR-10 with an AlexNet architecture, showing that even small persistency values (K=2 or 5) already lead to a significantly faster convergence and to a comparable (or even better) generalization than the standard "disposable minibatch" approach (K=1), in particular when large minibatches are used. The lesson learned is that minibatch persistency can be a simple yet effective way to deal with large minibatches.
Deterministic $O(1)$-Approximation Algorithms to 1-Center Clustering with Outliers
Venue: arXiv cs.DS 2018
Year: 2018

The 1-center clustering with outliers problem asks about identifying a prototypical robust statistic that approximates the location of a cluster of points. Given some constant $0 < α< 1$ and $n$ points such that $αn$ of them are in some (unknown) ball of radius $r,$ the goal is to compute a ball of radius $O(r)$ that also contains $αn$ points. This problem can be formulated with the points in a normed vector space such as $\mathbb{R}^d$ or in a general metric space. The problem has a simple randomized solution: a randomly selected point is a correct solution with constant probability, and its correctness can be verified in linear time. However, the deterministic complexity of this problem was not known. In this paper, for any $\ell_p$ vector space, we show an $O(nd)$-time solution with a ball of radius $O(r)$ for a fixed $α> \frac{1}{2},$ and for any normed vector space, we show an $O(nd)$-time solution with a ball of radius $O(r)$ when $α> \frac{1}{2}$ as well as an $O (nd \log^{(k)}(n))$-time solution with a ball of radius $O(r)$ for all $α> 0, k \in \mathbb{N},$ where $\log^{(k)}(n)$ represents the $k$th iterated logarithm, assuming distance computation and vector space operations take $O(d)$ time. For an arbitrary metric space, we show for any $C \in \mathbb{N}$ an $O(n^{1+1/C})$-time solution that finds a ball of radius $2Cr,$ assuming distance computation between any pair of points takes $O(1)$-time. Moreover, this algorithm is optimal for general metric spaces, as we show that for any fixed $α, C,$ there is no $o(n^{1+1/C})$-query and thus no $o(n^{1+1/C})$-time solution that deterministically finds a ball of radius $2Cr$.
Few-Shot Segmentation Propagation with Guided Networks
Venue: arXiv cs.CV 2018
Year: 2018

Learning-based methods for visual segmentation have made progress on particular types of segmentation tasks, but are limited by the necessary supervision, the narrow definitions of fixed tasks, and the lack of control during inference for correcting errors. To remedy the rigidity and annotation burden of standard approaches, we address the problem of few-shot segmentation: given few image and few pixel supervision, segment any images accordingly. We propose guided networks, which extract a latent task representation from any amount of supervision, and optimize our architecture end-to-end for fast, accurate few-shot segmentation. Our method can switch tasks without further optimization and quickly update when given more guidance. We report the first results for segmentation from one pixel per concept and show real-time interactive video segmentation. Our unified approach propagates pixel annotations across space for interactive segmentation, across time for video segmentation, and across scenes for semantic segmentation. Our guided segmentor is state-of-the-art in accuracy for the amount of annotation and time. See http://github.com/shelhamer/revolver for code, models, and more details.
Distinguishing Refracted Features using Light Field Cameras with Application to Structure from Motion
Venue: arXiv cs.CV 2018
Year: 2018

Robots must reliably interact with refractive objects in many applications; however, refractive objects can cause many robotic vision algorithms to become unreliable or even fail, particularly feature-based matching applications, such as structure-from-motion. We propose a method to distinguish between refracted and Lambertian image features using a light field camera. Specifically, we propose to use textural cross-correlation to characterise apparent feature motion in a single light field, and compare this motion to its Lambertian equivalent based on 4D light field geometry. Our refracted feature distinguisher has a 34.3% higher rate of detection compared to state-of-the-art for light fields captured with large baselines relative to the refractive object. Our method also applies to light field cameras with much smaller baselines than previously considered, yielding up to 2 times better detection for 2D-refractive objects, such as a sphere, and up to 8 times better for 1D-refractive objects, such as a cylinder. For structure from motion, we demonstrate that rejecting refracted features using our distinguisher yields up to 42.4% lower reprojection error, and lower failure rate when the robot is approaching refractive objects. Our method lead to more robust robot vision in the presence of refractive objects.
In situ TensorView: In situ Visualization of Convolutional Neural Networks
Venue: arXiv cs.CV 2018
Year: 2018

Convolutional Neural Networks(CNNs) are complex systems. They are trained so they can adapt their internal connections to recognize images, texts and more. It is both interesting and helpful to visualize the dynamics within such deep artificial neural networks so that people can understand how these artificial networks are learning and making predictions. In the field of scientific simulations, visualization tools like Paraview have long been utilized to provide insights and understandings. We present in situ TensorView to visualize the training and functioning of CNNs as if they are systems of scientific simulations. In situ TensorView is a loosely coupled in situ visualization open framework that provides multiple viewers to help users to visualize and understand their networks. It leverages the capability of co-processing from Paraview to provide real-time visualization during training and predicting phases. This avoid heavy I/O overhead for visualizing large dynamic systems. Only a small number of lines of codes are injected in TensorFlow framework. The visualization can provide guidance to adjust the architecture of networks, or compress the pre-trained networks. We showcase visualizing the training of LeNet-5 and VGG16 using in situ TensorView.
Contrastive Hebbian Learning with Random Feedback Weights
Venue: arXiv cs.LG 2018
Year: 2018

Neural networks are commonly trained to make predictions through learning algorithms. Contrastive Hebbian learning, which is a powerful rule inspired by gradient backpropagation, is based on Hebb's rule and the contrastive divergence algorithm. It operates in two phases, the forward (or free) phase, where the data are fed to the network, and a backward (or clamped) phase, where the target signals are clamped to the output layer of the network and the feedback signals are transformed through the transpose synaptic weight matrices. This implies symmetries at the synaptic level, for which there is no evidence in the brain. In this work, we propose a new variant of the algorithm, called random contrastive Hebbian learning, which does not rely on any synaptic weights symmetries. Instead, it uses random matrices to transform the feedback signals during the clamped phase, and the neural dynamics are described by first order non-linear differential equations. The algorithm is experimentally verified by solving a Boolean logic task, classification tasks (handwritten digits and letters), and an autoencoding task. This article also shows how the parameters affect learning, especially the random matrices. We use the pseudospectra analysis to investigate further how random matrices impact the learning process. Finally, we discuss the biological plausibility of the proposed algorithm, and how it can give rise to better computational models for learning.