OpenRaiser/Intern-Atlas huggingface dataset:
Building effective deep neural network architectures one feature at a time
Venue: arXiv cs.CV 2017
Year: 2017

Successful training of convolutional neural networks is often associated with sufficiently deep architectures composed of high amounts of features. These networks typically rely on a variety of regularization and pruning techniques to converge to less redundant states. We introduce a novel bottom-up approach to expand representations in fixed-depth architectures. These architectures start from just a single feature per layer and greedily increase width of individual layers to attain effective representational capacities needed for a specific task. While network growth can rely on a family of metrics, we propose a computationally efficient version based on feature time evolution and demonstrate its potency in determining feature importance and a networks' effective capacity. We demonstrate how automatically expanded architectures converge to similar topologies that benefit from lesser amount of parameters or improved accuracy and exhibit systematic correspondence in representational complexity with the specified task. In contrast to conventional design patterns with a typical monotonic increase in the amount of features with increased depth, we observe that CNNs perform better when there is more learnable parameters in intermediate, with falloffs to earlier and later layers.
Being even slightly shallow makes life hard
Venue: arXiv cs.CC 2017
Year: 2017

We study the computational complexity of identifying dense substructures, namely $r/2$-shallow topological minors and $r$-subdivisions. Of particular interest is the case when $r=1$, when these substructures correspond to very localized relaxations of subgraphs. Since Densest Subgraph can be solved in polynomial time, we ask whether these slight relaxations also admit efficient algorithms. In the following, we provide a negative answer: Dense $r/2$-Shallow Topological Minor and Dense $r$-Subdivsion are already NP-hard for $r = 1$ in very sparse graphs. Further, they do not admit algorithms with running time $2^{o(\mathbf{tw}^2)} n^{O(1)}$ when parameterized by the treewidth of the input graph for $r \geq 2$ unless ETH fails.
Closing the Blinds: Four Strategies for Protecting Smart Home Privacy from Network Observers
Venue: arXiv cs.CR 2017
Year: 2017

The growing market for smart home IoT devices promises new conveniences for consumers while presenting novel challenges for preserving privacy within the home. Specifically, Internet service providers or neighborhood WiFi eavesdroppers can measure Internet traffic rates from smart home devices and infer consumers' private in-home behaviors. Here we propose four strategies that device manufacturers and third parties can take to protect consumers from side-channel traffic rate privacy threats: 1) blocking traffic, 2) concealing DNS, 3) tunneling traffic, and 4) shaping and injecting traffic. We hope that these strategies, and the implementation nuances we discuss, will provide a foundation for the future development of privacy-sensitive smart homes.
Syndrome-Coupled Rate-Compatible Error-Correcting Codes
Venue: arXiv cs.IT 2017
Year: 2017

Rate-compatible error-correcting codes (ECCs), which consist of a set of extended codes, are of practical interest in both wireless communications and data storage. In this work, we first study the lower bounds for rate-compatible ECCs, thus proving the existence of good rate-compatible codes. Then, we propose a general framework for constructing rate-compatible ECCs based on cosets and syndromes of a set of nested linear codes. We evaluate our construction from two points of view. From a combinatorial perspective, we show that we can construct rate-compatible codes with increasing minimum distances. From a probabilistic point of view, we prove that we are able to construct capacity-achieving rate-compatible codes.
Online Signature Verification using Recurrent Neural Network and Length-normalized Path Signature
Venue: arXiv cs.CV 2017
Year: 2017

Inspired by the great success of recurrent neural networks (RNNs) in sequential modeling, we introduce a novel RNN system to improve the performance of online signature verification. The training objective is to directly minimize intra-class variations and to push the distances between skilled forgeries and genuine samples above a given threshold. By back-propagating the training signals, our RNN network produced discriminative features with desired metrics. Additionally, we propose a novel descriptor, called the length-normalized path signature (LNPS), and apply it to online signature verification. LNPS has interesting properties, such as scale invariance and rotation invariance after linear combination, and shows promising results in online signature verification. Experiments on the publicly available SVC-2004 dataset yielded state-of-the-art performance of 2.37% equal error rate (EER).
Using a Hamiltonian cycle problem algorithm to assist in solving difficult instances of Traveling Salesman Problem
Venue: arXiv cs.DS 2017
Year: 2017

We describe a hybrid procedure for solving the traveling salesman problem (TSP) to provable optimality. We first sparsify the instance, and then use a hybrid algorithm that combines a branch-and-cut TSP solver with a Hamiltonian cycle problem solver. We demonstrate that this procedure enables us to solve difficult instances to optimality, including one which had remained unsolved since its construction in 2002.
Beyond Massive-MIMO: The Potential of Positioning with Large Intelligent Surfaces
Venue: arXiv cs.IT 2017
Year: 2017

We consider the potential for positioning with a system where antenna arrays are deployed as a large intelligent surface (LIS), which is a newly proposed concept beyond massive-MIMO where future man-made structures are electronically active with integrated electronics and wireless communication making the entire environment \lq\lq{}intelligent\rq\rq{}. In a first step, we derive Fisher-information and Cramér-Rao lower bounds (CRLBs) in closed-form for positioning a terminal located perpendicular to the center of the LIS, whose location we refer to as being on the central perpendicular line (CPL) of the LIS. For a terminal that is not on the CPL, closed-form expressions of the Fisher-information and CRLB seem out of reach, and we alternatively find approximations of them which are shown to be accurate. Under mild conditions, we show that the CRLB for all three Cartesian dimensions ($x$, $y$ and $z$) decreases quadratically in the surface-area of the LIS, except for a terminal exactly on the CPL where the CRLB for the $z$-dimension (distance from the LIS) decreases linearly in the same. In a second step, we analyze the CRLB for positioning when there is an unknown phase $φ$ presented in the analog circuits of the LIS. We then show that the CRLBs are dramatically increased for all three dimensions but decrease in the third-order of the surface-area. Moreover, with an infinitely large LIS the CRLB for the $z$-dimension with an unknown $φ$ is 6 dB higher than the case without phase uncertainty, and the CRLB for estimating $φ$ converges to a constant that is independent of the wavelength $λ$. At last, we extensively discuss the impact of centralized and distributed deployments of LIS, and show that a distributed deployment of LIS can enlarge the coverage for terminal-positioning and improve the overall positioning performance.
Low-Complexity Iterative Algorithms for (Discrete) Compressed Sensing
Venue: arXiv cs.IT 2017
Year: 2017

We consider iterative (`turbo') algorithms for compressed sensing. First, a unified exposition of the different approaches available in the literature is given, thereby enlightening the general principles and main differences. In particular we discuss i) the estimation step (matched filter vs. optimum MMSE estimator), ii) the unbiasing operation (implicitly or explicitly done and equivalent to the calculation of extrinsic information), and iii) thresholding vs. the calculation of soft values. Based on these insights we propose a low-complexity but well-performing variant utilizing a Krylov space approximation of the optimum linear MMSE estimator. The derivations are valid for any probability density of the signal vector. However, numerical results are shown for the discrete case. The novel algorithms shows very good performance and even slightly faster convergence compared to approximative message passing.
Ultra-Reliable and Low Latency Communication in mmWave-Enabled Massive MIMO Networks
Venue: arXiv cs.IT 2017
Year: 2017

Ultra-reliability and low-latency are two key components in 5G networks. In this letter, we investigate the problem of ultra-reliable and low-latency communication (URLLC) in millimeter wave (mmWave)-enabled massive multiple-input multiple-output (MIMO) networks. The problem is cast as a network utility maximization subject to probabilistic latency and reliability constraints. To solve this problem, we resort to the Lyapunov technique whereby a utility-delay control approach is proposed, which adapts to channel variations and queue dynamics. Numerical results demonstrate that our proposed approach ensures reliable communication with a guaranteed probability of 99.99%, and reduces latency by 28.41% and 77.11% as compared to baselines with and without probabilistic latency constraints, respectively.
Spectral-graph Based Classifications: Linear Regression for Classification and Normalized Radial Basis Function Network
Venue: arXiv cs.LG 2017
Year: 2017

Spectral graph theory has been widely applied in unsupervised and semi-supervised learning. In this paper, we find for the first time, to our knowledge, that it also plays a concrete role in supervised classification. It turns out that two classifiers are inherently related to the theory: linear regression for classification (LRC) and normalized radial basis function network (nRBFN), corresponding to linear and nonlinear kernel respectively. The spectral graph theory provides us with a new insight into a fundamental aspect of classification: the tradeoff between fitting error and overfitting risk. With the theory, ideal working conditions for LRC and nRBFN are presented, which ensure not only zero fitting error but also low overfitting risk. For quantitative analysis, two concepts, the fitting error and the spectral risk (indicating overfitting), have been defined. Their bounds for nRBFN and LRC are derived. A special result shows that the spectral risk of nRBFN is lower bounded by the number of classes and upper bounded by the size of radial basis. When the conditions are not met exactly, the classifiers will pursue the minimum fitting error, running into the risk of overfitting. It turns out that $\ell_2$-norm regularization can be applied to control overfitting. Its effect is explored under the spectral context. It is found that the two terms in the $\ell_2$-regularized objective are one-one correspondent to the fitting error and the spectral risk, revealing a tradeoff between the two quantities. Concerning practical performance, we devise a basis selection strategy to address the main problem hindering the applications of (n)RBFN. With the strategy, nRBFN is easy to implement yet flexible. Experiments on 14 benchmark data sets show the performance of nRBFN is comparable to that of SVM, whereas the parameter tuning of nRBFN is much easier, leading to reduction of model selection time.
Atari games and Intel processors
Venue: arXiv cs.DC 2017
Year: 2017

The asynchronous nature of the state-of-the-art reinforcement learning algorithms such as the Asynchronous Advantage Actor-Critic algorithm, makes them exceptionally suitable for CPU computations. However, given the fact that deep reinforcement learning often deals with interpreting visual information, a large part of the train and inference time is spent performing convolutions. In this work we present our results on learning strategies in Atari games using a Convolutional Neural Network, the Math Kernel Library and TensorFlow 0.11rc0 machine learning framework. We also analyze effects of asynchronous computations on the convergence of reinforcement learning algorithms.
How Does Knowledge Come By?
Venue: arXiv cs.CY 2017
Year: 2017

Although the amount of knowledge that the humans possess has been gradually increasing, we still do not know the procedure and conditions that lead to the creation of new knowledge. An understanding of the modus operandi for the creation of knowledge may help in accelerating the existing pace of building knowledge. Our state of ignorance regarding various aspects of the process of knowledge building is highlighted by the existing literature in the domain. The reason behind it has been our inability to acquire the underlying data of this complex process. However, current time shows great promise of improvements in the knowledge building domain due to the availability of several online knowledge building portals. In this report, we emphasise that these portals act as prototypes for universal knowledge building process. The analysis of big data availed from these portals may equip the knowledge building researchers with the much needed meta-knowledge.
The Web Centipede: Understanding How Web Communities Influence Each Other Through the Lens of Mainstream and Alternative News Sources
Venue: arXiv cs.SI 2017
Year: 2017

As the number and the diversity of news outlets on the Web grow, so does the opportunity for "alternative" sources of information to emerge. Using large social networks like Twitter and Facebook, misleading, false, or agenda-driven information can quickly and seamlessly spread online, deceiving people or influencing their opinions. Also, the increased engagement of tightly knit communities, such as Reddit and 4chan, further compounds the problem, as their users initiate and propagate alternative information, not only within their own communities, but also to different ones as well as various social media. In fact, these platforms have become an important piece of the modern information ecosystem, which, thus far, has not been studied as a whole. In this paper, we begin to fill this gap by studying mainstream and alternative news shared on Twitter, Reddit, and 4chan. By analyzing millions of posts around several axes, we measure how mainstream and alternative news flows between these platforms. Our results indicate that alt-right communities within 4chan and Reddit can have a surprising level of influence on Twitter, providing evidence that "fringe" communities often succeed in spreading alternative news to mainstream social networks and the greater Web.
Parameter Adaptation and Criticality in Particle Swarm Optimization
Venue: arXiv cs.NE 2017
Year: 2017

Generality is one of the main advantages of heuristic algorithms, as such, multiple parameters are exposed to the user with the objective of allowing them to shape the algorithms to their specific needs. Parameter selection, therefore, becomes an intrinsic problem of every heuristic algorithm. Selecting good parameter values relies not only on knowledge related to the problem at hand, but to the algorithms themselves. This research explores the usage of self-organized criticality to reduce user interaction in the process of selecting suitable parameters for particle swarm optimization (PSO) heuristics. A particle swarm variant (named Adaptive PSO) with self-organized criticality is developed and benchmarked against the standard PSO. Criticality is observed in the dynamic behaviour of this swarm and excellent results are observed in the long run. In contrast with the standard PSO, the Adaptive PSO does not stagnate at any point in time, balancing the concepts of exploration and exploitation better. A software platform for experimenting with particle swarms, called PSO Laboratory, is also developed. This software is used to test the standard PSO as well as all other PSO variants developed in the process of creating the Adaptive PSO. As the software is intended to be of aid to future and related research, special attention has been put in the development of a friendly graphical user interface. Particle swarms are executed in real time, allowing users to experiment by changing parameters on-the-fly.
Segmentation of 3D High-frequency Ultrasound Images of Human Lymph Nodes Using Graph Cut with Energy Functional Adapted to Local Intensity Distribution
Venue: arXiv cs.CV 2017
Year: 2017

Previous studies by our group have shown that three-dimensional high-frequency quantitative ultrasound methods have the potential to differentiate metastatic lymph nodes from cancer-free lymph nodes dissected from human cancer patients. To successfully perform these methods inside the lymph node parenchyma, an automatic segmentation method is highly desired to exclude the surrounding thin layer of fat from quantitative ultrasound processing and accurately correct for ultrasound attenuation. In high-frequency ultrasound images of lymph nodes, the intensity distribution of lymph node parenchyma and fat varies spatially because of acoustic attenuation and focusing effects. Thus, the intensity contrast between two object regions (e.g., lymph node parenchyma and fat) is also spatially varying. In our previous work, nested graph cut demonstrated its ability to simultaneously segment lymph node parenchyma, fat, and the outer phosphate-buffered saline bath even when some boundaries are lost because of acoustic attenuation and focusing effects. This paper describes a novel approach called graph cut with locally adaptive energy to further deal with spatially varying distributions of lymph node parenchyma and fat caused by inhomogeneous acoustic attenuation. The proposed method achieved Dice similarity coefficients of 0.937+-0.035 when compared to expert manual segmentation on a representative dataset consisting of 115 three-dimensional lymph node images obtained from colorectal cancer patients.
Simultaneous Multiple Surface Segmentation Using Deep Learning
Venue: arXiv cs.CV 2017
Year: 2017

The task of automatically segmenting 3-D surfaces representing boundaries of objects is important for quantitative analysis of volumetric images, and plays a vital role in biomedical image analysis. Recently, graph-based methods with a global optimization property have been developed and optimized for various medical imaging applications. Despite their widespread use, these require human experts to design transformations, image features, surface smoothness priors, and re-design for a different tissue, organ or imaging modality. Here, we propose a Deep Learning based approach for segmentation of the surfaces in volumetric medical images, by learning the essential features and transformations from training data, without any human expert intervention. We employ a regional approach to learn the local surface profiles. The proposed approach was evaluated on simultaneous intraretinal layer segmentation of optical coherence tomography (OCT) images of normal retinas and retinas affected by age related macular degeneration (AMD). The proposed approach was validated on 40 retina OCT volumes including 20 normal and 20 AMD subjects. The experiments showed statistically significant improvement in accuracy for our approach compared to state-of-the-art graph based optimal surface segmentation with convex priors (G-OSC). A single Convolution Neural Network (CNN) was used to learn the surfaces for both normal and diseased images. The mean unsigned surface positioning errors obtained by G-OSC method 2.31 voxels (95% CI 2.02-2.60 voxels) was improved to $1.27$ voxels (95% CI 1.14-1.40 voxels) using our new approach. On average, our approach takes 94.34 s, requiring 95.35 MB memory, which is much faster than the 2837.46 s and 6.87 GB memory required by the G-OSC method on the same computer system.
Sparse Coding on Stereo Video for Object Detection
Venue: arXiv cs.CV 2017
Year: 2017

Deep Convolutional Neural Networks (DCNN) require millions of labeled training examples for image classification and object detection tasks, which restrict these models to domains where such datasets are available. In this paper, we explore the use of unsupervised sparse coding applied to stereo-video data to help alleviate the need for large amounts of labeled data. We show that replacing a typical supervised convolutional layer with an unsupervised sparse-coding layer within a DCNN allows for better performance on a car detection task when only a limited number of labeled training examples is available. Furthermore, the network that incorporates sparse coding allows for more consistent performance over varying initializations and ordering of training examples when compared to a fully supervised DCNN. Finally, we compare activations between the unsupervised sparse-coding layer and the supervised convolutional layer, and show that the sparse representation exhibits an encoding that is depth selective, whereas encodings from the convolutional layer do not exhibit such selectivity. These result indicates promise for using unsupervised sparse-coding approaches in real-world computer vision tasks in domains with limited labeled training data.
Local Information with Feedback Perturbation Suffices for Dictionary Learning in Neural Circuits
Venue: arXiv cs.LG 2017
Year: 2017

While the sparse coding principle can successfully model information processing in sensory neural systems, it remains unclear how learning can be accomplished under neural architectural constraints. Feasible learning rules must rely solely on synaptically local information in order to be implemented on spatially distributed neurons. We describe a neural network with spiking neurons that can address the aforementioned fundamental challenge and solve the L1-minimizing dictionary learning problem, representing the first model able to do so. Our major innovation is to introduce feedback synapses to create a pathway to turn the seemingly non-local information into local ones. The resulting network encodes the error signal needed for learning as the change of network steady states caused by feedback, and operates akin to the classical stochastic gradient descent method.
Towards Communication-Aware Robust Topologies
Venue: arXiv cs.NI 2017
Year: 2017

We currently witness the emergence of interesting new network topologies optimized towards the traffic matrices they serve, such as demand-aware datacenter interconnects (e.g., ProjecToR) and demand-aware overlay networks (e.g., SplayNets). This paper introduces a formal framework and approach to reason about and design such topologies. We leverage a connection between the communication frequency of two nodes and the path length between them in the network, which depends on the entropy of the communication matrix. Our main contribution is a novel robust, yet sparse, family of network topologies which guarantee an expected path length that is proportional to the entropy of the communication patterns.