OpenRaiser/Intern-Atlas huggingface dataset:
A Strongly Quasiconvex PAC-Bayesian Bound
Venue: arXiv cs.LG 2017
Year: 2017

We propose a new PAC-Bayesian bound and a way of constructing a hypothesis space, so that the bound is convex in the posterior distribution and also convex in a trade-off parameter between empirical performance of the posterior distribution and its complexity. The complexity is measured by the Kullback-Leibler divergence to a prior. We derive an alternating procedure for minimizing the bound. We show that the bound can be rewritten as a one-dimensional function of the trade-off parameter and provide sufficient conditions under which the function has a single global minimum. When the conditions are satisfied the alternating minimization is guaranteed to converge to the global minimum of the bound. We provide experimental results demonstrating that rigorous minimization of the bound is competitive with cross-validation in tuning the trade-off between complexity and empirical performance. In all our experiments the trade-off turned to be quasiconvex even when the sufficient conditions were violated.
DEBH: Detection and Elimination Black Holes in Mobile Ad Hoc Network
Venue: arXiv cs.NI 2017
Year: 2017

Security in Mobile Ad hoc Network (MANET) is one of the key challenges due to its special features e.g. hop-by-hop communications, dynamic topology, and open network boundary that received tremendous attention by scholars. Traditional security methods are not applicable in MANET due to its special properties. In this paper, a novel approach called Detecting and Eliminating Black Holes (DEBH) is proposed that uses a data control packet and an additional Black hole Check (BCh) table for detecting and eliminating malicious nodes. Benefiting from trustable nodes, the processing overhead of the security method decreases by passing time. Ad hoc On-demand Distance Vector (AODV) routing protocol is used as the routing protocol in our design. After finding the freshest path using AODV, our design checks the safety of selected path. In case of detecting any malicious node, it is isolated from the entire network by broadcasting a packet that contains the ID of malicious nodes. Simulation results show that DEBH increases network throughput and decreases packet overhead and delay in comparison with other studied approaches. Moreover, DEBH is able to detect all active malicious nodes which generates fault routing information.
Infinite-Label Learning with Semantic Output Codes
Venue: arXiv cs.LG 2017
Year: 2017

We develop a new statistical machine learning paradigm, named infinite-label learning, to annotate a data point with more than one relevant labels from a candidate set, which pools both the finite labels observed at training and a potentially infinite number of previously unseen labels. The infinite-label learning fundamentally expands the scope of conventional multi-label learning, and better models the practical requirements in various real-world applications, such as image tagging, ads-query association, and article categorization. However, how can we learn a labeling function that is capable of assigning to a data point the labels omitted from the training set? To answer the question, we seek some clues from the recent work on zero-shot learning, where the key is to represent a class/label by a vector of semantic codes, as opposed to treating them as atomic labels. We validate the infinite-label learning by a PAC bound in theory and some empirical studies on both synthetic and real data.
Automatic Synchronization of Multi-User Photo Galleries
Venue: arXiv cs.MM 2017
Year: 2017

In this paper we address the issue of photo galleries synchronization, where pictures related to the same event are collected by different users. Existing solutions to address the problem are usually based on unrealistic assumptions, like time consistency across photo galleries, and often heavily rely on heuristics, limiting therefore the applicability to real-world scenarios. We propose a solution that achieves better generalization performance for the synchronization task compared to the available literature. The method is characterized by three stages: at first, deep convolutional neural network features are used to assess the visual similarity among the photos; then, pairs of similar photos are detected across different galleries and used to construct a graph; eventually, a probabilistic graphical model is used to estimate the temporal offset of each pair of galleries, by traversing the minimum spanning tree extracted from this graph. The experimental evaluation is conducted on four publicly available datasets covering different types of events, demonstrating the strength of our proposed method. A thorough discussion of the obtained results is provided for a critical assessment of the quality in synchronization.
Bulk-synchronous pseudo-streaming algorithms for many-core accelerators
Venue: arXiv cs.DC 2017
Year: 2017

The bulk-synchronous parallel (BSP) model provides a framework for writing parallel programs with predictable performance. In this paper we extend the BSP model to support what we will call pseudo-streaming algorithms for accelerators. We also generalize the BSP cost function to these algorithms, so that it is possible to predict the running time for programs targeting many-core accelerators and to identify possible bottlenecks. Several examples of algorithms within this new framework will be explored. We extend the BSPlib standard by proposing a small number of new BSP primitives to create and use streams in a portable way. We will introduce a software library called Epiphany BSP that implements these ideas for the Parallella development board. Finally we will give experimental results for pseudo-streaming algorithms on the Parallella platform.
TriCheck: Memory Model Verification at the Trisection of Software, Hardware, and ISA
Venue: arXiv cs.AR 2017
Year: 2017

Memory consistency models (MCMs) which govern inter-module interactions in a shared memory system, are a significant, yet often under-appreciated, aspect of system design. MCMs are defined at the various layers of the hardware-software stack, requiring thoroughly verified specifications, compilers, and implementations at the interfaces between layers. Current verification techniques evaluate segments of the system stack in isolation, such as proving compiler mappings from a high-level language (HLL) to an ISA or proving validity of a microarchitectural implementation of an ISA. This paper makes a case for full-stack MCM verification and provides a toolflow, TriCheck, capable of verifying that the HLL, compiler, ISA, and implementation collectively uphold MCM requirements. The work showcases TriCheck's ability to evaluate a proposed ISA MCM in order to ensure that each layer and each mapping is correct and complete. Specifically, we apply TriCheck to the open source RISC-V ISA, seeking to verify accurate, efficient, and legal compilations from C11. We uncover under-specifications and potential inefficiencies in the current RISC-V ISA documentation and identify possible solutions for each. As an example, we find that a RISC-V-compliant microarchitecture allows 144 outcomes forbidden by C11 to be observed out of 1,701 litmus tests examined. Overall, this paper demonstrates the necessity of full-stack verification for detecting MCM-related bugs in the hardware-software stack.
3D Object Proposals using Stereo Imagery for Accurate Object Class Detection
Venue: arXiv cs.CV 2017
Year: 2017

The goal of this paper is to perform 3D object detection in the context of autonomous driving. Our method first aims at generating a set of high-quality 3D object proposals by exploiting stereo imagery. We formulate the problem as minimizing an energy function that encodes object size priors, placement of objects on the ground plane as well as several depth informed features that reason about free space, point cloud densities and distance to the ground. We then exploit a CNN on top of these proposals to perform object detection. In particular, we employ a convolutional neural net (CNN) that exploits context and depth information to jointly regress to 3D bounding box coordinates and object pose. Our experiments show significant performance gains over existing RGB and RGB-D object proposal methods on the challenging KITTI benchmark. When combined with the CNN, our approach outperforms all existing results in object detection and orientation estimation tasks for all three KITTI object classes. Furthermore, we experiment also with the setting where LIDAR information is available, and show that using both LIDAR and stereo leads to the best result.
A Stackelberg Game Approach for Two-Level Distributed Energy Management in Smart Grids
Venue: arXiv cs.SY 2017
Year: 2017

The pursuit of sustainability motivates microgrids that depend on distributed resources to produce more renewable energies. An efficient operation and planning relies on a holistic framework that takes into account the interdependent decision-making of the generators of the existing power grids and the distributed resources of the microgrid in the integrated system. To this end, we use a Stackelberg game-theoretic framework to study the interactions between generators (leaders) and microgrids (followers). Entities on both sides make strategic decisions on the amount of power generation to maximize their payoffs. Our framework not only takes into account the economic factors but also incorporates the stability and efficiency of the smart grid, such as the power flow constraints and voltage angle regulations. We develop three update schemes for both generators and microgrids, respectively, and among which a fully distributed algorithm enabled by phasor measurement units is presented. The distributed algorithm merely requires the information of voltage angles at local buses for updates, and its convergence to the unique equilibrium is shown. We further develop the implementation architectures of the update schemes in the smart grid. Finally, case studies are used to corroborate the effectiveness of the proposed algorithms.
The $(h,k)$-Server Problem on Bounded Depth Trees
Venue: arXiv cs.DS 2017
Year: 2017

We study the $k$-server problem in the resource augmentation setting i.e., when the performance of the online algorithm with $k$ servers is compared to the offline optimal solution with $h \leq k$ servers. The problem is very poorly understood beyond uniform metrics. For this special case, the classic $k$-server algorithms are roughly $(1+1/ε)$-competitive when $k=(1+ε) h$, for any $ε>0$. Surprisingly however, no $o(h)$-competitive algorithm is known even for HSTs of depth 2 and even when $k/h$ is arbitrarily large. We obtain several new results for the problem. First we show that the known $k$-server algorithms do not work even on very simple metrics. In particular, the Double Coverage algorithm has competitive ratio $Ω(h)$ irrespective of the value of $k$, even for depth-2 HSTs. Similarly the Work Function Algorithm, that is believed to be optimal for all metric spaces when $k=h$, has competitive ratio $Ω(h)$ on depth-3 HSTs even if $k=2h$. Our main result is a new algorithm that is $O(1)$-competitive for constant depth trees, whenever $k =(1+ε)h$ for any $ε> 0$. Finally, we give a general lower bound that any deterministic online algorithm has competitive ratio at least 2.4 even for depth-2 HSTs and when $k/h$ is arbitrarily large. This gives a surprising qualitative separation between uniform metrics and depth-2 HSTs for the $(h,k)$-server problem, and gives the strongest known lower bound for the problem on general metrics.
Ternary Neural Networks for Resource-Efficient AI Applications
Venue: arXiv cs.LG 2017
Year: 2017

The computation and storage requirements for Deep Neural Networks (DNNs) are usually high. This issue limits their deployability on ubiquitous computing devices such as smart phones, wearables and autonomous drones. In this paper, we propose ternary neural networks (TNNs) in order to make deep learning more resource-efficient. We train these TNNs using a teacher-student approach based on a novel, layer-wise greedy methodology. Thanks to our two-stage training procedure, the teacher network is still able to use state-of-the-art methods such as dropout and batch normalization to increase accuracy and reduce training time. Using only ternary weights and activations, the student ternary network learns to mimic the behavior of its teacher network without using any multiplication. Unlike its -1,1 binary counterparts, a ternary neural network inherently prunes the smaller weights by setting them to zero during training. This makes them sparser and thus more energy-efficient. We design a purpose-built hardware architecture for TNNs and implement it on FPGA and ASIC. We evaluate TNNs on several benchmark datasets and demonstrate up to 3.1x better energy efficiency with respect to the state of the art while also improving accuracy.
Crowdsourcing with Unsure Option
Venue: arXiv cs.AI 2017
Year: 2017

One of the fundamental problems in crowdsourcing is the trade-off between the number of the workers needed for high-accuracy aggregation and the budget to pay. For saving budget, it is important to ensure high quality of the crowd-sourced labels, hence the total cost on label collection will be reduced. Since the self-confidence of the workers often has a close relationship with their abilities, a possible way for quality control is to request the workers to return the labels only when they feel confident, by means of providing unsure option to them. On the other hand, allowing workers to choose unsure option also leads to the potential danger of budget waste. In this work, we propose the analysis towards understanding when providing the unsure option indeed leads to significant cost reduction, as well as how the confidence threshold is set. We also propose an online mechanism, which is alternative for threshold selection when the estimation of the crowd ability distribution is difficult.
Deep-Anomaly: Fully Convolutional Neural Network for Fast Anomaly Detection in Crowded Scenes
Venue: arXiv cs.CV 2017
Year: 2017

The detection of abnormal behaviours in crowded scenes has to deal with many challenges. This paper presents an efficient method for detection and localization of anomalies in videos. Using fully convolutional neural networks (FCNs) and temporal data, a pre-trained supervised FCN is transferred into an unsupervised FCN ensuring the detection of (global) anomalies in scenes. High performance in terms of speed and accuracy is achieved by investigating the cascaded detection as a result of reducing computation complexities. This FCN-based architecture addresses two main tasks, feature representation and cascaded outlier detection. Experimental results on two benchmarks suggest that detection and localization of the proposed method outperforms existing methods in terms of accuracy.
A Deep Multi-Level Network for Saliency Prediction
Venue: arXiv cs.CV 2017
Year: 2017

This paper presents a novel deep architecture for saliency prediction. Current state of the art models for saliency prediction employ Fully Convolutional networks that perform a non-linear combination of features extracted from the last convolutional layer to predict saliency maps. We propose an architecture which, instead, combines features extracted at different levels of a Convolutional Neural Network (CNN). Our model is composed of three main blocks: a feature extraction CNN, a feature encoding network, that weights low and high level feature maps, and a prior learning network. We compare our solution with state of the art saliency models on two public benchmarks datasets. Results show that our model outperforms under all evaluation metrics on the SALICON dataset, which is currently the largest public dataset for saliency prediction, and achieves competitive results on the MIT300 benchmark.
OpenCL/CUDA algorithms for parallel decoding of any irregular LDPC code using GPU
Venue: arXiv cs.IT 2017
Year: 2017

The development of multicore architectures supporting parallel data processing has led to a paradigm shift, which affects communication systems significantly. This article provides a scalable parallel approach of an iterative LDPC decoder, presented in a tutorial-based style. It is suitable for decoding any irregular LDPC code without the limitation of the maximum node degree, and it includes a parallel calculation of the syndrome. This is the main difference from algorithms presented so far. The proposed approach can be implemented in applications supporting massive parallel computing, such as GPU or FPGA devices. The implementation of the LDPC decoder with the use the OpenCL and CUDA frameworks is discussed and the performance evaluation is given at the end of this contribution.
MC^2: A Two-Phase Algorithm for Leveraged Matrix Completion
Venue: arXiv cs.IT 2017
Year: 2017

Leverage scores, loosely speaking, reflect the importance of the rows and columns of a matrix. Ideally, given the leverage scores of a rank-$r$ matrix $M\in\mathbb{R}^{n\times n}$, that matrix can be reliably completed from just $O(rn\log^{2}n)$ samples if the samples are chosen randomly from a nonuniform distribution induced by the leverage scores. In practice, however, the leverage scores are often unknown a priori. As such, the sample complexity in uniform matrix completion---using uniform random sampling---increases to $O(η(M)\cdot rn\log^{2}n)$, where $η(M)$ is the largest leverage score of $M$. In this paper, we propose a two-phase algorithm called MC$^2$ for matrix completion: in the first phase, the leverage scores are estimated based on uniform random samples, and then in the second phase the matrix is resampled nonuniformly based on the estimated leverage scores and then completed. For well-conditioned matrices, the total sample complexity of MC$^2$ is no worse than uniform matrix completion, and for certain classes of well-conditioned matrices---namely, reasonably coherent matrices whose leverage scores exhibit mild decay---MC$^2$ requires substantially fewer samples. Numerical simulations suggest that the algorithm outperforms uniform matrix completion in a broad class of matrices, and in particular, is much less sensitive to the condition number than our theory currently requires.
Fundamental Storage-Latency Tradeoff in Cache-Aided MIMO Interference Networks
Venue: arXiv cs.IT 2017
Year: 2017

Caching is an effective technique to improve user perceived experience for content delivery in wireless networks. Wireless caching differs from traditional web caching in that it can exploit the broadcast nature of wireless medium and hence opportunistically change the network topologies. This paper studies a cache-aided MIMO interference network with 3 transmitters each equipped with M antennas and 3 receivers each with N antennas. With caching at both the transmitter and receiver sides, the network is changed to hybrid forms of MIMO broadcast channel, MIMO X channel, and MIMO multicast channels. We analyze the degrees of freedom (DoF) of these new channel models using practical interference management schemes. Based on the collective use of these DoF results, we then obtain an achievable normalized delivery time (NDT) of the network, an information-theoretic metric that evaluates the worst-case delivery time at given cache sizes. The obtained NDT is for arbitrary M, N and any feasible cache sizes. It is shown to be optimal in certain cases and within a multiplicative gap of 3 from the optimum in other cases. The extension to the network with arbitrary number of transmitters and receivers is also discussed.
Vertex coloring of graphs via phase dynamics of coupled oscillatory networks
Venue: arXiv cs.ET 2017
Year: 2017

While Boolean logic has been the backbone of digital information processing, there are classes of computationally hard problems wherein this conventional paradigm is fundamentally inefficient. Vertex coloring of graphs, belonging to the class of combinatorial optimization represents such a problem; and is well studied for its wide spectrum of applications in data sciences, life sciences, social sciences and engineering and technology. This motivates alternate, and more efficient non-Boolean pathways to their solution. Here, we demonstrate a coupled relaxation oscillator based dynamical system that exploits the insulator-metal transition in vanadium dioxide (VO2), to efficiently solve the vertex coloring of graphs. By harnessing the natural analogue between optimization, pertinent to graph coloring solutions, and energy minimization processes in highly parallel, interconnected dynamical systems, we harness the physical manifestation of the latter process to approximate the optimal coloring of k-partite graphs. We further indicate a fundamental connection between the eigen properties of a linear dynamical system and the spectral algorithms that can solve approximate graph coloring. Our work not only elucidates a physics-based computing approach but also presents tantalizing opportunities for building customized analog co-processors for solving hard problems efficiently.
Optimality of the Johnson-Lindenstrauss Lemma
Venue: arXiv cs.IT 2017
Year: 2017

For any integers $d, n \geq 2$ and $1/({\min\{n,d\}})^{0.4999} < \varepsilon<1$, we show the existence of a set of $n$ vectors $X\subset \mathbb{R}^d$ such that any embedding $f:X\rightarrow \mathbb{R}^m$ satisfying $$ \forall x,y\in X,\ (1-\varepsilon)\|x-y\|_2^2\le \|f(x)-f(y)\|_2^2 \le (1+\varepsilon)\|x-y\|_2^2 $$ must have $$ m = Ω(\varepsilon^{-2} \lg n). $$ This lower bound matches the upper bound given by the Johnson-Lindenstrauss lemma [JL84]. Furthermore, our lower bound holds for nearly the full range of $\varepsilon$ of interest, since there is always an isometric embedding into dimension $\min\{d, n\}$ (either the identity map, or projection onto $\mathop{span}(X)$). Previously such a lower bound was only known to hold against linear maps $f$, and not for such a wide range of parameters $\varepsilon, n, d$ [LN16]. The best previously known lower bound for general $f$ was $m = Ω(\varepsilon^{-2}\lg n/\lg(1/\varepsilon))$ [Wel74, Lev83, Alo03], which is suboptimal for any $\varepsilon = o(1)$.
Optimizing Codes for Source Separation in Color Image Demosaicing and Compressive Video Recovery
Venue: arXiv cs.CV 2017
Year: 2017

There exist several applications in image processing (eg: video compressed sensing [Hitomi, Y. et al, "Video from a single coded exposure photograph using a learned overcomplete dictionary"] and color image demosaicing [Moghadam, A. A. et al, "Compressive Framework for Demosaicing of Natural Images"]) which require separation of constituent images given measurements in the form of a coded superposition of those images. Physically practical code patterns in these applications are non-negative, systematically structured, and do not always obey the nice incoherence properties of other patterns such as Gaussian codes, which can adversely affect reconstruction performance. The contribution of this paper is to design code patterns for video compressed sensing and demosaicing by minimizing the mutual coherence of the matrix $\boldsymbol{ΦΨ}$ where $\boldsymbolΦ$ represents the sensing matrix created from the code, and $\boldsymbolΨ$ is the signal representation matrix. Our main contribution is that we explicitly take into account the special structure of those code patterns as required by these applications: (1)~non-negativity, (2)~block-diagonal nature, and (3)~circular shifting. In particular, the last property enables for accurate and seamless patch-wise reconstruction for some important compressed sensing architectures.