OpenRaiser/Intern-Atlas huggingface dataset:
A Generic Deep Architecture for Single Image Reflection Removal and Image Smoothing
Venue: arXiv cs.CV 2018
Year: 2018

This paper proposes a deep neural network structure that exploits edge information in addressing representative low-level vision tasks such as layer separation and image filtering. Unlike most other deep learning strategies applied in this context, our approach tackles these challenging problems by estimating edges and reconstructing images using only cascaded convolutional layers arranged such that no handcrafted or application-specific image-processing components are required. We apply the resulting transferrable pipeline to two different problem domains that are both sensitive to edges, namely, single image reflection removal and image smoothing. For the former, using a mild reflection smoothness assumption and a novel synthetic data generation method that acts as a type of weak supervision, our network is able to solve much more difficult reflection cases that cannot be handled by previous methods. For the latter, we also exceed the state-of-the-art quantitative and qualitative results by wide margins. In all cases, the proposed framework is simple, fast, and easy to transfer across disparate domains.
Efficient Systematic Encoding of Non-binary VT Codes
Venue: arXiv cs.IT 2018
Year: 2018

Varshamov-Tenengolts (VT) codes are a class of codes which can correct a single deletion or insertion with a linear-time decoder. This paper addresses the problem of efficient encoding of non-binary VT codes, defined over an alphabet of size $q >2$. We propose a simple linear-time encoding method to systematically map binary message sequences onto VT codewords. The method provides a new lower bound on the size of $q$-ary VT codes of length $n$.
Solving Hard Stable Matching Problems Involving Groups of Similar Agents
Venue: arXiv cs.GT 2018
Year: 2018

Many important stable matching problems are known to be NP-hard, even when strong restrictions are placed on the input. In this paper we seek to identify structural properties of instances of stable matching problems which will allow us to design efficient algorithms using elementary techniques. We focus on the setting in which all agents involved in some matching problem can be partitioned into k different types, where the type of an agent determines his or her preferences, and agents have preferences over types (which may be refined by more detailed preferences within a single type). This situation would arise in practice if agents form preferences solely based on some small collection of agents' attributes. We also consider a generalisation in which each agent may consider some small collection of other agents to be exceptional, and rank these in a way that is not consistent with their types; this could happen in practice if agents have prior contact with a small number of candidates. We show that (for the case without exceptions), several well-studied NP-hard stable matching problems including Max SMTI (that of finding the maximum cardinality stable matching in an instance of stable marriage with ties and incomplete lists) belong to the parameterised complexity class FPT when parameterised by the number of different types of agents needed to describe the instance. For Max SMTI this tractability result can be extended to the setting in which each agent promotes at most one `exceptional' candidate to the top of his/her list (when preferences within types are not refined), but the problem remains NP-hard if preference lists can contain two or more exceptions and the exceptional candidates can be placed anywhere in the preference lists, even if the number of types is bounded by a constant.
Taxonomy of Blockchain Technologies. Principles of Identification and Classification
Venue: arXiv cs.CY 2018
Year: 2018

A comparative study across the most widely known blockchain technologies is conducted with a bottom-up approach. Blockchains are disentangled into building blocks. Each building block is then hierarchically classified in main and subcomponents. Then, alternative layouts for the subcomponents are identified and compared between them. Finally, a taxonomy tree summarises the study and provides a navigation tool across different blockchain architectural configurations.
Finding Subcube Heavy Hitters in Analytics Data Streams
Venue: arXiv cs.DS 2018
Year: 2018

Data streams typically have items of large number of dimensions. We study the fundamental heavy-hitters problem in this setting. Formally, the data stream consists of $d$-dimensional items $x_1,\ldots,x_m \in [n]^d$. A $k$-dimensional subcube $T$ is a subset of distinct coordinates $\{ T_1,\cdots,T_k \} \subseteq [d]$. A subcube heavy hitter query ${\rm Query}(T,v)$, $v \in [n]^k$, outputs YES if $f_T(v) \geq γ$ and NO if $f_T(v) < γ/4$, where $f_T$ is the ratio of number of stream items whose coordinates $T$ have joint values $v$. The all subcube heavy hitters query ${\rm AllQuery}(T)$ outputs all joint values $v$ that return YES to ${\rm Query}(T,v)$. The one dimensional version of this problem where $d=1$ was heavily studied in data stream theory, databases, networking and signal processing. The subcube heavy hitters problem is applicable in all these cases. We present a simple reservoir sampling based one-pass streaming algorithm to solve the subcube heavy hitters problem in $\tilde{O}(kd/γ)$ space. This is optimal up to poly-logarithmic factors given the established lower bound. In the worst case, this is $Θ(d^2/γ)$ which is prohibitive for large $d$, and our goal is to circumvent this quadratic bottleneck. Our main contribution is a model-based approach to the subcube heavy hitters problem. In particular, we assume that the dimensions are related to each other via the Naive Bayes model, with or without a latent dimension. Under this assumption, we present a new two-pass, $\tilde{O}(d/γ)$-space algorithm for our problem, and a fast algorithm for answering ${\rm AllQuery}(T)$ in $O(k/γ^2)$ time. Our work develops the direction of model-based data stream analysis, with much that remains to be explored.
Wearable Communications in 5G: Challenges and Enabling Technologies
Venue: arXiv cs.NI 2018
Year: 2018

As wearable devices become more ingrained in our daily lives, traditional communication networks primarily designed for human being-oriented applications are facing tremendous challenges. The upcoming 5G wireless system aims to support unprecedented high capacity, low latency, and massive connectivity. In this article, we evaluate key challenges in wearable communications. A cloud/edge communication architecture that integrates the cloud radio access network, software defined network, device to device communications, and cloud/edge technologies is presented. Computation offloading enabled by this multi-layer communications architecture can offload computation-excessive and latency-stringent applications to nearby devices through device to device communications or to nearby edge nodes through cellular or other wireless technologies. Critical issues faced by wearable communications such as short battery life, limited computing capability, and stringent latency can be greatly alleviated by this cloud/edge architecture. Together with the presented architecture, current transmission and networking technologies, including non-orthogonal multiple access, mobile edge computing, and energy harvesting, can greatly enhance the performance of wearable communication in terms of spectral efficiency, energy efficiency, latency, and connectivity.
The Linear Complexity of a Class of Binary Sequences With Optimal Autocorrelation
Venue: arXiv cs.IT 2018
Year: 2018

Binary sequences with optimal autocorrelation and large linear complexity have important applications in cryptography and communications. Very recently, a class of binary sequences of period $4p$ with optimal autocorrelation was proposed via interleaving four suitable Ding-Helleseth-Lam sequences (Des. Codes Cryptogr., DOI 10.1007/s10623-017-0398-5), where $p$ is an odd prime with $p\equiv 1(\bmod~4)$. The objective of this paper is to determine the minimal polynomial and the linear complexity of this class of binary optimal sequences via sequence polynomial approach. It turns out that this class of sequences has quite good linear complexity.
On privacy amplification, lossy compression, and their duality to channel coding
Venue: arXiv cs.IT 2018
Year: 2018

We examine the task of privacy amplification from information-theoretic and coding-theoretic points of view. In the former, we give a one-shot characterization of the optimal rate of privacy amplification against classical adversaries in terms of the optimal type-II error in asymmetric hypothesis testing. This formulation can be easily computed to give finite-blocklength bounds and turns out to be equivalent to smooth min-entropy bounds by Renner and Wolf [Asiacrypt 2005] and Watanabe and Hayashi [ISIT 2013], as well as a bound in terms of the $E_γ$ divergence by Yang, Schaefer, and Poor [arXiv:1706.03866 [cs.IT]]. In the latter, we show that protocols for privacy amplification based on linear codes can be easily repurposed for channel simulation. Combined with known relations between channel simulation and lossy source coding, this implies that privacy amplification can be understood as a basic primitive for both channel simulation and lossy compression. Applied to symmetric channels or lossy compression settings, our construction leads to proto- cols of optimal rate in the asymptotic i.i.d. limit. Finally, appealing to the notion of channel duality recently detailed by us in [IEEE Trans. Info. Theory 64, 577 (2018)], we show that linear error-correcting codes for symmetric channels with quantum output can be transformed into linear lossy source coding schemes for classical variables arising from the dual channel. This explains a "curious duality" in these problems for the (self-dual) erasure channel observed by Martinian and Yedidia [Allerton 2003; arXiv:cs/0408008] and partly anticipates recent results on optimal lossy compression by polar and low-density generator matrix codes.
The Spatial Outage Capacity of Wireless Networks
Venue: arXiv cs.IT 2018
Year: 2018

We address a fundamental question in wireless networks that, surprisingly, has not been studied before: what is the maximum density of concurrently active links that satisfy a certain outage constraint? We call this quantity the spatial outage capacity (SOC), give a rigorous definition, and analyze it for Poisson bipolar networks with ALOHA. Specifically, we provide exact analytical and approximate expressions for the density of links satisfying an outage constraint and give simple upper and lower bounds on the SOC. In the high-reliability regime where the target outage probability is close to zero, we obtain an exact closed-form expression of the SOC, which reveals the interesting and perhaps counter-intuitive result that all transmitters need to be always active to achieve the SOC, i.e., the transmit probability needs to be set to 1 to achieve the SOC.
Adaptive Sampling for Rapidly Matching Histograms
Venue: arXiv cs.DB 2018
Year: 2018

In exploratory data analysis, analysts often have a need to identify histograms that possess a specific distribution, among a large class of candidate histograms, e.g., find countries whose income distribution is most similar to that of Greece. This distribution could be a new one that the user is curious about, or a known distribution from an existing histogram visualization. At present, this process of identification is brute-force, requiring the manual generation and evaluation of a large number of histograms. We present FastMatch: an end-to-end approach for interactively retrieving the histogram visualizations most similar to a user-specified target, from a large collection of histograms. The primary technical contribution underlying FastMatch is a probabilistic algorithm, HistSim, a theoretically sound sampling-based approach to identify the top-$k$ closest histograms under $\ell_1$ distance. While HistSim can be used independently, within FastMatch we couple HistSim with a novel system architecture that is aware of practical considerations, employing asynchronous block-based sampling policies, building on lightweight sampling engines developed in recent work. FastMatch obtains near-perfect accuracy with up to $35\times$ speedup over approaches that do not use sampling on several real-world datasets.
Neural Network-based Graph Embedding for Cross-Platform Binary Code Similarity Detection
Venue: arXiv cs.CR 2018
Year: 2018

The problem of cross-platform binary code similarity detection aims at detecting whether two binary functions coming from different platforms are similar or not. It has many security applications, including plagiarism detection, malware detection, vulnerability search, etc. Existing approaches rely on approximate graph matching algorithms, which are inevitably slow and sometimes inaccurate, and hard to adapt to a new task. To address these issues, in this work, we propose a novel neural network-based approach to compute the embedding, i.e., a numeric vector, based on the control flow graph of each binary function, then the similarity detection can be done efficiently by measuring the distance between the embeddings for two functions. We implement a prototype called Gemini. Our extensive evaluation shows that Gemini outperforms the state-of-the-art approaches by large margins with respect to similarity detection accuracy. Further, Gemini can speed up prior art's embedding generation time by 3 to 4 orders of magnitude and reduce the required training time from more than 1 week down to 30 minutes to 10 hours. Our real world case studies demonstrate that Gemini can identify significantly more vulnerable firmware images than the state-of-the-art, i.e., Genius. Our research showcases a successful application of deep learning on computer security problems.
Constructions of Coded Caching Schemes with Flexible Memory Size
Venue: arXiv cs.IT 2018
Year: 2018

Coded caching scheme recently has become quite popular in the wireless network due to its effectively reducing the transmission amount (denote such an amount by $R$) during peak traffic times. However to realize a coded caching scheme, each file must be divided into $F$ packets which usually increases the computation complexity of a coded caching scheme. So we prefer to construct a caching scheme that decreases the order of $F$ for practical implementations. In this paper, we construct four classes of new schemes where two classes can significantly reduce the value of $F$ by increasing a little $R$ comparing with the well known scheme proposed by Maddah-Ali and Niesen, and $F$ in the other two classes grows sub-exponentially with $K$ by sacrificing more $R$. It is worth noting that a tradeoff between $R$ and $F$, which is a hot topic in the field of caching scheme, is proposed by our constructions. In addition, our constructions include all the results constructed by Yan et al., (IEEE Trans. Inf. Theory 63, 5821-5833, 2017) and some main results obtained by Shangguan et al., (arXiv preprint arXiv:1608.03989v1) as the special cases.
Analysis of the Impact of Negative Sampling on Link Prediction in Knowledge Graphs
Venue: arXiv cs.AI 2018
Year: 2018

Knowledge graphs are large, useful, but incomplete knowledge repositories. They encode knowledge through entities and relations which define each other through the connective structure of the graph. This has inspired methods for the joint embedding of entities and relations in continuous low-dimensional vector spaces, that can be used to induce new edges in the graph, i.e., link prediction in knowledge graphs. Learning these representations relies on contrasting positive instances with negative ones. Knowledge graphs include only positive relation instances, leaving the door open for a variety of methods for selecting negative examples. In this paper we present an empirical study on the impact of negative sampling on the learned embeddings, assessed through the task of link prediction. We use state-of-the-art knowledge graph embeddings -- \rescal , TransE, DistMult and ComplEX -- and evaluate on benchmark datasets -- FB15k and WN18. We compare well known methods for negative sampling and additionally propose embedding based sampling methods. We note a marked difference in the impact of these sampling methods on the two datasets, with the "traditional" corrupting positives method leading to best results on WN18, while embedding based methods benefiting the task on FB15k.
Learning Anytime Predictions in Neural Networks via Adaptive Loss Balancing
Venue: arXiv cs.LG 2018
Year: 2018

This work considers the trade-off between accuracy and test-time computational cost of deep neural networks (DNNs) via \emph{anytime} predictions from auxiliary predictions. Specifically, we optimize auxiliary losses jointly in an \emph{adaptive} weighted sum, where the weights are inversely proportional to average of each loss. Intuitively, this balances the losses to have the same scale. We demonstrate theoretical considerations that motivate this approach from multiple viewpoints, including connecting it to optimizing the geometric mean of the expectation of each loss, an objective that ignores the scale of losses. Experimentally, the adaptive weights induce more competitive anytime predictions on multiple recognition data-sets and models than non-adaptive approaches including weighing all losses equally. In particular, anytime neural networks (ANNs) can achieve the same accuracy faster using adaptive weights on a small network than using static constant weights on a large one. For problems with high performance saturation, we also show a sequence of exponentially deepening ANNscan achieve near-optimal anytime results at any budget, at the cost of a const fraction of extra computation.
Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates
Venue: arXiv cs.LG 2018
Year: 2018

In this paper, we describe a phenomenon, which we named "super-convergence", where neural networks can be trained an order of magnitude faster than with standard training methods. The existence of super-convergence is relevant to understanding why deep networks generalize well. One of the key elements of super-convergence is training with one learning rate cycle and a large maximum learning rate. A primary insight that allows super-convergence training is that large learning rates regularize the training, hence requiring a reduction of all other forms of regularization in order to preserve an optimal regularization balance. We also derive a simplification of the Hessian Free optimization method to compute an estimate of the optimal learning rate. Experiments demonstrate super-convergence for Cifar-10/100, MNIST and Imagenet datasets, and resnet, wide-resnet, densenet, and inception architectures. In addition, we show that super-convergence provides a greater boost in performance relative to standard training when the amount of labeled training data is limited. The architectures and code to replicate the figures in this paper are available at github.com/lnsmith54/super-convergence. See http://www.fast.ai/2018/04/30/dawnbench-fastai/ for an application of super-convergence to win the DAWNBench challenge (see https://dawn.cs.stanford.edu/benchmark/).
Accurate parameter estimation for Bayesian Network Classifiers using Hierarchical Dirichlet Processes
Venue: arXiv cs.LG 2018
Year: 2018

This paper introduces a novel parameter estimation method for the probability tables of Bayesian network classifiers (BNCs), using hierarchical Dirichlet processes (HDPs). The main result of this paper is to show that improved parameter estimation allows BNCs to outperform leading learning methods such as Random Forest for both 0-1 loss and RMSE, albeit just on categorical datasets. As data assets become larger, entering the hyped world of "big", efficient accurate classification requires three main elements: (1) classifiers with low-bias that can capture the fine-detail of large datasets (2) out-of-core learners that can learn from data without having to hold it all in main memory and (3) models that can classify new data very efficiently. The latest Bayesian network classifiers (BNCs) satisfy these requirements. Their bias can be controlled easily by increasing the number of parents of the nodes in the graph. Their structure can be learned out of core with a limited number of passes over the data. However, as the bias is made lower to accurately model classification tasks, so is the accuracy of their parameters' estimates, as each parameter is estimated from ever decreasing quantities of data. In this paper, we introduce the use of Hierarchical Dirichlet Processes for accurate BNC parameter estimation. We conduct an extensive set of experiments on 68 standard datasets and demonstrate that our resulting classifiers perform very competitively with Random Forest in terms of prediction, while keeping the out-of-core capability and superior classification time.
Fast Locality-Sensitive Hashing Frameworks for Approximate Near Neighbor Search
Venue: arXiv cs.DS 2018
Year: 2018

The Indyk-Motwani Locality-Sensitive Hashing (LSH) framework (STOC 1998) is a general technique for constructing a data structure to answer approximate near neighbor queries by using a distribution $\mathcal{H}$ over locality-sensitive hash functions that partition space. For a collection of $n$ points, after preprocessing, the query time is dominated by $O(n^ρ \log n)$ evaluations of hash functions from $\mathcal{H}$ and $O(n^ρ)$ hash table lookups and distance computations where $ρ\in (0,1)$ is determined by the locality-sensitivity properties of $\mathcal{H}$. It follows from a recent result by Dahlgaard et al. (FOCS 2017) that the number of locality-sensitive hash functions can be reduced to $O(\log^2 n)$, leaving the query time to be dominated by $O(n^ρ)$ distance computations and $O(n^ρ \log n)$ additional word-RAM operations. We state this result as a general framework and provide a simpler analysis showing that the number of lookups and distance computations closely match the Indyk-Motwani framework, making it a viable replacement in practice. Using ideas from another locality-sensitive hashing framework by Andoni and Indyk (SODA 2006) we are able to reduce the number of additional word-RAM operations to $O(n^ρ)$.
Machine Translation in Indian Languages: Challenges and Resolution
Venue: arXiv cs.CL 2018
Year: 2018

English to Indian language machine translation poses the challenge of structural and morphological divergence. This paper describes English to Indian language statistical machine translation using pre-ordering and suffix separation. The pre-ordering uses rules to transfer the structure of the source sentences prior to training and translation. This syntactic restructuring helps statistical machine translation to tackle the structural divergence and hence better translation quality. The suffix separation is used to tackle the morphological divergence between English and highly agglutinative Indian languages. We demonstrate that the use of pre-ordering and suffix separation helps in improving the quality of English to Indian Language machine translation.