OpenRaiser/Intern-Atlas huggingface dataset:
Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networks
Venue: arXiv cs.LG 2019
Year: 2019

Empirical studies show that gradient-based methods can learn deep neural networks (DNNs) with very good generalization performance in the over-parameterization regime, where DNNs can easily fit a random labeling of the training data. Very recently, a line of work explains in theory that with over-parameterization and proper random initialization, gradient-based methods can find the global minima of the training loss for DNNs. However, existing generalization error bounds are unable to explain the good generalization performance of over-parameterized DNNs. The major limitation of most existing generalization bounds is that they are based on uniform convergence and are independent of the training algorithm. In this work, we derive an algorithm-dependent generalization error bound for deep ReLU networks, and show that under certain assumptions on the data distribution, gradient descent (GD) with proper random initialization is able to train a sufficiently over-parameterized DNN to achieve arbitrarily small generalization error. Our work sheds light on explaining the good generalization performance of over-parameterized deep neural networks.
Blaze: Simplified High Performance Cluster Computing
Venue: arXiv cs.DC 2019
Year: 2019

MapReduce and its variants have significantly simplified and accelerated the process of developing parallel programs. However, most MapReduce implementations focus on data-intensive tasks while many real-world tasks are compute intensive and their data can fit distributedly into the memory. For these tasks, the speed of MapReduce programs can be much slower than those hand-optimized ones. We present Blaze, a C++ library that makes it easy to develop high performance parallel programs for such compute intensive tasks. At the core of Blaze is a highly-optimized in-memory MapReduce function, which has three main improvements over conventional MapReduce implementations: eager reduction, fast serialization, and special treatment for a small fixed key range. We also offer additional conveniences that make developing parallel programs similar to developing serial programs. These improvements make Blaze an easy-to-use cluster computing library that approaches the speed of hand-optimized parallel code. We apply Blaze to some common data mining tasks, including word frequency count, PageRank, k-means, expectation maximization (Gaussian mixture model), and k-nearest neighbors. Blaze outperforms Apache Spark by more than 10 times on average for these tasks, and the speed of Blaze scales almost linearly with the number of nodes. In addition, Blaze uses only the MapReduce function and 3 utility functions in its implementation while Spark uses almost 30 different parallel primitives in its official implementation.
On the Complexity of Toric Ideals
Venue: arXiv cs.SC 2019
Year: 2019

We investigate the computational complexity of problems on toric ideals such as normal forms, Gröbner bases, and Graver bases. We show that all these problems are strongly NP-hard in the general case. Nonetheless, we can derive efficient algorithms by taking advantage of the sparsity pattern of the matrix. We describe this sparsity pattern with a graph, and study the parameterized complexity of toric ideals in terms of graph parameters such as treewidth and treedepth. In particular, we show that the normal form problem can be solved in parameter-tractable time in terms of the treedepth. An important application of this result is in multiway ideals arising in algebraic statistics. We also give a parameter-tractable membership test to the reduced Gröbner basis. This test leads to an efficient procedure for computing the reduced Gröbner basis. Similar results hold for Graver bases computation.
Symbolic QED Pre-silicon Verification for Automotive Microcontroller Cores: Industrial Case Study
Venue: arXiv cs.LO 2019
Year: 2019

We present an industrial case study that demonstrates the practicality and effectiveness of Symbolic Quick Error Detection (Symbolic QED) in detecting logic design flaws (logic bugs) during pre-silicon verification. Our study focuses on several microcontroller core designs (~1,800 flip-flops, ~70,000 logic gates) that have been extensively verified using an industrial verification flow and used for various commercial automotive products. The results of our study are as follows: 1. Symbolic QED detected all logic bugs in the designs that were detected by the industrial verification flow (which includes various flavors of simulation-based verification and formal verification). 2. Symbolic QED detected additional logic bugs that were not recorded as detected by the industrial verification flow. (These additional bugs were also perhaps detected by the industrial verification flow.) 3. Symbolic QED enables significant design productivity improvements: (a) 8X improved (i.e., reduced) verification effort for a new design (8 person-weeks for Symbolic QED vs. 17 person-months using the industrial verification flow). (b) 60X improved verification effort for subsequent designs (2 person-days for Symbolic QED vs. 4-7 person-months using the industrial verification flow). (c) Quick bug detection (runtime of 20 seconds or less), together with short counterexamples (10 or fewer instructions) for quick debug, using Symbolic QED.
Differentially Private Release of High-Dimensional Datasets using the Gaussian Copula
Venue: arXiv cs.CR 2019
Year: 2019

We propose a generic mechanism to efficiently release differentially private synthetic versions of high-dimensional datasets with high utility. The core technique in our mechanism is the use of copulas. Specifically, we use the Gaussian copula to define dependencies of attributes in the input dataset, whose rows are modelled as samples from an unknown multivariate distribution, and then sample synthetic records through this copula. Despite the inherently numerical nature of Gaussian correlations we construct a method that is applicable to both numerical and categorical attributes alike. Our mechanism is efficient in that it only takes time proportional to the square of the number of attributes in the dataset. We propose a differentially private way of constructing the Gaussian copula without compromising computational efficiency. Through experiments on three real-world datasets, we show that we can obtain highly accurate answers to the set of all one-way marginal, and two-and three-way positive conjunction queries, with 99\% of the query answers having absolute (fractional) error rates between 0.01 to 3\%. Furthermore, for a majority of two-way and three-way queries, we outperform independent noise addition through the well-known Laplace mechanism. In terms of computational time we demonstrate that our mechanism can output synthetic datasets in around 6 minutes 47 seconds on average with an input dataset of about 200 binary attributes and more than 32,000 rows, and about 2 hours 30 mins to execute a much larger dataset of about 700 binary attributes and more than 5 million rows. To further demonstrate scalability, we ran the mechanism on larger (artificial) datasets with 1,000 and 2,000 binary attributes (and 5 million rows) obtaining synthetic outputs in approximately 6 and 19 hours, respectively.
Bitcoin Selfish Mining and Dyck Words
Venue: arXiv cs.CR 2019
Year: 2019

We give a straightforward proof for the formula giving the long-term apparent hashrate of the Selfish Mining strategy in Bitcoin using only elementary probabilities and combinatorics, and more precisely, Dyck words. There is no need to compute stationary probabilities on Markov chain nor stopping times for Poisson processes as it was previously done. We consider also several other block withholding strategies.
An Ensemble SVM-based Approach for Voice Activity Detection
Venue: arXiv cs.SD 2019
Year: 2019

Voice activity detection (VAD), used as the front end of speech enhancement, speech and speaker recognition algorithms, determines the overall accuracy and efficiency of the algorithms. Therefore, a VAD with low complexity and high accuracy is highly desirable for speech processing applications. In this paper, we propose a novel training method on large dataset for supervised learning-based VAD system using support vector machine (SVM). Despite of high classification accuracy of support vector machines (SVM), trivial SVM is not suitable for classification of large data sets needed for a good VAD system because of high training complexity. To overcome this problem, a novel ensemble-based approach using SVM has been proposed in this paper.The performance of the proposed ensemble structure has been compared with a feedforward neural network (NN). Although NN performs better than single SVM-based VAD trained on a small portion of the training data, ensemble SVM gives accuracy comparable to neural network-based VAD. Ensemble SVM and NN give 88.74% and 86.28% accuracy respectively whereas the stand-alone SVM shows 57.05% accuracy on average on the test dataset.
Revisiting a single-stage method for face detection
Venue: arXiv cs.CV 2019
Year: 2019

Although accurate, two-stage face detectors usually require more inference time than single-stage detectors do. This paper proposes a simple yet effective single-stage model for real-time face detection with a prominently high accuracy. We build our single-stage model on the top of the ResNet-101 backbone and analyze some problems with the baseline single-stage detector in order to design several strategies for reducing the false positive rate. The design leverages the context information from the deeper layers in order to increase recall rate while maintaining a low false positive rate. In addition, we reduce the detection time by an improved inference procedure for decoding outputs faster. The inference time of a VGA ($640{\times}480$) image was only approximately 26 ms with a Titan X GPU. The effectiveness of our proposed method was evaluated on several face detection benchmarks (Wider Face, AFW, Pascal Face, and FDDB). The experiments show that our method achieved competitive results on these popular datasets with a faster runtime than the current best two-stage practices.
Dynamic Real-time Multimodal Routing with Hierarchical Hybrid Planning
Venue: arXiv cs.AI 2019
Year: 2019

We introduce the problem of Dynamic Real-time Multimodal Routing (DREAMR), which requires planning and executing routes under uncertainty for an autonomous agent. The agent has access to a time-varying transit vehicle network in which it can use multiple modes of transportation. For instance, a drone can either fly or ride on terrain vehicles for segments of their routes. DREAMR is a difficult problem of sequential decision making under uncertainty with both discrete and continuous variables. We design a novel hierarchical hybrid planning framework to solve the DREAMR problem that exploits its structural decomposability. Our framework consists of a global open-loop planning layer that invokes and monitors a local closed-loop execution layer. Additional abstractions allow efficient and seamless interleaving of planning and execution. We create a large-scale simulation for DREAMR problems, with each scenario having hundreds of transportation routes and thousands of connection points. Our algorithmic framework significantly outperforms a receding horizon control baseline, in terms of elapsed time to reach the destination and energy expended by the agent.
Detection of Violent Extremists in Social Media
Venue: arXiv cs.SI 2019
Year: 2019

The ease of use of the Internet has enabled violent extremists such as the Islamic State of Iraq and Syria (ISIS) to easily reach large audience, build personal relationships and increase recruitment. Social media are primarily based on the reports they receive from their own users to mitigate the problem. Despite efforts of social media in suspending many accounts, this solution is not guaranteed to be effective, because not all extremists are caught this way, or they can simply return with another account or migrate to other social networks. In this paper, we design an automatic detection scheme that using as little as three groups of information related to usernames, profile, and textual content of users, determines whether or not a given username belongs to an extremist user. We first demonstrate that extremists are inclined to adopt usernames that are similar to the ones that their like-minded have adopted in the past. We then propose a detection framework that deploys features which are highly indicative of potential online extremism. Results on a real-world ISIS-related dataset from Twitter demonstrate the effectiveness of the methodology in identifying extremist users.
An Optimal Algorithm for Online Freeze-tag
Venue: arXiv cs.DS 2019
Year: 2019

In the freeze-tag problem, one active robot must wake up many frozen robots. The robots are considered as points in a metric space, where active robots move at a constant rate and activate other robots by visiting them. In the (time-dependent) online variant of the problem, frozen robots are not revealed until a specified time. Hammar, Nilsson, and Persson have shown that no online algorithm can achieve a competitive ratio better than $7/3$ for online freeze-tag, and asked whether there is any $O(1)$-competitive algorithm. In this paper, we provide a $(1+\sqrt{2})$-competitive algorithm for online time-dependent freeze-tag, and show that no algorithm can achieve a lower competitive ratio on every metric space.
Mobility-Aware Joint Service Placement and Routing in Space-Air-Ground Integrated Networks
Venue: arXiv cs.NI 2019
Year: 2019

People desire to be connected, no matter where they are. Recently, providing Internet access to on-board passengers has received a lot of attention from both industry and academia. However, in order to guarantee an acceptable Quality of Service (QoS) for the passenger services with low incurred cost, the path to route the services, as well as the datacenter (DC) to deploy the services should be carefully determined. This problem is challenging, due to different types of Air-to-Ground (A2G) connections, i.e., satellites and Direct-Air-to-Ground (DA2G) links. These A2G connection types differ in terms of cost, bandwidth, and latency. Furthermore, due to the flights' movements, it is important to consider adapting the service location accordingly. In this work, we formulate two Mixed Integer Linear Programs (MILPs) for the problem of Joint Service Placement and Routing (JSPR): i) Static (S-JSPR), and ii) Mobility-Aware~(MA-JSPR) in Space-Air-Ground Integrated Networks (SAGIN), with the objective of minimizing the total cost. We compare S-JSPR and MA-JSPR using comprehensive evaluations in a realistic European-based SAGIN. The obtained results show that the MA-JSPR model, by considering the future flight positions and using a service migration control, reduces the long-term total cost notably. Also, we show S-JSPR benefits from a low time-complexity and it achieves lower end-to-end delays comparing to MA-JSPR model.
Classifying Convex Bodies by their Contact and Intersection Graphs
Venue: arXiv cs.CG 2019
Year: 2019

Suppose that $A$ is a convex body in the plane and that $A_1,\dots,A_n$ are translates of $A$. Such translates give rise to an intersection graph of $A$, $G=(V,E)$, with vertices $V=\{1,\dots,n\}$ and edges $E=\{uv\mid A_u\cap A_v\neq \emptyset\}$. The subgraph $G'=(V, E')$ satisfying that $E'\subset E$ is the set of edges $uv$ for which the interiors of $A_u$ and $A_v$ are disjoint is a unit distance graph of $A$. If furthermore $G'=G$, i.e., if the interiors of $A_u$ and $A_v$ are disjoint whenever $u\neq v$, then $G$ is a contact graph of $A$. In this paper we study which pairs of convex bodies have the same contact, unit distance, or intersection graphs. We say that two convex bodies $A$ and $B$ are equivalent if there exists a linear transformation $B'$ of $B$ such that for any slope, the longest line segments with that slope contained in $A$ and $B'$, respectively, are equally long. For a broad class of convex bodies, including all strictly convex bodies and linear transformations of regular polygons, we show that the contact graphs of $A$ and $B$ are the same if and only if $A$ and $B$ are equivalent. We prove the same statement for unit distance and intersection graphs.
Metric Learning on Manifolds
Venue: arXiv cs.LG 2019
Year: 2019

Recent literature has shown that symbolic data, such as text and graphs, is often better represented by points on a curved manifold, rather than in Euclidean space. However, geometrical operations on manifolds are generally more complicated than in Euclidean space, and thus many techniques for processing and analysis taken for granted in Euclidean space are difficult on manifolds. A priori, it is not obvious how we may generalize such methods to manifolds. We consider specifically the problem of distance metric learning, and present a framework that solves it on a large class of manifolds, such that similar data are located in closer proximity with respect to the manifold distance function. In particular, we extend the existing metric learning algorithms, and derive the corresponding sample complexity rates for the case of manifolds. Additionally, we demonstrate an improvement of performance in $k$-means clustering and $k$-nearest neighbor classification on real-world complex networks using our methods.
Face Alignment using a 3D Deeply-initialized Ensemble of Regression Trees
Venue: arXiv cs.CV 2019
Year: 2019

Face alignment algorithms locate a set of landmark points in images of faces taken in unrestricted situations. State-of-the-art approaches typically fail or lose accuracy in the presence of occlusions, strong deformations, large pose variations and ambiguous configurations. In this paper we present 3DDE, a robust and efficient face alignment algorithm based on a coarse-to-fine cascade of ensembles of regression trees. It is initialized by robustly fitting a 3D face model to the probability maps produced by a convolutional neural network. With this initialization we address self-occlusions and large face rotations. Further, the regressor implicitly imposes a prior face shape on the solution, addressing occlusions and ambiguous face configurations. Its coarse-to-fine structure tackles the combinatorial explosion of parts deformation. In the experiments performed, 3DDE improves the state-of-the-art in 300W, COFW, AFLW and WFLW data sets. Finally, we perform cross-dataset experiments that reveal the existence of a significant data set bias in these benchmarks.
Disguised-Nets: Image Disguising for Privacy-preserving Outsourced Deep Learning
Venue: arXiv cs.LG 2019
Year: 2019

Deep learning model developers often use cloud GPU resources to experiment with large data and models that need expensive setups. However, this practice raises privacy concerns. Adversaries may be interested in: 1) personally identifiable information or objects encoded in the training images, and 2) the models trained with sensitive data to launch model-based attacks. Learning deep neural networks (DNN) from encrypted data is still impractical due to the large training data and the expensive learning process. A few recent studies have tried to provide efficient, practical solutions to protect data privacy in outsourced deep-learning. However, we find out that they are vulnerable under certain attacks. In this paper, we specifically identify two types of unique attacks on outsourced deep-learning: 1) the visual re-identification attack on the training data, and 2) the class membership attack on the learned models, which can break existing privacy-preserving solutions. We develop an image disguising approach to address these attacks and design a suite of methods to evaluate the levels of attack resilience for a privacy-preserving solution for outsourced deep learning. The experimental results show that our image-disguising mechanisms can provide a high level of protection against the two attacks while still generating high-quality DNN models for image classification.
Exponentiated Gradient Meets Gradient Descent
Venue: arXiv cs.LG 2019
Year: 2019

The (stochastic) gradient descent and the multiplicative update method are probably the most popular algorithms in machine learning. We introduce and study a new regularization which provides a unification of the additive and multiplicative updates. This regularization is derived from an hyperbolic analogue of the entropy function, which we call hypentropy. It is motivated by a natural extension of the multiplicative update to negative numbers. The hypentropy has a natural spectral counterpart which we use to derive a family of matrix-based updates that bridge gradient methods and the multiplicative method for matrices. While the latter is only applicable to positive semi-definite matrices, the spectral hypentropy method can naturally be used with general rectangular matrices. We analyze the new family of updates by deriving tight regret bounds. We study empirically the applicability of the new update for settings such as multiclass learning, in which the parameters constitute a general rectangular matrix.
An Enhanced Interleaving Frame Loss Concealment Method for Voice Over IP Network Services
Venue: arXiv cs.NI 2019
Year: 2019

This paper focuses on AMR WB G.722.2 speech codec, and discusses the unused bandwidth resources of the senders by using a Word16(16 bit) to encode the sent frames. A packet loss concealment (PLC) method for G.722.2 speech codec is proposed in order to overcome this problem and increases the efficiency of this codec by improving the quality of decoded speech under burst frame loss conditions over frame-switched networks. Objective and subjective experimental results confirm that our proposed algorithm could achieve better speech quality. Our proposed method achieves a PESQ value higher than 2 at 20% frame erasure rate and ensure the compatibility between our modified decoder and the non-modified G.722.2 coder.
Virtually the Same: Comparing Physical and Virtual Testbeds
Venue: arXiv cs.NI 2019
Year: 2019

Network designers, planners, and security professionals increasingly rely on large-scale virtual testbeds to emulate networks and make decisions about real-world deployments. However, there has been limited research on how well these virtual testbeds match their physical counterparts. Specifically, does the virtualization that these testbeds depend on actually capture real-world behaviors sufficiently well to support decisions? As a first step, we perform simple experiments on both physical and virtual testbeds to begin to understand where and how the testbeds differ. We set up a web service on one host and run ApacheBench against this service from a different host, instrumenting each system during these tests. We define an initial repeatable methodology to quantitatively compare testbeds. Specifically, we compare the testbeds at three levels of abstraction: application, operating system (OS) and network. For the application level, we use the ApacheBench results. For OS behavior, we compare patterns of system call orderings using Markov chains. This provides a unique visual representation of the workload and OS behavior in our testbeds. We also drill down into read-system-call behaviors and show how at one level both systems are deterministic and identical, but as we move up in abstractions that consistency declines. Finally, we use packet captures to compare network behaviors and performance. We reconstruct flows and compare per-flow and per-experiment statistics. We find that the behavior of the workload in the testbeds is similar but that the underlying processes to support it do vary. The low-level network behavior can vary widely in packetization depending on the virtual network driver. While these differences can be important, and knowing about them will help experiment designers, the core application and OS behaviors still represent similar processes.