OpenRaiser/Intern-Atlas huggingface dataset:
Single Machine Graph Analytics on Massive Datasets Using Intel Optane DC Persistent Memory
Venue: arXiv cs.DC 2020
Year: 2020

Intel Optane DC Persistent Memory (Optane PMM) is a new kind of byte-addressable memory with higher density and lower cost than DRAM. This enables the design of affordable systems that support up to 6TB of randomly accessible memory. In this paper, we present key runtime and algorithmic principles to consider when performing graph analytics on extreme-scale graphs on large-memory platforms of this sort. To demonstrate the importance of these principles, we evaluate four existing shared-memory graph frameworks on large real-world web-crawls, using a machine with 6TB of Optane PMM. Our results show that frameworks based on the runtime and algorithmic principles advocated in this paper (i) perform significantly better than the others, and (ii) are competitive with graph analytics frameworks running on large production clusters.
PrivacyGuard: Enforcing Private Data Usage Control with Blockchain and Attested Off-chain Contract Execution
Venue: arXiv cs.CR 2020
Year: 2020

The abundance and rich varieties of data are enabling many transformative applications of big data analytics that have profound societal impacts. However, there are also increasing concerns regarding the improper use of individual data owner's private data. In this paper, we propose PrivacyGuard, a system that leverages blockchain smart contract and trusted execution environment (TEE) to enable individual's control over the access and usage of their private data. Smart contracts are used to specify data usage policy, i.e., who can use what data under which conditions and what analytics to perform, while the distributed blockchain ledger is used to keep an irreversible and non-repudiable data usage record. To address the efficiency problem of on-chain contract execution and to prevent exposing private data on the publicly viewable blockchain, PrivacyGuard incorporates a novel TEE-based off-chain contract execution engine along with a protocol to securely commit the execution result onto blockchain. We have built and deployed a prototype of PrivacyGuard with Ethereum and Intel SGX. Our experiment result demonstrates that PrivacyGuard fulfills the promised privacy goal and supports analytics on data from a considerable number of data owners.
Combinatorial Conversion and Moment Bisimulation for Stochastic Rewriting Systems
Venue: arXiv cs.LO 2020
Year: 2020

We develop a novel method to analyze the dynamics of stochastic rewriting systems evolving over finitary adhesive, extensive categories. Our formalism is based on the so-called rule algebra framework and exhibits an intimate relationship between the combinatorics of the rewriting rules (as encoded in the rule algebra) and the dynamics which these rules generate on observables (as encoded in the stochastic mechanics formalism). We introduce the concept of combinatorial conversion, whereby under certain technical conditions the evolution equation for (the exponential generating function of) the statistical moments of observables can be expressed as the action of certain differential operators on formal power series. This permits us to formulate the novel concept of moment-bisimulation, whereby two dynamical systems are compared in terms of their evolution of sets of observables that are in bijection. In particular, we exhibit non-trivial examples of graphical rewriting systems that are moment-bisimilar to certain discrete rewriting systems (such as branching processes or the larger class of stochastic chemical reaction systems). Our results point towards applications of a vast number of existing well-established exact and approximate analysis techniques developed for chemical reaction systems to the far richer class of general stochastic rewriting systems.
Real Image Denoising with Feature Attention
Venue: arXiv cs.CV 2020
Year: 2020

Deep convolutional neural networks perform better on images containing spatially invariant noise (synthetic noise); however, their performance is limited on real-noisy photographs and requires multiple stage network modeling. To advance the practicability of denoising algorithms, this paper proposes a novel single-stage blind real image denoising network (RIDNet) by employing a modular architecture. We use a residual on the residual structure to ease the flow of low-frequency information and apply feature attention to exploit the channel dependencies. Furthermore, the evaluation in terms of quantitative metrics and visual quality on three synthetic and four real noisy datasets against 19 state-of-the-art algorithms demonstrate the superiority of our RIDNet.
Adaptive Wing Loss for Robust Face Alignment via Heatmap Regression
Venue: arXiv cs.CV 2020
Year: 2020

Heatmap regression with a deep network has become one of the mainstream approaches to localize facial landmarks. However, the loss function for heatmap regression is rarely studied. In this paper, we analyze the ideal loss function properties for heatmap regression in face alignment problems. Then we propose a novel loss function, named Adaptive Wing loss, that is able to adapt its shape to different types of ground truth heatmap pixels. This adaptability penalizes loss more on foreground pixels while less on background pixels. To address the imbalance between foreground and background pixels, we also propose Weighted Loss Map, which assigns high weights on foreground and difficult background pixels to help training process focus more on pixels that are crucial to landmark localization. To further improve face alignment accuracy, we introduce boundary prediction and CoordConv with boundary coordinates. Extensive experiments on different benchmarks, including COFW, 300W and WFLW, show our approach outperforms the state-of-the-art by a significant margin on various evaluation metrics. Besides, the Adaptive Wing loss also helps other heatmap regression tasks. Code will be made publicly available at https://github.com/protossw512/AdaptiveWingLoss.
Compressed Indexes for Fast Search of Semantic Data
Venue: arXiv cs.IR 2020
Year: 2020

The sheer increase in volume of RDF data demands efficient solutions for the triple indexing problem, that is devising a compressed data structure to compactly represent RDF triples by guaranteeing, at the same time, fast pattern matching operations. This problem lies at the heart of delivering good practical performance for the resolution of complex SPARQL queries on large RDF datasets. In this work, we propose a trie-based index layout to solve the problem and introduce two novel techniques to reduce its space of representation for improved effectiveness. The extensive experimental analysis conducted over a wide range of publicly available real-world datasets, reveals that our best space/time trade-off configuration substantially outperforms existing solutions at the state-of-the-art, by taking 30-60% less space and speeding up query execution by a factor of 2-81x.
Causality Extraction based on Self-Attentive BiLSTM-CRF with Transferred Embeddings
Venue: arXiv cs.CL 2020
Year: 2020

Causality extraction from natural language texts is a challenging open problem in artificial intelligence. Existing methods utilize patterns, constraints, and machine learning techniques to extract causality, heavily depending on domain knowledge and requiring considerable human effort and time for feature engineering. In this paper, we formulate causality extraction as a sequence labeling problem based on a novel causality tagging scheme. On this basis, we propose a neural causality extractor with the BiLSTM-CRF model as the backbone, named SCITE (Self-attentive BiLSTM-CRF wIth Transferred Embeddings), which can directly extract cause and effect without extracting candidate causal pairs and identifying their relations separately. To address the problem of data insufficiency, we transfer contextual string embeddings, also known as Flair embeddings, which are trained on a large corpus in our task. In addition, to improve the performance of causality extraction, we introduce a multihead self-attention mechanism into SCITE to learn the dependencies between causal words. We evaluate our method on a public dataset, and experimental results demonstrate that our method achieves significant and consistent improvement compared to baselines.
AT-GAN: An Adversarial Generator Model for Non-constrained Adversarial Examples
Venue: arXiv cs.CV 2020
Year: 2020

Despite the rapid development of adversarial machine learning, most adversarial attack and defense researches mainly focus on the perturbation-based adversarial examples, which is constrained by the input images. In comparison with existing works, we propose non-constrained adversarial examples, which are generated entirely from scratch without any constraint on the input. Unlike perturbation-based attacks, or the so-called unrestricted adversarial attack which is still constrained by the input noise, we aim to learn the distribution of adversarial examples to generate non-constrained but semantically meaningful adversarial examples. Following this spirit, we propose a novel attack framework called AT-GAN (Adversarial Transfer on Generative Adversarial Net). Specifically, we first develop a normal GAN model to learn the distribution of benign data, and then transfer the pre-trained GAN model to estimate the distribution of adversarial examples for the target model. In this way, AT-GAN can learn the distribution of adversarial examples that is very close to the distribution of real data. To our knowledge, this is the first work of building an adversarial generator model that could produce adversarial examples directly from any input noise. Extensive experiments and visualizations show that the proposed AT-GAN can very efficiently generate diverse adversarial examples that are more realistic to human perception. In addition, AT-GAN yields higher attack success rates against adversarially trained models under white-box attack setting and exhibits moderate transferability against black-box models.
Text Classification Algorithms: A Survey
Venue: arXiv cs.LG 2020
Year: 2020

In recent years, there has been an exponential growth in the number of complex documents and texts that require a deeper understanding of machine learning methods to be able to accurately classify texts in many applications. Many machine learning approaches have achieved surpassing results in natural language processing. The success of these learning algorithms relies on their capacity to understand complex models and non-linear relationships within data. However, finding suitable structures, architectures, and techniques for text classification is a challenge for researchers. In this paper, a brief overview of text classification algorithms is discussed. This overview covers different text feature extractions, dimensionality reduction methods, existing algorithms and techniques, and evaluations methods. Finally, the limitations of each technique and their application in the real-world problem are discussed.
NWR: Rethinking Thomas Write Rule for Omittable Write Operations
Venue: arXiv cs.DB 2020
Year: 2020

Concurrency control protocols are the key to scaling current DBMS performances. They efficiently interleave read and write operations in transactions, but occasionally they restrict concurrency by using coordination such as exclusive lockings. Although exclusive lockings ensure the correctness of DBMS, it incurs serious performance penalties on multi-core environments. In particular, existing protocols generally suffer from emerging highly write contended workloads, since they use innumerable lockings for write operations. In this paper, we rethink the Thomas write rule (TWR), which allows the timestamp ordering (T/O) protocol to omit write operations without any lockings. We formalize the notion of omitting and decouple it from the T/O protocol implementation, in order to define a new rule named non-visible write rule (NWR). When the rules of NWR are satisfied, any protocol can in theory generate omittable write operations with preserving the correctness without any lockings. In the experiments, we implement three NWR-extended protocols: Silo+NWR, TicToc+NWR, and MVTO+NWR. Experimental results demonstrate the efficiency and the low-overhead property of the extended protocols. We confirm that NWR-extended protocols achieve more than 11x faster than the originals in the best case of highly write contended YCSB-A and comparable performance with the originals in the other workloads.
On conflict-free chromatic guarding of simple polygons
Venue: arXiv cs.CG 2020
Year: 2020

We study the problem of colouring the vertices of a polygon, such that every viewer in it can see a unique colour. The goal is to minimise the number of colours used. This is also known as the conflict-free chromatic guarding problem with vertex guards, and is motivated, e.g., by the problem of radio frequency assignment to sensors placed at the polygon vertices. We first study the scenario in which viewers can be all points of the polygon (such as a mobile robot which moves in the interior of the polygon). We efficiently solve the related problem of minimising the number of guards and approximate (up to only an additive error) the number of colours required in the special case of polygons called funnels. Then we give an upper bound of O(log^2 n) colours on n-vertex weak visibility polygons, by decomposing the problem into sub-funnels. This bound generalises to all simple polygons. We briefly look also at the second scenario, in which the viewers are only the vertices of the polygon. We show a lower bound of 3 colours in the general case of simple polygons and conjecture that this is tight. We also prove that already deciding whether 1 or 2 colours are enough is NP-complete.
On the Polarizing Behavior and Scaling Exponent of Polar Codes with Product Kernels
Venue: arXiv cs.IT 2020
Year: 2020

Polar codes, introduced by Arikan, achieve the capacity of arbitrary binary-input discrete memoryless channel $W$ under successive cancellation decoding. Any such channel having capacity $I(W)$ and for any coding scheme allowing transmission at rate $R$, scaling exponent is a parameter which characterizes how fast gap to capacity decreases as a function of code length $N$ for a fixed probability of error. The relation between them is given by $N\geqslant α/(I(W)-R)^μ$. Scaling exponent for kernels of small size up to $L=8$ have been exhaustively found. In this paper, we consider product kernels $T_{L}$ obtained by taking Kronecker product of component kernels. We derive the properties of polarizing product kernels relating to number of product kernels, self duality and partial distances in terms of the respective properties of the smaller component kernels. Subsequently, polarization behavior of component kernel $T_{l}$ is used to calculate scaling exponent of $T_{L}=T_{2}\otimes T_{l}$. Using this method, we show that $μ(T_{2}\otimes T_{5})=3.942.$ Further, we employ a heuristic approach to construct good kernel of $L=14$ from kernel having size $l=8$ having best $μ$ and find $μ(T_{2}\otimes T_{7})=3.485.$
Efficient Motion Planning for Automated Lane Change based on Imitation Learning and Mixed-Integer Optimization
Venue: arXiv cs.RO 2020
Year: 2020

Intelligent motion planning is one of the core components in automated vehicles, which has received extensive interests. Traditional motion planning methods suffer from several drawbacks in terms of optimality, efficiency and generalization capability. Sampling based methods cannot guarantee the optimality of the generated trajectories. Whereas the optimization-based methods are not able to perform motion planning in real-time, and limited by the simplified formalization. In this work, we propose a learning-based approach to handle those shortcomings. Mixed Integer Quadratic Problem based optimization (MIQP) is used to generate the optimal lane-change trajectories which served as the training dataset for learning-based action generation algorithms. A hierarchical supervised learning model is devised to make the fast lane-change decision. Numerous experiments have been conducted to evaluate the optimality, efficiency, and generalization capability of the proposed approach. The experimental results indicate that the proposed model outperforms several commonly used motion planning baselines.
OpenTapioca: Lightweight Entity Linking for Wikidata
Venue: arXiv cs.CL 2020
Year: 2020

We propose a simple Named Entity Linking system that can be trained from Wikidata only. This demonstrates the strengths and weaknesses of this data source for this task and provides an easily reproducible baseline to compare other systems against. Our model is lightweight to train, to run and to keep synchronous with Wikidata in real time.
Beyond Submodular Maximization via One-Sided Smoothness
Venue: arXiv cs.DS 2020
Year: 2020

The multilinear framework has achieved the breakthrough $1-1/e$ approximation for maximizing a monotone submodular function subject to a matroid constraint. This framework has a continuous optimization part and a rounding part. We extend both parts to a wider array of problems. In particular, we make a conceptual contribution by identifying a family of parameterized functions. As a running example we focus on solving diversity problems $\max f(S)=\frac{1}{2}\sum_{i,j\in A}A_{ij}:S\in\mathcal{M}$, where $\mathcal{M}$ is a matroid. These diversity functions have $A_{ij}\geq 0$ as a measure of dissimilarity of $i,j$, and $A$ has $0$-diagonal. The multilinear framework cannot be directly applied to the multilinear extension of such functions. We introduce a new parameter for functions $F\in{\bf C}^2$ which measures the approximability of the associated problem $\max\{F(x):x\in P\}$, for solvable downwards-closed polytopes $P$. A function $F$ is called one-sided $σ$-smooth if $\frac{1}{2}u^T\nabla^2 F(x) u\leqσ\cdot\frac{||u||_1}{||x||_1}u^T\nabla F(x)$ for all $u,x\geq 0$, $x\neq 0$. We give an $Ω(1/σ)$-approximation for the maximization problem of monotone, normalized one-sided $σ$-smooth $F$ with an additional property: non-positive third order partial derivatives. Using the multilinear framework and new matroid rounding techniques for quadratic objectives, we give an $Ω(1/σ^{3/2})$-approximation for maximizing a $σ$-semi-metric diversity function subject to matroid constraint. This improves upon the previous best bound of $Ω(1/σ^2)$ and we give evidence that it may be tight. For general one-sided smooth functions, we show the continuous process gives an $Ω(1/3^{2σ})$-approximation, independent of $n$. In this setting, by discretizing, we present a poly-time algorithm for multilinear one-sided $σ$-smooth functions.
Tripping through time: Efficient Localization of Activities in Videos
Venue: arXiv cs.CV 2020
Year: 2020

Localizing moments in untrimmed videos via language queries is a new and interesting task that requires the ability to accurately ground language into video. Previous works have approached this task by processing the entire video, often more than once, to localize relevant activities. In the real world applications of this approach, such as video surveillance, efficiency is a key system requirement. In this paper, we present TripNet, an end-to-end system that uses a gated attention architecture to model fine-grained textual and visual representations in order to align text and video content. Furthermore, TripNet uses reinforcement learning to efficiently localize relevant activity clips in long videos, by learning how to intelligently skip around the video. It extracts visual features for few frames to perform activity classification. In our evaluation over Charades-STA, ActivityNet Captions and the TACoS dataset, we find that TripNet achieves high accuracy and saves processing time by only looking at 32-41% of the entire video.
Statistical Learning and Estimation of Piano Fingering
Venue: arXiv cs.LG 2020
Year: 2020

Automatic estimation of piano fingering is important for understanding the computational process of music performance and applicable to performance assistance and education systems. While a natural way to formulate the quality of fingerings is to construct models of the constraints/costs of performance, it is generally difficult to find appropriate parameter values for these models. Here we study an alternative data-driven approach based on statistical modeling in which the appropriateness of a given fingering is described by probabilities. Specifically, we construct two types of hidden Markov models (HMMs) and their higher-order extensions. We also study deep neural network (DNN)-based methods for comparison. Using a newly released dataset of fingering annotations, we conduct systematic evaluations of these models as well as a representative constraint-based method. We find that the methods based on high-order HMMs outperform the other methods in terms of estimation accuracies. We also quantitatively study individual difference of fingering and propose evaluation measures that can be used with multiple ground truth data. We conclude that the HMM-based methods are currently state of the art and generate acceptable fingerings in most parts and that they have certain limitations such as ignorance of phrase boundaries and interdependence of the two hands.
Deconstructing the Filter Bubble: User Decision-Making and Recommender Systems
Venue: arXiv cs.CY 2020
Year: 2020

We study a model of user decision-making in the context of recommender systems via numerical simulation. Our model provides an explanation for the findings of Nguyen, et. al (2014), where, in environments where recommender systems are typically deployed, users consume increasingly similar items over time even without recommendation. We find that recommendation alleviates these natural filter-bubble effects, but that it also leads to an increase in homogeneity across users, resulting in a trade-off between homogenizing across-user consumption and diversifying within-user consumption. Finally, we discuss how our model highlights the importance of collecting data on user beliefs and their evolution over time both to design better recommendations and to further understand their impact.
Reviewing Data Access Patterns and Computational Redundancy for Machine Learning Algorithms
Venue: arXiv cs.LG 2020
Year: 2020

Machine learning (ML) is probably the first and foremost used technique to deal with the size and complexity of the new generation of data. In this paper, we analyze one of the means to increase the performances of ML algorithms which is exploiting data locality. Data locality and access patterns are often at the heart of performance issues in computing systems due to the use of certain hardware techniques to improve performance. Altering the access patterns to increase locality can dramatically increase performance of a given algorithm. Besides, repeated data access can be seen as redundancy in data movement. Similarly, there can also be redundancy in the repetition of calculations. This work also identifies some of the opportunities for avoiding these redundancies by directly reusing computation results. We document the possibilities of such reuse in some selected machine learning algorithms and give initial indicative results from our first experiments on data access improvement and algorithm redesign.