OpenRaiser/Intern-Atlas huggingface dataset:
A Robust Hierarchical Graph Convolutional Network Model for Collaborative Filtering
Venue: arXiv cs.IR 2020
Year: 2020

Graph Convolutional Network (GCN) has achieved great success and has been applied in various fields including recommender systems. However, GCN still suffers from many issues such as training difficulties, over-smoothing, vulnerable to adversarial attacks, etc. Distinct from current GCN-based methods which simply employ GCN for recommendation, in this paper we are committed to build a robust GCN model for collaborative filtering. Firstly, we argue that recursively incorporating messages from different order neighborhood mixes distinct node messages indistinguishably, which increases the training difficulty; instead we choose to separately aggregate different order neighbor messages with a simple GCN model which has been shown effective; then we accumulate them together in a hierarchical way without introducing additional model parameters. Secondly, we propose a solution to alleviate over-smoothing by randomly dropping out neighbor messages at each layer, which also well prevents over-fitting and enhances the robustness. Extensive experiments on three real-world datasets demonstrate the effectiveness and robustness of our model.
Cloud+: A safe and restrained data access control program for cloud
Venue: arXiv cs.DC 2020
Year: 2020

Secure distributed storage, which is a rising cloud administration, is planned to guarantee the mystery of re-appropriated data yet also to give versatile data access to cloud customers whose data is out of physical control. Ciphertext-Policy Attribute-Based Encryption (CP-ABE) is seen as a champion among the most reassuring frameworks that may be used to verify the confirmation of the administration. Be that as it may, the use of CP-ABE may yield an unavoidable security burst which is known as the abuse of access accreditation (for example decoding right). In this paper, we look at the two essential occurrences of access accreditation misuse: one is on the semi-believed specialist side, and the other is supportive of the cloud customer. To ease the abuse, we propose revocable CP-ABE based distributed storage structure with express renouncing, planned information getting to and numerous examining capacities alluded as Cloud+.
Self-Supervised and Controlled Multi-Document Opinion Summarization
Venue: arXiv cs.CL 2020
Year: 2020

We address the problem of unsupervised abstractive summarization of collections of user generated reviews with self-supervision and control. We propose a self-supervised setup that considers an individual document as a target summary for a set of similar documents. This setting makes training simpler than previous approaches by relying only on standard log-likelihood loss. We address the problem of hallucinations through the use of control codes, to steer the generation towards more coherent and relevant summaries.Finally, we extend the Transformer architecture to allow for multiple reviews as input. Our benchmarks on two datasets against graph-based and recent neural abstractive unsupervised models show that our proposed method generates summaries with a superior quality and relevance.This is confirmed in our human evaluation which focuses explicitly on the faithfulness of generated summaries We also provide an ablation study, which shows the importance of the control setup in controlling hallucinations and achieve high sentiment and topic alignment of the summaries with the input reviews.
Preventing Posterior Collapse with Levenshtein Variational Autoencoder
Venue: arXiv cs.LG 2020
Year: 2020

Variational autoencoders (VAEs) are a standard framework for inducing latent variable models that have been shown effective in learning text representations as well as in text generation. The key challenge with using VAEs is the {\it posterior collapse} problem: learning tends to converge to trivial solutions where the generators ignore latent variables. In our Levenstein VAE, we propose to replace the evidence lower bound (ELBO) with a new objective which is simple to optimize and prevents posterior collapse. Intuitively, it corresponds to generating a sequence from the autoencoder and encouraging the model to predict an optimal continuation according to the Levenshtein distance (LD) with the reference sentence at each time step in the generated sequence. We motivate the method from the probabilistic perspective by showing that it is closely related to optimizing a bound on the intractable Kullback-Leibler divergence of an LD-based kernel density estimator from the model distribution. With this objective, any generator disregarding latent variables will incur large penalties and hence posterior collapse does not happen. We relate our approach to policy distillation \cite{RossGB11} and dynamic oracles \cite{GoldbergN12}. By considering Yelp and SNLI benchmarks, we show that Levenstein VAE produces more informative latent representations than alternative approaches to preventing posterior collapse.
RAIN: A Simple Approach for Robust and Accurate Image Classification Networks
Venue: arXiv cs.CR 2020
Year: 2020

It has been shown that the majority of existing adversarial defense methods achieve robustness at the cost of sacrificing prediction accuracy. The undesirable severe drop in accuracy adversely affects the reliability of machine learning algorithms and prohibits their deployment in realistic applications. This paper aims to address this dilemma by proposing a novel preprocessing framework, which we term Robust and Accurate Image classificatioN(RAIN), to improve the robustness of given CNN classifiers and, at the same time, preserve their high prediction accuracies. RAIN introduces a new randomization-enhancement scheme. It applies randomization over inputs to break the ties between the model forward prediction path and the backward gradient path, thus improving the model robustness. However, similar to existing preprocessing-based methods, the randomized process will degrade the prediction accuracy. To understand why this is the case, we compare the difference between original and processed images, and find it is the loss of high-frequency components in the input image that leads to accuracy drop of the classifier. Based on this finding, RAIN enhances the input's high-frequency details to retain the CNN's high prediction accuracy. Concretely, RAIN consists of two novel randomization modules: randomized small circular shift (RdmSCS) and randomized down-upsampling (RdmDU). The RdmDU module randomly downsamples the input image, and then the RdmSCS module circularly shifts the input image along a randomly chosen direction by a small but random number of pixels. Finally, the RdmDU module performs upsampling with a detail-enhancement model, such as deep super-resolution networks. We conduct extensive experiments on the STL10 and ImageNet datasets to verify the effectiveness of RAIN against various types of adversarial attacks.
TACRED Revisited: A Thorough Evaluation of the TACRED Relation Extraction Task
Venue: arXiv cs.CL 2020
Year: 2020

TACRED (Zhang et al., 2017) is one of the largest, most widely used crowdsourced datasets in Relation Extraction (RE). But, even with recent advances in unsupervised pre-training and knowledge enhanced neural RE, models still show a high error rate. In this paper, we investigate the questions: Have we reached a performance ceiling or is there still room for improvement? And how do crowd annotations, dataset, and models contribute to this error rate? To answer these questions, we first validate the most challenging 5K examples in the development and test sets using trained annotators. We find that label errors account for 8% absolute F1 test error, and that more than 50% of the examples need to be relabeled. On the relabeled test set the average F1 score of a large baseline model set improves from 62.1 to 70.1. After validation, we analyze misclassifications on the challenging instances, categorize them into linguistically motivated error groups, and verify the resulting error hypotheses on three state-of-the-art RE models. We show that two groups of ambiguous relations are responsible for most of the remaining errors and that models may adopt shallow heuristics on the dataset when entities are not masked.
Flag Codes from Planar Spreads in Network Coding
Venue: arXiv cs.IT 2020
Year: 2020

In this paper we study a class of multishot network codes given by families of nested subspaces (flags) of a vector space $\mathbb{F}_q^n$, being $q$ a prime power and $\mathbb{F}_q$ the finite field of $q$ elements. In particular, we focus on flag codes having maximum minimum distance (optimum distance flag codes). We explore the existence of these codes from spreads, based on the good properties of the latter ones. For $n=2k$, we show that optimum distance full flag codes with the largest size are exactly those that can be constructed from a planar spread. We give a precise construction of them as well as a decoding algorithm.
The Complexity of Dynamic Data Race Prediction
Venue: arXiv cs.LO 2020
Year: 2020

Writing concurrent programs is notoriously hard due to scheduling non-determinism. The most common concurrency bugs are data races, which are accesses to a shared resource that can be executed concurrently. Dynamic data-race prediction is the most standard technique for detecting data races: given an observed, data-race-free trace $t$, the task is to determine whether $t$ can be reordered to a trace $t^*$ that exposes a data-race. Although the problem has received significant practical attention for over three decades, its complexity has remained elusive. In this work, we address this lacuna, identifying sources of intractability and conditions under which the problem is efficiently solvable. Given a trace $t$ of size $n$ over $k$ threads, our main results are as follows. First, we establish a general $O(k\cdot n^{2\cdot (k-1)})$ upper-bound, as well as an $O(n^k)$ upper-bound when certain parameters of $t$ are constant. In addition, we show that the problem is NP-hard and even W[1]-hard parameterized by $k$, and thus unlikely to be fixed-parameter tractable. Second, we study the problem over acyclic communication topologies, such as server-clients hierarchies. We establish an $O(k^2\cdot d\cdot n^2\cdot \log n)$ upper-bound, where $d$ is the number of shared variables accessed in $t$. In addition, we show that even for traces with $k=2$ threads, the problem has no $O(n^{2-ε})$ algorithm under Orthogonal Vectors. Since any trace with 2 threads defines an acyclic topology, our upper-bound for this case is optimal wrt polynomial improvements for up to moderate values of $k$ and $d$. Finally, we study a distance-bounded version of the problem, where the task is to expose a data race by a witness trace that is similar to $t$. We develop an algorithm that works in $O(n)$ time when certain parameters of $t$ are constant.
Improving Semantic Segmentation via Self-Training
Venue: arXiv cs.CV 2020
Year: 2020

Deep learning usually achieves the best results with complete supervision. In the case of semantic segmentation, this means that large amounts of pixelwise annotations are required to learn accurate models. In this paper, we show that we can obtain state-of-the-art results using a semi-supervised approach, specifically a self-training paradigm. We first train a teacher model on labeled data, and then generate pseudo labels on a large set of unlabeled data. Our robust training framework can digest human-annotated and pseudo labels jointly and achieve top performances on Cityscapes, CamVid and KITTI datasets while requiring significantly less supervision. We also demonstrate the effectiveness of self-training on a challenging cross-domain generalization task, outperforming conventional finetuning method by a large margin. Lastly, to alleviate the computational burden caused by the large amount of pseudo labels, we propose a fast training schedule to accelerate the training of segmentation models by up to 2x without performance degradation.
A more secure IPv6 neighborhood process
Venue: arXiv cs.NI 2020
Year: 2020

The process of neighborhood establishment in an IPv6 network is made out through the NDP (Neighbor Discovery Protocol). Using ICMPv6 messages (NS - Neighbor Solicitation and NA - Neighbor Advertisement) which contains the IP address to be resolved, is exposed during the exchange of messages, making communication vulnerable to various types of attacks. This paper presents a proposal for a safer neighborhood establishment using the DH and HMAC algorithms. Experiments were performed in virtualized environments with the objective of analyzing the efficiency of the modification proposed in the NDP.
Segatron: Segment-Aware Transformer for Language Modeling and Understanding
Venue: arXiv cs.CL 2020
Year: 2020

Transformers are powerful for sequence modeling. Nearly all state-of-the-art language models and pre-trained language models are based on the Transformer architecture. However, it distinguishes sequential tokens only with the token position index. We hypothesize that better contextual representations can be generated from the Transformer with richer positional information. To verify this, we propose a segment-aware Transformer (Segatron), by replacing the original token position encoding with a combined position encoding of paragraph, sentence, and token. We first introduce the segment-aware mechanism to Transformer-XL, which is a popular Transformer-based language model with memory extension and relative position encoding. We find that our method can further improve the Transformer-XL base model and large model, achieving 17.1 perplexity on the WikiText-103 dataset. We further investigate the pre-training masked language modeling task with Segatron. Experimental results show that BERT pre-trained with Segatron (SegaBERT) can outperform BERT with vanilla Transformer on various NLP tasks, and outperforms RoBERTa on zero-shot sentence representation learning.
CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization
Venue: arXiv cs.HC 2020
Year: 2020

Deep learning's great success motivates many practitioners and students to learn about this exciting technology. However, it is often challenging for beginners to take their first step due to the complexity of understanding and applying deep learning. We present CNN Explainer, an interactive visualization tool designed for non-experts to learn and examine convolutional neural networks (CNNs), a foundational deep learning model architecture. Our tool addresses key challenges that novices face while learning about CNNs, which we identify from interviews with instructors and a survey with past students. CNN Explainer tightly integrates a model overview that summarizes a CNN's structure, and on-demand, dynamic visual explanation views that help users understand the underlying components of CNNs. Through smooth transitions across levels of abstraction, our tool enables users to inspect the interplay between low-level mathematical operations and high-level model structures. A qualitative user study shows that CNN Explainer helps users more easily understand the inner workings of CNNs, and is engaging and enjoyable to use. We also derive design lessons from our study. Developed using modern web technologies, CNN Explainer runs locally in users' web browsers without the need for installation or specialized hardware, broadening the public's education access to modern deep learning techniques.
Probability of Pilot Interference in Pulsed Radar-Cellular Coexistence: Fundamental Insights on Demodulation and Limited CSI Feedback
Venue: arXiv cs.NI 2020
Year: 2020

This paper considers an underlay pulsed radar-cellular spectrum sharing scenario, where the cellular system uses pilot-aided demodulation, statistical channel state information (S-CSI) estimation and limited feedback schemes. Under a realistic system model, upper and lower bounds are derived on the probability that at least a specified number of pilot signals are interfered by a radar pulse train in a finite CSI estimation window. Exact probabilities are also derived for important special cases which reveal operational regimes where the lower bound is achieved. Using these results, this paper (a) provides insights on pilot interference-minimizing schemes for accurate coherent symbol demodulation, and (b) demonstrates that pilot-aided methods fail to accurately estimate S-CSI of the pulsed radar interference channel for a wide range of radar repetition intervals.
Hide-and-Seek: A Template for Explainable AI
Venue: arXiv cs.LG 2020
Year: 2020

Lack of transparency has been the Achilles heal of Neural Networks and their wider adoption in industry. Despite significant interest this shortcoming has not been adequately addressed. This study proposes a novel framework called Hide-and-Seek (HnS) for training Interpretable Neural Networks and establishes a theoretical foundation for exploring and comparing similar ideas. Extensive experimentation indicates that a high degree of interpretability can be imputed into Neural Networks, without sacrificing their predictive power.
Contextual Text Style Transfer
Venue: arXiv cs.CL 2020
Year: 2020

We introduce a new task, Contextual Text Style Transfer - translating a sentence into a desired style with its surrounding context taken into account. This brings two key challenges to existing style transfer approaches: ($i$) how to preserve the semantic meaning of target sentence and its consistency with surrounding context during transfer; ($ii$) how to train a robust model with limited labeled data accompanied with context. To realize high-quality style transfer with natural context preservation, we propose a Context-Aware Style Transfer (CAST) model, which uses two separate encoders for each input sentence and its surrounding context. A classifier is further trained to ensure contextual consistency of the generated sentence. To compensate for the lack of parallel data, additional self-reconstruction and back-translation losses are introduced to leverage non-parallel data in a semi-supervised fashion. Two new benchmarks, Enron-Context and Reddit-Context, are introduced for formality and offensiveness style transfer. Experimental results on these datasets demonstrate the effectiveness of the proposed CAST model over state-of-the-art methods across style accuracy, content preservation and contextual consistency metrics.
An Efficient Noisy Binary Search in Graphs via Median Approximation
Venue: arXiv cs.DS 2020
Year: 2020

Consider a generalization of the classical binary search problem in linearly sorted data to the graph-theoretic setting. The goal is to design an adaptive query algorithm, called a strategy, that identifies an initially unknown target vertex in a graph by asking queries. Each query is conducted as follows: the strategy selects a vertex $q$ and receives a reply $v$: if $q$ is the target, then $v=q$, and if $q$ is not the target, then $v$ is a neighbor of $q$ that lies on a shortest path to the target. Furthermore, there is a noise parameter $0\leq p<\frac{1}{2}$, which means that each reply can be incorrect with probability $p$. The optimization criterion to be minimized is the overall number of queries asked by the strategy, called the query complexity. The query complexity is well understood to be $O(\varepsilon^{-2}\log n)$ for general graphs, where $n$ is the order of the graph and $\varepsilon=\frac{1}{2}-p$. However, implementing such a strategy is computationally expensive, with each query requiring possibly $O(n^2)$ operations. In this work we propose two efficient strategies that keep the optimal query complexity. The first strategy achieves the overall complexity of $O(\varepsilon^{-1}n\log n)$ per a single query. The second strategy is dedicated to graphs of small diameter $D$ and maximum degree $Δ$ and has the average complexity of $O(n+\varepsilon^{-2}DΔ\log n)$ per query. We stress out that we develop an algorithmic tool of graph median approximation that is of independent interest: the median can be efficiently approximated by finding a vertex minimizing the sum of distances to a randomly sampled vertex subset of size $O(\varepsilon^{-2}\log n)$.
A Novel Structure of Advance Encryption Standard with 3-Dimensional Dynamic S-box and Key Generation Matrix
Venue: arXiv cs.CR 2020
Year: 2020

The study of sending and receiving secret messages is called cryptography. Generally, senders and receivers are unaware about the process of encryption and decryption. Hence, encryption plays an important role in data communication and data security. The meaning of encryption is not only to keep data confidential from unwanted access but also ensuring the data integrity through available way. As the capacity of breaking the security is increasing rapidly, so, the process that hides information is one of the most concerned topics. Advanced Encryption Standard is a popular, widely used and efficient encryption algorithm, which has been used since it was invented. This paper focuses on the AES key generation process and Substitution box. It modifies the conventional key generation technique and builds the dynamic 3-Dimensional S-box of Advance Encryption Standard. The proposed approach suggests 3-Dimensioanl Key Generation Matrix and S-box. As per shown this novel technique increases the amount of time it needs during encryption and decryption. The experimental result shows that it also enhances the strength of AES algorithm. The proposed approach illustrates the theoretical analysis and corresponding experimented results.
Combining predicate transformer semantics for effects: a case study in parsing regular languages
Venue: arXiv cs.LO 2020
Year: 2020

This paper describes how to verify a parser for regular expressions in a functional programming language using predicate transformer semantics for a variety of effects. Where our previous work in this area focused on the semantics for a single effect, parsing requires a combination of effects: non-determinism, general recursion and mutable state. Reasoning about such combinations of effects is notoriously difficult, yet our approach using predicate transformers enables the careful separation of program syntax, correctness proofs and termination proofs.
Multi-dimensional Arrays with Levels
Venue: arXiv cs.DS 2020
Year: 2020

We explore a data structure that generalises rectangular multi-dimensional arrays. The shape of an n-dimensional array is typically given by a tuple of n natural numbers. Each element in that tuple defines the length of the corresponding axis. If we treat this tuple as an array, the shape of that array is described by the single natural number n. A natural number itself can be also treated as an array with the shape described by the natural number 1 (or the element of any singleton set). This observation gives rise to the hierarchy of array types where the shape of an array of level l+1 is a level-l array of natural numbers. Such a hierarchy occurs naturally when treating arrays as containers, which makes it possible to define both rank- and level-polymorphic operations. The former can be found in most array languages, whereas the latter gives rise to partial selections on a large set of hyperplanes, which is often useful in practice. In this paper we present an Agda formalisation of arrays with levels. We show that the proposed formalism supports standard rank-polymorphic array operations, while type system gives static guarantees that indexing is within bounds. We generalise the notion of ranked operator so that it becomes applicable on arrays of arbitrary levels and we show why this may be useful in practice.