OpenRaiser/Intern-Atlas huggingface dataset:
FLAP -- A Federated Learning Framework for Attribute-based Access Control Policies
Venue: arXiv cs.CR 2020
Year: 2020

Technology advances in areas such as sensors, IoT, and robotics, enable new collaborative applications (e.g., autonomous devices). A primary requirement for such collaborations is to have a secure system which enables information sharing and information flow protection. Policy-based management system is a key mechanism for secure selective sharing of protected resources. However, policies in each party of such a collaborative environment cannot be static as they have to adapt to different contexts and situations. One advantage of collaborative applications is that each party in the collaboration can take advantage of knowledge of the other parties for learning or enhancing its own policies. We refer to this learning mechanism as policy transfer. The design of a policy transfer framework has challenges, including policy conflicts and privacy issues. Policy conflicts typically arise because of differences in the obligations of the parties, whereas privacy issues result because of data sharing constraints for sensitive data. Hence, the policy transfer framework should be able to tackle such challenges by considering minimal sharing of data and support policy adaptation to address conflict. In the paper we propose a framework that aims at addressing such challenges. We introduce a formal definition of the policy transfer problem for attribute-based policies. We then introduce the transfer methodology that consists of three sequential steps. Finally we report experimental results.
Evaluating the Cost of Atomic Operations on Modern Architectures
Venue: arXiv cs.DC 2020
Year: 2020

Atomic operations (atomics) such as Compare-and-Swap (CAS) or Fetch-and-Add (FAA) are ubiquitous in parallel programming. Yet, performance tradeoffs between these operations and various characteristics of such systems, such as the structure of caches, are unclear and have not been thoroughly analyzed. In this paper we establish an evaluation methodology, develop a performance model, and present a set of detailed benchmarks for latency and bandwidth of different atomics. We consider various state-of-the-art x86 architectures: Intel Haswell, Xeon Phi, Ivy Bridge, and AMD Bulldozer. The results unveil surprising performance relationships between the considered atomics and architectural properties such as the coherence state of the accessed cache lines. One key finding is that all the tested atomics have comparable latency and bandwidth even if they are characterized by different consensus numbers. Another insight is that the hardware implementation of atomics prevents any instruction-level parallelism even if there are no dependencies between the issued operations. Finally, we discuss solutions to the discovered performance issues in the analyzed architectures. Our analysis enables simpler and more effective parallel programming and accelerates data processing on various architectures deployed in both off-the-shelf machines and large compute systems.
Understanding YouTube Communities via Subscription-based Channel Embeddings
Venue: arXiv cs.LG 2020
Year: 2020

YouTube is an important source of news and entertainment worldwide, but the scale makes it challenging to study the ideas and topics being discussed on the platform. This paper presents new methods to discover and classify YouTube channels which enable the analysis of communities and categories on the platform using orders of magnitude more channels than have been used in previous studies. Instead of using channel and video data as features for classification as other researchers have, these methods use a self-supervised learning approach that leverages the public subscription pages of commenters. We test the classification method on the task of predicting the political lean of YouTube news channels and find that it outperforms the previous best model on the task. Further experiments also show that there are important advantages to using commenter subscriptions to discover channels. The subscription data, along with an iterative approach, is applied to discover, to our current understanding, the most comprehensive set of English language socio-political YouTube channels yet to be analyzed. We experiment with predicting more fine grained political tags for channels using a previously annotated dataset and find that our model performs better than the average individual human reviewer for most of the top tags. This fine grained political tag model is then applied to the newly discovered English language socio-political channels to create a new dataset to analyze the amount of traffic going to different political content. The data shows that some tags, such as "Partisan Right" and "Conspiracy", are significantly under represented when looking only at the most popular socio-political channels. Through the use of our methods, we are able to get a much more accurate picture of the size of these communities on YouTube.
SlimSell: A Vectorizable Graph Representation for Breadth-First Search
Venue: arXiv cs.DC 2020
Year: 2020

Vectorization and GPUs will profoundly change graph processing. Traditional graph algorithms tuned for 32- or 64-bit based memory accesses will be inefficient on architectures with 512-bit wide (or larger) instruction units that are already present in the Intel Knights Landing (KNL) manycore CPU. Anticipating this shift, we propose SlimSell: a vectorizable graph representation to accelerate Breadth-First Search (BFS) based on sparse-matrix dense-vector (SpMV) products. SlimSell extends and combines the state-of-the-art SIMD-friendly Sell-C-sigma matrix storage format with tropical, real, boolean, and sel-max semiring operations. The resulting design reduces the necessary storage (by up to 50%) and thus pressure on the memory subsystem. We augment SlimSell with the SlimWork and SlimChunk schemes that reduce the amount of work and improve load balance, further accelerating BFS. We evaluate all the schemes on Intel Haswell multicore CPUs, the state-of-the-art Intel Xeon Phi KNL manycore CPUs, and NVIDIA Tesla GPUs. Our experiments indicate which semiring offers highest speedups for BFS and illustrate that SlimSell accelerates a tuned Graph500 BFS code by up to 33%. This work shows that vectorization can secure high-performance in BFS based on SpMV products; the proposed principles and designs can be extended to other graph algorithms.
Software Engineering Practices for Scientific Software Development: A Systematic Mapping Study
Venue: arXiv cs.SE 2020
Year: 2020

Background: The development of scientific software applications is far from trivial, due to the constant increase in the necessary complexity of these applications, their increasing size, and their need for intensive maintenance and reuse. Aim: To this end, developers of scientific software (who usually lack a formal computer science background) need to use appropriate software engineering (SE) practices. This paper describes the results of a systematic mapping study on the use of SE for scientific application development and their impact on software quality. Method: To achieve this goal we have performed a systematic mapping study on 359 papers. We first describe a catalogue of SE practices used in scientific software development. Then, we discuss the quality attributes of interest that drive the application of these practices, as well as tentative side-effects of applying the practices on qualities. Results: The main findings indicate that scientific software developers are focusing on practices that improve implementation productivity, such as code reuse, use of third-party libraries, and the application of "good" programming techniques. In addition, apart from the finding that performance is a key-driver for many of these applications, scientific software developers also find maintainability and productivity to be important. Conclusions: The results of the study are compared to existing literature, are interpreted under a software engineering prism, and various implications for researchers and practitioners are provided. One of the key findings of the study, which is considered as important for driving future research endeavors is the lack of evidence on the trade-offs that need to be made when applying a software practice, i.e., negative (indirect) effects on other quality attributes.
Proximal Policy Gradient: PPO with Policy Gradient
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we propose a new algorithm PPG (Proximal Policy Gradient), which is close to both VPG (vanilla policy gradient) and PPO (proximal policy optimization). The PPG objective is a partial variation of the VPG objective and the gradient of the PPG objective is exactly same as the gradient of the VPG objective. To increase the number of policy update iterations, we introduce the advantage-policy plane and design a new clipping strategy. We perform experiments in OpenAI Gym and Bullet robotics environments for ten random seeds. The performance of PPG is comparable to PPO, and the entropy decays slower than PPG. Thus we show that performance similar to PPO can be obtained by using the gradient formula from the original policy gradient theorem.
Cue Me In: Content-Inducing Approaches to Interactive Story Generation
Venue: arXiv cs.CL 2020
Year: 2020

Automatically generating stories is a challenging problem that requires producing causally related and logical sequences of events about a topic. Previous approaches in this domain have focused largely on one-shot generation, where a language model outputs a complete story based on limited initial input from a user. Here, we instead focus on the task of interactive story generation, where the user provides the model mid-level sentence abstractions in the form of cue phrases during the generation process. This provides an interface for human users to guide the story generation. We present two content-inducing approaches to effectively incorporate this additional information. Experimental results from both automatic and human evaluations show that these methods produce more topically coherent and personalized stories compared to baseline methods.
Early Detection of Sepsis using Ensemblers
Venue: arXiv cs.LG 2020
Year: 2020

This paper describes a methodology to detect sepsis ahead of time by analyzing hourly patient records. The Physionet 2019 challenge consists of medical records of over 40,000 patients. Using imputation and weak ensembler technique to analyze these medical records and 3-fold validation, a model is created and validated internally. The model achieved an accuracy of 93.45% and a utility score of 0.271. The utility score as defined by the organizers takes into account true positives, negatives and false alarms.
L-RED: Efficient Post-Training Detection of Imperceptible Backdoor Attacks without Access to the Training Set
Venue: arXiv cs.CV 2020
Year: 2020

Backdoor attacks (BAs) are an emerging form of adversarial attack typically against deep neural network image classifiers. The attacker aims to have the classifier learn to classify to a target class when test images from one or more source classes contain a backdoor pattern, while maintaining high accuracy on all clean test images. Reverse-Engineering-based Defenses (REDs) against BAs do not require access to the training set but only to an independent clean dataset. Unfortunately, most existing REDs rely on an unrealistic assumption that all classes except the target class are source classes of the attack. REDs that do not rely on this assumption often require a large set of clean images and heavy computation. In this paper, we propose a Lagrangian-based RED (L-RED) that does not require knowledge of the number of source classes (or whether an attack is present). Our defense requires very few clean images to effectively detect BAs and is computationally efficient. Notably, we detect 56 out of 60 BAs using only two clean images per class in our experiments on CIFAR-10.
Real-time Localized Photorealistic Video Style Transfer
Venue: arXiv cs.CV 2020
Year: 2020

We present a novel algorithm for transferring artistic styles of semantically meaningful local regions of an image onto local regions of a target video while preserving its photorealism. Local regions may be selected either fully automatically from an image, through using video segmentation algorithms, or from casual user guidance such as scribbles. Our method, based on a deep neural network architecture inspired by recent work in photorealistic style transfer, is real-time and works on arbitrary inputs without runtime optimization once trained on a diverse dataset of artistic styles. By augmenting our video dataset with noisy semantic labels and jointly optimizing over style, content, mask, and temporal losses, our method can cope with a variety of imperfections in the input and produce temporally coherent videos without visual artifacts. We demonstrate our method on a variety of style images and target videos, including the ability to transfer different styles onto multiple objects simultaneously, and smoothly transition between styles in time.
PROP: Pre-training with Representative Words Prediction for Ad-hoc Retrieval
Venue: arXiv cs.IR 2020
Year: 2020

Recently pre-trained language representation models such as BERT have shown great success when fine-tuned on downstream tasks including information retrieval (IR). However, pre-training objectives tailored for ad-hoc retrieval have not been well explored. In this paper, we propose Pre-training with Representative wOrds Prediction (PROP) for ad-hoc retrieval. PROP is inspired by the classical statistical language model for IR, specifically the query likelihood model, which assumes that the query is generated as the piece of text representative of the "ideal" document. Based on this idea, we construct the representative words prediction (ROP) task for pre-training. Given an input document, we sample a pair of word sets according to the document language model, where the set with higher likelihood is deemed as more representative of the document. We then pre-train the Transformer model to predict the pairwise preference between the two word sets, jointly with the Masked Language Model (MLM) objective. By further fine-tuning on a variety of representative downstream ad-hoc retrieval tasks, PROP achieves significant improvements over baselines without pre-training or with other pre-training methods. We also show that PROP can achieve exciting performance under both the zero- and low-resource IR settings. The code and pre-trained models are available at https://github.com/Albert-Ma/PROP.
Text Classification of Manifestos and COVID-19 Press Briefings using BERT and Convolutional Neural Networks
Venue: arXiv cs.CL 2020
Year: 2020

We build a sentence-level political discourse classifier using existing human expert annotated corpora of political manifestos from the Manifestos Project (Volkens et al., 2020a) and applying them to a corpus ofCOVID-19Press Briefings (Chatsiou, 2020). We use manually annotated political manifestos as training data to train a local topic ConvolutionalNeural Network (CNN) classifier; then apply it to the COVID-19PressBriefings Corpus to automatically classify sentences in the test corpus.We report on a series of experiments with CNN trained on top of pre-trained embeddings for sentence-level classification tasks. We show thatCNN combined with transformers like BERT outperforms CNN combined with other embeddings (Word2Vec, Glove, ELMo) and that it is possible to use a pre-trained classifier to conduct automatic classification on different political texts without additional training.
Bi-directional Cognitive Thinking Network for Machine Reading Comprehension
Venue: arXiv cs.CL 2020
Year: 2020

We propose a novel Bi-directional Cognitive Knowledge Framework (BCKF) for reading comprehension from the perspective of complementary learning systems theory. It aims to simulate two ways of thinking in the brain to answer questions, including reverse thinking and inertial thinking. To validate the effectiveness of our framework, we design a corresponding Bi-directional Cognitive Thinking Network (BCTN) to encode the passage and generate a question (answer) given an answer (question) and decouple the bi-directional knowledge. The model has the ability to reverse reasoning questions which can assist inertial thinking to generate more accurate answers. Competitive improvement is observed in DuReader dataset, confirming our hypothesis that bi-directional knowledge helps the QA task. The novel framework shows an interesting perspective on machine reading comprehension and cognitive science.
Extracting Seasonal Gradual Patterns from Temporal Sequence Data Using Periodic Patterns Mining
Venue: arXiv cs.LG 2020
Year: 2020

Mining frequent episodes aims at recovering sequential patterns from temporal data sequences, which can then be used to predict the occurrence of related events in advance. On the other hand, gradual patterns that capture co-variation of complex attributes in the form of " when X increases/decreases, Y increases/decreases" play an important role in many real world applications where huge volumes of complex numerical data must be handled. Recently, these patterns have received attention from the data mining community exploring temporal data who proposed methods to automatically extract gradual patterns from temporal data. However, to the best of our knowledge, no method has been proposed to extract gradual patterns that regularly appear at identical time intervals in many sequences of temporal data, despite the fact that such patterns may add knowledge to certain applications, such as e-commerce. In this paper, we propose to extract co-variations of periodically repeating attributes from the sequences of temporal data that we call seasonal gradual patterns. For this purpose, we formulate the task of mining seasonal gradual patterns as the problem of mining periodic patterns in multiple sequences and then we exploit periodic pattern mining algorithms to extract seasonal gradual patterns. We discuss specific features of these patterns and propose an approach for their extraction based on mining periodic frequent patterns common to multiple sequences. We also propose a new anti-monotonous support definition associated to these seasonal gradual patterns. The illustrative results obtained from some real world data sets show that the proposed approach is efficient and that it can extract small sets of patterns by filtering numerous nonseasonal patterns to identify the seasonal ones.
Interpretable Deep Learning for Automatic Diagnosis of 12-lead Electrocardiogram
Venue: arXiv cs.LG 2020
Year: 2020

Electrocardiogram (ECG) is a widely used reliable, non-invasive approach for cardiovascular disease diagnosis. With the rapid growth of ECG examinations and the insufficiency of cardiologists, accurate and automatic diagnosis of ECG signals has become a hot research topic. Deep learning methods have demonstrated promising results in predictive healthcare tasks. In this paper, we developed a deep neural network for multi-label classification of cardiac arrhythmias in 12-lead ECG recordings. Experiments on a public 12-lead ECG dataset showed the effectiveness of our method. The proposed model achieved an average area under the receiver operating characteristic curve (AUC) of 0.970 and an average F1 score of 0.813. The deep model showed superior performance than 4 machine learning methods learned from extracted expert features. Besides, the deep models trained on single-lead ECGs produce lower performance than using all 12 leads simultaneously. The best-performing leads are lead I, aVR, and V5 among 12 leads. Finally, we employed the SHapley Additive exPlanations (SHAP) method to interpret the model's behavior at both patient level and population level. Our code is freely available at https://github.com/onlyzdd/ecg-diagnosis.
Snap-and-Chat Protocols: System Aspects
Venue: arXiv cs.CR 2020
Year: 2020

The availability-finality dilemma says that blockchain protocols cannot be both available under dynamic participation and safe under network partition. Snap-and-chat protocols have recently been proposed as a resolution to this dilemma. A snap-and-chat protocol produces an always available ledger containing a finalized prefix ledger which is always safe and catches up with the available ledger whenever network conditions permit. In contrast to existing handcrafted finality gadget based designs like Ethereum 2.0's consensus protocol Gasper, snap-and-chat protocols are constructed as a black-box composition of off-the-shelf BFT and longest chain protocols. In this paper, we consider system aspects of snap-and-chat protocols and show how they can provide two important features: 1) accountability, 2) support of light clients. Through this investigation, a deeper understanding of the strengths and challenges of snap-and-chat protocols is gained.
Tilting at windmills: Data augmentation for deep pose estimation does not help with occlusions
Venue: arXiv cs.CV 2020
Year: 2020

Occlusion degrades the performance of human pose estimation. In this paper, we introduce targeted keypoint and body part occlusion attacks. The effects of the attacks are systematically analyzed on the best performing methods. In addition, we propose occlusion specific data augmentation techniques against keypoint and part attacks. Our extensive experiments show that human pose estimation methods are not robust to occlusion and data augmentation does not solve the occlusion problems.
Compressed Super-Resolution of Positive Sources
Venue: arXiv cs.IT 2020
Year: 2020

Atomic norm minimization is a convex optimization framework to recover point sources from a subset of their low-pass observations, or equivalently the underlying frequencies of a spectrally-sparse signal. When the amplitudes of the sources are positive, a positive atomic norm can be formulated, and exact recovery can be ensured without imposing a separation between the sources, as long as the number of observations is greater than the number of sources. However, the classic formulation of the atomic norm requires to solve a semidefinite program involving a linear matrix inequality of a size on the order of the signal dimension, which can be prohibitive. In this letter, we introduce a novel "compressed" semidefinite program, which involves a linear matrix inequality of a reduced dimension on the order of the number of sources. We guarantee the tightness of this program under certain conditions on the operator involved in the dimensionality reduction. Finally, we apply the proposed method to direction finding over sparse arrays based on second-order statistics and achieve significant computational savings.
Dual Averaging is Surprisingly Effective for Deep Learning Optimization
Venue: arXiv cs.LG 2020
Year: 2020

First-order stochastic optimization methods are currently the most widely used class of methods for training deep neural networks. However, the choice of the optimizer has become an ad-hoc rule that can significantly affect the performance. For instance, SGD with momentum (SGD+M) is typically used in computer vision (CV) and Adam is used for training transformer models for Natural Language Processing (NLP). Using the wrong method can lead to significant performance degradation. Inspired by the dual averaging algorithm, we propose Modernized Dual Averaging (MDA), an optimizer that is able to perform as well as SGD+M in CV and as Adam in NLP. Our method is not adaptive and is significantly simpler than Adam. We show that MDA induces a decaying uncentered $L_2$-regularization compared to vanilla SGD+M and hypothesize that this may explain why it works on NLP problems where SGD+M fails.