OpenRaiser/Intern-Atlas huggingface dataset:
Sensitive Data Detection with High-Throughput Neural Network Models for Financial Institutions
Venue: arXiv cs.LG 2020
Year: 2020

Named Entity Recognition has been extensively investigated in many fields. However, the application of sensitive entity detection for production systems in financial institutions has not been well explored due to the lack of publicly available, labeled datasets. In this paper, we use internal and synthetic datasets to evaluate various methods of detecting NPI (Nonpublic Personally Identifiable) information commonly found within financial institutions, in both unstructured and structured data formats. Character-level neural network models including CNN, LSTM, BiLSTM-CRF, and CNN-CRF are investigated on two prediction tasks: (i) entity detection on multiple data formats, and (ii) column-wise entity prediction on tabular datasets. We compare these models with other standard approaches on both real and synthetic data, with respect to F1-score, precision, recall, and throughput. The real datasets include internal structured data and public email data with manually tagged labels. Our experimental results show that the CNN model is simple yet effective with respect to accuracy and throughput and thus, is the most suitable candidate model to be deployed in the production environment(s). Finally, we provide several lessons learned on data limitations, data labelling and the intrinsic overlap of data entities.
Learning to Solve AC Optimal Power Flow by Differentiating through Holomorphic Embeddings
Venue: arXiv cs.LG 2020
Year: 2020

Alternating current optimal power flow (AC-OPF) is one of the fundamental problems in power systems operation. AC-OPF is traditionally cast as a constrained optimization problem that seeks optimal generation set points whilst fulfilling a set of non-linear equality constraints -- the power flow equations. With increasing penetration of renewable generation, grid operators need to solve larger problems at shorter intervals. This motivates the research interest in learning OPF solutions with neural networks, which have fast inference time and is potentially scalable to large networks. The main difficulty in solving the AC-OPF problem lies in dealing with this equality constraint that has spurious roots, i.e. there are assignments of voltages that fulfill the power flow equations that however are not physically realizable. This property renders any method relying on projected-gradients brittle because these non-physical roots can act as attractors. In this paper, we show efficient strategies that circumvent this problem by differentiating through the operations of a power flow solver that embeds the power flow equations into a holomorphic function. The resulting learning-based approach is validated experimentally on a 200-bus system and we show that, after training, the learned agent produces optimized power flow solutions reliably and fast. Specifically, we report a 12x increase in speed and a 40% increase in robustness compared to a traditional solver. To the best of our knowledge, this approach constitutes the first learning-based approach that successfully respects the full non-linear AC-OPF equations.
Hate Speech detection in the Bengali language: A dataset and its baseline evaluation
Venue: arXiv cs.CL 2020
Year: 2020

Social media sites such as YouTube and Facebook have become an integral part of everyone's life and in the last few years, hate speech in the social media comment section has increased rapidly. Detection of hate speech on social media websites faces a variety of challenges including small imbalanced data sets, the findings of an appropriate model and also the choice of feature analysis method. further more, this problem is more severe for the Bengali speaking community due to the lack of gold standard labelled datasets. This paper presents a new dataset of 30,000 user comments tagged by crowd sourcing and varified by experts. All the comments are collected from YouTube and Facebook comment section and classified into seven categories: sports, entertainment, religion, politics, crime, celebrity and TikTok & meme. A total of 50 annotators annotated each comment three times and the majority vote was taken as the final annotation. Nevertheless, we have conducted base line experiments and several deep learning models along with extensive pre-trained Bengali word embedding such as Word2Vec, FastText and BengFastText on this dataset to facilitate future research opportunities. The experiment illustrated that although all deep learning models performed well, SVM achieved the best result with 87.5% accuracy. Our core contribution is to make this benchmark dataset available and accessible to facilitate further research in the field of in the field of Bengali hate speech detection.
Trellis Code Error Exponent From Results for Asynchronous Multiple Access Channels
Venue: arXiv cs.IT 2020
Year: 2020

An asynchronous multiple access error exponent result implicates a new result for time invariant trellis codes of memory 1.
On the performance of deep learning for numerical optimization: an application to protein structure prediction
Venue: arXiv cs.NE 2020
Year: 2020

Deep neural networks have recently drawn considerable attention to build and evaluate artificial learning models for perceptual tasks. Here, we present a study on the performance of the deep learning models to deal with global optimization problems. The proposed approach adopts the idea of the neural architecture search (NAS) to generate efficient neural networks for solving the problem at hand. The space of network architectures is represented using a directed acyclic graph and the goal is to find the best architecture to optimize the objective function for a new, previously unknown task. Different from proposing very large networks with GPU computational burden and long training time, we focus on searching for lightweight implementations to find the best architecture. The performance of NAS is first analyzed through empirical experiments on CEC 2017 benchmark suite. Thereafter, it is applied to a set of protein structure prediction (PSP) problems. The experiments reveal that the generated learning models can achieve competitive results when compared to hand-designed algorithms; given enough computational budget
Interpretable Image Clustering via Diffeomorphism-Aware K-Means
Venue: arXiv cs.CV 2020
Year: 2020

We design an interpretable clustering algorithm aware of the nonlinear structure of image manifolds. Our approach leverages the interpretability of $K$-means applied in the image space while addressing its clustering performance issues. Specifically, we develop a measure of similarity between images and centroids that encompasses a general class of deformations: diffeomorphisms, rendering the clustering invariant to them. Our work leverages the Thin-Plate Spline interpolation technique to efficiently learn diffeomorphisms best characterizing the image manifolds. Extensive numerical simulations show that our approach competes with state-of-the-art methods on various datasets.
Deep Learning Techniques for Super-Resolution in Video Games
Venue: arXiv cs.NE 2020
Year: 2020

The computational cost of video game graphics is increasing and hardware for processing graphics is struggling to keep up. This means that computer scientists need to develop creative new ways to improve the performance of graphical processing hardware. Deep learning techniques for video super-resolution can enable video games to have high quality graphics whilst offsetting much of the computational cost. These emerging technologies allow consumers to have improved performance and enjoyment from video games and have the potential to become standard within the game development industry.
Treatment Targeting by AUUC Maximization with Generalization Guarantees
Venue: arXiv cs.LG 2020
Year: 2020

We consider the task of optimizing treatment assignment based on individual treatment effect prediction. This task is found in many applications such as personalized medicine or targeted advertising and has gained a surge of interest in recent years under the name of Uplift Modeling. It consists in targeting treatment to the individuals for whom it would be the most beneficial. In real life scenarios, when we do not have access to ground-truth individual treatment effect, the capacity of models to do so is generally measured by the Area Under the Uplift Curve (AUUC), a metric that differs from the learning objectives of most of the Individual Treatment Effect (ITE) models. We argue that the learning of these models could inadvertently degrade AUUC and lead to suboptimal treatment assignment. To tackle this issue, we propose a generalization bound on the AUUC and present a novel learning algorithm that optimizes a derivable surrogate of this bound, called AUUC-max. Finally, we empirically demonstrate the tightness of this generalization bound, its effectiveness for hyper-parameter tuning and show the efficiency of the proposed algorithm compared to a wide range of competitive baselines on two classical benchmarks.
Individually Conditional Individual Mutual Information Bound on Generalization Error
Venue: arXiv cs.IT 2020
Year: 2020

We propose a new information-theoretic bound on generalization error based on a combination of the error decomposition technique of Bu et al. and the conditional mutual information (CMI) construction of Steinke and Zakynthinou. In a previous work, Haghifam et al. proposed a different bound combining the two aforementioned techniques, which we refer to as the conditional individual mutual information (CIMI) bound. However, in a simple Gaussian setting, both the CMI and the CIMI bounds are order-wise worse than that by Bu et al.. This observation motivated us to propose the new bound, which overcomes this issue by reducing the conditioning terms in the conditional mutual information. In the process of establishing this bound, a conditional decoupling lemma is established, which also leads to a meaningful dichotomy and comparison among these information-theoretic bounds.
Toward Transformer-Based Object Detection
Venue: arXiv cs.CV 2020
Year: 2020

Transformers have become the dominant model in natural language processing, owing to their ability to pretrain on massive amounts of data, then transfer to smaller, more specific tasks via fine-tuning. The Vision Transformer was the first major attempt to apply a pure transformer model directly to images as input, demonstrating that as compared to convolutional networks, transformer-based architectures can achieve competitive results on benchmark classification tasks. However, the computational complexity of the attention operator means that we are limited to low-resolution inputs. For more complex tasks such as detection or segmentation, maintaining a high input resolution is crucial to ensure that models can properly identify and reflect fine details in their output. This naturally raises the question of whether or not transformer-based architectures such as the Vision Transformer are capable of performing tasks other than classification. In this paper, we determine that Vision Transformers can be used as a backbone by a common detection task head to produce competitive COCO results. The model that we propose, ViT-FRCNN, demonstrates several known properties associated with transformers, including large pretraining capacity and fast fine-tuning performance. We also investigate improvements over a standard detection backbone, including superior performance on out-of-domain images, better performance on large objects, and a lessened reliance on non-maximum suppression. We view ViT-FRCNN as an important stepping stone toward a pure-transformer solution of complex vision tasks such as object detection.
Understanding The Top 10 OWASP Vulnerabilities
Venue: arXiv cs.CR 2020
Year: 2020

Understanding the common vulnerabilities in web applications help businesses be better prepared in protecting their data against such attacks. With the knowledge gained from research users and developers can be better equipped to deal with the most common attacks and form solutions to prevent future attacks against their web applications. Vulnerabilities exist in many forms within modern web applications which can be easily mitigated with investment of time and research.
Relightable 3D Head Portraits from a Smartphone Video
Venue: arXiv cs.CV 2020
Year: 2020

In this work, a system for creating a relightable 3D portrait of a human head is presented. Our neural pipeline operates on a sequence of frames captured by a smartphone camera with the flash blinking (flash-no flash sequence). A coarse point cloud reconstructed via structure-from-motion software and multi-view denoising is then used as a geometric proxy. Afterwards, a deep rendering network is trained to regress dense albedo, normals, and environmental lighting maps for arbitrary new viewpoints. Effectively, the proxy geometry and the rendering network constitute a relightable 3D portrait model, that can be synthesized from an arbitrary viewpoint and under arbitrary lighting, e.g. directional light, point light, or an environment map. The model is fitted to the sequence of frames with human face-specific priors that enforce the plausibility of albedo-lighting decomposition and operates at the interactive frame rate. We evaluate the performance of the method under varying lighting conditions and at the extrapolated viewpoints and compare with existing relighting methods.
Handling uncertainty using features from pathology: opportunities in primary care data for developing high risk cancer survival methods
Venue: arXiv cs.LG 2020
Year: 2020

More than 144 000 Australians were diagnosed with cancer in 2019. The majority will first present to their GP symptomatically, even for cancer for which screening programs exist. Diagnosing cancer in primary care is challenging due to the non-specific nature of cancer symptoms and its low prevalence. Understanding the epidemiology of cancer symptoms and patterns of presentation in patient's medical history from primary care data could be important to improve earlier detection and cancer outcomes. As past medical data about a patient can be incomplete, irregular or missing, this creates additional challenges when attempting to use the patient's history for any new diagnosis. Our research aims to investigate the opportunities in a patient's pathology history available to a GP, initially focused on the results within the frequently ordered full blood count to determine relevance to a future high-risk cancer prognosis, and treatment outcome. We investigated how past pathology test results can lead to deriving features that can be used to predict cancer outcomes, with emphasis on patients at risk of not surviving the cancer within 2-year period. This initial work focuses on patients with lung cancer, although the methodology can be applied to other types of cancer and other data within the medical record. Our findings indicate that even in cases of incomplete or obscure patient history, hematological measures can be useful in generating features relevant for predicting cancer risk and survival. The results strongly indicate to add the use of pathology test data for potential high-risk cancer diagnosis, and the utilize additional pathology metrics or other primary care datasets even more for similar purposes.
Representation-Free Model Predictive Control for Dynamic Motions in Quadrupeds
Venue: arXiv cs.RO 2020
Year: 2020

This paper presents a novel Representation-Free Model Predictive Control (RF-MPC) framework for controlling various dynamic motions of a quadrupedal robot in three dimensional (3D) space. Our formulation directly represents the rotational dynamics using the rotation matrix, which liberates us from the issues associated with the use of Euler angles and quaternion as the orientation representations. With a variation-based linearization scheme and a carefully constructed cost function, the MPC control law is transcribed to the standard Quadratic Program (QP) form. The MPC controller can operate at real-time rates of 250 Hz on a quadruped robot. Experimental results including periodic quadrupedal gaits and a controlled backflip validate that our control strategy could stabilize dynamic motions that involve singularity in 3D maneuvers.
Self-supervised Learning with Fully Convolutional Networks
Venue: arXiv cs.CV 2020
Year: 2020

Although deep learning based methods have achieved great success in many computer vision tasks, their performance relies on a large number of densely annotated samples that are typically difficult to obtain. In this paper, we focus on the problem of learning representation from unlabeled data for semantic segmentation. Inspired by two patch-based methods, we develop a novel self-supervised learning framework by formulating the Jigsaw Puzzle problem as a patch-wise classification process and solving it with a fully convolutional network. By learning to solve a Jigsaw Puzzle problem with 25 patches and transferring the learned features to semantic segmentation task on Cityscapes dataset, we achieve a 5.8 percentage point improvement over the baseline model that initialized from random values. Moreover, experiments show that our self-supervised learning method can be applied to different datasets and models. In particular, we achieved competitive performance with the state-of-the-art methods on the PASCAL VOC2012 dataset using significant fewer training images.
Fast and Efficient Parallel Breadth-First Search with Power-law Graph Transformation
Venue: arXiv cs.DS 2020
Year: 2020

In the big data era, graph computing is widely used to exploit the hidden value in real-world graphs in various scenarios such as social networks, knowledge graphs, web searching, and recommendation systems. However, the random memory accesses result in inefficient use of cache and the irregular degree distribution leads to substantial load imbalance. Breadth-First Search (BFS) is frequently utilized as a kernel for many important and complex graph algorithms. In this paper, we describe a preprocessing approach using Reverse Cuthill-Mckee (RCM) algorithm to improve data locality and demonstrate how to achieve an efficient load balancing for BFS. Computations on RCM-reordered graph data are also accelerated with SIMD executions. We evaluate the performance of the graph preprocessing approach on Kronecker graphs of the Graph500 benchmark and real-world graphs. Our BFS implementation on RCM-reordered graph data achieves 326.48 MTEPS/W (mega TEPS per watt) on an ARMv8 system, ranking 2nd on the Green Graph500 list in June 2020 (the 1st rank uses GPU acceleration).
Transfer Learning Based Automatic Model Creation Tool For Resource Constraint Devices
Venue: arXiv cs.LG 2020
Year: 2020

With the enhancement of Machine Learning, many tools are being designed to assist developers to easily create their Machine Learning models. In this paper, we propose a novel method for auto creation of such custom models for constraint devices using transfer learning without the need to write any machine learning code. We share the architecture of our automatic model creation tool and the CNN Model created by it using pretrained models such as YAMNet and MobileNetV2 as feature extractors. Finally, we demonstrate accuracy and memory footprint of the model created from the tool by creating an Automatic Image and Audio classifier and report the results of our experiments using Stanford Cars and ESC-50 dataset.
Fairness and Accuracy in Federated Learning
Venue: arXiv cs.LG 2020
Year: 2020

In the federated learning setting, multiple clients jointly train a model under the coordination of the central server, while the training data is kept on the client to ensure privacy. Normally, inconsistent distribution of data across different devices in a federated network and limited communication bandwidth between end devices impose both statistical heterogeneity and expensive communication as major challenges for federated learning. This paper proposes an algorithm to achieve more fairness and accuracy in federated learning (FedFa). It introduces an optimization scheme that employs a double momentum gradient, thereby accelerating the convergence rate of the model. An appropriate weight selection algorithm that combines the information quantity of training accuracy and training frequency to measure the weights is proposed. This procedure assists in addressing the issue of unfairness in federated learning due to preferences for certain clients. Our results show that the proposed FedFa algorithm outperforms the baseline algorithm in terms of accuracy and fairness.
Semantics and explanation: why counterfactual explanations produce adversarial examples in deep neural networks
Venue: arXiv cs.AI 2020
Year: 2020

Recent papers in explainable AI have made a compelling case for counterfactual modes of explanation. While counterfactual explanations appear to be extremely effective in some instances, they are formally equivalent to adversarial examples. This presents an apparent paradox for explainability researchers: if these two procedures are formally equivalent, what accounts for the explanatory divide apparent between counterfactual explanations and adversarial examples? We resolve this paradox by placing emphasis back on the semantics of counterfactual expressions. Producing satisfactory explanations for deep learning systems will require that we find ways to interpret the semantics of hidden layer representations in deep neural networks.
Petri Net Based Symbolic Model Checking for Computation Tree Logic of Knowledge
Venue: arXiv cs.SE 2020
Year: 2020

Computation Tree Logic of Knowledge (CTLK) can specify many design requirements of privacy and security of multi-agent systems (MAS). In our conference paper, we defined Knowledge-oriented Petri Nets (KPN) to model MAS and proposed Reachability Graphs with Equivalence Relations (RGER) to verify CTLK. In this paper, we use the technique of Ordered Binary Decision Diagrams (OBDD) to encode RGER in order to alleviate the state explosion problem and enhance the verification efficiency. We propose a heuristic method to order those variables in OBDD, which can well improve the time and space performance of producing, encoding and exploring a huge state space. More importantly, our method does not produce and encode any transition or equivalence relation of states when producing and encoding an RGER, and in fact it dynamically produces those transition or equivalence relations that are required in the verification process of CTLK formulas. This policy can save a lot of time and space since the number of transition or equivalence relations of states is much greater than the number of states themselves. We design symbolic model checking algorithms, develop a tool and apply them to two famous examples: Alice-Bob Protocol and Dining Cryptographers Protocol. We compare our tool with MCMAS which is the state-of-the-art model checker of verifying CTLK. The experimental results illustrate the advantages of our model and method. Our tool running in a general PC can totally spend less than 14 hours to verify Dining Cryptographers Protocol with 1200 concurrent cryptographers where there are about $10^{1080}$ states and the two verified CTLK formulas have more than 6000 atomic propositions and more than 3600 operators. These good performances are owed to a combination of the OBDD technique and the structure characteristics of KPN.