OpenRaiser/Intern-Atlas huggingface dataset:
Adaptive Anomaly Detection in Chaotic Time Series with a Spatially Aware Echo State Network
Venue: arXiv cs.NE 2019
Year: 2019

This work builds an automated anomaly detection method for chaotic time series, and more concretely for turbulent, high-dimensional, ocean simulations. We solve this task by extending the Echo State Network by spatially aware input maps, such as convolutions, gradients, cosine transforms, et cetera, as well as a spatially aware loss function. The spatial ESN is used to create predictions which reduce the detection problem to thresholding of the prediction error. We benchmark our detection framework on different tasks of increasing difficulty to show the generality of the framework before applying it to raw climate model output in the region of the Japanese ocean current Kuroshio, which exhibits a bimodality that is not easily detected by the naked eye. The code is available as an open source Python package, Torsk, available at https://github.com/nmheim/torsk, where we also provide supplementary material and programs that reproduce the results shown in this paper.
Beyond Human-Level Accuracy: Computational Challenges in Deep Learning
Venue: arXiv cs.LG 2019
Year: 2019

Deep learning (DL) research yields accuracy and product improvements from both model architecture changes and scale: larger data sets and models, and more computation. For hardware design, it is difficult to predict DL model changes. However, recent prior work shows that as dataset sizes grow, DL model accuracy and model size grow predictably. This paper leverages the prior work to project the dataset and model size growth required to advance DL accuracy beyond human-level, to frontier targets defined by machine learning experts. Datasets will need to grow $33$--$971 \times$, while models will need to grow $6.6$--$456\times$ to achieve target accuracies. We further characterize and project the computational requirements to train these applications at scale. Our characterization reveals an important segmentation of DL training challenges for recurrent neural networks (RNNs) that contrasts with prior studies of deep convolutional networks. RNNs will have comparatively moderate operational intensities and very large memory footprint requirements. In contrast to emerging accelerator designs, large-scale RNN training characteristics suggest designs with significantly larger memory capacity and on-chip caches.
Gröbner Bases with Reduction Machines
Venue: arXiv cs.SC 2019
Year: 2019

In this paper, we make a contribution to the computation of Gröbner bases. For polynomial reduction, instead of choosing the leading monomial of a polynomial as the monomial with respect to which the reduction process is carried out, we investigate what happens if we make that choice arbitrarily. It turns out not only this is possible (the fact that this produces a normal form being already known in the literature), but, for a fixed choice of reductors, the obtained normal form is the same no matter the order in which we reduce the monomials. To prove this, we introduce reduction machines, which work by reducing each monomial independently and then collecting the result. We show that such a machine can simulate any such reduction. We then discuss different implementations of these machines. Some of these implementations address inherent inefficiencies in reduction machines (repeating the same computations). We describe a first implementation and look at some experimental results.
SATURN -- Software Deobfuscation Framework Based on LLVM
Venue: arXiv cs.CR 2019
Year: 2019

The strength of obfuscated software has increased over the recent years. Compiler based obfuscation has become the de facto standard in the industry and recent papers also show that injection of obfuscation techniques is done at the compiler level. In this paper we discuss a generic approach for deobfuscation and recompilation of obfuscated code based on the compiler framework LLVM. We show how binary code can be lifted back into the compiler intermediate language LLVM-IR and explain how we recover the control flow graph of an obfuscated binary function with an iterative control flow graph construction algorithm based on compiler optimizations and SMT solving. Our approach does not make any assumptions about the obfuscated code, but instead uses strong compiler optimizations available in LLVM and Souper Optimizer to simplify away the obfuscation. Our experimental results show that this approach can be effective to weaken or even remove the applied obfuscation techniques like constant unfolding, certain arithmetic-based opaque expressions, dead code insertions, bogus control flow or integer encoding found in public and commercial obfuscators. The recovered LLVM-IR can be further processed by custom deobfuscation passes that are now applied at the same level as the injected obfuscation techniques or recompiled with one of the available LLVM backends. The presented work is implemented in a deobfuscation tool called SATURN.
Spiking Neural Networks for Inference and Learning: A Memristor-based Design Perspective
Venue: arXiv cs.ET 2019
Year: 2019

On metrics of density and power efficiency, neuromorphic technologies have the potential to surpass mainstream computing technologies in tasks where real-time functionality, adaptability, and autonomy are essential. While algorithmic advances in neuromorphic computing are proceeding successfully, the potential of memristors to improve neuromorphic computing have not yet born fruit, primarily because they are often used as a drop-in replacement to conventional memory. However, interdisciplinary approaches anchored in machine learning theory suggest that multifactor plasticity rules matching neural and synaptic dynamics to the device capabilities can take better advantage of memristor dynamics and its stochasticity. Furthermore, such plasticity rules generally show much higher performance than that of classical Spike Time Dependent Plasticity (STDP) rules. This chapter reviews the recent development in learning with spiking neural network models and their possible implementation with memristor-based hardware.
GPU-based parallelism for ASP-solving
Venue: arXiv cs.AI 2019
Year: 2019

Answer Set Programming (ASP) has become, the paradigm of choice in the field of logic programming and non-monotonic reasoning. Thanks to the availability of efficient solvers, ASP has been successfully employed in a large number of application domains. The term GPU-computing indicates a recent programming paradigm aimed at enabling the use of modern parallel Graphical Processing Units (GPUs) for general purpose computing. In this paper we describe an approach to ASP-solving that exploits GPU parallelism. The design of a GPU-based solver poses various challenges due to the peculiarities of GPUs' software and hardware architectures and to the intrinsic nature of the satisfiability problem.
Employ Multimodal Machine Learning for Content quality analysis
Venue: arXiv cs.IR 2019
Year: 2019

The task of identifying high-quality content becomes increasingly important, and it can improve overall reading time and CTR(click-through rate estimates). Generalizes quality analysis only focused on single Modal,such as image or text,but in today's mainstream media sites a lot of information is presented in graphic form.In this paper we propose a MultiModal quality recognition approach for the quality score. First we use two feature extractors,one for image and another for the text. After that we use an Siamese Network with the rank loss as the optimization objective.Compare with other approach,our approach get a more accuracy result.
ICDM 2019 Knowledge Graph Contest: Team UWA
Venue: arXiv cs.CL 2019
Year: 2019

We present an overview of our triple extraction system for the ICDM 2019 Knowledge Graph Contest. Our system uses a pipeline-based approach to extract a set of triples from a given document. It offers a simple and effective solution to the challenge of knowledge graph construction from domain-specific text. It also provides the facility to visualise useful information about each triple such as the degree, betweenness, structured relation type(s), and named entity types.
PISEP^2: Pseudo Image Sequence Evolution based 3D Pose Prediction
Venue: arXiv cs.CV 2019
Year: 2019

Pose prediction is to predict future poses given a window of previous poses. In this paper, we propose a new problem that predicts poses using 3D joint coordinate sequences. Different from the traditional pose prediction based on Mocap frames, this problem is convenient to use in real applications due to its simple sensors to capture data. We also present a new framework, PISEP^2 (Pseudo Image Sequence Evolution based 3D Pose Prediction), to address this new problem. Specifically, a skeletal representation is proposed by transforming the joint coordinate sequence into an image sequence, which can model the different correlations of different joints. With this image based skeletal representation, we model the pose prediction as the evolution of image sequence. Moreover, a novel inference network is proposed to predict all future poses in one step by decoupling the decoders in a non-recursive manner. Compared with the recursive sequence to sequence model, we can improve the computational efficiency and avoid error accumulation significantly. Extensive experiments are carried out on two benchmark datasets (e.g. G3D and FNTU). The proposed method achieves the state-of-the-art performance on both datasets, which demonstrates the effectiveness of our proposed method.
Blockchain-Powered Software Defined Network-Enabled Networking Infrastructure for Cloud Management
Venue: arXiv cs.CR 2019
Year: 2019

Cloud architecture has become a valuable solution for different applications, such as big data analytics, due to its high-degree of availability, scalability and strategic value. However, there still remain challenges in managing cloud architecture, in areas such as cloud security. In this paper, we exploit software-defined networking (SDN) and blockchain technologies to secure cloud management platforms from a networking perspective. We develop a blockchain-powered SDN-enabled networking infrastructure in which the integration between blockchain-based security and autonomy management layer and multi-controller SDN networking layer is defined to enhance the integrity of the control and management messages. Furthermore, our proposed networking infrastructure also enables the autonomous bandwidth provisioning to enhance the availability of cloud architecture. In the simulation section, we evaluate the performance of our proposed blockchain-powered SDN-enabled networking infrastructure by considering different scenarios.
Understanding Bias in Machine Learning
Venue: arXiv cs.LG 2019
Year: 2019

Bias is known to be an impediment to fair decisions in many domains such as human resources, the public sector, health care etc. Recently, hope has been expressed that the use of machine learning methods for taking such decisions would diminish or even resolve the problem. At the same time, machine learning experts warn that machine learning models can be biased as well. In this article, our goal is to explain the issue of bias in machine learning from a technical perspective and to illustrate the impact that biased data can have on a machine learning model. To reach such a goal, we develop interactive plots to visualizing the bias learned from synthetic data.
3D Bounding Box Estimation for Autonomous Vehicles by Cascaded Geometric Constraints and Depurated 2D Detections Using 3D Results
Venue: arXiv cs.CV 2019
Year: 2019

3D object detection is one of the most important tasks in 3D vision perceptual system of autonomous vehicles. In this paper, we propose a novel two stage 3D object detection method aimed at get the optimal solution of object location in 3D space based on regressing two additional 3D object properties by a deep convolutional neural network and combined with cascaded geometric constraints between the 2D and 3D boxes. First, we modify the existing 3D properties regressing network by adding two additional components, viewpoints classification and the center projection of the 3D bounding box s bottom face. Second, we use the predicted center projection combined with similar triangle constraint to acquire an initial 3D bounding box by a closed-form solution. Then, the location predicted by previous step is used as the initial value of the over-determined equations constructed by 2D and 3D boxes fitting constraint with the configuration determined with the classified viewpoint. Finally, we use the recovered physical world information by the 3D detections to filter out the false detection and false alarm in 2D detections. We compare our method with the state-of-the-arts on the KITTI dataset show that although conceptually simple, our method outperforms more complex and computational expensive methods not only by improving the overall precision of 3D detections, but also increasing the orientation estimation precision. Furthermore our method can deal with the truncated objects to some extent and remove the false alarm and false detections in both 2D and 3D detections.
On Orthogonal Vector Edge Coloring
Venue: arXiv cs.DM 2019
Year: 2019

Given a graph $G$ and a positive integer $d$, an orthogonal vector $d$-coloring of $G$ is an assignment $f$ of vectors of $\mathbb{R}^d$ to $V(G)$ in such a way that adjacent vertices receive orthogonal vectors. The orthogonal chromatic number of $G$, denoted by $χ_v(G)$, is the minimum $d$ for which $G$ admits an orthogonal vector $d$-coloring. This notion has close ties with the notions of Lovász Theta Function, quantum chromatic number, and many other problems, and even though this and related metrics have been extensively studied over the years, we have found that there is a gap in the knowledge concerning the edge version of the problem. In this article, we discuss this version and its relation with other insteresting known facts, and pose a question about the orthogonal chromatic index of cubic graphs.
Stochastic Linear Optimization with Adversarial Corruption
Venue: arXiv cs.LG 2019
Year: 2019

We extend the model of stochastic bandits with adversarial corruption (Lykouriset al., 2018) to the stochastic linear optimization problem (Dani et al., 2008). Our algorithm is agnostic to the amount of corruption chosen by the adaptive adversary. The regret of the algorithm only increases linearly in the amount of corruption. Our algorithm involves using Löwner-John's ellipsoid for exploration and dividing time horizon into epochs with exponentially increasing size to limit the influence of corruption.
Future Frame Prediction Using Convolutional VRNN for Anomaly Detection
Venue: arXiv cs.CV 2019
Year: 2019

Anomaly detection in videos aims at reporting anything that does not conform the normal behaviour or distribution. However, due to the sparsity of abnormal video clips in real life, collecting annotated data for supervised learning is exceptionally cumbersome. Inspired by the practicability of generative models for semi-supervised learning, we propose a novel sequential generative model based on variational autoencoder (VAE) for future frame prediction with convolutional LSTM (ConvLSTM). To the best of our knowledge, this is the first work that considers temporal information in future frame prediction based anomaly detection framework from the model perspective. Our experiments demonstrate that our approach is superior to the state-of-the-art methods on three benchmark datasets.
Towards Task-Oriented Dialogue in Mixed Domains
Venue: arXiv cs.CL 2019
Year: 2019

This work investigates the task-oriented dialogue problem in mixed-domain settings. We study the effect of alternating between different domains in sequences of dialogue turns using two related state-of-the-art dialogue systems. We first show that a specialized state tracking component in multiple domains plays an important role and gives better results than an end-to-end task-oriented dialogue system. We then propose a hybrid system which is able to improve the belief tracking accuracy of about 28% of average absolute point on a standard multi-domain dialogue dataset. These experimental results give some useful insights for improving our commercial chatbot platform FPT.AI, which is currently deployed for many practical chatbot applications.
Accelerating Transformer Decoding via a Hybrid of Self-attention and Recurrent Neural Network
Venue: arXiv cs.CL 2019
Year: 2019

Due to the highly parallelizable architecture, Transformer is faster to train than RNN-based models and popularly used in machine translation tasks. However, at inference time, each output word requires all the hidden states of the previously generated words, which limits the parallelization capability, and makes it much slower than RNN-based ones. In this paper, we systematically analyze the time cost of different components of both the Transformer and RNN-based model. Based on it, we propose a hybrid network of self-attention and RNN structures, in which, the highly parallelizable self-attention is utilized as the encoder, and the simpler RNN structure is used as the decoder. Our hybrid network can decode 4-times faster than the Transformer. In addition, with the help of knowledge distillation, our hybrid network achieves comparable translation quality to the original Transformer.
Depth Map Estimation for Free-Viewpoint Television
Venue: arXiv cs.CV 2019
Year: 2019

The paper presents a new method of depth estimation dedicated for free-viewpoint television (FTV). The estimation is performed for segments and thus their size can be used to control a trade-off between the quality of depth maps and the processing time of their estimation. The proposed algorithm can take as its input multiple arbitrarily positioned views which are simultaneously used to produce multiple inter view consistent output depth maps. The presented depth estimation method uses novel parallelization and temporal consistency enhancement methods that significantly reduce the processing time of depth estimation. An experimental assessment of the proposals has been performed, based on the analysis of virtual view quality in FTV. The results show that the proposed method provides an improvement of the depth map quality over the state of-the-art method, simultaneously reducing the complexity of depth estimation. The consistency of depth maps, which is crucial for the quality of the synthesized video and thus the quality of experience of navigating through a 3D scene, is also vastly improved.
An Active Learning Approach for Reducing Annotation Cost in Skin Lesion Analysis
Venue: arXiv cs.CV 2019
Year: 2019

Automated skin lesion analysis is very crucial in clinical practice, as skin cancer is among the most common human malignancy. Existing approaches with deep learning have achieved remarkable performance on this challenging task, however, heavily relying on large-scale labelled datasets. In this paper, we present a novel active learning framework for cost-effective skin lesion analysis. The goal is to effectively select and utilize much fewer labelled samples, while the network can still achieve state-of-the-art performance. Our sample selection criteria complementarily consider both informativeness and representativeness, derived from decoupled aspects of measuring model certainty and covering sample diversity. To make wise use of the selected samples, we further design a simple yet effective strategy to aggregate intra-class images in pixel space, as a new form of data augmentation. We validate our proposed method on data of ISIC 2017 Skin Lesion Classification Challenge for two tasks. Using only up to 50% of samples, our approach can achieve state-of-the-art performances on both tasks, which are comparable or exceeding the accuracies with full-data training, and outperform other well-known active learning methods by a large margin.
A complex network approach to political analysis: application to the Brazilian Chamber of Deputies
Venue: arXiv cs.SI 2019
Year: 2019

In this paper, we introduce a network-based methodology to study how clusters represented by political entities evolve over time. We constructed networks of voting data from the Brazilian Chamber of Deputies, where deputies are nodes and edges are represented by voting similarity among deputies. The Brazilian Chamber of deputies is characterized by a multi-party political system. Thus, we would expect a broad spectrum of ideas to be represented. Our results, however, revealed that plurality of ideas is not present at all: the effective number of communities representing ideas based on agreement/disagreement in propositions is about 3 over the entire studied time span. The obtained results also revealed different patterns of coalitions between distinct parties. Finally, we also found signs of early party isolation before presidential impeachment proceedings effectively started. We believe that the proposed framework could be used to complement the study of political dynamics and even applied in similar social networks where individuals are organized in a complex manner.
A Discussion on Influence of Newspaper Headlines on Social Media
Venue: arXiv cs.CL 2019
Year: 2019

Newspaper headlines contribute severely and have an influence on the social media. This work studies the durability of impact of verbs and adjectives on headlines and determine the factors which are responsible for its nature of influence on the social media. Each headline has been categorized into positive, negative or neutral based on its sentiment score. Initial results show that intensity of a sentiment nature is positively correlated with the social media impression. Additionally, verbs and adjectives show a relation with the sentiment scores
Duet: An Expressive Higher-order Language and Linear Type System for Statically Enforcing Differential Privacy
Venue: arXiv cs.PL 2019
Year: 2019

During the past decade, differential privacy has become the gold standard for protecting the privacy of individuals. However, verifying that a particular program provides differential privacy often remains a manual task to be completed by an expert in the field. Language-based techniques have been proposed for fully automating proofs of differential privacy via type system design, however these results have lagged behind advances in differentially-private algorithms, leaving a noticeable gap in programs which can be automatically verified while also providing state-of-the-art bounds on privacy. We propose Duet, an expressive higher-order language, linear type system and tool for automatically verifying differential privacy of general-purpose higher-order programs. In addition to general purpose programming, Duet supports encoding machine learning algorithms such as stochastic gradient descent, as well as common auxiliary data analysis tasks such as clipping, normalization and hyperparameter tuning - each of which are particularly challenging to encode in a statically verified differential privacy framework. We present a core design of the Duet language and linear type system, and complete key proofs about privacy for well-typed programs. We then show how to extend Duet to support realistic machine learning applications and recent variants of differential privacy which result in improved accuracy for many practical differentially private algorithms. Finally, we implement several differentially private machine learning algorithms in Duet which have never before been automatically verified by a language-based tool, and we present experimental results which demonstrate the benefits of Duet's language design in terms of accuracy of trained machine learning models.