OpenRaiser/Intern-Atlas huggingface dataset:
Scene Recognition with Prototype-agnostic Scene Layout
Venue: arXiv cs.CV 2019
Year: 2019

Abstract--- Exploiting the spatial structure in scene images is a key research direction for scene recognition. Due to the large intra-class structural diversity, building and modeling flexible structural layout to adapt various image characteristics is a challenge. Existing structural modeling methods in scene recognition either focus on predefined grids or rely on learned prototypes, which all have limited representative ability. In this paper, we propose Prototype-agnostic Scene Layout (PaSL) construction method to build the spatial structure for each image without conforming to any prototype. Our PaSL can flexibly capture the diverse spatial characteristic of scene images and have considerable generalization capability. Given a PaSL, we build Layout Graph Network (LGN) where regions in PaSL are defined as nodes and two kinds of independent relations between regions are encoded as edges. The LGN aims to incorporate two topological structures (formed in spatial and semantic similarity dimensions) into image representations through graph convolution. Extensive experiments show that our approach achieves state-of-the-art results on widely recognized MIT67 and SUN397 datasets without multi-model or multi-scale fusion. Moreover, we also conduct the experiments on one of the largest scale datasets, Places365. The results demonstrate the proposed method can be well generalized and obtains competitive performance.
Active learning to optimise time-expensive algorithm selection
Venue: arXiv cs.LG 2019
Year: 2019

Hard optimisation problems such as Boolean Satisfiability typically have long solving times and can usually be solved by many algorithms, although the performance can vary widely in practice. Research has shown that no single algorithm outperforms all the others; thus, it is crucial to select the best algorithm for a given problem. Supervised machine learning models can accurately predict which solver is best for a given problem, but they require first to run every solver in the portfolio for all examples available to create labelled data. As this approach cannot scale, we developed an active learning framework that addresses this problem by constructing an optimal training set, so that the learner can achieve higher or equal performances with less training data. Our work proves that active learning is beneficial for algorithm selection techniques and provides practical guidance to incorporate into existing systems.
Exploring Temporal Differences in 3D Convolutional Neural Networks
Venue: arXiv cs.CV 2019
Year: 2019

Traditional 3D convolutions are computationally expensive, memory intensive, and due to large number of parameters, they often tend to overfit. On the other hand, 2D CNNs are less computationally expensive and less memory intensive than 3D CNNs and have shown remarkable results in applications like image classification and object recognition. However, in previous works, it has been observed that they are inferior to 3D CNNs when applied on a spatio-temporal input. In this work, we propose a convolutional block which extracts the spatial information by performing a 2D convolution and extracts the temporal information by exploiting temporal differences, i.e., the change in the spatial information at different time instances, using simple operations of shift, subtract and add without utilizing any trainable parameters. The proposed convolutional block has same number of parameters as of a 2D convolution kernel of size nxn, i.e. n^2, and has n times lesser parameters than an nxnxn 3D convolution kernel. We show that the 3D CNNs perform better when the 3D convolution kernels are replaced by the proposed convolutional blocks. We evaluate the proposed convolutional block on UCF101 and ModelNet datasets.
Minimum status, matching and domination of graphs
Venue: arXiv cs.DM 2019
Year: 2019

The minimum status of a graph is the minimum of statuses of all vertices of this graph. We give a sharp upper bound for the minimum status of a connected graph with fixed order and matching number (domination number, respectively), and characterize the unique trees achieving the bound. We also determine the unique tree such that its minimum status is as small as possible when order and matching number (domination number, respectively) are fixed.
Kernel Node Embeddings
Venue: arXiv cs.SI 2019
Year: 2019

Learning representations of nodes in a low dimensional space is a crucial task with many interesting applications in network analysis, including link prediction and node classification. Two popular approaches for this problem include matrix factorization and random walk-based models. In this paper, we aim to bring together the best of both worlds, towards learning latent node representations. In particular, we propose a weighted matrix factorization model which encodes random walk-based information about the nodes of the graph. The main benefit of this formulation is that it allows to utilize kernel functions on the computation of the embeddings. We perform an empirical evaluation on real-world networks, showing that the proposed model outperforms baseline node embedding algorithms in two downstream machine learning tasks.
When Explainability Meets Adversarial Learning: Detecting Adversarial Examples using SHAP Signatures
Venue: arXiv cs.LG 2019
Year: 2019

State-of-the-art deep neural networks (DNNs) are highly effective in solving many complex real-world problems. However, these models are vulnerable to adversarial perturbation attacks, and despite the plethora of research in this domain, to this day, adversaries still have the upper hand in the cat and mouse game of adversarial example generation methods vs. detection and prevention methods. In this research, we present a novel detection method that uses Shapley Additive Explanations (SHAP) values computed for the internal layers of a DNN classifier to discriminate between normal and adversarial inputs. We evaluate our method by building an extensive dataset of adversarial examples over the popular CIFAR-10 and MNIST datasets, and training a neural network-based detector to distinguish between normal and adversarial inputs. We evaluate our detector against adversarial examples generated by diverse state-of-the-art attacks and demonstrate its high detection accuracy and strong generalization ability to adversarial inputs generated with different attack methods.
Order-free Learning Alleviating Exposure Bias in Multi-label Classification
Venue: arXiv cs.LG 2019
Year: 2019

Multi-label classification (MLC) assigns multiple labels to each sample. Prior studies show that MLC can be transformed to a sequence prediction problem with a recurrent neural network (RNN) decoder to model the label dependency. However, training a RNN decoder requires a predefined order of labels, which is not directly available in the MLC specification. Besides, RNN thus trained tends to overfit the label combinations in the training set and have difficulty generating unseen label sequences. In this paper, we propose a new framework for MLC which does not rely on a predefined label order and thus alleviates exposure bias. The experimental results on three multi-label classification benchmark datasets show that our method outperforms competitive baselines by a large margin. We also find the proposed approach has a higher probability of generating label combinations not seen during training than the baseline models. The result shows that the proposed approach has better generalization capability.
May I Check Again? -- A simple but efficient way to generate and use contextual dictionaries for Named Entity Recognition. Application to French Legal Texts
Venue: arXiv cs.CL 2019
Year: 2019

In this paper we present a new method to learn a model robust to typos for a Named Entity Recognition task. Our improvement over existing methods helps the model to take into account the context of the sentence inside a court decision in order to recognize an entity with a typo. We used state-of-the-art models and enriched the last layer of the neural network with high-level information linked with the potential of the word to be a certain type of entity. More precisely, we utilized the similarities between the word and the potential entity candidates in the tagged sentence context. The experiments on a dataset of French court decisions show a reduction of the relative F1-score error of 32%, upgrading the score obtained with the most competitive fine-tuned state-of-the-art system from 94.85% to 96.52%.
Disease Labeling via Machine Learning is NOT quite the same as Medical Diagnosis
Venue: arXiv cs.AI 2019
Year: 2019

A key step in medical diagnosis is giving the patient a universally recognized label (e.g. Appendicitis) which essentially assigns the patient to a class(es) of patients with similar body failures. However, two patients having the same disease label(s) with high probability may still have differences in their feature manifestation patterns implying differences in the required treatments. Additionally, in many cases, the labels of the primary diagnoses leave some findings unexplained. Medical diagnosis is only partially about probability calculations for label X or Y. Diagnosis is not complete until the patient overall situation is clinically understood to the level that enables the best therapeutic decisions. Most machine learning models are data centric models, and evidence so far suggest they can reach expert level performance in the disease labeling phase. Nonetheless, like any other mathematical technique, they have their limitations and applicability scope. Primarily, data centric algorithms are knowledge blind and lack anatomy and physiology knowledge that physicians leverage to achieve complete diagnosis. This article advocates to complement them with intelligence to overcome their inherent limitations as knowledge blind algorithms. Machines can learn many things from data, but data is not the only source that machines can learn from. Historic patient data only tells us what the possible manifestations of a certain body failure are. Anatomy and physiology knowledge tell us how the body works and fails. Both are needed for complete diagnosis. The proposed Double Deep Learning approach, along with the initiative for Medical Wikipedia for Smart Machines, leads to AI diagnostic support solutions for complete diagnosis beyond the limited data only labeling solutions we see today. AI for medicine will forever be limited until their intelligence also integrates anatomy and physiology.
Personalized HeartSteps: A Reinforcement Learning Algorithm for Optimizing Physical Activity
Venue: arXiv cs.LG 2019
Year: 2019

With the recent evolution of mobile health technologies, health scientists are increasingly interested in developing just-in-time adaptive interventions (JITAIs), typically delivered via notification on mobile device and designed to help the user prevent negative health outcomes and promote the adoption and maintenance of healthy behaviors. A JITAI involves a sequence of decision rules (i.e., treatment policy) that takes the user's current context as input and specifies whether and what type of an intervention should be provided at the moment. In this paper, we develop a Reinforcement Learning (RL) algorithm that continuously learns and improves the treatment policy embedded in the JITAI as the data is being collected from the user. This work is motivated by our collaboration on designing the RL algorithm in HeartSteps V2 based on data from HeartSteps V1. HeartSteps is a physical activity mobile health application. The RL algorithm developed in this paper is being used in HeartSteps V2 to decide, five times per day, whether to deliver a context-tailored activity suggestion.
Cost-aware Multi-objective Bayesian optimisation
Venue: arXiv cs.LG 2019
Year: 2019

The notion of expense in Bayesian optimisation generally refers to the uniformly expensive cost of function evaluations over the whole search space. However, in some scenarios, the cost of evaluation for black-box objective functions is non-uniform since different inputs from search space may incur different costs for function evaluations. We introduce a cost-aware multi-objective Bayesian optimisation with non-uniform evaluation cost over objective functions by defining cost-aware constraints over the search space. The cost-aware constraints are a sorted tuple of indexes that demonstrate the ordering of dimensions of the search space based on the user's prior knowledge about their cost of usage. We formulate a new multi-objective Bayesian optimisation acquisition function with detailed analysis of the convergence that incorporates this cost-aware constraints while optimising the objective functions. We demonstrate our algorithm based on synthetic and real-world problems in hyperparameter tuning of neural networks and random forests.
Formulating Manipulable Argumentation with Intra-/Inter-Agent Preferences
Venue: arXiv cs.AI 2019
Year: 2019

From marketing to politics, exploitation of incomplete information through selective communication of arguments is ubiquitous. In this work, we focus on development of an argumentation-theoretic model for manipulable multi-agent argumentation, where each agent may transmit deceptive information to others for tactical motives. In particular, we study characterisation of epistemic states, and their roles in deception/honesty detection and (mis)trust-building. To this end, we propose the use of intra-agent preferences to handle deception/honesty detection and inter-agent preferences to determine which agent(s) to believe in more. We show how deception/honesty in an argumentation of an agent, if detected, would alter the agent's perceived trustworthiness, and how that may affect their judgement as to which arguments should be acceptable.
CBNet: A Novel Composite Backbone Network Architecture for Object Detection
Venue: arXiv cs.CV 2019
Year: 2019

In existing CNN based detectors, the backbone network is a very important component for basic feature extraction, and the performance of the detectors highly depends on it. In this paper, we aim to achieve better detection performance by building a more powerful backbone from existing backbones like ResNet and ResNeXt. Specifically, we propose a novel strategy for assembling multiple identical backbones by composite connections between the adjacent backbones, to form a more powerful backbone named Composite Backbone Network (CBNet). In this way, CBNet iteratively feeds the output features of the previous backbone, namely high-level features, as part of input features to the succeeding backbone, in a stage-by-stage fashion, and finally the feature maps of the last backbone (named Lead Backbone) are used for object detection. We show that CBNet can be very easily integrated into most state-of-the-art detectors and significantly improve their performances. For example, it boosts the mAP of FPN, Mask R-CNN and Cascade R-CNN on the COCO dataset by about 1.5 to 3.0 percent. Meanwhile, experimental results show that the instance segmentation results can also be improved. Specially, by simply integrating the proposed CBNet into the baseline detector Cascade Mask R-CNN, we achieve a new state-of-the-art result on COCO dataset (mAP of 53.3) with single model, which demonstrates great effectiveness of the proposed CBNet architecture. Code will be made available on https://github.com/PKUbahuangliuhe/CBNet.
Improving the scalabiliy of neutron cross-section lookup codes on multicore NUMA system
Venue: arXiv cs.DC 2019
Year: 2019

We use the XSBench proxy application, a memory-intensive OpenMP program, to explore the source of on-node scalability degradation of a popular Monte Carlo (MC) reactor physics benchmark on non-uniform memory access (NUMA) systems. As background, we present the details of XSBench, a performance abstraction "proxy app" for the full MC simulation, as well as the internal design of the Linux kernel. We explain how the physical memory allocation inside the kernel affects the multicore scalability of XSBench. On a sixteen-core, two-socket NUMA testbed, the scaling efficiency is improved from a nonoptimized 70% to an optimized 95%, and the optimized version consumes 25% less energy than does the nonoptimized version. In addition to the NUMA optimization we evaluate a page-size optimization to XSBench and observe a 1.5x performance improvement, compared with a nonoptimized one.
Outlier Detection in High Dimensional Data
Venue: arXiv cs.LG 2019
Year: 2019

High-dimensional data poses unique challenges in outlier detection process. Most of the existing algorithms fail to properly address the issues stemming from a large number of features. In particular, outlier detection algorithms perform poorly on data set of small size with a large number of features. In this paper, we propose a novel outlier detection algorithm based on principal component analysis and kernel density estimation. The proposed method is designed to address the challenges of dealing with high-dimensional data by projecting the original data onto a smaller space and using the innate structure of the data to calculate anomaly scores for each data point. Numerical experiments on synthetic and real-life data show that our method performs well on high-dimensional data. In particular, the proposed method outperforms the benchmark methods as measured by the $F_1$-score. Our method also produces better-than-average execution times compared to the benchmark methods.
Security in Process: Detecting Attacks in Industrial Process Data
Venue: arXiv cs.CR 2019
Year: 2019

Due to the fourth industrial revolution, industrial applications make use of the progress in communication and embedded devices. This allows industrial users to increase efficiency and manageability while reducing cost and effort. Furthermore, the fourth industrial revolution, creating the so-called Industry 4.0, opens a variety of novel use and business cases in the industrial environment. However, this progress comes at the cost of an enlarged attack surface of industrial companies. Operational networks that have previously been phyiscally separated from public networks are now connected in order to make use of new communication capabilites. This motivates the need for industrial intrusion detection solutions that are compatible to the long-term operation machines in industry as well as the heterogeneous and fast-changing networks. In this work, process data is analysed. The data is created and monitored on real-world hardware. After a set up phase, attacks are introduced into the systems that influence the process behaviour. A time series-based anomaly detection approach, the Matrix Profiles, are adapted to the specific needs and applied to the intrusion detection. The results indicate an applicability of these methods to detect attacks in the process behaviour. Furthermore, they are easily integrated into existing process environments. Additionally, one-class classifiers One-Class Support Vector Machines and Isolation Forest are applied to the data without a notion of timing. While Matrix Profiles perform well in terms of creating and visualising results, the one-class classifiers perform poorly.
Nearly-Unsupervised Hashcode Representations for Relation Extraction
Venue: arXiv cs.LG 2019
Year: 2019

Recently, kernelized locality sensitive hashcodes have been successfully employed as representations of natural language text, especially showing high relevance to biomedical relation extraction tasks. In this paper, we propose to optimize the hashcode representations in a nearly unsupervised manner, in which we only use data points, but not their class labels, for learning. The optimized hashcode representations are then fed to a supervised classifier following the prior work. This nearly unsupervised approach allows fine-grained optimization of each hash function, which is particularly suitable for building hashcode representations generalizing from a training set to a test set. We empirically evaluate the proposed approach for biomedical relation extraction tasks, obtaining significant accuracy improvements w.r.t. state-of-the-art supervised and semi-supervised approaches.
A Wide and Deep Neural Network for Survival Analysis from Anatomical Shape and Tabular Clinical Data
Venue: arXiv cs.LG 2019
Year: 2019

We introduce a wide and deep neural network for prediction of progression from patients with mild cognitive impairment to Alzheimer's disease. Information from anatomical shape and tabular clinical data (demographics, biomarkers) are fused in a single neural network. The network is invariant to shape transformations and avoids the need to identify point correspondences between shapes. To account for right censored time-to-event data, i.e., when it is only known that a patient did not develop Alzheimer's disease up to a particular time point, we employ a loss commonly used in survival analysis. Our network is trained end-to-end to combine information from a patient's hippocampus shape and clinical biomarkers. Our experiments on data from the Alzheimer's Disease Neuroimaging Initiative demonstrate that our proposed model is able to learn a shape descriptor that augments clinical biomarkers and outperforms a deep neural network on shape alone and a linear model on common clinical biomarkers.
Association, Blockage and Handoffs in IEEE 802.11ad based 60GHz Picocells- A Closer Look
Venue: arXiv cs.NI 2019
Year: 2019

The link misalignment and high susceptibility to blockages are the biggest hurdles in realizing 60GHz based wireless local area networks (WLANs). However, much of the previous studies investigating 60GHz alignment and blockage issues do not provide an accurate quantitative evaluation from the perspective of WLANs. In this paper, we present an in-depth quantitative evaluation of commodity IEEE 802.11ad devices by forming a 60GHz WLAN with two docking stations mimicking as access points (APs). Through extensive experiments, we provide important insights about directional coverage pattern of antennas, communication range and co-channel interference and blockages. We are able to measure the IEEE 802.11ad link alignment and association overheads in absolute time units. With a very high accuracy (96-97%), our blockage characterization can differentiate between temporary and permanent blockages caused by humans in the indoor environment, which is a key insight. Utilizing our blockage characterization, we also demonstrate intelligent handoff to alternate APs using consumergrade IEEE 802.11ad devices. Our blockage-induced handoff experiments provide important insights that would be helpful in integrating millimeter wave based WLANs into future wireless networks.