OpenRaiser/Intern-Atlas huggingface dataset:
One-dimensional Deep Image Prior for Time Series Inverse Problems
Venue: arXiv cs.LG 2019
Year: 2019

We extend the Deep Image Prior (DIP) framework to one-dimensional signals. DIP is using a randomly initialized convolutional neural network (CNN) to solve linear inverse problems by optimizing over weights to fit the observed measurements. Our main finding is that properly tuned one-dimensional convolutional architectures provide an excellent Deep Image Prior for various types of temporal signals including audio, biological signals, and sensor measurements. We show that our network can be used in a variety of recovery tasks including missing value imputation, blind denoising, and compressed sensing from random Gaussian projections. The key challenge is how to avoid overfitting by carefully tuning early stopping, total variation, and weight decay regularization. Our method requires up to 4 times fewer measurements than Lasso and outperforms NLM-VAMP for random Gaussian measurements on audio signals, has similar imputation performance to a Kalman state-space model on a variety of data, and outperforms wavelet filtering in removing additive noise from air-quality sensor readings.
Investigating the Dirac operator evaluation with FPGAs
Venue: arXiv cs.DC 2019
Year: 2019

In recent years the computational capacity of single Field Programmable Gate Arrays (FPGA) devices as well as their versatility has increased significantly. Adding to that the High Level Synthesis frameworks allowing to program such processors in a high level language like C++, makes modern FPGA devices a serious candidate as building blocks of a general purpose High Performance Computing solution. In this contribution we describe benchmarks which we performed using a Lattice QCD code, a highly compute-demanding HPC academic code for elementary particle simulations. We benchmark the performance of a single FPGA device running in two modes: using the external or embedded memory. We discuss both approaches in detail using the Xilinx U250 device and provide estimates for the necessary memory throughput and the minimal amount of resources needed to deliver optimal performance depending on the available hardware platform.
Doping Tests for Cyber-Physical Systems
Venue: arXiv cs.LO 2019
Year: 2019

The software running in embedded or cyber-physical systems (CPS) is typically of proprietary nature, so users do not know precisely what the systems they own are (in)capable of doing. Most malfunctionings of such systems are not intended by the manufacturer, but some are, which means these cannot be classified as bugs or security loopholes. The most prominent examples have become public in the diesel emissions scandal, where millions of cars were found to be equipped with software violating the law, altogether polluting the environment and putting human health at risk. The behaviour of the software embedded in these cars was intended by the manufacturer, but it was not in the interest of society, a phenomenon that has been called software doping. Doped software is significantly different from buggy or insecure software and hence it is not possible to use classical verification and testing techniques to discover and mitigate software doping. The work presented in this paper builds on existing definitions of software doping and lays the theoretical foundations for conducting software doping tests, so as to enable attacking evil manufacturers. The complex nature of software doping makes it very hard to effectuate doping tests in practice. We explain the biggest challenges and provide efficient solutions to realise doping tests despite this complexity.
Towards whole program generation of quadrature-free discontinuous Galerkin methods for the shallow water equations
Venue: arXiv cs.MS 2019
Year: 2019

The shallow water equations (SWE) are a commonly used model to study tsunamis, tides, and coastal ocean circulation. However, there exist various approaches to discretize and solve them efficiently. Which of them is best for a certain scenario is often not known and, in addition, depends heavily on the used HPC platform. From a simulation software perspective, this places a premium on the ability to adapt easily to different numerical methods and hardware architectures. One solution to this problem is to apply code generation techniques and to express methods and specific hardware-dependent implementations on different levels of abstraction. This allows for a separation of concerns and makes it possible, e.g., to exchange the discretization scheme without having to rewrite all low-level optimized routines manually. In this paper, we show how code for an advanced quadrature-free discontinuous Galerkin (DG) discretized shallow water equation solver can be generated. Here, we follow the multi-layered approach from the ExaStencils project that starts from the continuous problem formulation, moves to the discrete scheme, spells out the numerical algorithms, and, finally, maps to a representation that can be transformed to a distributed memory parallel implementation by our in-house Scala-based source-to-source compiler. Our contributions include: A new quadrature-free discontinuous Galerkin formulation, an extension of the class of supported computational grids, and an extension of our toolchain allowing to evaluate discrete integrals stemming from the DG discretization implemented in Python. As first results we present the whole toolchain and also demonstrate the convergence of our method for higher order DG discretizations.
Knowledge-rich Image Gist Understanding Beyond Literal Meaning
Venue: arXiv cs.IR 2019
Year: 2019

We investigate the problem of understanding the message (gist) conveyed by images and their captions as found, for instance, on websites or news articles. To this end, we propose a methodology to capture the meaning of image-caption pairs on the basis of large amounts of machine-readable knowledge that has previously been shown to be highly effective for text understanding. Our method identifies the connotation of objects beyond their denotation: where most approaches to image understanding focus on the denotation of objects, i.e., their literal meaning, our work addresses the identification of connotations, i.e., iconic meanings of objects, to understand the message of images. We view image understanding as the task of representing an image-caption pair on the basis of a wide-coverage vocabulary of concepts such as the one provided by Wikipedia, and cast gist detection as a concept-ranking problem with image-caption pairs as queries. To enable a thorough investigation of the problem of gist understanding, we produce a gold standard of over 300 image-caption pairs and over 8,000 gist annotations covering a wide variety of topics at different levels of abstraction. We use this dataset to experimentally benchmark the contribution of signals from heterogeneous sources, namely image and text. The best result with a Mean Average Precision (MAP) of 0.69 indicate that by combining both dimensions we are able to better understand the meaning of our image-caption pairs than when using language or vision information alone. We test the robustness of our gist detection approach when receiving automatically generated input, i.e., using automatically generated image tags or generated captions, and prove the feasibility of an end-to-end automated process.
Societal Controversies in Wikipedia Articles
Venue: arXiv cs.CL 2019
Year: 2019

Collaborative content creation inevitably reaches situations where different points of view lead to conflict. We focus on Wikipedia, the free encyclopedia anyone may edit, where disputes about content in controversial articles often reflect larger societal debates. While Wikipedia has a public edit history and discussion section for every article, the substance of these sections is difficult to phantom for Wikipedia users interested in the development of an article and in locating which topics were most controversial. In this paper we present Contropedia, a tool that augments Wikipedia articles and gives insight into the development of controversial topics. Contropedia uses an efficient language agnostic measure based on the edit history that focuses on wiki links to easily identify which topics within a Wikipedia article have been most controversial and when.
Uncovering convolutional neural network decisions for diagnosing multiple sclerosis on conventional MRI using layer-wise relevance propagation
Venue: arXiv cs.CV 2019
Year: 2019

Machine learning-based imaging diagnostics has recently reached or even superseded the level of clinical experts in several clinical domains. However, classification decisions of a trained machine learning system are typically non-transparent, a major hindrance for clinical integration, error tracking or knowledge discovery. In this study, we present a transparent deep learning framework relying on convolutional neural networks (CNNs) and layer-wise relevance propagation (LRP) for diagnosing multiple sclerosis (MS). MS is commonly diagnosed utilizing a combination of clinical presentation and conventional magnetic resonance imaging (MRI), specifically the occurrence and presentation of white matter lesions in T2-weighted images. We hypothesized that using LRP in a naive predictive model would enable us to uncover relevant image features that a trained CNN uses for decision-making. Since imaging markers in MS are well-established this would enable us to validate the respective CNN model. First, we pre-trained a CNN on MRI data from the Alzheimer's Disease Neuroimaging Initiative (n = 921), afterwards specializing the CNN to discriminate between MS patients and healthy controls (n = 147). Using LRP, we then produced a heatmap for each subject in the holdout set depicting the voxel-wise relevance for a particular classification decision. The resulting CNN model resulted in a balanced accuracy of 87.04% and an area under the curve of 96.08% in a receiver operating characteristic curve. The subsequent LRP visualization revealed that the CNN model focuses indeed on individual lesions, but also incorporates additional information such as lesion location, non-lesional white matter or gray matter areas such as the thalamus, which are established conventional and advanced MRI markers in MS. We conclude that LRP and the proposed framework have the capability to make diagnostic decisions of...
Intrusion Detection Mechanism Using Fuzzy Rule Interpolation
Venue: arXiv cs.CR 2019
Year: 2019

Fuzzy Rule Interpolation (FRI) methods can serve deducible (interpolated) conclusions even in case if some situations are not explicitly defined in a fuzzy rule based knowledge representation. This property can be beneficial in partial heuristically solved applications; there the efficiency of expert knowledge representation is mixed with the precision of machine learning methods. The goal of this paper is to introduce the benefits of FRI in the Intrusion Detection Systems (IDS) application area, in the design and implementation of the detection mechanism for Distributed Denial of Service (DDOS) attacks. In the example of the paper as a test-bed environment an open source DDOS dataset and the General Public License (GNU) FRI Toolbox was applied. The performance of the FRI-IDS example application is compared to other common classification algorithms used for detecting DDOS attacks on the same open source test-bed environment. According to the results, the overall detection rate of the FRI-IDS is in pair with other methods. On the example dataset it outperforms the detection rate of the support vector machine algorithm, whereas other algorithms (neural network, random forest and decision tree) recorded lightly higher detection rate. Consequently, the FRI inference system could be a suitable approach to be implemented as a detection mechanism for IDS; it effectively decreases the false positive rate value. Moreover, because of its fuzzy rule base knowledge representation nature, it can easily adapt expert knowledge, and also be-suitable for predicting the level of degree for threat possibility.
A Framework to Control Functional Connectivity in the Human Brain
Venue: arXiv cs.SY 2019
Year: 2019

In this paper, we propose a framework to control brain-wide functional connectivity by selectively acting on the brain's structure and parameters. Functional connectivity, which measures the degree of correlation between neural activities in different brain regions, can be used to distinguish between healthy and certain diseased brain dynamics and, possibly, as a control parameter to restore healthy functions. In this work, we use a collection of interconnected Kuramoto oscillators to model oscillatory neural activity, and show that functional connectivity is essentially regulated by the degree of synchronization between different clusters of oscillators. Then, we propose a minimally invasive method to correct the oscillators' interconnections and frequencies to enforce arbitrary and stable synchronization patterns among the oscillators and, consequently, a desired pattern of functional connectivity. Additionally, we show that our synchronization-based framework is robust to parameter mismatches and numerical inaccuracies, and validate it using a realistic neurovascular model to simulate neural activity and functional connectivity in the human brain.
(De)Constructing Bias on Skin Lesion Datasets
Venue: arXiv cs.CV 2019
Year: 2019

Melanoma is the deadliest form of skin cancer. Automated skin lesion analysis plays an important role for early detection. Nowadays, the ISIC Archive and the Atlas of Dermoscopy dataset are the most employed skin lesion sources to benchmark deep-learning based tools. However, all datasets contain biases, often unintentional, due to how they were acquired and annotated. Those biases distort the performance of machine-learning models, creating spurious correlations that the models can unfairly exploit, or, contrarily destroying cogent correlations that the models could learn. In this paper, we propose a set of experiments that reveal both types of biases, positive and negative, in existing skin lesion datasets. Our results show that models can correctly classify skin lesion images without clinically-meaningful information: disturbingly, the machine-learning model learned over images where no information about the lesion remains, presents an accuracy above the AI benchmark curated with dermatologists' performances. That strongly suggests spurious correlations guiding the models. We fed models with additional clinically meaningful information, which failed to improve the results even slightly, suggesting the destruction of cogent correlations. Our main findings raise awareness of the limitations of models trained and evaluated in small datasets such as the ones we evaluated, and may suggest future guidelines for models intended for real-world deployment.
A Passivity-based Nonlinear Admittance Control with Application to Powered Upper-limb Control under Unknown Environmental Interactions
Venue: arXiv cs.RO 2019
Year: 2019

This paper presents an admittance controller based on the passivity theory for a powered upper-limb exoskeleton robot which is governed by the nonlinear equation of motion. Passivity allows us to include a human operator and environmental interaction in the control loop. The robot interacts with the human operator via F/T sensor and interacts with the environment mainly via end-effectors. Although the environmental interaction cannot be detected by any sensors (hence unknown), passivity allows us to have natural interaction. An analysis shows that the behavior of the actual system mimics that of a nominal model as the control gain goes to infinity, which implies that the proposed approach is an admittance controller. However, because the control gain cannot grow infinitely in practice, the performance limitation according to the achievable control gain is also analyzed. The result of this analysis indicates that the performance in the sense of infinite norm increases linearly with the control gain. In the experiments, the proposed properties were verified using 1 degree-of-freedom testbench, and an actual powered upper-limb exoskeleton was used to lift and maneuver the unknown payload.
On the validity of memristor modeling in the neural network literature
Venue: arXiv cs.NE 2019
Year: 2019

An analysis of the literature shows that there are two types of non-memristive models that have been widely used in the modeling of so-called "memristive" neural networks. Here, we demonstrate that such models have nothing in common with the concept of memristive elements: they describe either non-linear resistors or certain bi-state systems, which all are devices without memory. Therefore, the results presented in a significant number of publications are at least questionable, if not completely irrelevant to the actual field of memristive neural networks.
Solvability of Power Flow Equations Through Existence and Uniqueness of Complex Fixed Point
Venue: arXiv cs.SY 2019
Year: 2019

Variations of loading level and changes in system topological property may cause the operating point of an electric power systems to move gradually towards the verge of its transmission capability, which can lead to catastrophic outcomes such as voltage collapse blackout. From a modeling perspective, voltage collapse is closely related to the solvability of power flow equations. Determining conditions for existence and uniqueness of solution to power flow equations is one of the fundamental problems in power systems that has great theoretical and practical significance. In this paper, we provide strong sufficient condition certifying the existence and uniqueness of power flow solutions in a subset of state (voltage) space. The novel analytical approach heavily exploits the contractive properties of the fixed-point form in complex domain, which leads to much sharper analytical conditions than previous ones based primarily on analysis in the real domain. Extensive computational experiments are performed which validate the correctness and demonstrate the effectiveness of the proposed condition.
Enhanced Center Coding for Cell Detection with Convolutional Neural Networks
Venue: arXiv cs.CV 2019
Year: 2019

Cell imaging and analysis are fundamental to biomedical research because cells are the basic functional units of life. Among different cell-related analysis, cell counting and detection are widely used. In this paper, we focus on one common step of learning-based cell counting approaches: coding the raw dot labels into more suitable maps for learning. Two criteria of coding raw dot labels are discussed, and a new coding scheme is proposed in this paper. The two criteria measure how easy it is to train the model with a coding scheme, and how robust the recovered raw dot labels are when predicting. The most compelling advantage of the proposed coding scheme is the ability to distinguish neighboring cells in crowded regions. Cell counting and detection experiments are conducted for five coding schemes on four types of cells and two network architectures. The proposed coding scheme improves the counting accuracy versus the widely-used Gaussian and rectangle kernels up to 12%, and also improves the detection accuracy versus the common proximity coding up to 14%.
School management information systems: Challenges to educational decision-making in the big data era
Venue: arXiv cs.CY 2019
Year: 2019

Despite the benefits of school management information systems (SMIS), the concept of data-driven school culture failed to materialize for many educational institutions. Challenges posed by the quality of data in the big data era have prevented many schools from realizing the real potential of the SMIS. The paper analyses the uses, features, and inhibiting factors of SMIS. The paper proposes a five-phase conceptual model that assist administrators with making timely, quality decisions. The paper enriches the theoretical landscape of SMIS usage in the era of big data and lays a foundation for the future by establishing an educational decision-making model.
VoteNet: A Deep Learning Label Fusion Method for Multi-Atlas Segmentation
Venue: arXiv cs.CV 2019
Year: 2019

Deep learning (DL) approaches are state-of-the-art for many medical image segmentation tasks. They offer a number of advantages: they can be trained for specific tasks, computations are fast at test time, and segmentation quality is typically high. In contrast, previously popular multi-atlas segmentation (MAS) methods are relatively slow (as they rely on costly registrations) and even though sophisticated label fusion strategies have been proposed, DL approaches generally outperform MAS. In this work, we propose a DL-based label fusion strategy (VoteNet) which locally selects a set of reliable atlases whose labels are then fused via plurality voting. Experiments on 3D brain MRI data show that by selecting a good initial atlas set MAS with VoteNet significantly outperforms a number of other label fusion strategies as well as a direct DL segmentation approach. We also provide an experimental analysis of the upper performance bound achievable by our method. While unlikely achievable in practice, this bound suggests room for further performance improvements. Lastly, to address the runtime disadvantage of standard MAS, all our results make use of a fast DL registration approach.
HEPCloud, an Elastic Hybrid HEP Facility using an Intelligent Decision Support System
Venue: arXiv cs.DC 2019
Year: 2019

HEPCloud is rapidly becoming the primary system for provisioning compute resources for all Fermilab-affiliated experiments. In order to reliably meet the peak demands of the next generation of High Energy Physics experiments, Fermilab must plan to elastically expand its computational capabilities to cover the forecasted need. Commercial cloud and allocation-based High Performance Computing (HPC) resources both have explicit and implicit costs that must be considered when deciding when to provision these resources, and at which scale. In order to support such provisioning in a manner consistent with organizational business rules and budget constraints, we have developed a modular intelligent decision support system (IDSS) to aid in the automatic provisioning of resources spanning multiple cloud providers, multiple HPC centers, and grid computing federations. In this paper, we discuss the goals and architecture of the HEPCloud Facility, the architecture of the IDSS, and our early experience in using the IDSS for automated facility expansion both at Fermi and Brookhaven National Laboratory.
Distribution LMP-based Transactive Day-ahead Market with Variable Renewable Generation
Venue: arXiv cs.SY 2019
Year: 2019

The largescale penetration of variable renewable energy (VRE) and their generation uncertainties poses a major challenge for the distribution system operator (DSO) to efficiently determine the day-ahead real and reactive power distribution locational marginal prices (DLMPs) and their underlying components. In this paper, we propose a DLMP-based transactive day-ahead market (DAM) model, that in addition to energy and losses, determines prices for creating congestions and voltage violations under peak-load and large-scale stochastic VRE penetration conditions. To account for the VRE uncertainties and the effect of their large-scale penetration on the DLMP components and distributed energy resources' (DERs) schedules, we propose a novel data-driven probability efficient point (PEP) method that computes the optimal total VRE generation at different confidence (risk) levels to incorporate in the proposed transactive DAM model. We perform a wide range of simulation studies on a modified IEEE 69-node system to validate the proposed methods and demonstrate the effect of peak load conditions, large-scale VRE penetration, and inclusion of battery energy storage systems (BESS) on the resulting positive or negative real and reactive power DLMPs and their components.