OpenRaiser/Intern-Atlas huggingface dataset:
Real-time parameter inference in reduced-order flame models with heteroscedastic Bayesian neural network ensembles
Venue: arXiv cs.LG 2020
Year: 2020

The estimation of model parameters with uncertainties from observed data is a ubiquitous inverse problem in science and engineering. In this paper, we suggest an inexpensive and easy to implement parameter estimation technique that uses a heteroscedastic Bayesian Neural Network trained using anchored ensembling. The heteroscedastic aleatoric error of the network models the irreducible uncertainty due to parameter degeneracies in our inverse problem, while the epistemic uncertainty of the Bayesian model captures uncertainties which may arise from an input observation's out-of-distribution nature. We use this tool to perform real-time parameter inference in a 6 parameter G-equation model of a ducted, premixed flame from observations of acoustically excited flames. We train our networks on a library of 2.1 million simulated flame videos. Results on the test dataset of simulated flames show that the network recovers flame model parameters, with the correlation coefficient between predicted and true parameters ranging from 0.97 to 0.99, and well-calibrated uncertainty estimates. The trained neural networks are then used to infer model parameters from real videos of a premixed Bunsen flame captured using a high-speed camera in our lab. Re-simulation using inferred parameters shows excellent agreement between the real and simulated flames. Compared to Ensemble Kalman Filter-based tools that have been proposed for this problem in the combustion literature, our neural network ensemble achieves better data-efficiency and our sub-millisecond inference times represent a savings on computational costs by several orders of magnitude. This allows us to calibrate our reduced-order flame model in real-time and predict the thermoacoustic instability behaviour of the flame more accurately.
Chasing Carbon: The Elusive Environmental Footprint of Computing
Venue: arXiv cs.AR 2020
Year: 2020

Given recent algorithm, software, and hardware innovation, computing has enabled a plethora of new applications. As computing becomes increasingly ubiquitous, however, so does its environmental impact. This paper brings the issue to the attention of computer-systems researchers. Our analysis, built on industry-reported characterization, quantifies the environmental effects of computing in terms of carbon emissions. Broadly, carbon emissions have two sources: operational energy consumption, and hardware manufacturing and infrastructure. Although carbon emissions from the former are decreasing thanks to algorithmic, software, and hardware innovations that boost performance and power efficiency, the overall carbon footprint of computer systems continues to grow. This work quantifies the carbon output of computer systems to show that most emissions related to modern mobile and data-center equipment come from hardware manufacturing and infrastructure. We therefore outline future directions for minimizing the environmental impact of computing systems.
Data Augmentation via Structured Adversarial Perturbations
Venue: arXiv cs.LG 2020
Year: 2020

Data augmentation is a major component of many machine learning methods with state-of-the-art performance. Common augmentation strategies work by drawing random samples from a space of transformations. Unfortunately, such sampling approaches are limited in expressivity, as they are unable to scale to rich transformations that depend on numerous parameters due to the curse of dimensionality. Adversarial examples can be considered as an alternative scheme for data augmentation. By being trained on the most difficult modifications of the inputs, the resulting models are then hopefully able to handle other, presumably easier, modifications as well. The advantage of adversarial augmentation is that it replaces sampling with the use of a single, calculated perturbation that maximally increases the loss. The downside, however, is that these raw adversarial perturbations appear rather unstructured; applying them often does not produce a natural transformation, contrary to a desirable data augmentation technique. To address this, we propose a method to generate adversarial examples that maintain some desired natural structure. We first construct a subspace that only contains perturbations with the desired structure. We then project the raw adversarial gradient onto this space to select a structured transformation that would maximally increase the loss when applied. We demonstrate this approach through two types of image transformations: photometric and geometric. Furthermore, we show that training on such structured adversarial images improves generalization.
Efficient Hyperparameter Tuning with Dynamic Accuracy Derivative-Free Optimization
Venue: arXiv cs.LG 2020
Year: 2020

Many machine learning solutions are framed as optimization problems which rely on good hyperparameters. Algorithms for tuning these hyperparameters usually assume access to exact solutions to the underlying learning problem, which is typically not practical. Here, we apply a recent dynamic accuracy derivative-free optimization method to hyperparameter tuning, which allows inexact evaluations of the learning problem while retaining convergence guarantees. We test the method on the problem of learning elastic net weights for a logistic classifier, and demonstrate its robustness and efficiency compared to a fixed accuracy approach. This demonstrates a promising approach for hyperparameter tuning, with both convergence guarantees and practical performance.
Toward an Automated HPC Pipeline for Processing Large Scale Electron Microscopy Data
Venue: arXiv cs.DC 2020
Year: 2020

We present a fully modular and scalable software pipeline for processing electron microscope (EM) images of brain slices into 3D visualization of individual neurons and demonstrate an end-to-end segmentation of a large EM volume using a supercomputer. Our pipeline scales multiple packages used by the EM community with minimal changes to the original source codes. We tested each step of the pipeline individually, on a workstation, a cluster, and a supercomputer. Furthermore, we can compose workflows from these operations using a Balsam database that can be triggered during the data acquisition or with the use of different front ends and control the granularity of the pipeline execution. We describe the implementation of our pipeline and modifications required to integrate and scale up existing codes. The modular nature of our environment enables diverse research groups to contribute to the pipeline without disrupting the workflow, i.e. new individual codes can be easily integrated for each step on the pipeline.
Task-relevant Representation Learning for Networked Robotic Perception
Venue: arXiv cs.RO 2020
Year: 2020

Today, even the most compute-and-power constrained robots can measure complex, high data-rate video and LIDAR sensory streams. Often, such robots, ranging from low-power drones to space and subterranean rovers, need to transmit high-bitrate sensory data to a remote compute server if they are uncertain or cannot scalably run complex perception or mapping tasks locally. However, today's representations for sensory data are mostly designed for human, not robotic, perception and thus often waste precious compute or wireless network resources to transmit unimportant parts of a scene that are unnecessary for a high-level robotic task. This paper presents an algorithm to learn task-relevant representations of sensory data that are co-designed with a pre-trained robotic perception model's ultimate objective. Our algorithm aggressively compresses robotic sensory data by up to 11x more than competing methods. Further, it achieves high accuracy and robust generalization on diverse tasks including Mars terrain classification with low-power deep learning accelerators, neural motion planning, and environmental timeseries classification.
BlockSim-Net: A Network Based Blockchain Simulator
Venue: arXiv cs.CR 2020
Year: 2020

Since its proposal by Eyal and Sirer (CACM '13), selfish mining attack on proof-of-work blockchains has been studied extensively in terms of both improving its impact and defending against it. Before any defense is deployed in a real world blockchain system, it needs to be tested for security and dependability. However, real blockchain systems are too complex to conduct any test on or benchmark the developed protocols. Some simulation environments have been proposed recently, such as BlockSim (Maher et al., '20). However, BlockSim is developed for the simulation of an entire network on a single CPU. Therefore, it is insufficient to capture the essence of a real blockchain network, as it is not distributed and the complications such as propagation delays that occur in reality cannot be simulated realistically enough. In this work, we propose BlockSim-Net, a simple, efficient, high performance, network-based blockchain simulator, to better reflect reality.
ASFGNN: Automated Separated-Federated Graph Neural Network
Venue: arXiv cs.LG 2020
Year: 2020

Graph Neural Networks (GNNs) have achieved remarkable performance by taking advantage of graph data. The success of GNN models always depends on rich features and adjacent relationships. However, in practice, such data are usually isolated by different data owners (clients) and thus are likely to be Non-Independent and Identically Distributed (Non-IID). Meanwhile, considering the limited network status of data owners, hyper-parameters optimization for collaborative learning approaches is time-consuming in data isolation scenarios. To address these problems, we propose an Automated Separated-Federated Graph Neural Network (ASFGNN) learning paradigm. ASFGNN consists of two main components, i.e., the training of GNN and the tuning of hyper-parameters. Specifically, to solve the data Non-IID problem, we first propose a separated-federated GNN learning model, which decouples the training of GNN into two parts: the message passing part that is done by clients separately, and the loss computing part that is learnt by clients federally. To handle the time-consuming parameter tuning problem, we leverage Bayesian optimization technique to automatically tune the hyper-parameters of all the clients. We conduct experiments on benchmark datasets and the results demonstrate that ASFGNN significantly outperforms the naive federated GNN, in terms of both accuracy and parameter-tuning efficiency.
Alquist 3.0: Alexa Prize Bot Using Conversational Knowledge Graph
Venue: arXiv cs.CL 2020
Year: 2020

The third version of the open-domain dialogue system Alquist developed within the Alexa Prize 2020 competition is designed to conduct coherent and engaging conversations on popular topics. The main novel contribution is the introduction of a system leveraging an innovative approach based on a conversational knowledge graph and adjacency pairs. The conversational knowledge graph allows the system to utilize knowledge expressed during the dialogue in consequent turns and across conversations. Dialogue adjacency pairs divide the conversation into small conversational structures, which can be combined and allow the system to react to a wide range of user inputs flexibly. We discuss and describe Alquist's pipeline, data acquisition and processing, dialogue manager, NLG, knowledge aggregation, and a hierarchy of adjacency pairs. We present the experimental results of the individual parts of the system.
Modular Primitives for High-Performance Differentiable Rendering
Venue: arXiv cs.GR 2020
Year: 2020

We present a modular differentiable renderer design that yields performance superior to previous methods by leveraging existing, highly optimized hardware graphics pipelines. Our design supports all crucial operations in a modern graphics pipeline: rasterizing large numbers of triangles, attribute interpolation, filtered texture lookups, as well as user-programmable shading and geometry processing, all in high resolutions. Our modular primitives allow custom, high-performance graphics pipelines to be built directly within automatic differentiation frameworks such as PyTorch or TensorFlow. As a motivating application, we formulate facial performance capture as an inverse rendering problem and show that it can be solved efficiently using our tools. Our results indicate that this simple and straightforward approach achieves excellent geometric correspondence between rendered results and reference imagery.
Secure Information Flow Connections
Venue: arXiv cs.CR 2020
Year: 2020

Denning's lattice model provided secure information flow analyses with an intuitive mathematical foundation: the lattice ordering determines permitted flows. We examine how this framework may be extended to support the flow of information between autonomous organisations, each employing possibly quite different security lattices and information flow policies. We propose a connection framework that permits different organisations to exchange information while maintaining both security of information flow as well as their autonomy in formulating and maintaining security policies. Our prescriptive framework is based on the rigorous mathematical framework of Lagois connections proposed by Melton, together with a simple operational model for transferring object data between domains. The merit of this formulation is that it is simple, minimal, adaptable and intuitive. We show that our framework is semantically sound, by proving that the connections proposed preserve standard correctness notions such as non-interference. We then illustrate how Lagois theory also provides a robust framework and methodology for negotiating and maintaining secure agreements on information flow between autonomous organisations, even when either or both organisations change their security lattices. Composition and decomposition properties indicate support for a modular approach to secure flow frameworks in complex organisations. We next show that this framework extends naturally and conservatively to the Decentralised Labels Model of Myers et al. - a Lagois connection between the hierarchies of principals in two organisations naturally induces a Lagois connection between the corresponding security label lattices, thus extending the security guarantees ensured by the decentralised model to encompass bidirectional inter-organisational flows.
Understanding Work Practices of Autonomous Agile Teams: A Social-psychological Review
Venue: arXiv cs.SE 2020
Year: 2020

The purpose of this paper is to suggest additional aspects of social psychology that could help when making sense of autonomous agile teams. To make use of well-tested theories in social psychology and instead see how they replicated and differ in the autonomous agile team context would avoid reinventing the wheel. This was done, as an initial step, through looking at some very common agile practices and relate them to existing findings in social-psychological research. The two theories found that I argue could be more applied to the software engineering context are social identity theory and group socialization theory. The results show that literature provides social-psychological reasons for the popularity of some agile practices, but that scientific studies are needed to gather empirical evidence on these under-researched topics. Understanding deeper psychological theories could provide a better understanding of the psychological processes when building autonomous agile team, which could then lead to better predictability and intervention in relation to human factors.
Safe trajectory of a piece moved by a robot
Venue: arXiv cs.RO 2020
Year: 2020

In this work, we propose a mathematical model for a physical problem based on the movement of a metal piece held by a robot. Using the principles of Kirchoff plate theory, a set of equations determining stresses and deformations caused during the motion, have been provided. We also discuss possible numerical treatment of these equations and finally, a solution to the one-dimensional analog of the problem has been presented.
Underspecification Presents Challenges for Credibility in Modern Machine Learning
Venue: arXiv cs.LG 2020
Year: 2020

ML models often exhibit unexpectedly poor behavior when they are deployed in real-world domains. We identify underspecification as a key reason for these failures. An ML pipeline is underspecified when it can return many predictors with equivalently strong held-out performance in the training domain. Underspecification is common in modern ML pipelines, such as those based on deep learning. Predictors returned by underspecified pipelines are often treated as equivalent based on their training domain performance, but we show here that such predictors can behave very differently in deployment domains. This ambiguity can lead to instability and poor model behavior in practice, and is a distinct failure mode from previously identified issues arising from structural mismatch between training and deployment domains. We show that this problem appears in a wide variety of practical ML pipelines, using examples from computer vision, medical imaging, natural language processing, clinical risk prediction based on electronic health records, and medical genomics. Our results show the need to explicitly account for underspecification in modeling pipelines that are intended for real-world deployment in any domain.
Maximum Matchings and Popularity
Venue: arXiv cs.DS 2020
Year: 2020

Let $G$ be a bipartite graph where every node has a strict ranking of its neighbors. For every node, its preferences over neighbors extend naturally to preferences over matchings. Matching $N$ is more popular than matching $M$ if the number of nodes that prefer $N$ to $M$ is more than the number that prefer $M$ to $N$. A maximum matching $M$ in $G$ is a "popular max-matching" if there is no maximum matching in $G$ that is more popular than $M$. Such matchings are relevant in applications where the set of admissible solutions is the set of maximum matchings and we wish to find a best maximum matching as per node preferences. It is known that a popular max-matching always exists in $G$. Here we show a compact extended formulation for the popular max-matching polytope. So when there are edge costs, a min-cost popular max-matching in $G$ can be computed in polynomial time. This is in contrast to the min-cost popular matching problem which is known to be NP-hard. We also consider Pareto-optimality, which is a relaxation of popularity, and show that computing a min-cost Pareto-optimal matching/max-matching is NP-hard.
Fixed Parameter Approximation Scheme for Min-max $k$-cut
Venue: arXiv cs.DS 2020
Year: 2020

We consider the graph $k$-partitioning problem under the min-max objective, termed as Minmax $k$-cut. The input here is a graph $G=(V,E)$ with non-negative edge weights $w:E\rightarrow \mathbb{R}_+$ and an integer $k\geq 2$ and the goal is to partition the vertices into $k$ non-empty parts $V_1, \ldots, V_k$ so as to minimize $\max_{i=1}^k w(δ(V_i))$. Although minimizing the sum objective $\sum_{i=1}^k w(δ(V_i))$, termed as Minsum $k$-cut, has been studied extensively in the literature, very little is known about minimizing the max objective. We initiate the study of Minmax $k$-cut by showing that it is NP-hard and W[1]-hard when parameterized by $k$, and design a parameterized approximation scheme when parameterized by $k$. The main ingredient of our parameterized approximation scheme is an exact algorithm for Minmax $k$-cut that runs in time $(λk)^{O(k^2)}n^{O(1)}$, where $λ$ is value of the optimum and $n$ is the number of vertices. Our algorithmic technique builds on the technique of Lokshtanov, Saurabh, and Surianarayanan (FOCS, 2020) who showed a similar result for Minsum $k$-cut. Our algorithmic techniques are more general and can be used to obtain parameterized approximation schemes for minimizing $\ell_p$-norm measures of $k$-partitioning for every $p\geq 1$.
Towards Obfuscated Malware Detection for Low Powered IoT Devices
Venue: arXiv cs.CR 2020
Year: 2020

With the increased deployment of IoT and edge devices into commercial and user networks, these devices have become a new threat vector for malware authors. It is imperative to protect these devices as they become more prevalent in commercial and personal networks. However, due to their limited computational power and storage space, especially in the case of battery-powered devices, it is infeasible to deploy state-of-the-art malware detectors onto these systems. In this work, we propose using and extracting features from Markov matrices constructed from opcode traces as a low cost feature for unobfuscated and obfuscated malware detection. We empirically show that our approach maintains a high detection rate while consuming less power than similar work.
Massively Parallel Graph Drawing and Representation Learning
Venue: arXiv cs.LG 2020
Year: 2020

To fully exploit the performance potential of modern multi-core processors, machine learning and data mining algorithms for big data must be parallelized in multiple ways. Today's CPUs consist of multiple cores, each following an independent thread of control, and each equipped with multiple arithmetic units which can perform the same operation on a vector of multiple data objects. Graph embedding, i.e. converting the vertices of a graph into numerical vectors is a data mining task of high importance and is useful for graph drawing (low-dimensional vectors) and graph representation learning (high-dimensional vectors). In this paper, we propose MulticoreGEMPE (Graph Embedding by Minimizing the Predictive Entropy), an information-theoretic method which can generate low and high-dimensional vectors. MulticoreGEMPE applies MIMD (Multiple Instructions Multiple Data, using OpenMP) and SIMD (Single Instructions Multiple Data, using AVX-512) parallelism. We propose general ideas applicable in other graph-based algorithms like \emph{vectorized hashing} and \emph{vectorized reduction}. Our experimental evaluation demonstrates the superiority of our approach.
Amalgamating Different JIT Compilations in a Meta-tracing JIT Compiler Framework
Venue: arXiv cs.PL 2020
Year: 2020

Many modern virtual machines, such as JVMs, .NET Framework, and V8, employ a just-in-time (JIT) compiler to achieve their high-performance. There are two major compilation strategies; trace-based compilation and method-based compilation. They have their own advantages and disadvantages, so we presume that applying suitable strategies for different program parts is essential for faster execution. This paper proposes a new approach called the meta-hybrid JIT compiler framework, which combined the two strategies in a single meta-JIT compiler framework. We implemented the BacCaml framework for proof-of-concept. We also report that some programs actually ran faster by the hybrid compilation in our experiments.
Fast XML/HTML for Haskell: XML TypeLift
Venue: arXiv cs.PL 2020
Year: 2020

The paper presents and compares a range of parsers with and without data mapping for conversion between XML and Haskell. The best performing parser competes favorably with the fastest tools available in other languages and is, thus, suitable for use in large-scale data analysis. The best performing parser also allows software developers of intermediate-level Haskell programming skills to start processing large numbers of XML documents soon after finding the relevant XML Schema from a simple internet search, without the need for specialist prior knowledge or skills. We hope that this unique combination of parser performance and usability will provide a new standard for XML mapping to high-level languages.