OpenRaiser/Intern-Atlas huggingface dataset:
RUAD: unsupervised anomaly detection in HPC systems
Venue: arXiv cs.LG 2022
Year: 2022

The increasing complexity of modern high-performance computing (HPC) systems necessitates the introduction of automated and data-driven methodologies to support system administrators' effort toward increasing the system's availability. Anomaly detection is an integral part of improving the availability as it eases the system administrator's burden and reduces the time between an anomaly and its resolution. However, current state-of-the-art (SoA) approaches to anomaly detection are supervised and semi-supervised, so they require a human-labelled dataset with anomalies - this is often impractical to collect in production HPC systems. Unsupervised anomaly detection approaches based on clustering, aimed at alleviating the need for accurate anomaly data, have so far shown poor performance. In this work, we overcome these limitations by proposing RUAD, a novel Recurrent Unsupervised Anomaly Detection model. RUAD achieves better results than the current semi-supervised and unsupervised SoA approaches. This is achieved by considering temporal dependencies in the data and including long-short term memory cells in the model architecture. The proposed approach is assessed on a complete ten-month history of a Tier-0 system (Marconi100 from CINECA with 980 nodes). RUAD achieves an area under the curve (AUC) of 0.763 in semi-supervised training and an AUC of 0.767 in unsupervised training, which improves upon the SoA approach that achieves an AUC of 0.747 in semi-supervised training and an AUC of 0.734 in unsupervised training. It also vastly outperforms the current SoA unsupervised anomaly detection approach based on clustering, achieving the AUC of 0.548.
On Evaluating Self-Adaptive and Self-Healing Systems using Chaos Engineering
Venue: arXiv cs.SE 2022
Year: 2022

With the growing adoption of self-adaptive systems in various domains, there is an increasing need for strategies to assess their correct behavior. In particular self-healing systems, which aim to provide resilience and fault-tolerance, often deal with unanticipated failures in critical and highly dynamic environments. Their reactive and complex behavior makes it challenging to assess if these systems execute according to the desired goals. Recently, several studies have expressed concern about the lack of systematic evaluation methods for self-healing behavior. In this paper, we propose CHESS, an approach for the systematic evaluation of self-adaptive and self-healing systems that builds on chaos engineering. Chaos engineering is a methodology for subjecting a system to unexpected conditions and scenarios. It has shown great promise in helping developers build resilient microservice architectures and cyber-physical systems. CHESS turns this idea around by using chaos engineering to evaluate how well a self-healing system can withstand such perturbations. We investigate the viability of this approach through an exploratory study on a self-healing smart office environment. The study helps us explore the promises and limitations of the approach, as well as identify directions where additional work is needed. We conclude with a summary of lessons learned.
Multi-Message Private Information Retrieval: A Scalar Linear Solution
Venue: arXiv cs.IT 2022
Year: 2022

In recent years, the Multi-message Private Information Retrieval (MPIR) problem has received significant attention from the research community. In this problem, a user wants to privately retrieve $D$ messages out of $K$ messages whose identical copies are stored on $N$ remote servers, while maximizing the download rate. The MPIR schemes can find applications in many practical scenarios and can serve as an important building block for private computation and private machine learning applications. The existing solutions for MPIR require a large degree of subpacketization, which can result in large overheads, high complexity, and impose constraints on the system parameters. These factors can limit practical applications of the existing solutions. In this paper, we present a methodology for the design of scalar-linear MPIR schemes. Such schemes are easy to implement in practical systems as they do not require partitioning of messages into smaller size sub-messages and do not impose any constraints on the minimum required size of the messages. Focusing on the case of $N=D+1$, we show that when $D$ divides $K$, our scheme achieves the capacity, where the capacity is defined as the maximum achievable download rate. When the divisibility condition does not hold, the performance of our scheme is the same or within a small additive margin compared to the best known scheme that requires a high degree of subpacketization.
FFCNN: Fast FPGA based Acceleration for Convolution neural network inference
Venue: arXiv cs.LG 2022
Year: 2022

We present a new efficient OpenCL-based Accelerator for large scale Convolutional Neural Networks called Fast Inference on FPGAs for Convolution Neural Network (FFCNN). FFCNN is based on a deeply pipelined OpenCL kernels architecture. As pointed out before, high-level synthesis tools such as the OpenCL framework can easily port codes originally designed for CPUs and GPUs to FPGAs, but it is still difficult to make OpenCL codes run efficiently on FPGAs. This work aims to propose an efficient FPGA implementation of OpenCL High-Performance Computing Applications. To do so, a Data reuse and task mapping techniques are also presented to improve design efficiency. In addition, the following motivations were taken into account when developing FFCNN: 1) FFCNN has been designed to be easily implemented on Intel OpenCL SDK based FPGA design flow. 2) In FFFCN, different techniques have been integrated to improve the memory band with and throughput. A performance analysis is conducted on two deep CNN for Large-Scale Images classification. The obtained results, and the comparison with other works designed to accelerate the same types of architectures, show the efficiency and the competitiveness of the proposed accelerator design by significantly improved performance and resource utilization.
Podcast Summary Assessment: A Resource for Evaluating Summary Assessment Methods
Venue: arXiv cs.CL 2022
Year: 2022

Automatic summary assessment is useful for both machine-generated and human-produced summaries. Automatically evaluating the summary text given the document enables, for example, summary generation system development and detection of inappropriate summaries. Summary assessment can be run in a number of modes: ranking summary generation systems; ranking summaries of a particular document; and estimating the quality of a document-summary pair on an absolute scale. Existing datasets with annotation for summary assessment are usually based on news summarization datasets such as CNN/DailyMail or XSum. In this work, we describe a new dataset, the podcast summary assessment corpus, a collection of podcast summaries that were evaluated by human experts at TREC2020. Compared to existing summary assessment data, this dataset has two unique aspects: (i) long-input, speech podcast based, documents; and (ii) an opportunity to detect inappropriate reference summaries in podcast corpus. First, we examine existing assessment methods, including model-free and model-based methods, and provide benchmark results for this long-input summary assessment dataset. Second, with the aim of filtering reference summary-document pairings for training, we apply summary assessment for data selection. The experimental results on these two aspects provide interesting insights on the summary assessment and generation tasks. The podcast summary assessment data is available.
ECP SOLLVE: Validation and Verification Testsuite Status Update and Compiler Insight for OpenMP
Venue: arXiv cs.DC 2022
Year: 2022

The OpenMP language continues to evolve with every new specification release, as does the need to validate and verify the new features that have been introduced. With the release of OpenMP 5.0 and OpenMP 5.1, plenty of new target offload and host-based features have been introduced to the programming model. While OpenMP continues to grow in maturity, there is an observable growth in the number of compiler and hardware vendors that support OpenMP. In this manuscript, we focus on evaluating the conformity and implementation progress of various compiler vendors such as Cray, IBM, GNU, Clang/LLVM, NVIDIA, Intel and AMD. We specifically address the 4.5, 5.0, and 5.1 versions of the specification.
The PWLR Graph Representation: A Persistent Weisfeiler-Lehman scheme with Random Walks for Graph Classification
Venue: arXiv cs.LG 2022
Year: 2022

This paper presents the Persistent Weisfeiler-Lehman Random walk scheme (abbreviated as PWLR) for graph representations, a novel mathematical framework which produces a collection of explainable low-dimensional representations of graphs with discrete and continuous node features. The proposed scheme effectively incorporates normalized Weisfeiler-Lehman procedure, random walks on graphs, and persistent homology. We thereby integrate three distinct properties of graphs, which are local topological features, node degrees, and global topological invariants, while preserving stability from graph perturbations. This generalizes many variants of Weisfeiler-Lehman procedures, which are primarily used to embed graphs with discrete node labels. Empirical results suggest that these representations can be efficiently utilized to produce comparable results to state-of-the-art techniques in classifying graphs with discrete node labels, and enhanced performances in classifying those with continuous node features.
Planar Confluent Orthogonal Drawings of 4-Modal Digraphs
Venue: arXiv cs.CG 2022
Year: 2022

In a planar confluent orthogonal drawing (PCOD) of a directed graph (digraph) vertices are drawn as points in the plane and edges as orthogonal polylines starting with a vertical segment and ending with a horizontal segment. Edges may overlap in their first or last segment, but must not intersect otherwise. PCODs can be seen as a directed variant of Kandinsky drawings or as planar L-drawings of subdivisions of digraphs. The maximum number of subdivision vertices in an edge is then the split complexity. A PCOD is upward if each edge is drawn with monotonically increasing y-coordinates and quasi-upward if no edge starts with decreasing y-coordinates. We study the split complexity of PCODs and (quasi-)upward PCODs for various classes of graphs.
Price of MEV: Towards a Game Theoretical Approach to MEV
Venue: arXiv cs.GT 2022
Year: 2022

Maximal (also miner) extractable value, or MEV, usually refers to the value that privileged players can extract by strategically ordering, censoring, and placing transactions in a blockchain. Each blockchain network, which we refer to as a domain, has its own consensus, ordering, and block-creation mechanisms, which gives rise to different optimal strategies to extract MEV. The strategic behaviour of rational players, known as searchers, lead to MEV games that have different impacts and externalities in each domain. Several ordering mechanisms, which determine the inclusion and position of transactions in a block, have been considered to construct alternative games to organise MEV extraction, and minimize negative externalities; examples include sealed bid auctions, first input first output, and private priority gas auctions. However, to date, no sufficiently formal and abstract definition of MEV games have been made. In this paper, we take a step toward the formalization of MEV games and compare different ordering mechanisms and their externalities. In particular, we attempt to formalize games that arise from common knowledge MEV opportunities, such as arbitrage and sandwich attacks. In defining these games, we utilise a theoretical framework that provides groundwork for several important roles and concepts, such as the searcher, sequencer, domain, and bundle. We also introduce the price of MEV as the price of anarchy of MEV games, a measure that provides formal comparison between different ordering mechanisms.
Generalization In Multi-Objective Machine Learning
Venue: arXiv cs.LG 2022
Year: 2022

Modern machine learning tasks often require considering not just one but multiple objectives. For example, besides the prediction quality, this could be the efficiency, robustness or fairness of the learned models, or any of their combinations. Multi-objective learning offers a natural framework for handling such problems without having to commit to early trade-offs. Surprisingly, statistical learning theory so far offers almost no insight into the generalization properties of multi-objective learning. In this work, we make first steps to fill this gap: we establish foundational generalization bounds for the multi-objective setting as well as generalization and excess bounds for learning with scalarizations. We also provide the first theoretical analysis of the relation between the Pareto-optimal sets of the true objectives and the Pareto-optimal sets of their empirical approximations from training data. In particular, we show a surprising asymmetry: all Pareto-optimal solutions can be approximated by empirically Pareto-optimal ones, but not vice versa.
Quality of Experience Optimization in IoT Energy Services
Venue: arXiv cs.DC 2022
Year: 2022

We propose a novel Quality of Experience (QoE) metric as a key criterion to optimize the composition of energy services in a crowdsourced IoT environment. A novel importance-based composition algorithm is proposed to ensure the highest QoE for consumers. A set of experiments is conducted to evaluate the proposed approaches' effectiveness and efficiency.
UWB Role Allocation with Distributed Ledger Technologies for Scalable Relative Localization in Multi-Robot Systems
Venue: arXiv cs.RO 2022
Year: 2022

Systems for relative localization in multi-robot systems based on ultra-wideband (UWB) ranging have recently emerged as robust solutions for GNSS-denied environments. Scalability remains one of the key challenges, particularly in ad-hoc deployments. Recent solutions include dynamic allocation of active and passive localization modes for different robots or nodes in the system. With larger-scale systems becoming more distributed, key research questions arise in the areas of security and trustability of such localization systems. This paper studies the potential integration of collaborative-decision making processes with distributed ledger technologies. Specifically, we investigate the design and implementation of a methodology for running an UWB role allocation algorithm within smart contracts in a blockchain. In previous works, we have separately studied the integration of ROS2 with the Hyperledger Fabric blockchain, and introduced a new algorithm for scalable UWB-based localization. In this paper, we extend these works by (i) running experiments with larger number of mobile robots switching between different spatial configurations and (ii) integrating the dynamic UWB role allocation algorithm into Fabric smart contracts for distributed decision-making in a system of multiple mobile robots. This enables us to deliver the same functionality within a secure and trustable process, with enhanced identity and data access management. Our results show the effectiveness of the UWB role allocation for continuously varying spatial formations of six autonomous mobile robots, while demonstrating a low impact on latency and computational resources of adding the blockchain layer that does not affect the localization process.
PECAN: A Product-Quantized Content Addressable Memory Network
Venue: arXiv cs.LG 2022
Year: 2022

A novel deep neural network (DNN) architecture is proposed wherein the filtering and linear transform are realized solely with product quantization (PQ). This results in a natural implementation via content addressable memory (CAM), which transcends regular DNN layer operations and requires only simple table lookup. Two schemes are developed for the end-to-end PQ prototype training, namely, through angle- and distance-based similarities, which differ in their multiplicative and additive natures with different complexity-accuracy tradeoffs. Even more, the distance-based scheme constitutes a truly multiplier-free DNN solution. Experiments confirm the feasibility of such Product-Quantized Content Addressable Memory Network (PECAN), which has strong implication on hardware-efficient deployments especially for in-memory computing.
Spatio-Temporal Wind Speed Forecasting using Graph Networks and Novel Transformer Architectures
Venue: arXiv cs.LG 2022
Year: 2022

This study focuses on multi-step spatio-temporal wind speed forecasting for the Norwegian continental shelf. The study aims to leverage spatial dependencies through the relative physical location of different measurement stations to improve local wind forecasts. Our multi-step forecasting models produce either 10-minute, 1- or 4-hour forecasts, with 10-minute resolution, meaning that the models produce more informative time series for predicted future trends. A graph neural network (GNN) architecture was used to extract spatial dependencies, with different update functions to learn temporal correlations. These update functions were implemented using different neural network architectures. One such architecture, the Transformer, has become increasingly popular for sequence modelling in recent years. Various alterations have been proposed to better facilitate time series forecasting, of which this study focused on the Informer, LogSparse Transformer and Autoformer. This is the first time the LogSparse Transformer and Autoformer have been applied to wind forecasting and the first time any of these or the Informer have been formulated in a spatio-temporal setting for wind forecasting. By comparing against spatio-temporal Long Short-Term Memory (LSTM) and Multi-Layer Perceptron (MLP) models, the study showed that the models using the altered Transformer architectures as update functions in GNNs were able to outperform these. Furthermore, we propose the Fast Fourier Transformer (FFTransformer), which is a novel Transformer architecture based on signal decomposition and consists of two separate streams that analyse the trend and periodic components separately. The FFTransformer and Autoformer were found to achieve superior results for the 10-minute and 1-hour ahead forecasts, with the FFTransformer significantly outperforming all other models for the 4-hour ahead forecasts.
Cortical-inspired placement and routing: minimizing the memory resources in multi-core neuromorphic processors
Venue: arXiv cs.NE 2022
Year: 2022

Brain-inspired event-based neuromorphic processing systems have emerged as a promising technology in particular for bio-medical circuits and systems. However, both neuromorphic and biological implementations of neural networks have critical energy and memory constraints. To minimize the use of memory resources in multi-core neuromorphic processors, we propose a network design approach inspired by biological neural networks. We use this approach to design a new routing scheme optimized for small-world networks and, at the same time, to present a hardware-aware placement algorithm that optimizes the allocation of resources for small-world network models. We validate the algorithm with a canonical small-world network and present preliminary results for other networks derived from it
A Data-Centric Approach For Dual-Arm Robotic Garment Flattening
Venue: arXiv cs.RO 2022
Year: 2022

Due to the high dimensionality of object states, a garment flattening pipeline requires recognising the configurations of garments for a robot to produce/select manipulation plans to flatten garments. In this paper, we propose a data-centric approach to identify known configurations of garments based on a known configuration network (KCNet) trained on depth images that capture the known configurations of garments and prior knowledge of garment shapes. In this paper, we propose a data-centric approach to identify the known configurations of garments based on a known configuration network (KCNet) trained on the depth images that capture the known configurations of garments and prior knowledge of garment shapes. The known configurations of garments are the configurations of garments when a robot hangs garments in the middle of the air. We found that it is possible to achieve 92\% accuracy if we let the robot recognise the common hanging configurations (the known configurations) of garments. We also demonstrate an effective robot garment flattening pipeline with our proposed approach on a dual-arm Baxter robot. The robot achieved an average operating time of 221.6 seconds and successfully manipulated garments of five different shapes.
A Lattice Boltzmann Method for nonlinear solid mechanics in the reference configuration
Venue: arXiv cs.CE 2022
Year: 2022

With a sufficiently fine discretisation, the Lattice Boltzmann Method (LBM) mimics a second order Crank-Nicolson scheme for certain types of balance laws (Farag et al. [2021]). This allows the explicit, highly parallelisable LBM to efficiently solve the fundamental equations of solid mechanics: the conservation of mass, the balance of linear momentum, and constitutive relations. To date, all LBM algorithms for solid simulation - see e.g. Murthy et al. [2017], Escande et al. [2020], Schlüter et al. [2021] - have been limited to the small strain case. Furthermore, the typical interpretation of the LBM in the current (Eulerian) configuration is not easily extensible to large strains, as large topological changes complicate the treatment of boundary conditions. In this publication, we propose a large deformation Lattice Boltzmann Method for geometrically and constitutively nonlinear solid mechanics. To facilitate versatile boundary modelling, the algorithm is defined in the reference (Lagrangian) configuration.
DR-DSGD: A Distributionally Robust Decentralized Learning Algorithm over Graphs
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, we propose to solve a regularized distributionally robust learning problem in the decentralized setting, taking into account the data distribution shift. By adding a Kullback-Liebler regularization function to the robust min-max optimization problem, the learning problem can be reduced to a modified robust minimization problem and solved efficiently. Leveraging the newly formulated optimization problem, we propose a robust version of Decentralized Stochastic Gradient Descent (DSGD), coined Distributionally Robust Decentralized Stochastic Gradient Descent (DR-DSGD). Under some mild assumptions and provided that the regularization parameter is larger than one, we theoretically prove that DR-DSGD achieves a convergence rate of $\mathcal{O}\left(1/\sqrt{KT} + K/T\right)$, where $K$ is the number of devices and $T$ is the number of iterations. Simulation results show that our proposed algorithm can improve the worst distribution test accuracy by up to $10\%$. Moreover, DR-DSGD is more communication-efficient than DSGD since it requires fewer communication rounds (up to $20$ times less) to achieve the same worst distribution test accuracy target. Furthermore, the conducted experiments reveal that DR-DSGD results in a fairer performance across devices in terms of test accuracy.