OpenRaiser/Intern-Atlas huggingface dataset:
Epistemic Networks
Venue: arXiv cs.SI 2021
Year: 2021

We show how important phenomena in social networks like coordination, trust and the communication of unsubstantiated information (gossip) can be modelled and understood using epistemic networks or epinets: directed graphs comprising networked agents and the key facts, statements or other kinds of propositional beliefs relevant to their actions. We use epinets to sharpen the explanatory and reach of social network analysis to situations problematic to both network-structural approaches and epistemic game theory.
Software Engineering for Robotic Systems:a systematic mapping study
Venue: arXiv cs.RO 2021
Year: 2021

Robots are being applied in a vast range of fields, leading researchers and practitioners to write tasks more complex than in the past. The robot software complexity increases the difficulty of engineering the robot's software components with quality requirements. Researchers and practitioners have applied software engineering (SE) approaches and robotic domains to address this issue in the last two decades. This study aims to identify, classify and evaluate the current state-of-the-art Software Engineering for Robotic Systems (SERS). We systematically selected and analyzed 50 primary studies extracted from an automated search on Scopus digital library and manual search on the two editions of the RoSE workshop. We present three main contributions. Firstly, we provide an analysis from three following perspectives: demographics of publication, SE areas applied in robotics domains, and RSE findings. Secondly, we show a catalogue of research studies that apply software engineering techniques in the robotic domain, classified with the SWEBOK guide. We have identified 5 of 15 software engineering areas from the SWEBOK guide applied explicitly in robotic domains. The majority of the studies focused on the development phase (design, models and methods and construction). Testing and quality software areas have little coverage in SERS. Finally, we identify research opportunities and gaps in software engineering for robotic systems for future studies.
Benchmarking Graph Neural Networks on Link Prediction
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, we benchmark several existing graph neural network (GNN) models on different datasets for link predictions. In particular, the graph convolutional network (GCN), GraphSAGE, graph attention network (GAT) as well as variational graph auto-encoder (VGAE) are implemented dedicated to link prediction tasks, in-depth analysis are performed, and results from several different papers are replicated, also a more fair and systematic comparison are provided. Our experiments show these GNN architectures perform similarly on various benchmarks for link prediction tasks.
Directional Bias Amplification
Venue: arXiv cs.LG 2021
Year: 2021

Mitigating bias in machine learning systems requires refining our understanding of bias propagation pathways: from societal structures to large-scale data to trained models to impact on society. In this work, we focus on one aspect of the problem, namely bias amplification: the tendency of models to amplify the biases present in the data they are trained on. A metric for measuring bias amplification was introduced in the seminal work by Zhao et al. (2017); however, as we demonstrate, this metric suffers from a number of shortcomings including conflating different types of bias amplification and failing to account for varying base rates of protected attributes. We introduce and analyze a new, decoupled metric for measuring bias amplification, $\text{BiasAmp}_{\rightarrow}$ (Directional Bias Amplification). We thoroughly analyze and discuss both the technical assumptions and normative implications of this metric. We provide suggestions about its measurement by cautioning against predicting sensitive attributes, encouraging the use of confidence intervals due to fluctuations in the fairness of models across runs, and discussing the limitations of what this metric captures. Throughout this paper, we work to provide an interrogative look at the technical measurement of bias amplification, guided by our normative ideas of what we want it to encompass. Code is located at https://github.com/princetonvisualai/directional-bias-amp
Railway Anomaly detection model using synthetic defect images generated by CycleGAN
Venue: arXiv cs.CV 2021
Year: 2021

Although training data is essential for machine learning, railway companies are facing difficulties in gathering adequate images of defective equipment due to their proactive replacement of would be defective equipment. Nevertheless, proactive replacement is indispensable for safe and undisturbed operation of public transport. In this research, we have developed a model using CycleGAN to generate artificial images of defective equipment instead of real images. By adopting these generated images as training data, we verified that these images are indistinguishable from real images and they play a vital role in enhancing the accuracy of the defect detection models.
Simultaneously Reconciled Quantile Forecasting of Hierarchically Related Time Series
Venue: arXiv cs.LG 2021
Year: 2021

Many real-life applications involve simultaneously forecasting multiple time series that are hierarchically related via aggregation or disaggregation operations. For instance, commercial organizations often want to forecast inventories simultaneously at store, city, and state levels for resource planning purposes. In such applications, it is important that the forecasts, in addition to being reasonably accurate, are also consistent w.r.t one another. Although forecasting such hierarchical time series has been pursued by economists and data scientists, the current state-of-the-art models use strong assumptions, e.g., all forecasts being unbiased estimates, noise distribution being Gaussian. Besides, state-of-the-art models have not harnessed the power of modern nonlinear models, especially ones based on deep learning. In this paper, we propose using a flexible nonlinear model that optimizes quantile regression loss coupled with suitable regularization terms to maintain the consistency of forecasts across hierarchies. The theoretical framework introduced herein can be applied to any forecasting model with an underlying differentiable loss function. A proof of optimality of our proposed method is also provided. Simulation studies over a range of datasets highlight the efficacy of our approach.
Neuroevolution of a Recurrent Neural Network for Spatial and Working Memory in a Simulated Robotic Environment
Venue: arXiv cs.NE 2021
Year: 2021

Animals ranging from rats to humans can demonstrate cognitive map capabilities. We evolved weights in a biologically plausible recurrent neural network (RNN) using an evolutionary algorithm to replicate the behavior and neural activity observed in rats during a spatial and working memory task in a triple T-maze. The rat was simulated in the Webots robot simulator and used vision, distance and accelerometer sensors to navigate a virtual maze. After evolving weights from sensory inputs to the RNN, within the RNN, and from the RNN to the robot's motors, the Webots agent successfully navigated the space to reach all four reward arms with minimal repeats before time-out. Our current findings suggest that it is the RNN dynamics that are key to performance, and that performance is not dependent on any one sensory type, which suggests that neurons in the RNN are performing mixed selectivity and conjunctive coding. Moreover, the RNN activity resembles spatial information and trajectory-dependent coding observed in the hippocampus. Collectively, the evolved RNN exhibits navigation skills, spatial memory, and working memory. Our method demonstrates how the dynamic activity in evolved RNNs can capture interesting and complex cognitive behavior and may be used to create RNN controllers for robotic applications.
Adapting legacy robotic machinery to industry 4: a ciot experiment version 1
Venue: arXiv cs.RO 2021
Year: 2021

This paper presents an experimental adaptation of a non-collaborative robot arm to collaborate with the environment, as one step towards adapting legacy robotic machinery to fit in industry 4.0 requirements. A cloud-based internet of things (CIoT) service is employed to connect, supervise and control a robotic arm's motion using the added wireless sensing devices to the environment. A programmable automation controller (PAC) unit, connected to the robot arm receives the most recent changes and updates the motion of the robot arm. The experimental results show that the proposed non-expensive service is tractable and adaptable to higher level for machine to machine collaboration. The proposed approach in this paper has industrial and educational applications. In the proposed approach, the CIoT technology is added as a technology interface between the sensors to the environment and the robotic arm. The proposed approach is versatile and fits to variety of applications to meet the flexible requirements of industry 4.0. The proposed approach has been implemented in an experiment using MECA 500 robot arm and AMAX 5580 programmable automation controller and ultrasonic proximity wireless sensor.
Distributionally Robust Federated Averaging
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, we study communication efficient distributed algorithms for distributionally robust federated learning via periodic averaging with adaptive sampling. In contrast to standard empirical risk minimization, due to the minimax structure of the underlying optimization problem, a key difficulty arises from the fact that the global parameter that controls the mixture of local losses can only be updated infrequently on the global stage. To compensate for this, we propose a Distributionally Robust Federated Averaging (DRFA) algorithm that employs a novel snapshotting scheme to approximate the accumulation of history gradients of the mixing parameter. We analyze the convergence rate of DRFA in both convex-linear and nonconvex-linear settings. We also generalize the proposed idea to objectives with regularization on the mixture parameter and propose a proximal variant, dubbed as DRFA-Prox, with provable convergence rates. We also analyze an alternative optimization method for regularized cases in strongly-convex-strongly-concave and non-convex (under PL condition)-strongly-concave settings. To the best of our knowledge, this paper is the first to solve distributionally robust federated learning with reduced communication, and to analyze the efficiency of local descent methods on distributed minimax problems. We give corroborating experimental evidence for our theoretical results in federated learning settings.
Strapdown Inertial Navigation System Initial Alignment based on Group of Double Direct Spatial Isometries
Venue: arXiv cs.RO 2021
Year: 2021

The task of strapdown inertial navigation system (SINS) initial alignment is to calculate the attitude transformation matrix from body frame to navigation frame. In this paper, such attitude transformation matrix is divided into two parts through introducing the initial inertially fixed navigation frame as inertial frame. The attitude changes of the navigation frame corresponding to the defined inertial frame can be exactly calculated with known velocity and position provided by GNSS. The attitude from body frame to the defined inertial frame is estimated based on the SINS mechanization in inertial frame. The attitude, velocity and position in inertial frame are formulated together as element of the group of double direct spatial isometries.It is proven that the group state model in inertial frame satisfies a particular "group affine" property and the corresponding error model satisfies a "log linear" autonomous differential equation on the Lie algebra. Based on such striking property, the attitude from body frame to the defined inertial frame can be estimated based on the linear error model with even extreme large misalignments. Two different error state vectors are extracted based on right and left matrix multiplications and the detailed linear state space models are derived based on the right and left errors for SINS mechanization in inertial frame. With the derived linear state space models, the explicit initial alignment procedures have been presented. Extensive simulation and field tests indicate that the initial alignment based on the left error model can perform quite well within a wide range of initial attitude errors, although the used filter is still a type of linear Kalman filter. This method is promising in practical products abandoning the traditional coarse alignment stage.
LazyFormer: Self Attention with Lazy Update
Venue: arXiv cs.CL 2021
Year: 2021

Improving the efficiency of Transformer-based language pre-training is an important task in NLP, especially for the self-attention module, which is computationally expensive. In this paper, we propose a simple but effective solution, called \emph{LazyFormer}, which computes the self-attention distribution infrequently. LazyFormer composes of multiple lazy blocks, each of which contains multiple Transformer layers. In each lazy block, the self-attention distribution is only computed once in the first layer and then is reused in all upper layers. In this way, the cost of computation could be largely saved. We also provide several training tricks for LazyFormer. Extensive experiments demonstrate the effectiveness of the proposed method.
On Interpretability and Similarity in Concept-Based Machine Learning
Venue: arXiv cs.LG 2021
Year: 2021

Machine Learning (ML) provides important techniques for classification and predictions. Most of these are black-box models for users and do not provide decision-makers with an explanation. For the sake of transparency or more validity of decisions, the need to develop explainable/interpretable ML-methods is gaining more and more importance. Certain questions need to be addressed: How does an ML procedure derive the class for a particular entity? Why does a particular clustering emerge from a particular unsupervised ML procedure? What can we do if the number of attributes is very large? What are the possible reasons for the mistakes for concrete cases and models? For binary attributes, Formal Concept Analysis (FCA) offers techniques in terms of intents of formal concepts, and thus provides plausible reasons for model prediction. However, from the interpretable machine learning viewpoint, we still need to provide decision-makers with the importance of individual attributes to the classification of a particular object, which may facilitate explanations by experts in various domains with high-cost errors like medicine or finance. We discuss how notions from cooperative game theory can be used to assess the contribution of individual attributes in classification and clustering processes in concept-based machine learning. To address the 3rd question, we present some ideas on how to reduce the number of attributes using similarities in large contexts.
Distributed Online Learning with Multiple Kernels
Venue: arXiv cs.LG 2021
Year: 2021

We consider the problem of learning a nonlinear function over a network of learners in a fully decentralized fashion. Online learning is additionally assumed, where every learner receives continuous streaming data locally. This learning model is called a fully distributed online learning (or a fully decentralized online federated learning). For this model, we propose a novel learning framework with multiple kernels, which is named DOMKL. The proposed DOMKL is devised by harnessing the principles of an online alternating direction method of multipliers and a distributed Hedge algorithm. We theoretically prove that DOMKL over T time slots can achieve an optimal sublinear regret, implying that every learner in the network can learn a common function which has a diminishing gap from the best function in hindsight. Our analysis also reveals that DOMKL yields the same asymptotic performance of the state-of-the-art centralized approach while keeping local data at edge learners. Via numerical tests with real datasets, we demonstrate the effectiveness of the proposed DOMKL on various online regression and time-series prediction tasks.
A Comprehensive Survey on the Multiple Travelling Salesman Problem: Applications, Approaches and Taxonomy
Venue: arXiv cs.CC 2021
Year: 2021

The Multiple Travelling Salesman Problem (MTSP) is among the most interesting combinatorial optimization problems because it is widely adopted in real-life applications, including robotics, transportation, networking, etc. Although the importance of this optimization problem, there is no survey dedicated to reviewing recent MTSP contributions. In this paper, we aim to fill this gap by providing a comprehensive review of existing studies on MTSP. In this survey, we focus on MTSP's recent contributions to both classical vehicles/robots and unmanned aerial vehicles. We highlight the approaches applied to solve the MTSP as well as its application domains. We analyze the MTSP variants and propose a taxonomy and a classification of recent studies.
A New Neuromorphic Computing Approach for Epileptic Seizure Prediction
Venue: arXiv cs.NE 2021
Year: 2021

Several high specificity and sensitivity seizure prediction methods with convolutional neural networks (CNNs) are reported. However, CNNs are computationally expensive and power hungry. These inconveniences make CNN-based methods hard to be implemented on wearable devices. Motivated by the energy-efficient spiking neural networks (SNNs), a neuromorphic computing approach for seizure prediction is proposed in this work. This approach uses a designed gaussian random discrete encoder to generate spike sequences from the EEG samples and make predictions in a spiking convolutional neural network (Spiking-CNN) which combines the advantages of CNNs and SNNs. The experimental results show that the sensitivity, specificity and AUC can remain 95.1%, 99.2% and 0.912 respectively while the computation complexity is reduced by 98.58% compared to CNN, indicating that the proposed Spiking-CNN is hardware friendly and of high precision.
libtxsize -- a library for automated Bitcoin transaction-size estimates
Venue: arXiv cs.DC 2021
Year: 2021

This paper presents libtxsize, a library to estimate the size requirements of arbitrary Bitcoin transactions. To account for different use cases, the library provides estimates in bytes, virtual bytes, and weight units. In addition to all currently existing input, output, and witness types, the library also supports estimates for the anticipated Pay-to-Taproot transaction type, so that estimates can be used as input for models attempting to quantify the impact of Taproot on Bitcoin's scalability. libtxsize is based on analytic models, whose credibility is established through first-principle analysis of transaction types as well as exhaustive empirical validation. Consequently, the paper can also serve as reference for different Bitcoin data and transaction types, their semantics, and their size requirements (both from an analytic and empirical point of view).
Hyperparameter Transfer Learning with Adaptive Complexity
Venue: arXiv cs.LG 2021
Year: 2021

Bayesian optimization (BO) is a sample efficient approach to automatically tune the hyperparameters of machine learning models. In practice, one frequently has to solve similar hyperparameter tuning problems sequentially. For example, one might have to tune a type of neural network learned across a series of different classification problems. Recent work on multi-task BO exploits knowledge gained from previous tuning tasks to speed up a new tuning task. However, previous approaches do not account for the fact that BO is a sequential decision making procedure. Hence, there is in general a mismatch between the number of evaluations collected in the current tuning task compared to the number of evaluations accumulated in all previously completed tasks. In this work, we enable multi-task BO to compensate for this mismatch, such that the transfer learning procedure is able to handle different data regimes in a principled way. We propose a new multi-task BO method that learns a set of ordered, non-linear basis functions of increasing complexity via nested drop-out and automatic relevance determination. Experiments on a variety of hyperparameter tuning problems show that our method improves the sample ef
A Fragile multi-CPR Game
Venue: arXiv cs.GT 2021
Year: 2021

A Fragile CPR Game is an instance of a resource sharing game where a common-pool resource, which is prone to failure due to overuse, is shared among several players. Each player has a fixed initial endowment and is faced with the task of investing in the common-pool resource without forcing it to fail. The return from the common-pool resource is subject to uncertainty and is perceived by the players in a prospect-theoretic manner. It is shown in [A.~R.~Hota, S.~Garg, S.~Sundaram, \textit{Fragility of the commons under prospect-theoretic risk attitudes}, Games and Economic Behavior \textbf{98} (2016) 135--164.] that, under some mild assumptions, a Fragile CPR Game admits a unique Nash equilibrium. In this article we investigate an extended version of a Fragile CPR Game, in which players are allowed to share multiple common-pool resources that are also prone to failure due to overuse. We refer to this game as a Fragile multi-CPR Game. Our main result states that, under some mild assumptions, a Fragile multi-CPR Game admits a Generalized Nash equilibrium. Moreover, we show that, when there are more players than common-pool resources, the set consisting of all Generalized Nash equilibria of a Fragile multi-CPR Game is of Lebesgue measure zero.
A deep perceptual metric for 3D point clouds
Venue: arXiv cs.CV 2021
Year: 2021

Point clouds are essential for storage and transmission of 3D content. As they can entail significant volumes of data, point cloud compression is crucial for practical usage. Recently, point cloud geometry compression approaches based on deep neural networks have been explored. In this paper, we evaluate the ability to predict perceptual quality of typical voxel-based loss functions employed to train these networks. We find that the commonly used focal loss and weighted binary cross entropy are poorly correlated with human perception. We thus propose a perceptual loss function for 3D point clouds which outperforms existing loss functions on the ICIP2020 subjective dataset. In addition, we propose a novel truncated distance field voxel grid representation and find that it leads to sparser latent spaces and loss functions that are more correlated with perceived visual quality compared to a binary representation. The source code is available at https://github.com/mauriceqch/2021_pc_perceptual_loss.
New Singly and Doubly Even Binary [72,36,12] Self-Dual Codes from $M_2(R)G$ -- Group Matrix Rings
Venue: arXiv cs.IT 2021
Year: 2021

In this work, we present a number of generator matrices of the form $[I_{2n} \ | \ τ_k(v)],$ where $I_{kn}$ is the $kn \times kn$ identity matrix, $v$ is an element in the group matrix ring $M_2(R)G$ and where $R$ is a finite commutative Frobenius ring and $G$ is a finite group of order 18. We employ these generator matrices and search for binary $[72,36,12]$ self-dual codes directly over the finite field $\mathbb{F}_2.$ As a result, we find 134 Type I and 1 Type II codes of this length, with parameters in their weight enumerators that were not known in the literature before. We tabulate all of our findings.