OpenRaiser/Intern-Atlas huggingface dataset:
Topic Allocation Method on Edge Servers for Latency-sensitive Notification Service
Venue: arXiv cs.NI 2021
Year: 2021

The importance of real-time notification has been growing for social services and Intelligent Transporting System (ITS). As an advanced version of Pub/Sub systems, publish-process-subscribe systems, where published messages are spooled and processed on edge servers, have been proposed to achieve data-driven intelligent notifications. In this paper, we present a system that allows a topic to be managed on multiple edge servers so that messages are processed near the publishers, even when publishers are spread over a wide area. Duplicating messages on geographically distributed servers could enable immediate notification to neighboring subscribers. However, the duplicated message spool may cause exhaustion of resources. We prepare a formal model of our publish-process-subscribe system and formulate the topic allocation as an optimization problem under the resource constraints of edge servers. As the optimization problem is NP-hard, we propose heuristics leveraging the locality and the pub/sub relationships observed between clients to use the edge server resources efficiently. Our performance evaluation shows that our method reduces the delay to deliver notifications and the effectiveness of the strategy exploiting the relationships between clients.
Forward and Backward Bellman equations improve the efficiency of EM algorithm for DEC-POMDP
Venue: arXiv cs.LG 2021
Year: 2021

Decentralized partially observable Markov decision process (DEC-POMDP) models sequential decision making problems by a team of agents. Since the planning of DEC-POMDP can be interpreted as the maximum likelihood estimation for the latent variable model, DEC-POMDP can be solved by the EM algorithm. However, in EM for DEC-POMDP, the forward--backward algorithm needs to be calculated up to the infinite horizon, which impairs the computational efficiency. In this paper, we propose the Bellman EM algorithm (BEM) and the modified Bellman EM algorithm (MBEM) by introducing the forward and backward Bellman equations into EM. BEM can be more efficient than EM because BEM calculates the forward and backward Bellman equations instead of the forward--backward algorithm up to the infinite horizon. However, BEM cannot always be more efficient than EM when the size of problems is large because BEM calculates an inverse matrix. We circumvent this shortcoming in MBEM by calculating the forward and backward Bellman equations without the inverse matrix. Our numerical experiments demonstrate that the convergence of MBEM is faster than that of EM.
Tools and Algorithms for SoC Communication Traces
Venue: arXiv cs.DC 2021
Year: 2021

In this paper, we study seven well-known trace analysis techniques both from the hardware and software domain and discuss their performance on communication-centric system-on-chip (SoC) traces. SoC traces are usually huge in size and concurrent in nature, therefore mining SoC traces poses additional challenges. We provide a hands-on discussion of the selected tools/algorithms in terms of the input, output, and analysis methods they employ. Hardware traces also varies in nature when observed in different level, this work can help developers/academicians to pick up the right techniques for their work. We take advantage of a synthetic trace generator to find the interestingness of the mined outcomes for each tool as well as we work with a realistic GEM5 set up to find the performance of these tools on more realistic SoC traces. Comprehensive analysis of the tool's performance and a benchmark trace dataset are also presented.
LSDAT: Low-Rank and Sparse Decomposition for Decision-based Adversarial Attack
Venue: arXiv cs.CV 2021
Year: 2021

We propose LSDAT, an image-agnostic decision-based black-box attack that exploits low-rank and sparse decomposition (LSD) to dramatically reduce the number of queries and achieve superior fooling rates compared to the state-of-the-art decision-based methods under given imperceptibility constraints. LSDAT crafts perturbations in the low-dimensional subspace formed by the sparse component of the input sample and that of an adversarial sample to obtain query-efficiency. The specific perturbation of interest is obtained by traversing the path between the input and adversarial sparse components. It is set forth that the proposed sparse perturbation is the most aligned sparse perturbation with the shortest path from the input sample to the decision boundary for some initial adversarial sample (the best sparse approximation of shortest path, likely to fool the model). Theoretical analyses are provided to justify the functionality of LSDAT. Unlike other dimensionality reduction based techniques aimed at improving query efficiency (e.g, ones based on FFT), LSD works directly in the image pixel domain to guarantee that non-$\ell_2$ constraints, such as sparsity, are satisfied. LSD offers better control over the number of queries and provides computational efficiency as it performs sparse decomposition of the input and adversarial images only once to generate all queries. We demonstrate $\ell_0$, $\ell_2$ and $\ell_\infty$ bounded attacks with LSDAT to evince its efficiency compared to baseline decision-based attacks in diverse low-query budget scenarios as outlined in the experiments.
GNNerator: A Hardware/Software Framework for Accelerating Graph Neural Networks
Venue: arXiv cs.AR 2021
Year: 2021

Graph Neural Networks (GNNs) use a fully-connected layer to extract features from the nodes of a graph and aggregate these features using message passing between nodes, combining two distinct computational patterns: dense, regular computations and sparse, irregular computations. To address this challenge, we propose GNNerator, an accelerator with heterogeneous compute engines optimized for these two patterns. Further, GNNerator implements feature-blocking, a novel GNN dataflow that beneficially trades off irregular memory accesses during aggregation for regular memory accesses during feature extraction. We show GNNerator achieves speedups of 5.7-37x over an NVIDIA RTX 2080-Ti, and 2.3x-3.8x over HyGCN, a state-of-the-art GNN accelerator.
Joint Parameter Discovery and Generative Modeling of Dynamic Systems
Venue: arXiv cs.LG 2021
Year: 2021

Given an unknown dynamic system such as a coupled harmonic oscillator with $n$ springs and point masses. We are often interested in gaining insights into its physical parameters, i.e. stiffnesses and masses, by observing trajectories of motion. How do we achieve this from video frames or time-series data and without the knowledge of the dynamics model? We present a neural framework for estimating physical parameters in a manner consistent with the underlying physics. The neural framework uses a deep latent variable model to disentangle the system physical parameters from canonical coordinate observations. It then returns a Hamiltonian parameterization that generalizes well with respect to the discovered physical parameters. We tested our framework with simple harmonic oscillators, $n=1$, and noisy observations and show that it discovers the underlying system parameters and generalizes well with respect to these discovered parameters. Our model also extrapolates the dynamics of the system beyond the training interval and outperforms a non-physically constrained baseline model. Our source code and datasets can be found at this URL: https://github.com/gbarber94/ConSciNet.
Predicting Drug-Drug Interactions from Heterogeneous Data: An Embedding Approach
Venue: arXiv cs.LG 2021
Year: 2021

Predicting and discovering drug-drug interactions (DDIs) using machine learning has been studied extensively. However, most of the approaches have focused on text data or textual representation of the drug structures. We present the first work that uses multiple data sources such as drug structure images, drug structure string representation and relational representation of drug relationships as the input. To this effect, we exploit the recent advances in deep networks to integrate these varied sources of inputs in predicting DDIs. Our empirical evaluation against several state-of-the-art methods using standalone different data types for drugs clearly demonstrate the efficacy of combining heterogeneous data in predicting DDIs.
BERTSurv: BERT-Based Survival Models for Predicting Outcomes of Trauma Patients
Venue: arXiv cs.AI 2021
Year: 2021

Survival analysis is a technique to predict the times of specific outcomes, and is widely used in predicting the outcomes for intensive care unit (ICU) trauma patients. Recently, deep learning models have drawn increasing attention in healthcare. However, there is a lack of deep learning methods that can model the relationship between measurements, clinical notes and mortality outcomes. In this paper we introduce BERTSurv, a deep learning survival framework which applies Bidirectional Encoder Representations from Transformers (BERT) as a language representation model on unstructured clinical notes, for mortality prediction and survival analysis. We also incorporate clinical measurements in BERTSurv. With binary cross-entropy (BCE) loss, BERTSurv can predict mortality as a binary outcome (mortality prediction). With partial log-likelihood (PLL) loss, BERTSurv predicts the probability of mortality as a time-to-event outcome (survival analysis). We apply BERTSurv on Medical Information Mart for Intensive Care III (MIMIC III) trauma patient data. For mortality prediction, BERTSurv obtained an area under the curve of receiver operating characteristic curve (AUC-ROC) of 0.86, which is an improvement of 3.6% over baseline of multilayer perceptron (MLP) without notes. For survival analysis, BERTSurv achieved a concordance index (C-index) of 0.7. In addition, visualizations of BERT's attention heads help to extract patterns in clinical notes and improve model interpretability by showing how the model assigns weights to different inputs.
Probabilistic Multi-View Fusion of Active Stereo Depth Maps for Robotic Bin-Picking
Venue: arXiv cs.RO 2021
Year: 2021

The reliable fusion of depth maps from multiple viewpoints has become an important problem in many 3D reconstruction pipelines. In this work, we investigate its impact on robotic bin-picking tasks such as 6D object pose estimation. The performance of object pose estimation relies heavily on the quality of depth data. However, due to the prevalence of shiny surfaces and cluttered scenes, industrial grade depth cameras often fail to sense depth or generate unreliable measurements from a single viewpoint. To this end, we propose a novel probabilistic framework for scene reconstruction in robotic bin-picking. Based on active stereo camera data, we first explicitly estimate the uncertainty of depth measurements for mitigating the adverse effects of both noise and outliers. The uncertainty estimates are then incorporated into a probabilistic model for incrementally updating the scene. To extensively evaluate the traditional fusion approach alongside our own approach, we will release a novel representative dataset with multiple views for each bin and curated parts. Over the entire dataset, we demonstrate that our framework outperforms a traditional fusion approach by a 12.8% reduction in reconstruction error, and 6.1% improvement in detection rate. The dataset will be available at https://www.trailab.utias.utoronto.ca/robi.
An Exploratory Study of Project Activity Changepoints in Open Source Software Evolution
Venue: arXiv cs.SE 2021
Year: 2021

To explore the prevalence of abrupt changes (changepoints) in open source project activity, we assembled a dataset of 8,919 projects from the World of Code. Projects were selected based on age, number of commits, and number of authors. Using the nonparametric PELT algorithm, we identified changepoints in project activity time series, finding that more than 90% of projects had between one and six changepoints. Increases and decreases in project activity occurred with roughly equal frequency. While most changes are relatively small, on the order of a few authors or few dozen commits per month, there were long tails of much larger project activity changes. In future work, we plan to focus on larger changes to search for common open source lifecycle patterns as well as common responses to external events.
Relational Operations in FOLE
Venue: arXiv cs.DB 2021
Year: 2021

This paper discusses relational operations in the first-order logical environment {FOLE}. Here we demonstrate how FOLE expresses the relational operations of database theory in a clear and implementable representation. An analysis of the representation of database tables/relations in FOLE reveals a principled way to express the relational operations. This representation is expressed in terms of a distinction between basic components versus composite relational operations. The 9 basic components fall into three categories: reflection (2), Booleans or basic operations (3), and adjoint flow (4). Adjoint flow is given for signatures (2) and for type domains (2), which are then combined into full adjoint flow. The basic components are used to express various composite operations, where we illustrate each of these with a flowchart. Implementation of the composite operations is then expressed in an input/output table containing four parts: constraint, construction, input, and output. We explain how limits and colimits are constructed from diagrams of tables, and then classify composite relational operations into three categories: limit-like, colimit-like and unorthodox.
AI Specialization for Pathways of Economic Diversification
Venue: arXiv cs.CY 2021
Year: 2021

The growth in AI is rapidly transforming the structure of economic production. However, very little is known about how within-AI specialization may relate to broad-based economic diversification. This paper provides a data-driven framework to integrate the interconnection between AI-based specialization with goods and services export specialization to help design future comparative advantage based on the inherent capabilities of nations. Using detailed data on private investment in AI and export specialization for more than 80 countries, we propose a systematic framework to help identify the connection from AI to goods and service sector specialization. The results are instructive for nations that aim to harness AI specialization to help guide sources of future competitive advantage. The operational framework could help inform the public and private sector to uncover connections with nearby areas of specialization.
A first step towards automated species recognition from camera trap images of mammals using AI in a European temperate forest
Venue: arXiv cs.CV 2021
Year: 2021

Camera traps are used worldwide to monitor wildlife. Despite the increasing availability of Deep Learning (DL) models, the effective usage of this technology to support wildlife monitoring is limited. This is mainly due to the complexity of DL technology and high computing requirements. This paper presents the implementation of the light-weight and state-of-the-art YOLOv5 architecture for automated labeling of camera trap images of mammals in the Bialowieza Forest (BF), Poland. The camera trapping data were organized and harmonized using TRAPPER software, an open source application for managing large-scale wildlife monitoring projects. The proposed image recognition pipeline achieved an average accuracy of 85% F1-score in the identification of the 12 most commonly occurring medium-size and large mammal species in BF using a limited set of training and testing data (a total 2659 images with animals). Based on the preliminary results, we concluded that the YOLOv5 object detection and classification model is a promising light-weight DL solution after the adoption of transfer learning technique. It can be efficiently plugged in via an API into existing web-based camera trapping data processing platforms such as e.g. TRAPPER system. Since TRAPPER is already used to manage and classify (manually) camera trapping datasets by many research groups in Europe, the implementation of AI-based automated species classification may significantly speed up the data processing workflow and thus better support data-driven wildlife monitoring and conservation. Moreover, YOLOv5 developers perform better performance on edge devices which may open a new chapter in animal population monitoring in real time directly from camera trap devices.
Encrypted Value Iteration and Temporal Difference Learning over Leveled Homomorphic Encryption
Venue: arXiv cs.CR 2021
Year: 2021

We consider an architecture of confidential cloud-based control synthesis based on Homomorphic Encryption (HE). Our study is motivated by the recent surge of data-driven control such as deep reinforcement learning, whose heavy computational requirements often necessitate an outsourcing to the third party server. To achieve more flexibility than Partially Homomorphic Encryption (PHE) and less computational overhead than Fully Homomorphic Encryption (FHE), we consider a Reinforcement Learning (RL) architecture over Leveled Homomorphic Encryption (LHE). We first show that the impact of the encryption noise under the Cheon-Kim-Kim-Song (CKKS) encryption scheme on the convergence of the model-based tabular Value Iteration (VI) can be analytically bounded. We also consider secure implementations of TD(0), SARSA(0) and Z-learning algorithms over the CKKS scheme, where we numerically demonstrate that the effects of the encryption noise on these algorithms are also minimal.
3DMNDT:3D multi-view registration method based on the normal distributions transform
Venue: arXiv cs.CV 2021
Year: 2021

The normal distributions transform (NDT) is an effective paradigm for the point set registration. This method is originally designed for pair-wise registration and it will suffer from great challenges when applied to multi-view registration. Under the NDT framework, this paper proposes a novel multi-view registration method, named 3D multi-view registration based on the normal distributions transform (3DMNDT), which integrates the K-means clustering and Lie algebra solver to achieve multi-view registration. More specifically, the multi-view registration is cast into the problem of maximum likelihood estimation. Then, the K-means algorithm is utilized to divide all data points into different clusters, where a normal distribution is computed to locally models the probability of measuring a data point in each cluster. Subsequently, the registration problem is formulated by the NDT-based likelihood function. To maximize this likelihood function, the Lie algebra solver is developed to sequentially optimize each rigid transformation. The proposed method alternately implements data point clustering, NDT computing, and likelihood maximization until desired registration results are obtained. Experimental results tested on benchmark data sets illustrate that the proposed method can achieve state-of-the-art performance for multi-view registration.
An Efficient Calibration Method for Triaxial Gyroscope
Venue: arXiv cs.RO 2021
Year: 2021

This paper presents an efficient servomotor-aided calibration method for the triaxial gyroscope. The entire calibration process only requires approximately one minute, and does not require high-precision equipment. This method is based on the idea that the measurement of the gyroscope should be equal to the rotation speed of the servomotor. A six-observation experimental design is proposed to minimize the maximum variance of the estimated scale factors and biases. In addition, a fast converging recursive linear least square estimation method is presented to reduce computational complexity. The simulation results reflect the robustness of the calibration method under normal and extreme conditions. We experimentally demonstrate the feasibility of the proposed method on a robot arm, and implement the method on a microcontroller. We verify the calibration results of the proposed method by comparing with a traditional turntable approach, and the experiment indicates that the results of these two methods are comparable. By comparing the calibrated low-cost gyroscope reading with the reading from a high-precision gyroscope, we can conclude that our method significantly increases the gyroscope's accuracy.
A Novel Upsampling and Context Convolution for Image Semantic Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Semantic segmentation, which refers to pixel-wise classification of an image, is a fundamental topic in computer vision owing to its growing importance in robot vision and autonomous driving industries. It provides rich information about objects in the scene such as object boundary, category, and location. Recent methods for semantic segmentation often employ an encoder-decoder structure using deep convolutional neural networks. The encoder part extracts feature of the image using several filters and pooling operations, whereas the decoder part gradually recovers the low-resolution feature maps of the encoder into a full input resolution feature map for pixel-wise prediction. However, the encoder-decoder variants for semantic segmentation suffer from severe spatial information loss, caused by pooling operations or convolutions with stride, and does not consider the context in the scene. In this paper, we propose a dense upsampling convolution method based on guided filtering to effectively preserve the spatial information of the image in the network. We further propose a novel local context convolution method that not only covers larger-scale objects in the scene but covers them densely for precise object boundary delineation. Theoretical analyses and experimental results on several benchmark datasets verify the effectiveness of our method. Qualitatively, our approach delineates object boundaries at a level of accuracy that is beyond the current excellent methods. Quantitatively, we report a new record of 82.86% and 81.62% of pixel accuracy on ADE20K and Pascal-Context benchmark datasets, respectively. In comparison with the state-of-the-art methods, the proposed method offers promising improvements.
Multi-Axis Force Sensing in Robotic Minimally Invasive Surgery With No Instrument Modification
Venue: arXiv cs.RO 2021
Year: 2021

This paper presents a novel multi-axis force-sensing approach in robotic minimally invasive surgery with no modification to the surgical instrument. Thus, it is adaptable to different surgical instruments. A novel 6-axis optical force sensor, with local signal conditioning and digital electronics, was mounted onto the proximal shaft of a da Vinci EndoWrist instrument. A new cannula design comprising an inner tube and an outer tube was proposed. The inner tube is attached to the cannula interface to the robot base through a compliant leaf spring with adjustable stiffness. It allows bending of the instrument shaft due to the tip forces. The outer tube mechanically filters out the body forces from affecting the instrument bending behavior. A mathematical model of the sensing principle was developed and used for model-based calibration. A data-driven calibration based on a shallow neural network architecture comprising a single 5-nodes hidden layer and a 5x1 output layer is discussed. Extensive testing was conducted to validate that the sensor can successfully measure the lateral forces and moments and the axial torque applied to the instruments distal end within the desired resolution, accuracy, and range requirements.