OpenRaiser/Intern-Atlas huggingface dataset:
Spatio-Temporal Neural Network for Fitting and Forecasting COVID-19
Venue: arXiv cs.LG 2021
Year: 2021

We established a Spatio-Temporal Neural Network, namely STNN, to forecast the spread of the coronavirus COVID-19 outbreak worldwide in 2020. The basic structure of STNN is similar to the Recurrent Neural Network (RNN) incorporating with not only temporal data but also spatial features. Two improved STNN architectures, namely the STNN with Augmented Spatial States (STNN-A) and the STNN with Input Gate (STNN-I), are proposed, which ensure more predictability and flexibility. STNN and its variants can be trained using Stochastic Gradient Descent (SGD) algorithm and its improved variants (e.g., Adam, AdaGrad and RMSProp). Our STNN models are compared with several classical epidemic prediction models, including the fully-connected neural network (BPNN), and the recurrent neural network (RNN), the classical curve fitting models, as well as the SEIR dynamical system model. Numerical simulations demonstrate that STNN models outperform many others by providing more accurate fitting and prediction, and by handling both spatial and temporal data.
DeepViT: Towards Deeper Vision Transformer
Venue: arXiv cs.CV 2021
Year: 2021

Vision transformers (ViTs) have been successfully applied in image classification tasks recently. In this paper, we show that, unlike convolution neural networks (CNNs)that can be improved by stacking more convolutional layers, the performance of ViTs saturate fast when scaled to be deeper. More specifically, we empirically observe that such scaling difficulty is caused by the attention collapse issue: as the transformer goes deeper, the attention maps gradually become similar and even much the same after certain layers. In other words, the feature maps tend to be identical in the top layers of deep ViT models. This fact demonstrates that in deeper layers of ViTs, the self-attention mechanism fails to learn effective concepts for representation learning and hinders the model from getting expected performance gain. Based on above observation, we propose a simple yet effective method, named Re-attention, to re-generate the attention maps to increase their diversity at different layers with negligible computation and memory cost. The pro-posed method makes it feasible to train deeper ViT models with consistent performance improvements via minor modification to existing ViT models. Notably, when training a deep ViT model with 32 transformer blocks, the Top-1 classification accuracy can be improved by 1.6% on ImageNet. Code is publicly available at https://github.com/zhoudaquan/dvit_repo.
Enhanced Principal Component Analysis under A Collaborative-Robust Framework
Venue: arXiv cs.LG 2021
Year: 2021

Principal component analysis (PCA) frequently suffers from the disturbance of outliers and thus a spectrum of robust extensions and variations of PCA have been developed. However, existing extensions of PCA treat all samples equally even those with large noise. In this paper, we first introduce a general collaborative-robust weight learning framework that combines weight learning and robust loss in a non-trivial way. More significantly, under the proposed framework, only a part of well-fitting samples are activated which indicates more importance during training, and others, whose errors are large, will not be ignored. In particular, the negative effects of inactivated samples are alleviated by the robust loss function. Then we furthermore develop an enhanced PCA which adopts a point-wise sigma-loss function that interpolates between L_2,1-norm and squared Frobenius-norm and meanwhile retains the rotational invariance property. Extensive experiments are conducted on occluded datasets from two aspects including reconstructed errors and clustering accuracy. The experimental results prove the superiority and effectiveness of our model.
Kokkos Kernels: Performance Portable Sparse/Dense Linear Algebra and Graph Kernels
Venue: arXiv cs.MS 2021
Year: 2021

As hardware architectures are evolving in the push towards exascale, developing Computational Science and Engineering (CSE) applications depend on performance portable approaches for sustainable software development. This paper describes one aspect of performance portability with respect to developing a portable library of kernels that serve the needs of several CSE applications and software frameworks. We describe Kokkos Kernels, a library of kernels for sparse linear algebra, dense linear algebra and graph kernels. We describe the design principles of such a library and demonstrate portable performance of the library using some selected kernels. Specifically, we demonstrate the performance of four sparse kernels, three dense batched kernels, two graph kernels and one team level algorithm.
On the Robustness of Monte Carlo Dropout Trained with Noisy Labels
Venue: arXiv cs.LG 2021
Year: 2021

The memorization effect of deep learning hinders its performance to effectively generalize on test set when learning with noisy labels. Prior study has discovered that epistemic uncertainty techniques are robust when trained with noisy labels compared with neural networks without uncertainty estimation. They obtain prolonged memorization effect and better generalization performance under the adversarial setting of noisy labels. Due to its superior performance amongst other selected epistemic uncertainty methods under noisy labels, we focus on Monte Carlo Dropout (MCDropout) and investigate why it is robust when trained with noisy labels. Through empirical studies on datasets MNIST, CIFAR-10, Animal-10n, we deep dive into three aspects of MCDropout under noisy label setting: 1. efficacy: understanding the learning behavior and test accuracy of MCDropout when training set contains artificially generated or naturally embedded label noise; 2. representation volatility: studying the responsiveness of neurons by examining the mean and standard deviation on each neuron's activation; 3. network sparsity: investigating the network support of MCDropout in comparison with deterministic neural networks. Our findings suggest that MCDropout further sparsifies and regularizes the deterministic neural networks and thus provides higher robustness against noisy labels.
Groovy Parallel Patterns: A Process oriented Parallelization Library
Venue: arXiv cs.DC 2021
Year: 2021

A novel parallel patterns library, Groovy Parallel Patterns, is presented which, from the outset, has been designed to exploit more general process parallelism than the usual data and task parallel architectures. The library executes on a standard Java Virtual Machine. The library provides a collection of processes that can be plugged together to form a variety of parallel architectures and is intrinsically its own DSL. A network of processes is guaranteed to be deadlock and livelock free and terminate correctly and this is proved by the use of formal methods. Error capture and a basic logging mechanism have been incorporated. The library enables effective refinement of solutions between process networks which can be checked also using formal methods. A library user is only required to create the required methods as pieces of sequential code, typically taken from extant sequential solutions, which can then be invoked by the processes as required. The utility of the library is demonstrated by several examples including; Monte Carlo Methods, Concordance, Jacobi solutions, N-body problems and Mandelbrot, which is implemented on both a multicore processor and a workstation cluster. The examples are analysed for speedup and efficiency, which show good and consistent performance improvement up to the number of available processor cores and workstations.
LaneAF: Robust Multi-Lane Detection with Affinity Fields
Venue: arXiv cs.CV 2021
Year: 2021

This study presents an approach to lane detection involving the prediction of binary segmentation masks and per-pixel affinity fields. These affinity fields, along with the binary masks, can then be used to cluster lane pixels horizontally and vertically into corresponding lane instances in a post-processing step. This clustering is achieved through a simple row-by-row decoding process with little overhead; such an approach allows LaneAF to detect a variable number of lanes without assuming a fixed or maximum number of lanes. Moreover, this form of clustering is more interpretable in comparison to previous visual clustering approaches, and can be analyzed to identify and correct sources of error. Qualitative and quantitative results obtained on popular lane detection datasets demonstrate the model's ability to detect and cluster lanes effectively and robustly. Our proposed approach sets a new state-of-the-art on the challenging CULane dataset and the recently introduced Unsupervised LLAMAS dataset.
Exemplars can Reciprocate Principal Components
Venue: arXiv cs.LG 2021
Year: 2021

This paper presents a clustering algorithm that is an extension of the Category Trees algorithm. Category Trees is a clustering method that creates tree structures that branch on category type and not feature. The development in this paper is to consider a secondary order of clustering that is not the category to which the data row belongs, but the tree, representing a single classifier, that it is eventually clustered with. Each tree branches to store subsets of other categories, but the rows in those subsets may also be related. This paper is therefore concerned with looking at that second level of clustering between the other category subsets, to try to determine if there is any consistency over it. It is argued that Principal Components may be a related and reciprocal type of structure, and there is an even bigger question about the relation between exemplars and principal components, in general. The theory is demonstrated using the Portugal Forest Fires dataset as a case study. The Category Trees are then combined with other Self-Organising algorithms from the author and it is suggested that they all belong to the same family type, which is an Entropy-style of classifier.
The quest for scaling BFT Consensus through Tree-Based Vote Aggregation
Venue: arXiv cs.DC 2021
Year: 2021

With the growing commercial interest in blockchain, permissioned implementations have received increasing attention. Unfortunately, existing BFT consensus protocols that are the backbone of permissioned blockchains, either scale poorly or offer limited throughput. Most of these algorithms require at least one process to receive and validate the votes from all other processes and then broadcast the result, which is inherently non-scalable. Some algorithms avoid this bottleneck by using aggregation trees to collect and validate votes. However, to the best of our knowledge, such algorithms offer limited throughput and degrade quickly in the presence of faults. In this paper we propose \thesystem, the first BFT communication abstraction that organizes participants in a tree to perform scalable vote aggregation and that, in faulty runs, is able to terminate the protocol within an optimal number of reconfigurations ($f+1$). We define precisely which aggregation trees allow for optimal reconfiguration and show that, unlike previous protocols, when using these configurations, \thesystem scales to large number of processes and outperforms HotStuff's throughput by up to 38x.
The IoT Exchange
Venue: arXiv cs.DC 2021
Year: 2021

The IoT ecosystem suffers from a variety of problems around security, identity, access control, data flow and data storage that introduce friction into interactions between various parties. In many respects, the situation is similar to the early days of the Internet, where, prior to the establishment of Internet Exchanges, routing between different BGP autonomous systems was often point to point. We propose a similar solution, the IoT Exchange, where IoT device owners can register their devices and offer data for sale or can upload data into the IoT services of any of the big hyperscale cloud platforms for further processing. The goal of the IoT Exchange is to break down the silos within which device wireless connectivity types and cloud provider IoT systems constrain users to operate. In addition, if the device owner needs to maintain the data close to the edge to reduce access latency, the MillenniumDB service running in an edge data center with minimal latency to the edge device, provides a database with a variety of schema engines (SQL, noSQL, etc). The IoT exchange uses decentralized identifiers for identity management and verifiable credentials for authorizing software updates and to control access to the devices, to avoid dependence on certificate authorities and other centralized identity and authorization management systems. In addition, verifiable credentials provide a way whereby privacy preserving processing can be applied to traffic between a device and an end data or control customer, if some risk of privacy compromise exists.
Performance Bounds for Neural Network Estimators: Applications in Fault Detection
Venue: arXiv cs.LG 2021
Year: 2021

We exploit recent results in quantifying the robustness of neural networks to input variations to construct and tune a model-based anomaly detector, where the data-driven estimator model is provided by an autoregressive neural network. In tuning, we specifically provide upper bounds on the rate of false alarms expected under normal operation. To accomplish this, we provide a theory extension to allow for the propagation of multiple confidence ellipsoids through a neural network. The ellipsoid that bounds the output of the neural network under the input variation informs the sensitivity - and thus the threshold tuning - of the detector. We demonstrate this approach on a linear and nonlinear dynamical system.
Combining Reward Information from Multiple Sources
Venue: arXiv cs.LG 2021
Year: 2021

Given two sources of evidence about a latent variable, one can combine the information from both by multiplying the likelihoods of each piece of evidence. However, when one or both of the observation models are misspecified, the distributions will conflict. We study this problem in the setting with two conflicting reward functions learned from different sources. In such a setting, we would like to retreat to a broader distribution over reward functions, in order to mitigate the effects of misspecification. We assume that an agent will maximize expected reward given this distribution over reward functions, and identify four desiderata for this setting. We propose a novel algorithm, Multitask Inverse Reward Design (MIRD), and compare it to a range of simple baselines. While all methods must trade off between conservatism and informativeness, through a combination of theory and empirical results on a toy environment, we find that MIRD and its variant MIRD-IF strike a good balance between the two.
Prediction of lung and colon cancer through analysis of histopathological images by utilizing Pre-trained CNN models with visualization of class activation and saliency maps
Venue: arXiv cs.CV 2021
Year: 2021

Colon and Lung cancer is one of the most perilous and dangerous ailments that individuals are enduring worldwide and has become a general medical problem. To lessen the risk of death, a legitimate and early finding is particularly required. In any case, it is a truly troublesome task that depends on the experience of histopathologists. If a histologist is under-prepared it may even hazard the life of a patient. As of late, deep learning has picked up energy, and it is being valued in the analysis of Medical Imaging. This paper intends to utilize and alter the current pre-trained CNN-based model to identify lung and colon cancer utilizing histopathological images with better augmentation techniques. In this paper, eight distinctive Pre-trained CNN models, VGG16, NASNetMobile, InceptionV3, InceptionResNetV2, ResNet50, Xception, MobileNet, and DenseNet169 are trained on LC25000 dataset. The model performances are assessed on precision, recall, f1score, accuracy, and auroc score. The results exhibit that all eight models accomplished noteworthy results ranging from 96% to 100% accuracy. Subsequently, GradCAM and SmoothGrad are also used to picture the attention images of Pre-trained CNN models classifying malignant and benign images.
3D Reconstruction and Alignment by Consumer RGB-D Sensors and Fiducial Planar Markers for Patient Positioning in Radiation Therapy
Venue: arXiv cs.CV 2021
Year: 2021

BACKGROUND AND OBJECTIVE: Patient positioning is a crucial step in radiation therapy, for which non-invasive methods have been developed based on surface reconstruction using optical 3D imaging. However, most solutions need expensive specialized hardware and a careful calibration procedure that must be repeated over time.This paper proposes a fast and cheap patient positioning method based on inexpensive consumer level RGB-D sensors. METHODS: The proposed method relies on a 3D reconstruction approach that fuses, in real-time, artificial and natural visual landmarks recorded from a hand-held RGB-D sensor. The video sequence is transformed into a set of keyframes with known poses, that are later refined to obtain a realistic 3D reconstruction of the patient. The use of artificial landmarks allows our method to automatically align the reconstruction to a reference one, without the need of calibrating the system with respect to the linear accelerator coordinate system. RESULTS:The experiments conducted show that our method obtains a median of 1 cm in translational error, and 1 degree of rotational error with respect to reference pose. Additionally, the proposed method shows as visual output overlayed poses (from the reference and the current scene) and an error map that can be used to correct the patient's current pose to match the reference pose. CONCLUSIONS: A novel approach to obtain 3D body reconstructions for patient positioning without requiring expensive hardware or dedicated graphic cards is proposed. The method can be used to align in real time the patient's current pose to a preview pose, which is a relevant step in radiation therapy.
Average Rate Analysis of RIS-aided Short Packet Communication in URLLC Systems
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, the average achievable rate of a re-configurable intelligent surface (RIS) aided factory automation is investigated in finite blocklength (FBL) regime. First, the composite channel containing the direct path plus the product of reflected paths through the RIS is characterized. Then, the distribution of the received signal-to-noise ratio (SNR) is matched to a Gamma random variable whose parameters depend on the total number of RIS elements as well as the channel pathloss. Next, by assuming FBL model, the achievable rate expression is identified and the corresponding average rate is elaborated based on the proposed SNR distribution. The phase error due to quantizing the phase shifts is considered in the simulation. The numerical results show that Monte Carlo simulations conform to the matched Gamma distribution for the received SNR for large number of RIS elements. In addition, the system reliability indicated by the tightness of the SNR distribution increases when RIS is leveraged particularly when only the reflected channel exists. This highlights the advantages of RIS-aided communications for ultra-reliable low-latency communications (URLLC) systems. The reduction of average achievable rate due to working in FBL regime with respect to Shannon capacity is also investigated as a function of total RIS elements.
Using an Epidemiological Model to Study the Spread of Misinformation during the Black Lives Matter Movement
Venue: arXiv cs.SI 2021
Year: 2021

The proliferation of social media platforms like Twitter has heightened the consequences of the spread of misinformation. To understand and model the spread of misinformation, in this paper, we leveraged the SEIZ (Susceptible, Exposed, Infected, Skeptics) epidemiological model to describe the underlying process that delineates the spread of misinformation on Twitter. Compared to the other epidemiological models, this model produces broader results because it includes the additional Skeptics (Z) compartment, wherein a user may be exposed to an item of misinformation but not engage in any reaction to it, and the additional Exposed (E) compartment, wherein the user may need some time before deciding to spread a misinformation item. We analyzed misinformation regarding the unrest in Washington, D.C. in the month of March 2020 which was propagated by the use of the #DCblackout hashtag by different users across the U.S. on Twitter. Our analysis shows that misinformation can be modeled using the concept of epidemiology. To the best of our knowledge, this research is the first to attempt to apply the SEIZ epidemiological model to the spread of a specific item of misinformation, which is a category distinct from that of rumor, and a hoax on online social media platforms. Applying a mathematical model can help to understand the trends and dynamics of the spread of misinformation on Twitter and ultimately help to develop techniques to quickly identify and control it.
Reward-Reinforced Reinforcement Learning for Multi-agent Systems
Venue: arXiv cs.MA 2021
Year: 2021

Reinforcement learning algorithms in multi-agent systems deliver highly resilient and adaptable solutions for common problems in telecommunications,aerospace, and industrial robotics. However, achieving an optimal global goal remains a persistent obstacle for collaborative multi-agent systems, where learning affects the behaviour of more than one agent. A number of nonlinear function approximation methods have been proposed for solving the Bellman equation, which describe a recursive format of an optimal policy. However, how to leverage the value distribution based on reinforcement learning, and how to improve the efficiency and efficacy of such systems remain a challenge. In this work, we developed a reward-reinforced generative adversarial network to represent the distribution of the value function, replacing the approximation of Bellman updates. We demonstrated our method is resilient and outperforms other conventional reinforcement learning methods. This method is also applied to a practical case study: maximising the number of user connections to autonomous airborne base stations in a mobile communication network. Our method maximises the data likelihood using a cost function under which agents have optimal learned behaviours. This reward-reinforced generative adversarial network can be used as ageneric framework for multi-agent learning at the system level
Multimodal Data Fusion for Power-On-and-Go Robotic Systems in Retail
Venue: arXiv cs.RO 2021
Year: 2021

Robotic systems for retail have gained a lot of attention due to the labor-intensive nature of such business environments. Many tasks have the potential to be automated via intelligent robotic systems that have manipulation capabilities. For example, empty shelves can be replenished, stray products can be picked up or new items can be delivered. However, many challenges make the realization of this vision a challenge. In particular, robots are still too expensive and do not work out of the box. In this paper, we discuss a work-in-progress approach for enabling power-on-and-go robots in retail environments through a combination of active, physical sensors and passive, artificial sensors. In particular, we use low-cost hardware sensors in conjunction with machine learning techniques in order to generate high-quality environmental information. More specifically, we present a setup in which a standard monocular camera and Bluetooth low-energy yield a reliable robot system that can immediately be used after placing a couple of sensors in the environment. The camera information is used to synthesize accurate 3D point clouds, whereas the BLE data is used to integrate the data into a complex map of the environment. The combination of active and passive sensing enables high-quality sensing capabilities at a fraction of the costs traditionally associated with such tasks.