OpenRaiser/Intern-Atlas huggingface dataset:
Detection of E-scooter Riders in Naturalistic Scenes
Venue: arXiv cs.CV 2021
Year: 2021

E-scooters have become ubiquitous vehicles in major cities around the world.The numbers of e-scooters keep escalating, increasing their interactions with other cars on the road. Normal behavior of an e-scooter rider varies enormously to other vulnerable road users. This situation creates new challenges for vehicle active safety systems and automated driving functionalities, which require the detection of e-scooter riders as the first step. To our best knowledge, there is no existing computer vision model to detect these e-scooter riders. This paper presents a novel vision-based system to differentiate between e-scooter riders and regular pedestrians and a benchmark data set for e-scooter riders in natural scenes. We propose an efficient pipeline built over two existing state-of-the-art convolutional neural networks (CNN), You Only Look Once (YOLOv3) and MobileNetV2. We fine-tune MobileNetV2 over our dataset and train the model to classify e-scooter riders and pedestrians. We obtain a recall of around 0.75 on our raw test sample to classify e-scooter riders with the whole pipeline. Moreover, the classification accuracy of trained MobileNetV2 on top of YOLOv3 is over 91%, with precision and recall over 0.9.
PAPooling: Graph-based Position Adaptive Aggregation of Local Geometry in Point Clouds
Venue: arXiv cs.CV 2021
Year: 2021

Fine-grained geometry, captured by aggregation of point features in local regions, is crucial for object recognition and scene understanding in point clouds. Nevertheless, existing preeminent point cloud backbones usually incorporate max/average pooling for local feature aggregation, which largely ignores points' positional distribution, leading to inadequate assembling of fine-grained structures. To mitigate this bottleneck, we present an efficient alternative to max pooling, Position Adaptive Pooling (PAPooling), that explicitly models spatial relations among local points using a novel graph representation, and aggregates features in a position adaptive manner, enabling position-sensitive representation of aggregated features. Specifically, PAPooling consists of two key steps, Graph Construction and Feature Aggregation, respectively in charge of constructing a graph with edges linking the center point with every neighboring point in a local region to map their relative positional information to channel-wise attentive weights, and adaptively aggregating local point features based on the generated weights through Graph Convolution Network (GCN). PAPooling is simple yet effective, and flexible enough to be ready to use for different popular backbones like PointNet++ and DGCNN, as a plug-andplay operator. Extensive experiments on various tasks ranging from 3D shape classification, part segmentation to scene segmentation well demonstrate that PAPooling can significantly improve predictive accuracy, while with minimal extra computational overhead. Code will be released.
Rate-Splitting Multiple Access for Satellite-Terrestrial Integrated Networks:Benefits of Coordination and Cooperation
Venue: arXiv cs.IT 2021
Year: 2021

This work studies the joint beamforming design problem of achieving max-min rate fairness in a satellite-terrestrial integrated network (STIN) where the satellite provides wide coverage to multibeam multicast satellite users (SUs), and the terrestrial base station (BS) serves multiple cellular users (CUs) in a densely populated area. Both the satellite and BS operate in the same frequency band. Since rate-splitting multiple access (RSMA) has recently emerged as a promising strategy for non-orthogonal transmission and robust interference management in multi-antenna wireless networks, we present two RSMA-based STIN schemes, namely the coordinated scheme relying on channel state information (CSI) sharing and the cooperative scheme relying on CSI and data sharing. Our objective is to maximize the minimum fairness rate amongst all SUs and CUs subject to transmit power constraints at the satellite and the BS. A joint beamforming algorithm is proposed to reformulate the original problem into an approximately equivalent convex one which can be iteratively solved. Moreover, an expectation-based robust joint beamforming algorithm is proposed against the practical environment when satellite channel phase uncertainties are considered. Simulation results demonstrate the effectiveness and robustness of our proposed RSMA schemes for STIN, and exhibit significant performance gains compared with various traditional transmission strategies.
A Grounded Well-being Conversational Agent with Multiple Interaction Modes: Preliminary Results
Venue: arXiv cs.CL 2021
Year: 2021

Technologies for enhancing well-being, healthcare vigilance and monitoring are on the rise. However, despite patient interest, such technologies suffer from low adoption. One hypothesis for this limited adoption is loss of human interaction that is central to doctor-patient encounters. In this paper we seek to address this limitation via a conversational agent that adopts one aspect of in-person doctor-patient interactions: A human avatar to facilitate medical grounded question answering. This is akin to the in-person scenario where the doctor may point to the human body or the patient may point to their own body to express their conditions. Additionally, our agent has multiple interaction modes, that may give more options for the patient to use the agent, not just for medical question answering, but also to engage in conversations about general topics and current events. Both the avatar, and the multiple interaction modes could help improve adherence. We present a high level overview of the design of our agent, Marie Bot Wellbeing. We also report implementation details of our early prototype , and present preliminary results.
EffCNet: An Efficient CondenseNet for Image Classification on NXP BlueBox
Venue: arXiv cs.CV 2021
Year: 2021

Intelligent edge devices with built-in processors vary widely in terms of capability and physical form to perform advanced Computer Vision (CV) tasks such as image classification and object detection, for example. With constant advances in the field of autonomous cars and UAVs, embedded systems and mobile devices, there has been an ever-growing demand for extremely efficient Artificial Neural Networks (ANN) for real-time inference on these smart edge devices with constrained computational resources. With unreliable network connections in remote regions and an added complexity of data transmission, it is of an utmost importance to capture and process data locally instead of sending the data to cloud servers for remote processing. Edge devices on the other hand, offer limited processing power due to their inexpensive hardware, and limited cooling and computational resources. In this paper, we propose a novel deep convolutional neural network architecture called EffCNet which is an improved and an efficient version of CondenseNet Convolutional Neural Network (CNN) for edge devices utilizing self-querying data augmentation and depthwise separable convolutional strategies to improve real-time inference performance as well as reduce the final trained model size, trainable parameters, and Floating-Point Operations (FLOPs) of EffCNet CNN. Furthermore, extensive supervised image classification analyses are conducted on two benchmarking datasets: CIFAR-10 and CIFAR-100, to verify real-time inference performance of our proposed CNN. Finally, we deploy these trained weights on NXP BlueBox which is an intelligent edge development platform designed for self-driving vehicles and UAVs, and conclusions will be extrapolated accordingly.
Virtualizing Intermittent Computing
Venue: arXiv cs.PL 2021
Year: 2021

Intermittent computing requires custom programming models to ensure the correct execution of applications despite power failures. However, existing programming models lead to programs that are hardware-dependent and not reusable. This paper aims at virtualizing intermittent computing to remedy these problems. We introduce PureVM, a virtual machine that abstracts a transiently powered computer, and PureLANG, a continuation-passing-style programming language to develop programs that run on PureVM. This virtualization, for the first time, paves the way for portable and reusable transiently-powered applications.
Optimal Multi-Robot Motion Planning via Parabolic Relaxation
Venue: arXiv cs.RO 2021
Year: 2021

Multi-robot systems offer enhanced capability over their monolithic counterparts, but they come at a cost of increased complexity in coordination. To reduce complexity and to make the problem tractable, multi-robot motion planning (MRMP) methods in the literature adopt de-coupled approaches that sacrifice either optimality or dynamic feasibility. In this paper, we present a convexification method, namely "parabolic relaxation", to generate optimal and dynamically feasible trajectories for MRMP in the coupled joint-space of all robots. We leverage upon the proposed relaxation to tackle the problem complexity and to attain computational tractability for planning over one hundred robots in extremely clustered environments. We take a multi-stage optimization approach that consists of i) mathematically formulating MRMP as a non-convex optimization, ii) lifting the problem into a higher dimensional space, iii) convexifying the problem through the proposed computationally efficient parabolic relaxation, and iv) penalizing with iterative search to ensure feasibility and recovery of feasible and near-optimal solutions to the original problem. Our numerical experiments demonstrate that the proposed approach is capable of generating optimal and dynamically feasible trajectories for challenging motion planning problems with higher success rate than the state-of-the-art, yet remain computationally tractable for over one hundred robots in a highly dense environment.
Data Augmentation For Medical MR Image Using Generative Adversarial Networks
Venue: arXiv cs.CV 2021
Year: 2021

Computer-assisted diagnosis (CAD) based on deep learning has become a crucial diagnostic technology in the medical industry, effectively improving diagnosis accuracy. However, the scarcity of brain tumor Magnetic Resonance (MR) image datasets causes the low performance of deep learning algorithms. The distribution of transformed images generated by traditional data augmentation (DA) intrinsically resembles the original ones, resulting in a limited performance in terms of generalization ability. This work improves Progressive Growing of GANs with a structural similarity loss function (PGGAN-SSIM) to solve image blurriness problems and model collapse. We also explore other GAN-based data augmentation to demonstrate the effectiveness of the proposed model. Our results show that PGGAN-SSIM successfully generates 256x256 realistic brain tumor MR images which fill the real image distribution uncovered by the original dataset. Furthermore, PGGAN-SSIM exceeds other GAN-based methods, achieving promising performance improvement in Frechet Inception Distance (FID) and Multi-scale Structural Similarity (MS-SSIM).
How Can Creativity Occur in Multi-Agent Systems?
Venue: arXiv cs.MA 2021
Year: 2021

Complex systems show how surprising and beautiful phenomena can emerge from structures or agents following simple rules. With the recent success of deep reinforcement learning (RL), a natural path forward would be to use the capabilities of multiple deep RL agents to produce emergent behavior of greater benefit and sophistication. In general, this has proved to be an unreliable strategy without significant computation due to the difficulties inherent in multi-agent RL training. In this paper, we propose some criteria for creativity in multi-agent RL. We hope this proposal will give artists applying multi-agent RL a starting point, and provide a catalyst for further investigation guided by philosophical discussion.
Anomaly-Aware Semantic Segmentation by Leveraging Synthetic-Unknown Data
Venue: arXiv cs.CV 2021
Year: 2021

Anomaly awareness is an essential capability for safety-critical applications such as autonomous driving. While recent progress of robotics and computer vision has enabled anomaly detection for image classification, anomaly detection on semantic segmentation is less explored. Conventional anomaly-aware systems assuming other existing classes as out-of-distribution (pseudo-unknown) classes for training a model will result in two drawbacks. (1) Unknown classes, which applications need to cope with, might not actually exist during training time. (2) Model performance would strongly rely on the class selection. Observing this, we propose a novel Synthetic-Unknown Data Generation, intending to tackle the anomaly-aware semantic segmentation task. We design a new Masked Gradient Update (MGU) module to generate auxiliary data along the boundary of in-distribution data points. In addition, we modify the traditional cross-entropy loss to emphasize the border data points. We reach the state-of-the-art performance on two anomaly segmentation datasets. Ablation studies also demonstrate the effectiveness of proposed modules.
A Causal Approach for Unfair Edge Prioritization and Discrimination Removal
Venue: arXiv cs.LG 2021
Year: 2021

In budget-constrained settings aimed at mitigating unfairness, like law enforcement, it is essential to prioritize the sources of unfairness before taking measures to mitigate them in the real world. Unlike previous works, which only serve as a caution against possible discrimination and de-bias data after data generation, this work provides a toolkit to mitigate unfairness during data generation, given by the Unfair Edge Prioritization algorithm, in addition to de-biasing data after generation, given by the Discrimination Removal algorithm. We assume that a non-parametric Markovian causal model representative of the data generation procedure is given. The edges emanating from the sensitive nodes in the causal graph, such as race, are assumed to be the sources of unfairness. We first quantify Edge Flow in any edge X -> Y, which is the belief of observing a specific value of Y due to the influence of a specific value of X along X -> Y. We then quantify Edge Unfairness by formulating a non-parametric model in terms of edge flows. We then prove that cumulative unfairness towards sensitive groups in a decision, like race in a bail decision, is non-existent when edge unfairness is absent. We prove this result for the non-trivial non-parametric model setting when the cumulative unfairness cannot be expressed in terms of edge unfairness. We then measure the Potential to mitigate the Cumulative Unfairness when edge unfairness is decreased. Based on these measurements, we propose the Unfair Edge Prioritization algorithm that can then be used by policymakers. We also propose the Discrimination Removal Procedure that de-biases a data distribution by eliminating optimization constraints that grow exponentially in the number of sensitive attributes and values taken by them. Extensive experiments validate the theorem and specifications used for quantifying the above measures.
Semi-supervised Domain Adaptation via Sample-to-Sample Self-Distillation
Venue: arXiv cs.CV 2021
Year: 2021

Semi-supervised domain adaptation (SSDA) is to adapt a learner to a new domain with only a small set of labeled samples when a large labeled dataset is given on a source domain. In this paper, we propose a pair-based SSDA method that adapts a model to the target domain using self-distillation with sample pairs. Each sample pair is composed of a teacher sample from a labeled dataset (i.e., source or labeled target) and its student sample from an unlabeled dataset (i.e., unlabeled target). Our method generates an assistant feature by transferring an intermediate style between the teacher and the student, and then train the model by minimizing the output discrepancy between the student and the assistant. During training, the assistants gradually bridge the discrepancy between the two domains, thus allowing the student to easily learn from the teacher. Experimental evaluation on standard benchmarks shows that our method effectively minimizes both the inter-domain and intra-domain discrepancies, thus achieving significant improvements over recent methods.
Exploring technologies to better link physical evidence and digital information for disaster victim identification
Venue: arXiv cs.HC 2021
Year: 2021

Disaster victim identification (DVI) entails a protracted process of evidence collection and data matching to reconcile physical remains with victim identity. Technology is critical to DVI by enabling the linkage of physical evidence to information. However, labelling physical remains and collecting data at the scene are dominated by low-technology paper-based practices. We ask, how can technology help us tag and track the victims of disaster? Our response has two parts. First, we conducted a human-computer interaction led investigation into the systematic factors impacting DVI tagging and tracking processes. Through interviews with Australian DVI practitioners, we explored how technologies to improve linkage might fit with prevailing work practices and preferences; practical and social considerations; and existing systems and processes. Using insights from these interviews and relevant literature, we identified four critical themes: protocols and training; stress and stressors; the plurality of information capture and management systems; and practicalities and constraints. Second, we applied the themes identified in the first part of the investigation to critically review technologies that could support DVI practitioners by enhancing DVI processes that link physical evidence to information. This resulted in an overview of candidate technologies matched with consideration of their key attributes. This study recognises the importance of considering human factors that can affect technology adoption into existing practices. We provide a searchable table (Supplementary Information) that relates technologies to the key attributes relevant to DVI practice, for the reader to apply to their own context. While this research directly contributes to DVI, it also has applications to other domains in which a physical/digital linkage is required, particularly within high-stress environments.
IB-MVS: An Iterative Algorithm for Deep Multi-View Stereo based on Binary Decisions
Venue: arXiv cs.CV 2021
Year: 2021

We present a novel deep-learning-based method for Multi-View Stereo. Our method estimates high resolution and highly precise depth maps iteratively, by traversing the continuous space of feasible depth values at each pixel in a binary decision fashion. The decision process leverages a deep-network architecture: this computes a pixelwise binary mask that establishes whether each pixel actual depth is in front or behind its current iteration individual depth hypothesis. Moreover, in order to handle occluded regions, at each iteration the results from different source images are fused using pixelwise weights estimated by a second network. Thanks to the adopted binary decision strategy, which permits an efficient exploration of the depth space, our method can handle high resolution images without trading resolution and precision. This sets it apart from most alternative learning-based Multi-View Stereo methods, where the explicit discretization of the depth space requires the processing of large cost volumes. We compare our method with state-of-the-art Multi-View Stereo methods on the DTU, Tanks and Temples and the challenging ETH3D benchmarks and show competitive results.
Robust Federated Learning for execution time-based device model identification under label-flipping attack
Venue: arXiv cs.CR 2021
Year: 2021

The computing device deployment explosion experienced in recent years, motivated by the advances of technologies such as Internet-of-Things (IoT) and 5G, has led to a global scenario with increasing cybersecurity risks and threats. Among them, device spoofing and impersonation cyberattacks stand out due to their impact and, usually, low complexity required to be launched. To solve this issue, several solutions have emerged to identify device models and types based on the combination of behavioral fingerprinting and Machine/Deep Learning (ML/DL) techniques. However, these solutions are not appropriated for scenarios where data privacy and protection is a must, as they require data centralization for processing. In this context, newer approaches such as Federated Learning (FL) have not been fully explored yet, especially when malicious clients are present in the scenario setup. The present work analyzes and compares the device model identification performance of a centralized DL model with an FL one while using execution time-based events. For experimental purposes, a dataset containing execution-time features of 55 Raspberry Pis belonging to four different models has been collected and published. Using this dataset, the proposed solution achieved 0.9999 accuracy in both setups, centralized and federated, showing no performance decrease while preserving data privacy. Later, the impact of a label-flipping attack during the federated model training is evaluated, using several aggregation mechanisms as countermeasure. Zeno and coordinate-wise median aggregation show the best performance, although their performance greatly degrades when the percentage of fully malicious clients (all training samples poisoned) grows over 50%.
K-nearest neighbour and dynamic time warping for online signature verification
Venue: arXiv cs.CV 2021
Year: 2021

Online signatures are one of the most commonly used biometrics. Several verification systems and public databases were presented in this field. This paper presents a combination of k-nearest neighbor and dynamic time warping algorithms as a verification system using the recently published DeepSignDB database. Our algorithm was applied on both finger and stylus input signatures which represent both office and mobile scenarios. The system was first tested on the development set of the database. It achieved an error rate of 6.04% for the stylus input signatures, 5.20% for the finger input signatures, and 6.00% for a combination of both types. The system was also applied to the evaluation set of the database and achieved very promising results, especially for finger input signatures.
Enhancing syntax expressiveness in domain-specific modelling
Venue: arXiv cs.SE 2021
Year: 2021

Domain-specific modelling helps tame the complexity of today's application domains by formalizing concepts and their relationships in modelling languages. While meta-editors are widely-used frameworks for implementing graphical editors for such modelling languages, they are best suitable for defining {novel} topological notations, i.e., syntaxes where the model layout does not contribute to the model semantics. In contrast, many engineering fields, e.g., railways systems or electrical engineering, use notations that, on the one hand, are standard and, on the other hand, are demanding more expressive power than topological syntaxes. In this paper, we discuss the problem of enhancing the expressiveness of modelling editors towards geometric/positional syntaxes. Several potential solutions are experimentally implemented on the jjodel web-based platform with the aim of identifying challenges and opportunities.