OpenRaiser/Intern-Atlas huggingface dataset:
Framework Artifact for the Road-Based Physical Internet based on Internet Protocols
Venue: arXiv cs.NI 2021
Year: 2021

The Physical Internet (PI) raises high expectations for efficiency gains in transport and logistics. The PI represents the network of logistics networks for physical objects in analogy to the Data Internet (DI). Road based traffic represents one of these logistics networks. Here, many empty runs and underutilized trips still take place. Hence, there is a lot of potential in the road-based Physical Internet (RBPI), which will have an impact on transport and logistics strategies, but also on vehicle design. On the DI, logistics strategies are implemented in protocols. In order to transfer such concepts to the RBPI, relevant protocols of the DI had been analyzed and transferred to the world of physical objects. However, not all functionalities can be transferred one-to-one, e.g. a data packet in the DI can simply be re-generated by a hub in case of damage or loss. To compensate for the challenges, a framework artifact has been designed with appropriate transformation customizations based on design science principles. From this, resulting requirements for future vehicles were derived. This paper makes a contribution to the implementation of the RBPI in order to fit road based vehicles to the future world of transport and logistics.
A White Paper on Neural Network Quantization
Venue: arXiv cs.LG 2021
Year: 2021

While neural networks have advanced the frontiers in many applications, they often come at a high computational cost. Reducing the power and latency of neural network inference is key if we want to integrate modern networks into edge devices with strict power and compute requirements. Neural network quantization is one of the most effective ways of achieving these savings but the additional noise it induces can lead to accuracy degradation. In this white paper, we introduce state-of-the-art algorithms for mitigating the impact of quantization noise on the network's performance while maintaining low-bit weights and activations. We start with a hardware motivated introduction to quantization and then consider two main classes of algorithms: Post-Training Quantization (PTQ) and Quantization-Aware-Training (QAT). PTQ requires no re-training or labelled data and is thus a lightweight push-button approach to quantization. In most cases, PTQ is sufficient for achieving 8-bit quantization with close to floating-point accuracy. QAT requires fine-tuning and access to labeled training data but enables lower bit quantization with competitive results. For both solutions, we provide tested pipelines based on existing literature and extensive experimentation that lead to state-of-the-art performance for common deep learning models and tasks.
Explaining decision of model from its prediction
Venue: arXiv cs.CV 2021
Year: 2021

This document summarizes different visual explanations methods such as CAM, Grad-CAM, Localization using Multiple Instance Learning - Saliency-based methods, Saliency-driven Class-Impressions, Muting pixels in input image - Adversarial methods and Activation visualization, Convolution filter visualization - Feature-based methods. We have also shown the results produced by different methods and a comparison between CAM, GradCAM, and Guided Backpropagation.
A Multi-Layered Approach for Measuring the Simulation-to-Reality Gap of Radar Perception for Autonomous Driving
Venue: arXiv cs.RO 2021
Year: 2021

With the increasing safety validation requirements for the release of a self-driving car, alternative approaches, such as simulation-based testing, are emerging in addition to conventional real-world testing. In order to rely on virtual tests the employed sensor models have to be validated. For this reason, it is necessary to quantify the discrepancy between simulation and reality in order to determine whether a certain fidelity is sufficient for a desired intended use. There exists no sound method to measure this simulation-to-reality gap of radar perception for autonomous driving. We address this problem by introducing a multi-layered evaluation approach, which consists of a combination of an explicit and an implicit sensor model evaluation. The former directly evaluates the realism of the synthetically generated sensor data, while the latter refers to an evaluation of a downstream target application. In order to demonstrate the method, we evaluated the fidelity of three typical radar model types (ideal, data-driven, ray tracing-based) and their applicability for virtually testing radar-based multi-object tracking. We have shown the effectiveness of the proposed approach in terms of providing an in-depth sensor model assessment that renders existing disparities visible and enables a realistic estimation of the overall model fidelity across different scenarios.
Plane and Sample: Maximizing Information about Autonomous Vehicle Performance using Submodular Optimization
Venue: arXiv cs.RO 2021
Year: 2021

As autonomous vehicles (AVs) take on growing Operational Design Domains (ODDs), they need to go through a systematic, transparent, and scalable evaluation process to demonstrate their benefits to society. Current scenario sampling techniques for AV performance evaluation usually focus on a specific functionality, such as lane changing, and do not accommodate a transfer of information about an AV system from one ODD to the next. In this paper, we reformulate the scenario sampling problem across ODDs and functionalities as a submodular optimization problem. To do so, we abstract AV performance as a Bayesian Hierarchical Model, which we use to infer information gained by revealing performance in new scenarios. We propose the information gain as a measure of scenario relevance and evaluation progress. Furthermore, we leverage the submodularity, or diminishing returns, property of the information gain not only to find a near-optimal scenario set, but also to propose a stopping criterion for an AV performance evaluation campaign. We find that we only need to explore about 7.5% of the scenario space to meet this criterion, a 23% improvement over Latin Hypercube Sampling.
Seeing Through Clouds in Satellite Images
Venue: arXiv cs.CV 2021
Year: 2021

This paper presents a neural-network-based solution to recover pixels occluded by clouds in satellite images. We leverage radio frequency (RF) signals in the ultra/super-high frequency band that penetrate clouds to help reconstruct the occluded regions in multispectral images. We introduce the first multi-modal multi-temporal cloud removal model. Our model uses publicly available satellite observations and produces daily cloud-free images. Experimental results show that our system significantly outperforms baselines by 8dB in PSNR. We also demonstrate use cases of our system in digital agriculture, flood monitoring, and wildfire detection. We will release the processed dataset to facilitate future research.
Pathological voice adaptation with autoencoder-based voice conversion
Venue: arXiv cs.SD 2021
Year: 2021

In this paper, we propose a new approach to pathological speech synthesis. Instead of using healthy speech as a source, we customise an existing pathological speech sample to a new speaker's voice characteristics. This approach alleviates the evaluation problem one normally has when converting typical speech to pathological speech, as in our approach, the voice conversion (VC) model does not need to be optimised for speech degradation but only for the speaker change. This change in the optimisation ensures that any degradation found in naturalness is due to the conversion process and not due to the model exaggerating characteristics of a speech pathology. To show a proof of concept of this method, we convert dysarthric speech using the UASpeech database and an autoencoder-based VC technique. Subjective evaluation results show reasonable naturalness for high intelligibility dysarthric speakers, though lower intelligibility seems to introduce a marginal degradation in naturalness scores for mid and low intelligibility speakers compared to ground truth. Conversion of speaker characteristics for low and high intelligibility speakers is successful, but not for mid. Whether the differences in the results for the different intelligibility levels is due to the intelligibility levels or due to the speakers needs to be further investigated.
Online Learning with Uncertain Feedback Graphs
Venue: arXiv cs.LG 2021
Year: 2021

Online learning with expert advice is widely used in various machine learning tasks. It considers the problem where a learner chooses one from a set of experts to take advice and make a decision. In many learning problems, experts may be related, henceforth the learner can observe the losses associated with a subset of experts that are related to the chosen one. In this context, the relationship among experts can be captured by a feedback graph, which can be used to assist the learner's decision making. However, in practice, the nominal feedback graph often entails uncertainties, which renders it impossible to reveal the actual relationship among experts. To cope with this challenge, the present work studies various cases of potential uncertainties, and develops novel online learning algorithms to deal with uncertainties while making use of the uncertain feedback graph. The proposed algorithms are proved to enjoy sublinear regret under mild conditions. Experiments on real datasets are presented to demonstrate the effectiveness of the novel algorithms.
Introducing Type Properties
Venue: arXiv cs.PL 2021
Year: 2021

In type theory, we can express many practical ideas by attributing some additional data to expressions we operate on during compilation. For instance, some substructural type theories augment variables' typing judgments with the information of their usage. That is, they allow one to explicitly state how many times - 0, 1, or many - a variable can be used. This solves the problem of resource usage control and allows us to treat variables as resources. What's more, it often happens that this attributed information is interpreted (used) during the same compilation and erased before we run a program. A case in the point is that in the same substructural type theories, their type checkers use these 0, 1, or many, to ensure that all variables are used as many times as these attributions say them to be. Yet, there wasn't any programming language concept whose concern would be to allow a programmer to express these attributions in the language itself. That is, to let the programmer express which data the one wants to attribute to what expressions and, most importantly, the meaning of the attributed data in their program. As it turned out, the presence of such a concept allows us to express many practical ideas in the language itself. For instance, with appropriate means for assigning the meaning of these attributions, this concept would allow one to express linear types as functionality in a separate program module, without the need to refine the whole type system to add them. In this paper, we present such a concept - we propose type properties. It allows a programmer to express these attributions while fulfilling the requirement of being fully on the static level. That is, it allows one to express how to interpret these attributions during compilation and erases them before a program is passed to the runtime.
Age of Information for Small Buffer Systems
Venue: arXiv cs.PF 2021
Year: 2021

Consider a message processing system whose objective is to produce the most current information as measured by the quantity known as "age of information". We have argued in previous papers that if we are allowed to design the message processing policy ad libitum, we should keep a small buffer and operate according to a LIFO policy. In this small note we provide an analysis for the AoI of the P_m system which uses a buffer of size m, a single server, operating without service preemption and in a LIFO manner for stored messages. Analytical expressions for the mean (or even distribution) of the AoI in steady-state are possible but with the aid computer algebra. We explain the the analysis for m=3.
ICDAR 2021 Competition on Components Segmentation Task of Document Photos
Venue: arXiv cs.CV 2021
Year: 2021

This paper describes the short-term competition on the Components Segmentation Task of Document Photos that was prepared in the context of the 16th International Conference on Document Analysis and Recognition (ICDAR 2021). This competition aims to bring together researchers working in the field of identification document image processing and provides them a suitable benchmark to compare their techniques on the component segmentation task of document images. Three challenge tasks were proposed entailing different segmentation assignments to be performed on a provided dataset. The collected data are from several types of Brazilian ID documents, whose personal information was conveniently replaced. There were 16 participants whose results obtained for some or all the three tasks show different rates for the adopted metrics, like Dice Similarity Coefficient ranging from 0.06 to 0.99. Different Deep Learning models were applied by the entrants with diverse strategies to achieve the best results in each of the tasks. Obtained results show that the currently applied methods for solving one of the proposed tasks (document boundary detection) are already well established. However, for the other two challenge tasks (text zone and handwritten sign detection) research and development of more robust approaches are still required to achieve acceptable results.
Watching Too Much Television is Good: Self-Supervised Audio-Visual Representation Learning from Movies and TV Shows
Venue: arXiv cs.CV 2021
Year: 2021

The abundance and ease of utilizing sound, along with the fact that auditory clues reveal so much about what happens in the scene, make the audio-visual space a perfectly intuitive choice for self-supervised representation learning. However, the current literature suggests that training on \textit{uncurated} data yields considerably poorer representations compared to the \textit{curated} alternatives collected in supervised manner, and the gap only narrows when the volume of data significantly increases. Furthermore, the quality of learned representations is known to be heavily influenced by the size and taxonomy of the curated datasets used for self-supervised training. This begs the question of whether we are celebrating too early on catching up with supervised learning when our self-supervised efforts still rely almost exclusively on curated data. In this paper, we study the efficacy of learning from Movies and TV Shows as forms of uncurated data for audio-visual self-supervised learning. We demonstrate that a simple model based on contrastive learning, trained on a collection of movies and TV shows, not only dramatically outperforms more complex methods which are trained on orders of magnitude larger uncurated datasets, but also performs very competitively with the state-of-the-art that learns from large-scale curated data. We identify that audiovisual patterns like the appearance of the main character or prominent scenes and mise-en-scène which frequently occur through the whole duration of a movie, lead to an overabundance of easy negative instances in the contrastive learning formulation. Capitalizing on such observation, we propose a hierarchical sampling policy, which despite its simplicity, effectively improves the performance, particularly when learning from TV shows which naturally face less semantic diversity.
SEEN: Sharpening Explanations for Graph Neural Networks using Explanations from Neighborhoods
Venue: arXiv cs.LG 2021
Year: 2021

Explaining the foundations for predictions obtained from graph neural networks (GNNs) is critical for credible use of GNN models for real-world problems. Owing to the rapid growth of GNN applications, recent progress in explaining predictions from GNNs, such as sensitivity analysis, perturbation methods, and attribution methods, showed great opportunities and possibilities for explaining GNN predictions. In this study, we propose a method to improve the explanation quality of node classification tasks that can be applied in a post hoc manner through aggregation of auxiliary explanations from important neighboring nodes, named SEEN. Applying SEEN does not require modification of a graph and can be used with diverse explainability techniques due to its independent mechanism. Experiments on matching motif-participating nodes from a given graph show great improvement in explanation accuracy of up to 12.71% and demonstrate the correlation between the auxiliary explanations and the enhanced explanation accuracy through leveraging their contributions. SEEN provides a simple but effective method to enhance the explanation quality of GNN model outputs, and this method is applicable in combination with most explainability techniques.
Source Separation-based Data Augmentation for Improved Joint Beat and Downbeat Tracking
Venue: arXiv cs.SD 2021
Year: 2021

Due to advances in deep learning, the performance of automatic beat and downbeat tracking in musical audio signals has seen great improvement in recent years. In training such deep learning based models, data augmentation has been found an important technique. However, existing data augmentation methods for this task mainly target at balancing the distribution of the training data with respect to their tempo. In this paper, we investigate another approach for data augmentation, to account for the composition of the training data in terms of the percussive and non-percussive sound sources. Specifically, we propose to employ a blind drum separation model to segregate the drum and non-drum sounds from each training audio signal, filtering out training signals that are drumless, and then use the obtained drum and non-drum stems to augment the training data. We report experiments on four completely unseen test sets, validating the effectiveness of the proposed method, and accordingly the importance of drum sound composition in the training data for beat and downbeat tracking.
Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translation
Venue: arXiv cs.RO 2021
Year: 2021

Simulation has recently become key for deep reinforcement learning to safely and efficiently acquire general and complex control policies from visual and proprioceptive inputs. Tactile information is not usually considered despite its direct relation to environment interaction. In this work, we present a suite of simulated environments tailored towards tactile robotics and reinforcement learning. A simple and fast method of simulating optical tactile sensors is provided, where high-resolution contact geometry is represented as depth images. Proximal Policy Optimisation (PPO) is used to learn successful policies across all considered tasks. A data-driven approach enables translation of the current state of a real tactile sensor to corresponding simulated depth images. This policy is implemented within a real-time control loop on a physical robot to demonstrate zero-shot sim-to-real policy transfer on several physically-interactive tasks requiring a sense of touch.
Contrastive Learning with Continuous Proxy Meta-Data for 3D MRI Classification
Venue: arXiv cs.CV 2021
Year: 2021

Traditional supervised learning with deep neural networks requires a tremendous amount of labelled data to converge to a good solution. For 3D medical images, it is often impractical to build a large homogeneous annotated dataset for a specific pathology. Self-supervised methods offer a new way to learn a representation of the images in an unsupervised manner with a neural network. In particular, contrastive learning has shown great promises by (almost) matching the performance of fully-supervised CNN on vision tasks. Nonetheless, this method does not take advantage of available meta-data, such as participant's age, viewed as prior knowledge. Here, we propose to leverage continuous proxy metadata, in the contrastive learning framework, by introducing a new loss called y-Aware InfoNCE loss. Specifically, we improve the positive sampling during pre-training by adding more positive examples with similar proxy meta-data with the anchor, assuming they share similar discriminative semantic features.With our method, a 3D CNN model pre-trained on $10^4$ multi-site healthy brain MRI scans can extract relevant features for three classification tasks: schizophrenia, bipolar diagnosis and Alzheimer's detection. When fine-tuned, it also outperforms 3D CNN trained from scratch on these tasks, as well as state-of-the-art self-supervised methods. Our code is made publicly available here.
"It's Viral!" -- A study of the behaviors, practices, and motivations of TikTok Social Activists
Venue: arXiv cs.HC 2021
Year: 2021

Social media platforms such as Facebook and Twitter are used for social activism purposes, and TikTok is no different. We conducted 9 qualitative semi-structured interviews with social activists who recently posted their videos on TikTok to understand. This study presents an initial look into why TikTok is used by social activists, and what processes they use to carry out this work. The interviews revealed the following main patterns: (1) content creation practices are typical and expected, (2) motivation and inspiration for posting social activist content comes from a wide range of personal sources, (3) social activism has communities on TikTok that provides encouragement and discussion, (4) engagement and interaction with other activists and viewers is a crucial part of content creation as well as social activism, and (5) the main driving force behind picking TikTok over other platforms is the ability to spread messages much farther with less effort. These findings provide insight into the unique factors that TikTok brings for social activists and corroborates previous findings in understanding how social activists use social media for their purposes.
How memory architecture affects learning in a simple POMDP: the two-hypothesis testing problem
Venue: arXiv cs.LG 2021
Year: 2021

Reinforcement learning is generally difficult for partially observable Markov decision processes (POMDPs), which occurs when the agent's observation is partial or noisy. To seek good performance in POMDPs, one strategy is to endow the agent with a finite memory, whose update is governed by the policy. However, policy optimization is non-convex in that case and can lead to poor training performance for random initialization. The performance can be empirically improved by constraining the memory architecture, then sacrificing optimality to facilitate training. Here we study this trade-off in a two-hypothesis testing problem, akin to the two-arm bandit problem. We compare two extreme cases: (i) the random access memory where any transitions between $M$ memory states are allowed and (ii) a fixed memory where the agent can access its last $m$ actions and rewards. For (i), the probability $q$ to play the worst arm is known to be exponentially small in $M$ for the optimal policy. Our main result is to show that similar performance can be reached for (ii) as well, despite the simplicity of the memory architecture: using a conjecture on Gray-ordered binary necklaces, we find policies for which $q$ is exponentially small in $2^m$, i.e. $q\simα^{2^m}$ with $α< 1$. In addition, we observe empirically that training from random initialization leads to very poor results for (i), and significantly better results for (ii) thanks to the constraints on the memory architecture.