OpenRaiser/Intern-Atlas huggingface dataset:
Integrated Speech and Gesture Synthesis
Venue: arXiv cs.HC 2021
Year: 2021

Text-to-speech and co-speech gesture synthesis have until now been treated as separate areas by two different research communities, and applications merely stack the two technologies using a simple system-level pipeline. This can lead to modeling inefficiencies and may introduce inconsistencies that limit the achievable naturalness. We propose to instead synthesize the two modalities in a single model, a new problem we call integrated speech and gesture synthesis (ISG). We also propose a set of models modified from state-of-the-art neural speech-synthesis engines to achieve this goal. We evaluate the models in three carefully-designed user studies, two of which evaluate the synthesized speech and gesture in isolation, plus a combined study that evaluates the models like they will be used in real-world applications -- speech and gesture presented together. The results show that participants rate one of the proposed integrated synthesis models as being as good as the state-of-the-art pipeline system we compare against, in all three tests. The model is able to achieve this with faster synthesis time and greatly reduced parameter count compared to the pipeline system, illustrating some of the potential benefits of treating speech and gesture synthesis together as a single, unified problem. Videos and code are available on our project page at https://swatsw.github.io/isg_icmi21/
Vision-based Autonomous Disinfection of High Touch Surfaces in Indoor Environments
Venue: arXiv cs.RO 2021
Year: 2021

Autonomous systems have played an important role in response to the Covid-19 pandemic. Notably, there have been multiple attempts to leverage Unmanned Aerial Vehicles (UAVs) to disinfect surfaces. Although recent research suggests that surface transmission is less significant than airborne transmission in the spread of Covid-19, surfaces and fomites can play, and have played, critical roles in the transmission of Covid-19 and many other viruses, especially in settings such as child daycares, schools, offices, and hospitals. Employing UAVs for mass spray disinfection offers several potential advantages, including high-throughput application of disinfectant, large scale deployment, and the minimization of health risks to sanitation workers. Despite these potential benefits and preliminary usage of UAVs for disinfection, there has been little research into their design and effectiveness. In this work, we present an autonomous UAV capable of effectively disinfecting indoor surfaces. We identify relevant parameters such as disinfectant type and concentration, and application time and distance required of the UAV to disinfect high-touch surfaces such as door handles. Finally, we develop a robotic system that enables the fully autonomous disinfection of door handles in an unstructured and previously unknown environment. To our knowledge, this is the smallest untethered UAV ever built with both full autonomy and spraying capabilities, allowing it to operate in confined indoor settings, and the first autonomous UAV to specifically target high-touch surfaces on an individual basis with spray disinfectant, resulting in more efficient use of disinfectant
Reducing Label Effort: Self-Supervised meets Active Learning
Venue: arXiv cs.CV 2021
Year: 2021

Active learning is a paradigm aimed at reducing the annotation effort by training the model on actively selected informative and/or representative samples. Another paradigm to reduce the annotation effort is self-training that learns from a large amount of unlabeled data in an unsupervised way and fine-tunes on few labeled samples. Recent developments in self-training have achieved very impressive results rivaling supervised learning on some datasets. The current work focuses on whether the two paradigms can benefit from each other. We studied object recognition datasets including CIFAR10, CIFAR100 and Tiny ImageNet with several labeling budgets for the evaluations. Our experiments reveal that self-training is remarkably more efficient than active learning at reducing the labeling effort, that for a low labeling budget, active learning offers no benefit to self-training, and finally that the combination of active learning and self-training is fruitful when the labeling budget is high. The performance gap between active learning trained either with self-training or from scratch diminishes as we approach to the point where almost half of the dataset is labeled.
Robust Long-Tailed Learning under Label Noise
Venue: arXiv cs.LG 2021
Year: 2021

Long-tailed learning has attracted much attention recently, with the goal of improving generalisation for tail classes. Most existing works use supervised learning without considering the prevailing noise in the training dataset. To move long-tailed learning towards more realistic scenarios, this work investigates the label noise problem under long-tailed label distribution. We first observe the negative impact of noisy labels on the performance of existing methods, revealing the intrinsic challenges of this problem. As the most commonly used approach to cope with noisy labels in previous literature, we then find that the small-loss trick fails under long-tailed label distribution. The reason is that deep neural networks cannot distinguish correctly-labeled and mislabeled examples on tail classes. To overcome this limitation, we establish a new prototypical noise detection method by designing a distance-based metric that is resistant to label noise. Based on the above findings, we propose a robust framework,~\algo, that realizes noise detection for long-tailed learning, followed by soft pseudo-labeling via both label smoothing and diverse label guessing. Moreover, our framework can naturally leverage semi-supervised learning algorithms to further improve the generalisation. Extensive experiments on benchmark and real-world datasets demonstrate the superiority of our methods over existing baselines. In particular, our method outperforms DivideMix by 3\% in test accuracy. Source code will be released soon.
Trajectory Following Strategies for Wireless Capsule Endoscopy under Reciprocally Rotating Magnetic Actuation in a Tubular Environment
Venue: arXiv cs.RO 2021
Year: 2021

Currently used wireless capsule endoscopy (WCE) is limited in terms of inspection time and flexibility since the capsule is passively moved by peristalsis and cannot be accurately positioned. Different methods have been proposed to facilitate active locomotion of WCE based on simultaneous magnetic actuation and localization technologies. In this work, we investigate the trajectory following problem of a robotic capsule under rotating magnetic actuation in a tubular environment, in order to realize safe, efficient and accurate inspection of the intestine at given points using wireless capsule endoscopes. Specifically, four trajectory following strategies are developed based on the PD controller, adaptive controller, model predictive controller and robust multi-stage model predictive controller. Moreover, our method takes into account the uncertainty in the intestinal environment by modeling the intestinal peristalsis and friction during the controller design. We validate our methods in simulation as well as in real-world experiments in various tubular environments, including plastic phantoms with different shapes and an ex-vivo pig colon. The results show that our approach can effectively actuate a reciprocally rotating capsule to follow a desired trajectory in complex tubular environments, thereby having the potential to enable accurate and repeatable inspection of the intestine for high-quality diagnosis.
A promotion for odd symmetric discrete Fourier transform
Venue: arXiv cs.IT 2021
Year: 2021

DFT is the numerical implementation of Fourier transform (FT), and it has many forms. Ordinary DFT (ODFT) and symmetric DFT (SDFT) are the two main forms of DFT. The most widely used DFT is ODFT, and the phase spectrum of this form is widely used in engineering applications. However, it is found ODFT has the problem of phase aliasing. Moreover, ODFT does not have many FT properties, such as symmetry, integration, and interpolation. When compared with ODFT, SDFT has more FT properties. Theoretically, the more properties a transformation has, the wider its application range. Hence, SDFT is more suitable as the discrete form of FT. In order to promote SDFT, the unique nature of SDFT is demonstrated. The time-domain of even-point SDFT is not symmetric to zero, and the author corrects it in this study. The author raises a new issue that should the signal length be odd or even when performing SDFT. The answer is odd. However, scientists and engineers are accustomed to using even-numbered sequences. At the end of this study, the reasons why the author advocates odd SDFT are given. Besides, even sampling function, discrete frequency Fourier transform, and the Gibbs phenomenon of the SDFT are introduced.
Improving the Reliability of Semantic Segmentation of Medical Images by Uncertainty Modeling with Bayesian Deep Networks and Curriculum Learning
Venue: arXiv cs.CV 2021
Year: 2021

In this paper we propose a novel method which leverages the uncertainty measures provided by Bayesian deep networks through curriculum learning so that the uncertainty estimates are fed back to the system to resample the training data more densely in areas where uncertainty is high. We show in the concrete setting of a semantic segmentation task (iPS cell colony segmentation) that the proposed system is able to increase significantly the reliability of the model.
P$ρ$Log: a system for rule-based programming
Venue: arXiv cs.LO 2021
Year: 2021

In this paper, we give a brief overview of the rule-based programming system, called P$ρ$Log and illustrate its capabilities.
An Underwater Image Semantic Segmentation Method Focusing on Boundaries and a Real Underwater Scene Semantic Segmentation Dataset
Venue: arXiv cs.CV 2021
Year: 2021

With the development of underwater object grabbing technology, underwater object recognition and segmentation of high accuracy has become a challenge. The existing underwater object detection technology can only give the general position of an object, unable to give more detailed information such as the outline of the object, which seriously affects the grabbing efficiency. To address this problem, we label and establish the first underwater semantic segmentation dataset of real scene(DUT-USEG:DUT Underwater Segmentation Dataset). The DUT- USEG dataset includes 6617 images, 1487 of which have semantic segmentation and instance segmentation annotations, and the remaining 5130 images have object detection box annotations. Based on this dataset, we propose a semi-supervised underwater semantic segmentation network focusing on the boundaries(US-Net: Underwater Segmentation Network). By designing a pseudo label generator and a boundary detection subnetwork, this network realizes the fine learning of boundaries between underwater objects and background, and improves the segmentation effect of boundary areas. Experiments show that the proposed method improves by 6.7% in three categories of holothurian, echinus, starfish in DUT-USEG dataset, and achieves state-of-the-art results. The DUT- USEG dataset will be released at https://github.com/baxiyi/DUT-USEG.
Local Exceptionality Detection in Time Series Using Subgroup Discovery
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, we present a novel approach for local exceptionality detection on time series data. This method provides the ability to discover interpretable patterns in the data, which can be used to understand and predict the progression of a time series. This being an exploratory approach, the results can be used to generate hypotheses about the relationships between the variables describing a specific process and its dynamics. We detail our approach in a concrete instantiation and exemplary implementation, specifically in the field of teamwork research. Using a real-world dataset of team interactions we include results from an example data analytics application of our proposed approach, showcase novel analysis options, and discuss possible implications of the results from the perspective of teamwork research.
AppSecure.nrw Software Security Study
Venue: arXiv cs.SE 2021
Year: 2021

In recent years, the World Economic Forum has identified software security as the most significant technological risk to the world's population, as software-intensive systems process critical data and provide critical services. This raises the question of the extent to which German companies are addressing software security in developing and operating their software products. This paper reports on the results of an extensive study among developers, product owners, and managers to answer this question. Our results show that ensuring security is a multi-faceted challenge for companies, involving low awareness, inaccurate self-assessment, and a lack of competence on the topic of secure software development among all stakeholders. The current situation in software development is therefore detrimental to the security of software products in the medium and long term.
ICM-3D: Instantiated Category Modeling for 3D Instance Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Separating 3D point clouds into individual instances is an important task for 3D vision. It is challenging due to the unknown and varying number of instances in a scene. Existing deep learning based works focus on a two-step pipeline: first learn a feature embedding and then cluster the points. Such a two-step pipeline leads to disconnected intermediate objectives. In this paper, we propose an integrated reformulation of 3D instance segmentation as a per-point classification problem. We propose ICM-3D, a single-step method to segment 3D instances via instantiated categorization. The augmented category information is automatically constructed from 3D spatial positions. We conduct extensive experiments to verify the effectiveness of ICM-3D and show that it obtains inspiring performance across multiple frameworks, backbones and benchmarks.
On the use of test smells for prediction of flaky tests
Venue: arXiv cs.SE 2021
Year: 2021

Regression testing is an important phase to deliver software with quality. However, flaky tests hamper the evaluation of test results and can increase costs. This is because a flaky test may pass or fail non-deterministically and to identify properly the flakiness of a test requires rerunning the test suite multiple times. To cope with this challenge, approaches have been proposed based on prediction models and machine learning. Existing approaches based on the use of the test case vocabulary may be context-sensitive and prone to overfitting, presenting low performance when executed in a cross-project scenario. To overcome these limitations, we investigate the use of test smells as predictors of flaky tests. We conducted an empirical study to understand if test smells have good performance as a classifier to predict the flakiness in the cross-project context, and analyzed the information gain of each test smell. We also compared the test smell-based approach with the vocabulary-based one. As a result, we obtained a classifier that had a reasonable performance (Random Forest, 0.83) to predict the flakiness in the testing phase. This classifier presented better performance than vocabulary-based model for cross-project prediction. The Assertion Roulette and Sleepy Test test smell types are the ones associated with the best information gain values.
A Hierarchical Assessment of Adversarial Severity
Venue: arXiv cs.LG 2021
Year: 2021

Adversarial Robustness is a growing field that evidences the brittleness of neural networks. Although the literature on adversarial robustness is vast, a dimension is missing in these studies: assessing how severe the mistakes are. We call this notion "Adversarial Severity" since it quantifies the downstream impact of adversarial corruptions by computing the semantic error between the misclassification and the proper label. We propose to study the effects of adversarial noise by measuring the Robustness and Severity into a large-scale dataset: iNaturalist-H. Our contributions are: (i) we introduce novel Hierarchical Attacks that harness the rich structured space of labels to create adversarial examples. (ii) These attacks allow us to benchmark the Adversarial Robustness and Severity of classification models. (iii) We enhance the traditional adversarial training with a simple yet effective Hierarchical Curriculum Training to learn these nodes gradually within the hierarchical tree. We perform extensive experiments showing that hierarchical defenses allow deep models to boost the adversarial Robustness by 1.85% and reduce the severity of all attacks by 0.17, on average.
State of the Art: Image Hashing
Venue: arXiv cs.CV 2021
Year: 2021

Perceptual image hashing methods are often applied in various objectives, such as image retrieval, finding duplicate or near-duplicate images, and finding similar images from large-scale image content. The main challenge in image hashing techniques is robust feature extraction, which generates the same or similar hashes in images that are visually identical. In this article, we present a short review of the state-of-the-art traditional perceptual hashing and deep learning-based perceptual hashing methods, identifying the best approaches.
AI at work -- Mitigating safety and discriminatory risk with technical standards
Venue: arXiv cs.CY 2021
Year: 2021

The use of artificial intelligence (AI) and AI methods in the workplace holds both great opportunities as well as risks to occupational safety and discrimination. In addition to legal regulation, technical standards will play a key role in mitigating such risk by defining technical requirements for development and testing of AI systems. This paper provides an overview and assessment of existing international, European and German standards as well as those currently under development. The paper is part of the research project "ExamAI - Testing and Auditing of AI systems" and focusses on the use of AI in an industrial production environment as well as in the realm of human resource management (HR).
Security and privacy for 6G: A survey on prospective technologies and challenges
Venue: arXiv cs.CR 2021
Year: 2021

Sixth-generation (6G) mobile networks will have to cope with diverse threats on a space-air-ground integrated network environment, novel technologies, and an accessible user information explosion. However, for now, security and privacy issues for 6G remain largely in concept. This survey provides a systematic overview of security and privacy issues based on prospective technologies for 6G in the physical, connection, and service layers, as well as through lessons learned from the failures of existing security architectures and state-of-the-art defenses. Two key lessons learned are as follows. First, other than inheriting vulnerabilities from the previous generations, 6G has new threat vectors from new radio technologies, such as the exposed location of radio stripes in ultra-massive MIMO systems at Terahertz bands and attacks against pervasive intelligence. Second, physical layer protection, deep network slicing, quantum-safe communications, artificial intelligence (AI) security, platform-agnostic security, real-time adaptive security, and novel data protection mechanisms such as distributed ledgers and differential privacy are the top promising techniques to mitigate the attack magnitude and personal data breaches substantially.
A Typed Programmatic Interface to Contracts on the Blockchain
Venue: arXiv cs.PL 2021
Year: 2021

Smart contract applications on the blockchain can only reach their full potential if they integrate seamlessly with traditional software systems via a programmatic interface. This interface should provide for originating and invoking contracts as well as observing the state of the blockchain. We propose a typed API for this purpose and establish some properties of the combined system. Specifically, we provide an execution model that enables us to prove type-safe interaction between programs and the blockchain. We establish further properties of the model that give rise to requirements on the API. A prototype of the interface is implemented in OCaml for the Tezos blockchain.
Federated Reinforcement Learning: Techniques, Applications, and Open Challenges
Venue: arXiv cs.LG 2021
Year: 2021

This paper presents a comprehensive survey of Federated Reinforcement Learning (FRL), an emerging and promising field in Reinforcement Learning (RL). Starting with a tutorial of Federated Learning (FL) and RL, we then focus on the introduction of FRL as a new method with great potential by leveraging the basic idea of FL to improve the performance of RL while preserving data-privacy. According to the distribution characteristics of the agents in the framework, FRL algorithms can be divided into two categories, i.e. Horizontal Federated Reinforcement Learning (HFRL) and Vertical Federated Reinforcement Learning (VFRL). We provide the detailed definitions of each category by formulas, investigate the evolution of FRL from a technical perspective, and highlight its advantages over previous RL algorithms. In addition, the existing works on FRL are summarized by application fields, including edge computing, communication, control optimization, and attack detection. Finally, we describe and discuss several key research directions that are crucial to solving the open problems within FRL.
Approximation algorithms for the random-field Ising model
Venue: arXiv cs.DS 2021
Year: 2021

Approximating the partition function of the ferromagnetic Ising model with general external fields is known to be #BIS-hard in the worst case, even for bounded-degree graphs, and it is widely believed that no polynomial-time approximation scheme exists. This motivates an average-case question: are there classes of instances for which polynomial-time approximation schemes exist? We investigate this question for the random field Ising model on graphs with maximum degree $Δ$. We establish the existence of fully polynomial-time approximation schemes and samplers with high probability over the random fields if the external fields are IID Gaussians with variance larger than a constant depending only on the inverse temperature and $Δ$. The main challenge comes from the positive density of vertices at which the external field is small. These regions, which may have connected components of size $Θ(\log n)$, are a barrier to algorithms based on establishing a zero-free region, and cause worst-case analyses of Glauber dynamics to fail. The analysis of our algorithm is based on percolation on a self-avoiding walk tree.