OpenRaiser/Intern-Atlas huggingface dataset:
Intra- & Extra-Source Exemplar-Based Style Synthesis for Improved Domain Generalization
Venue: arXiv cs.CV 2023
Year: 2023

The generalization with respect to domain shifts, as they frequently appear in applications such as autonomous driving, is one of the remaining big challenges for deep learning models. Therefore, we propose an exemplar-based style synthesis pipeline to improve domain generalization in semantic segmentation. Our method is based on a novel masked noise encoder for StyleGAN2 inversion. The model learns to faithfully reconstruct the image, preserving its semantic layout through noise prediction. Using the proposed masked noise encoder to randomize style and content combinations in the training set, i.e., intra-source style augmentation (ISSA) effectively increases the diversity of training data and reduces spurious correlation. As a result, we achieve up to $12.4\%$ mIoU improvements on driving-scene semantic segmentation under different types of data shifts, i.e., changing geographic locations, adverse weather conditions, and day to night. ISSA is model-agnostic and straightforwardly applicable with CNNs and Transformers. It is also complementary to other domain generalization techniques, e.g., it improves the recent state-of-the-art solution RobustNet by $3\%$ mIoU in Cityscapes to Dark Zürich. In addition, we demonstrate the strong plug-n-play ability of the proposed style synthesis pipeline, which is readily usable for extra-source exemplars e.g., web-crawled images, without any retraining or fine-tuning. Moreover, we study a new use case to indicate neural network's generalization capability by building a stylized proxy validation set. This application has significant practical sense for selecting models to be deployed in the open-world environment. Our code is available at \url{https://github.com/boschresearch/ISSA}.
More Synergy, Less Redundancy: Exploiting Joint Mutual Information for Self-Supervised Learning
Venue: arXiv cs.CV 2023
Year: 2023

Self-supervised learning (SSL) is now a serious competitor for supervised learning, even though it does not require data annotation. Several baselines have attempted to make SSL models exploit information about data distribution, and less dependent on the augmentation effect. However, there is no clear consensus on whether maximizing or minimizing the mutual information between representations of augmentation views practically contribute to improvement or degradation in performance of SSL models. This paper is a fundamental work where, we investigate role of mutual information in SSL, and reformulate the problem of SSL in the context of a new perspective on mutual information. To this end, we consider joint mutual information from the perspective of partial information decomposition (PID) as a key step in \textbf{reliable multivariate information measurement}. PID enables us to decompose joint mutual information into three important components, namely, unique information, redundant information and synergistic information. Our framework aims for minimizing the redundant information between views and the desired target representation while maximizing the synergistic information at the same time. Our experiments lead to a re-calibration of two redundancy reduction baselines, and a proposal for a new SSL training protocol. Extensive experimental results on multiple datasets and two downstream tasks show the effectiveness of this framework.
Accelerating Relational Database Analytical Processing with Bulk-Bitwise Processing-in-Memory
Venue: arXiv cs.DB 2023
Year: 2023

Online Analytical Processing (OLAP) for relational databases is a business decision support application. The application receives queries about the business database, usually requesting to summarize many database records, and produces few results. Existing OLAP requires transferring a large amount of data between the memory and the CPU, having a few operations per datum, and producing a small output. Hence, OLAP is a good candidate for processing-in-memory (PIM), where computation is performed where the data is stored, thus accelerating applications by reducing data movement between the memory and CPU. In particular, bulk-bitwise PIM, where the memory array is a bit-vector processing unit, seems a good match for OLAP. With the extensive inherent parallelism and minimal data movement of bulk-bitwise PIM, OLAP applications can process the entire database in parallel in memory, transferring only the results to the CPU. This paper shows a full stack adaptation of a bulk-bitwise PIM, from compiling SQL to hardware implementation, for supporting OLAP applications. Evaluating the Star Schema Benchmark (SSB), bulk-bitwise PIM achieves a 4.65X speedup over Monet-DB, a standard database system.
Solving Multi-Agent Target Assignment and Path Finding with a Single Constraint Tree
Venue: arXiv cs.AI 2023
Year: 2023

Combined Target-Assignment and Path-Finding problem (TAPF) requires simultaneously assigning targets to agents and planning collision-free paths for agents from their start locations to their assigned targets. As a leading approach to address TAPF, Conflict-Based Search with Target Assignment (CBS-TA) leverages both K-best target assignments to create multiple search trees and Conflict-Based Search (CBS) to resolve collisions in each search tree. While being able to find an optimal solution, CBS-TA suffers from scalability due to the duplicated collision resolution in multiple trees and the expensive computation of K-best assignments. We therefore develop Incremental Target Assignment CBS (ITA-CBS) to bypass these two computational bottlenecks. ITA-CBS generates only a single search tree and avoids computing K-best assignments by incrementally computing new 1-best assignments during the search. We show that, in theory, ITA-CBS is guaranteed to find an optimal solution and, in practice, is computationally efficient.
Efficient Visual Fault Detection for Freight Train Braking System via Heterogeneous Self Distillation in the Wild
Venue: arXiv cs.CV 2023
Year: 2023

Efficient visual fault detection of freight trains is a critical part of ensuring the safe operation of railways under the restricted hardware environment. Although deep learning-based approaches have excelled in object detection, the efficiency of freight train fault detection is still insufficient to apply in real-world engineering. This paper proposes a heterogeneous self-distillation framework to ensure detection accuracy and speed while satisfying low resource requirements. The privileged information in the output feature knowledge can be transferred from the teacher to the student model through distillation to boost performance. We first adopt a lightweight backbone to extract features and generate a new heterogeneous knowledge neck. Such neck models positional information and long-range dependencies among channels through parallel encoding to optimize feature extraction capabilities. Then, we utilize the general distribution to obtain more credible and accurate bounding box estimates. Finally, we employ a novel loss function that makes the network easily concentrate on values near the label to improve learning efficiency. Experiments on four fault datasets reveal that our framework can achieve over 37 frames per second and maintain the highest accuracy in comparison with traditional distillation approaches. Moreover, compared to state-of-the-art methods, our framework demonstrates more competitive performance with lower memory usage and the smallest model size.
Evaluation and Control Model Design of Human Factors for Autonomous Driving Systems
Venue: arXiv cs.RO 2023
Year: 2023

With the fast development of driving automation technologies, user psychological acceptance of driving automation has become one of the major obstacles to the adoption of the driving automation technology. The most basic function of a passenger car is to transport passengers or drivers to their destinations safely and comfortably. Thus, the design of the driving automation should not just guarantee the safety of vehicle operation but also ensure occupant subjective level of comfort. Hence this paper proposes a local path planning algorithm for obstacle avoidance with occupant subjective feelings considered. Firstly, turning and obstacle avoidance conditions are designed, and four classifiers in machine learning are used to respectively establish subjective and objective evaluation models that link the objective vehicle dynamics parameters and occupant subjective confidence. Then, two potential fields are established based on the artificial potential field, reflecting the psychological feeling of drivers on obstacles and road boundaries. Accordingly, a path planning algorithm and a path tracking algorithm are designed respectively based on model predictive control, and the psychological safety boundary and the optimal classifier are used as part of cost functions. Finally, co-simulations of MATLAB/Simulink and CarSim are carried out. The results confirm the effectiveness of the proposed control algorithm, which can avoid obstacles satisfactorily and improve the psychological feeling of occupants effectively.
Neural Polytopes
Venue: arXiv cs.LG 2023
Year: 2023

We find that simple neural networks with ReLU activation generate polytopes as an approximation of a unit sphere in various dimensions. The species of polytopes are regulated by the network architecture, such as the number of units and layers. For a variety of activation functions, generalization of polytopes is obtained, which we call neural polytopes. They are a smooth analogue of polytopes, exhibiting geometric duality. This finding initiates research of generative discrete geometry to approximate surfaces by machine learning.
ImDiffusion: Imputed Diffusion Models for Multivariate Time Series Anomaly Detection
Venue: arXiv cs.LG 2023
Year: 2023

Anomaly detection in multivariate time series data is of paramount importance for ensuring the efficient operation of large-scale systems across diverse domains. However, accurately detecting anomalies in such data poses significant challenges. Existing approaches, including forecasting and reconstruction-based methods, struggle to address these challenges effectively. To overcome these limitations, we propose a novel anomaly detection framework named ImDiffusion, which combines time series imputation and diffusion models to achieve accurate and robust anomaly detection. The imputation-based approach employed by ImDiffusion leverages the information from neighboring values in the time series, enabling precise modeling of temporal and inter-correlated dependencies, reducing uncertainty in the data, thereby enhancing the robustness of the anomaly detection process. ImDiffusion further leverages diffusion models as time series imputers to accurately capturing complex dependencies. We leverage the step-by-step denoised outputs generated during the inference process to serve as valuable signals for anomaly prediction, resulting in improved accuracy and robustness of the detection process. We evaluate the performance of ImDiffusion via extensive experiments on benchmark datasets. The results demonstrate that our proposed framework significantly outperforms state-of-the-art approaches in terms of detection accuracy and timeliness. ImDiffusion is further integrated into the real production system in Microsoft and observe a remarkable 11.4% increase in detection F1 score compared to the legacy approach. To the best of our knowledge, ImDiffusion represents a pioneering approach that combines imputation-based techniques with time series anomaly detection, while introducing the novel use of diffusion models to the field.
VINECS: Video-based Neural Character Skinning
Venue: arXiv cs.CV 2023
Year: 2023

Rigging and skinning clothed human avatars is a challenging task and traditionally requires a lot of manual work and expertise. Recent methods addressing it either generalize across different characters or focus on capturing the dynamics of a single character observed under different pose configurations. However, the former methods typically predict solely static skinning weights, which perform poorly for highly articulated poses, and the latter ones either require dense 3D character scans in different poses or cannot generate an explicit mesh with vertex correspondence over time. To address these challenges, we propose a fully automated approach for creating a fully rigged character with pose-dependent skinning weights, which can be solely learned from multi-view video. Therefore, we first acquire a rigged template, which is then statically skinned. Next, a coordinate-based MLP learns a skinning weights field parameterized over the position in a canonical pose space and the respective pose. Moreover, we introduce our pose- and view-dependent appearance field allowing us to differentiably render and supervise the posed mesh using multi-view imagery. We show that our approach outperforms state-of-the-art while not relying on dense 4D scans.
On the Definition of the Eta-long Normal Form in Type Systems of the Cube
Venue: arXiv cs.LO 2023
Year: 2023

The smallest transitive relation < on well-typed normal terms such that if t is a strict subterm of u then t < u and if T is the normal form of the type of t and the term t is not a sort then T < t is well-founded in the type systems of the cube. Thus every term admits a eta-long normal form.
Thompson Sampling under Bernoulli Rewards with Local Differential Privacy
Venue: arXiv cs.LG 2023
Year: 2023

This paper investigates the problem of regret minimization for multi-armed bandit (MAB) problems with local differential privacy (LDP) guarantee. Given a fixed privacy budget $ε$, we consider three privatizing mechanisms under Bernoulli scenario: linear, quadratic and exponential mechanisms. Under each mechanism, we derive stochastic regret bound for Thompson Sampling algorithm. Finally, we simulate to illustrate the convergence of different mechanisms under different privacy budgets.
Tales from the Git: Automating the detection of secrets on code and assessing developers' passwords choices
Venue: arXiv cs.SE 2023
Year: 2023

Typical users are known to use and reuse weak passwords. Yet, as cybersecurity concerns continue to rise, understanding the password practices of software developers becomes increasingly important. In this work, we examine developers' passwords on public repositories. Our dedicated crawler collected millions of passwords from public GitHub repositories; however, our focus is on their unique characteristics. To this end, this is the first study investigating the developer traits in password selection across different programming languages and contexts, e.g. email and database. Despite the fact that developers may have carelessly leaked their code on public repositories, our findings indicate that they tend to use significantly more secure passwords, regardless of the underlying programming language and context. Nevertheless, when the context allows, they often resort to similar password selection criteria as typical users. The public availability of such information in a cleartext format indicates that there is still much room for improvement and that further targeted awareness campaigns are necessary.
Generating Reliable Pixel-Level Labels for Source Free Domain Adaptation
Venue: arXiv cs.CV 2023
Year: 2023

This work addresses the challenging domain adaptation setting in which knowledge from the labelled source domain dataset is available only from the pretrained black-box segmentation model. The pretrained model's predictions for the target domain images are noisy because of the distributional differences between the source domain data and the target domain data. Since the model's predictions serve as pseudo labels during self-training, the noise in the predictions impose an upper bound on model performance. Therefore, we propose a simple yet novel image translation workflow, ReGEN, to address this problem. ReGEN comprises an image-to-image translation network and a segmentation network. Our workflow generates target-like images using the noisy predictions from the original target domain images. These target-like images are semantically consistent with the noisy model predictions and therefore can be used to train the segmentation network. In addition to being semantically consistent with the predictions from the original target domain images, the generated target-like images are also stylistically similar to the target domain images. This allows us to leverage the stylistic differences between the target-like images and the target domain image as an additional source of supervision while training the segmentation model. We evaluate our model with two benchmark domain adaptation settings and demonstrate that our approach performs favourably relative to recent state-of-the-art work. The source code will be made available.
Mega-cities dominate China's urban greening
Venue: arXiv cs.CV 2023
Year: 2023

Trees play a crucial role in urban environments, offering various ecosystem services that contribute to public health and human well-being. China has initiated a range of urban greening policies over the past decades, however, monitoring their impact on urban tree dynamics at a national scale has proven challenging. In this study, we deployed nano-satellites to quantify urban tree coverage in all major Chinese cities larger than 50 km2 in 2010 and 2019. Our findings indicate that approximately 6000 km2 (11%) of urban areas were covered by trees in 2019, and 76% of these cities experienced an increase in tree cover compared to 2010. Notably, the increase in tree cover in mega-cities such as Beijing, and Shanghai was approximately twice as large as in most other cities (7.69% vs 3.94%). The study employs a data-driven approach towards assessing urban tree cover changes in relation to greening policies, showing clear signs of tree cover increases but also suggesting an uneven implementation primarily benefiting a few mega-cities.
Energy-aware Time- and Event-triggered KVM Nodes
Venue: arXiv cs.OS 2023
Year: 2023

Industries are considering the adoption of cloud and edge computing for real-time applications due to current improvements in network latencies and the advent of Fog and Edge computing. Current cloud paradigms are not designed for real-time applications, as they neither provide low latencies/jitter nor the guarantees and determinism required by real-time applications. Experts estimate that data centers use 1% of global electricity for powering the equipment, and in turn, for dealing with the produced heat. Hence, energy consumption is a crucial metric in cloud technologies. Applying energy conservation techniques is not straightforward due to the increased scheduling overheads and application execution times. Inspired by slot shifting, we propose an algorithm to support energy-aware time-triggered execution of periodic real-time VMs while still providing the ability to execute aperiodic real-time and best-effort VMs in the slack of the time-triggered ones. The algorithm considers energy reduction techniques based on dynamic power management and dynamic voltage and frequency scaling. We implement our algorithm as an extension to the Linux kernel scheduler (for use with the KVM hypervisor) and evaluate it on a server-grade Intel Xeon node.
A Cross-Chain Query Language for Application-Level Interoperability Between Open and Permissionless Blockchains
Venue: arXiv cs.DC 2023
Year: 2023

Open and permissionless blockchains are distributed systems with thousands to tens of thousands of nodes, establishing novel platforms for decentralized applications. When realizing such an application, data might be stored and retrieved from one or more blockchains by distributed network nodes without relying on centralized coordination and trusted third parties. Data access could be provided through a query language such as SQL at the application level, establishing a unified view on application-level data that is verifiably stored. However, when accessing multiple blockchains through their node software and APIs, interoperability cannot be assumed today, resulting in challenges of inhomogeneous data access. In addition, different feature sets and trade-offs exist, e.g., regarding smart contract functionality, availability, distribution, scalability, and security. For increasing interoperability, the paper at hand suggests pursuing the development of a cross-chain query language at the application level. The language abstracts from implementation by providing a standardized syntax, an integrated data model, and a processing architecture for data queries. This research is an extended and updated paper of a prior publication demonstrating the language syntax, data model, and architecture with an evaluation of compatibility against the largest open and permissionless blockchains today.
OpenClinicalAI: An Open and Dynamic Model for Alzheimer's Disease Diagnosis
Venue: arXiv cs.LG 2023
Year: 2023

Although Alzheimer's disease (AD) cannot be reversed or cured, timely diagnosis can significantly reduce the burden of treatment and care. Current research on AD diagnosis models usually regards the diagnosis task as a typical classification task with two primary assumptions: 1) All target categories are known a priori; 2) The diagnostic strategy for each patient is consistent, that is, the number and type of model input data for each patient are the same. However, real-world clinical settings are open, with complexity and uncertainty in terms of both subjects and the resources of the medical institutions. This means that diagnostic models may encounter unseen disease categories and need to dynamically develop diagnostic strategies based on the subject's specific circumstances and available medical resources. Thus, the AD diagnosis task is tangled and coupled with the diagnosis strategy formulation. To promote the application of diagnostic systems in real-world clinical settings, we propose OpenClinicalAI for direct AD diagnosis in complex and uncertain clinical settings. This is the first powerful end-to-end model to dynamically formulate diagnostic strategies and provide diagnostic results based on the subject's conditions and available medical resources. OpenClinicalAI combines reciprocally coupled deep multiaction reinforcement learning (DMARL) for diagnostic strategy formulation and multicenter meta-learning (MCML) for open-set recognition. The experimental results show that OpenClinicalAI achieves better performance and fewer clinical examinations than the state-of-the-art model. Our method provides an opportunity to embed the AD diagnostic system into the current health care system to cooperate with clinicians to improve current health care.