OpenRaiser/Intern-Atlas huggingface dataset:
Conformal Disentanglement: A Neural Framework for Perspective Synthesis and Differentiation
Venue: arXiv cs.LG 2024
Year: 2024

For multiple scientific endeavors it is common to measure a phenomenon of interest in more than one ways. We make observations of objects from several different perspectives in space, at different points in time; we may also measure different properties of a mixture using different types of instruments. After collecting this heterogeneous information, it is necessary to be able to synthesize a complete picture of what is `common' across its sources: the subject we ultimately want to study. However, isolated (`clean') observations of a system are not always possible: observations often contain information about other systems in its environment, or about the measuring instruments themselves. In that sense, each observation may contain information that `does not matter' to the original object of study; this `uncommon' information between sensors observing the same object may still be important, and decoupling it from the main signal(s) useful. We introduce a neural network autoencoder framework capable of both tasks: it is structured to identify `common' variables, and, making use of orthogonality constraints to define geometric independence, to also identify disentangled `uncommon' information originating from the heterogeneous sensors. We demonstrate applications in several computational examples.
Handling Geometric Domain Shifts in Semantic Segmentation of Surgical RGB and Hyperspectral Images
Venue: arXiv cs.CV 2024
Year: 2024

Robust semantic segmentation of intraoperative image data holds promise for enabling automatic surgical scene understanding and autonomous robotic surgery. While model development and validation are primarily conducted on idealistic scenes, geometric domain shifts, such as occlusions of the situs, are common in real-world open surgeries. To close this gap, we (1) present the first analysis of state-of-the-art (SOA) semantic segmentation models when faced with geometric out-of-distribution (OOD) data, and (2) propose an augmentation technique called "Organ Transplantation", to enhance generalizability. Our comprehensive validation on six different OOD datasets, comprising 600 RGB and hyperspectral imaging (HSI) cubes from 33 pigs, each annotated with 19 classes, reveals a large performance drop in SOA organ segmentation models on geometric OOD data. This performance decline is observed not only in conventional RGB data (with a dice similarity coefficient (DSC) drop of 46 %) but also in HSI data (with a DSC drop of 45 %), despite the richer spectral information content. The performance decline increases with the spatial granularity of the input data. Our augmentation technique improves SOA model performance by up to 67 % for RGB data and 90 % for HSI data, achieving performance at the level of in-distribution performance on real OOD test data. Given the simplicity and effectiveness of our augmentation method, it is a valuable tool for addressing geometric domain shifts in surgical scene segmentation, regardless of the underlying model. Our code and pre-trained models are publicly available at https://github.com/IMSY-DKFZ/htc.
Analysis of the Performance of the Matrix Multiplication Algorithm on the Cirrus Supercomputer
Venue: arXiv cs.DC 2024
Year: 2024

Matrix multiplication is integral to various scientific and engineering disciplines, including machine learning, image processing, and gaming. With the increasing data volumes in areas like machine learning, the demand for efficient parallel processing of large matrices has grown significantly.This study explores the performance of both serial and parallel matrix multiplication on the Cirrus supercomputer at the University of Edinburgh. The results demonstrate the scalability and efficiency of these methods, providing insights for optimizing matrixmultiplication in real-world applications.
Evaluating Pre-Training Bias on Severe Acute Respiratory Syndrome Dataset
Venue: arXiv cs.LG 2024
Year: 2024

Machine learning (ML) is a growing field of computer science that has found many practical applications in several domains, including Health. However, as data grows in size and availability, and the number of models that aim to aid or replace human decisions, it raises the concern that these models can be susceptible to bias, which can lead to harm to specific individuals by basing its decisions on protected attributes such as gender, religion, sexual orientation, ethnicity, and others. Visualization techniques might generate insights and help summarize large datasets, enabling data scientists to understand the data better before training a model by evaluating pre-training metrics applied to the datasets before training, which might contribute to identifying potential harm before any effort is put into training and deploying the models. This work uses the severe acute respiratory syndrome dataset from OpenDataSUS to visualize three pre-training bias metrics and their distribution across different regions in Brazil. A random forest model is trained in each region and applied to the others. The aim is to compare the bias for the different regions, focusing on their protected attributes and comparing the model's performance with the metric values.
Weighted Matching in the Random-Order Streaming and Robust Communication Models
Venue: arXiv cs.DS 2024
Year: 2024

We study the maximum weight matching problem in the random-order semi-streaming model and in the robust communication model. Unlike many other sublinear models, in these two frameworks, there is a large gap between the guarantees of the best known algorithms for the unweighted and weighted versions of the problem. In the random-order semi-streaming setting, the edges of an $n$-vertex graph arrive in a stream in a random order. The goal is to compute an approximate maximum weight matching with a single pass over the stream using $O(n\text{ polylog } n)$ space. Our main result is a $(2/3-ε)$-approximation algorithm for maximum weight matching in random-order streams, using space $O(n \log n \log R)$, where $R$ is the ratio between the heaviest and the lightest edge in the graph. Our result nearly matches the best known unweighted $(2/3+ε_0)$-approximation (where $ε_0 \sim 10^{-14}$ is a small constant) achieved by Assadi and Behnezhad [ICALP 2021], and significantly improves upon previous weighted results. Our techniques also extend to the related robust communication model, in which the edges of a graph are partitioned randomly between Alice and Bob. Alice sends a single message of size $O(n\text{ polylog }n)$ to Bob, who must compute an approximate maximum weight matching. We achieve a $(5/6-ε)$-approximation using $O(n \log n \log R)$ words of communication, matching the results of Azarmehr and Behnezhad [ICALP 2023] for unweighted graphs.
Towards observability of scientific applications
Venue: arXiv cs.DC 2024
Year: 2024

As software systems increase in complexity, conventional monitoring methods struggle to provide a comprehensive overview or identify performance issues, often missing unexpected problems. Observability, however, offers a holistic approach, providing methods and tools that gather and analyze detailed telemetry data to uncover hidden issues. Originally developed for cloud-native systems, modern observability is less prevalent in scientific computing, particularly in HPC clusters, due to differences in application architecture, execution environments, and technology stacks. This paper proposes and evaluates an end-to-end observability solution tailored for scientific computing in HPC environments. We address several challenges, including collection of application-level metrics, instrumentation, context propagation, and tracing. We argue that typical dashboards with charts are not sufficient for advanced observability-driven analysis of scientific applications. Consequently, we propose a different approach based on data analysis using DataFrames and a Jupyter environment. The proposed solution is implemented and evaluated on two medical scientific pipelines running on an HPC cluster.
Continual-learning-based framework for structural damage recognition
Venue: arXiv cs.CV 2024
Year: 2024

Multi-damage is common in reinforced concrete structures and leads to the requirement of large number of neural networks, parameters and data storage, if convolutional neural network (CNN) is used for damage recognition. In addition, conventional CNN experiences catastrophic forgetting and training inefficiency as the number of tasks increases during continual learning, leading to large accuracy decrease of previous learned tasks. To address these problems, this study proposes a continuallearning-based damage recognition model (CLDRM) which integrates the learning without forgetting continual learning method into the ResNet-34 architecture for the recognition of damages in RC structures as well as relevant structural components. Three experiments for four recognition tasks were designed to validate the feasibility and effectiveness of the CLDRM framework. In this way, it reduces both the prediction time and data storage by about 75% in four tasks of continuous learning. Three experiments for four recognition tasks were designed to validate the feasibility and effectiveness of the CLDRM framework. By gradual feature fusion, CLDRM outperformed other methods by managed to achieve high accuracy in the damage recognition and classification. As the number of recognition tasks increased, CLDRM also experienced smaller decrease of the previous learned tasks. Results indicate that the CLDRM framework successfully performs damage recognition and classification with reasonable accuracy and effectiveness.
Predicting Parameter Change's Effect on Cellular Network Time Series
Venue: arXiv cs.NI 2024
Year: 2024

The cellular network provides convenient network access for ever-growing mobile phones. During the continuous optimization, operators can adjust cell parameters to enhance the Quality of Service (QoS) flexibly. A precise prediction of the parameter change's effect can help operators make proper parameter adjustments. This work focuses on predicting cell status (like the workload and QoS) after adjusting the cell parameters. The prediction will be conducted before an adjustment is actually applied to provide an early inspection. As it can be hard for available parameter adjustments with a limited number to cover all the parameter and user behavior combinations, we propose ParaSeer fusing domain knowledge on parameter adjustments into data-driven time series forecasting. ParaSeer organizes several pre-trained Transformers for adjustment-free time series forecasting, utilizing plenty of adjustment-free data. On the other hand, ParaSeer models the effect of adjusting the transmission power and cell individual offset (CIO) as a multiplier for the workload. We derive a formula to calculate the multiplier from the underlying mechanism of those two parameters, helping ParaSeer eliminate the thirst for data with parameter adjustments. We compare ParaSeer with baselines on two real-world datasets, where ParaSeer outperforms the best baseline by more than 25.8% in terms of RMSE. The extensive experiments further illustrate the contributions of ParaSeer's components.
Order-preserving pattern mining with forgetting mechanism
Venue: arXiv cs.DB 2024
Year: 2024

Order-preserving pattern (OPP) mining is a type of sequential pattern mining method in which a group of ranks of time series is used to represent an OPP. This approach can discover frequent trends in time series. Existing OPP mining algorithms consider data points at different time to be equally important; however, newer data usually have a more significant impact, while older data have a weaker impact. We therefore introduce the forgetting mechanism into OPP mining to reduce the importance of older data. This paper explores the mining of OPPs with forgetting mechanism (OPF) and proposes an algorithm called OPF-Miner that can discover frequent OPFs. OPF-Miner performs two tasks, candidate pattern generation and support calculation. In candidate pattern generation, OPF-Miner employs a maximal support priority strategy and a group pattern fusion strategy to avoid redundant pattern fusions. For support calculation, we propose an algorithm called support calculation with forgetting mechanism, which uses prefix and suffix pattern pruning strategies to avoid redundant support calculations. The experiments are conducted on nine datasets and 12 alternative algorithms. The results verify that OPF-Miner is superior to other competitive algorithms. More importantly, OPF-Miner yields good clustering performance for time series, since the forgetting mechanism is employed.
Skills Regularized Task Decomposition for Multi-task Offline Reinforcement Learning
Venue: arXiv cs.LG 2024
Year: 2024

Reinforcement learning (RL) with diverse offline datasets can have the advantage of leveraging the relation of multiple tasks and the common skills learned across those tasks, hence allowing us to deal with real-world complex problems efficiently in a data-driven way. In offline RL where only offline data is used and online interaction with the environment is restricted, it is yet difficult to achieve the optimal policy for multiple tasks, especially when the data quality varies for the tasks. In this paper, we present a skill-based multi-task RL technique on heterogeneous datasets that are generated by behavior policies of different quality. To learn the shareable knowledge across those datasets effectively, we employ a task decomposition method for which common skills are jointly learned and used as guidance to reformulate a task in shared and achievable subtasks. In this joint learning, we use Wasserstein auto-encoder (WAE) to represent both skills and tasks on the same latent space and use the quality-weighted loss as a regularization term to induce tasks to be decomposed into subtasks that are more consistent with high-quality skills than others. To improve the performance of offline RL agents learned on the latent space, we also augment datasets with imaginary trajectories relevant to high-quality skills for each task. Through experiments, we show that our multi-task offline RL approach is robust to the mixed configurations of different-quality datasets and it outperforms other state-of-the-art algorithms for several robotic manipulation tasks and drone navigation tasks.
ES-PTAM: Event-based Stereo Parallel Tracking and Mapping
Venue: arXiv cs.RO 2024
Year: 2024

Visual Odometry (VO) and SLAM are fundamental components for spatial perception in mobile robots. Despite enormous progress in the field, current VO/SLAM systems are limited by their sensors' capability. Event cameras are novel visual sensors that offer advantages to overcome the limitations of standard cameras, enabling robots to expand their operating range to challenging scenarios, such as high-speed motion and high dynamic range illumination. We propose a novel event-based stereo VO system by combining two ideas: a correspondence-free mapping module that estimates depth by maximizing ray density fusion and a tracking module that estimates camera poses by maximizing edge-map alignment. We evaluate the system comprehensively on five real-world datasets, spanning a variety of camera types (manufacturers and spatial resolutions) and scenarios (driving, flying drone, hand-held, egocentric, etc). The quantitative and qualitative results demonstrate that our method outperforms the state of the art in majority of the test sequences by a margin, e.g., trajectory error reduction of 45% on RPG dataset, 61% on DSEC dataset, and 21% on TUM-VIE dataset. To benefit the community and foster research on event-based perception systems, we release the source code and results: https://github.com/tub-rip/ES-PTAM
Geometry-guided Feature Learning and Fusion for Indoor Scene Reconstruction
Venue: arXiv cs.CV 2024
Year: 2024

In addition to color and textural information, geometry provides important cues for 3D scene reconstruction. However, current reconstruction methods only include geometry at the feature level thus not fully exploiting the geometric information. In contrast, this paper proposes a novel geometry integration mechanism for 3D scene reconstruction. Our approach incorporates 3D geometry at three levels, i.e. feature learning, feature fusion, and network supervision. First, geometry-guided feature learning encodes geometric priors to contain view-dependent information. Second, a geometry-guided adaptive feature fusion is introduced which utilizes the geometric priors as a guidance to adaptively generate weights for multiple views. Third, at the supervision level, taking the consistency between 2D and 3D normals into account, a consistent 3D normal loss is designed to add local constraints. Large-scale experiments are conducted on the ScanNet dataset, showing that volumetric methods with our geometry integration mechanism outperform state-of-the-art methods quantitatively as well as qualitatively. Volumetric methods with ours also show good generalization on the 7-Scenes and TUM RGB-D datasets.
CSAD: Unsupervised Component Segmentation for Logical Anomaly Detection
Venue: arXiv cs.CV 2024
Year: 2024

To improve logical anomaly detection, some previous works have integrated segmentation techniques with conventional anomaly detection methods. Although these methods are effective, they frequently lead to unsatisfactory segmentation results and require manual annotations. To address these drawbacks, we develop an unsupervised component segmentation technique that leverages foundation models to autonomously generate training labels for a lightweight segmentation network without human labeling. Integrating this new segmentation technique with our proposed Patch Histogram module and the Local-Global Student-Teacher (LGST) module, we achieve a detection AUROC of 95.3% in the MVTec LOCO AD dataset, which surpasses previous SOTA methods. Furthermore, our proposed method provides lower latency and higher throughput than most existing approaches.
Omega: The Power of Visual Simplicity
Venue: arXiv cs.PL 2024
Year: 2024

We are currently developing an innovative and visually-driven programming language called Omega.Although the Omega code is stored in text files, these files are not intended for manual editing or traditional printing.Furthermore, parsing these files using a context-free grammar is not possible.The parsing of the code and the facilitation of user-friendly manual editing both necessitate a global knowledge of the codebase.Strictly speaking, code visualization is not an integral part of the Omega language; instead, this task is delegated to the editing tools.Thanks to the global knowledge of the code, the editing process becomes remarkably straightforward, with numerous automatic completion features that enhance usability.Omega leverages a visual-oriented approach to encompass all fundamental aspects of software engineering.It offers robust features, including safe static typing, design by contracts, rules for accessing slots, operator definitions, and more,all presented in an intuitively and visually comprehensible manner, eliminating the need for obscure syntax.
Can SAR improve RSVQA performance?
Venue: arXiv cs.CV 2024
Year: 2024

Remote sensing visual question answering (RSVQA) has been involved in several research in recent years, leading to an increase in new methods. RSVQA automatically extracts information from satellite images, so far only optical, and a question to automatically search for the answer in the image and provide it in a textual form. In our research, we study whether Synthetic Aperture Radar (SAR) images can be beneficial to this field. We divide our study into three phases which include classification methods and VQA. In the first one, we explore the classification results of SAR alone and investigate the best method to extract information from SAR data. Then, we study the combination of SAR and optical data. In the last phase, we investigate how SAR images and a combination of different modalities behave in RSVQA compared to a method only using optical images. We conclude that adding the SAR modality leads to improved performances, although further research on using SAR data to automatically answer questions is needed as well as more balanced datasets.
RIDE: Boosting 3D Object Detection for LiDAR Point Clouds via Rotation-Invariant Analysis
Venue: arXiv cs.CV 2024
Year: 2024

The rotation robustness property has drawn much attention to point cloud analysis, whereas it still poses a critical challenge in 3D object detection. When subjected to arbitrary rotation, most existing detectors fail to produce expected outputs due to the poor rotation robustness. In this paper, we present RIDE, a pioneering exploration of Rotation-Invariance for the 3D LiDAR-point-based object DEtector, with the key idea of designing rotation-invariant features from LiDAR scenes and then effectively incorporating them into existing 3D detectors. Specifically, we design a bi-feature extractor that extracts (i) object-aware features though sensitive to rotation but preserve geometry well, and (ii) rotation-invariant features, which lose geometric information to a certain extent but are robust to rotation. These two kinds of features complement each other to decode 3D proposals that are robust to arbitrary rotations. Particularly, our RIDE is compatible and easy to plug into the existing one-stage and two-stage 3D detectors, and boosts both detection performance and rotation robustness. Extensive experiments on the standard benchmarks showcase that the mean average precision (mAP) and rotation robustness can be significantly boosted by integrating with our RIDE, with +5.6% mAP and 53% rotation robustness improvement on KITTI, +5.1% and 28% improvement correspondingly on nuScenes. The code will be available soon.
NeuroVE: Brain-inspired Linear-Angular Velocity Estimation with Spiking Neural Networks
Venue: arXiv cs.RO 2024
Year: 2024

Vision-based ego-velocity estimation is a fundamental problem in robot state estimation. However, the constraints of frame-based cameras, including motion blur and insufficient frame rates in dynamic settings, readily lead to the failure of conventional velocity estimation techniques. Mammals exhibit a remarkable ability to accurately estimate their ego-velocity during aggressive movement. Hence, integrating this capability into robots shows great promise for addressing these challenges. In this paper, we propose a brain-inspired framework for linear-angular velocity estimation, dubbed NeuroVE. The NeuroVE framework employs an event camera to capture the motion information and implements spiking neural networks (SNNs) to simulate the brain's spatial cells' function for velocity estimation. We formulate the velocity estimation as a time-series forecasting problem. To this end, we design an Astrocyte Leaky Integrate-and-Fire (ALIF) neuron model to encode continuous values. Additionally, we have developed an Astrocyte Spiking Long Short-term Memory (ASLSTM) structure, which significantly improves the time-series forecasting capabilities, enabling an accurate estimate of ego-velocity. Results from both simulation and real-world experiments indicate that NeuroVE has achieved an approximate 60% increase in accuracy compared to other SNN-based approaches.