OpenRaiser/Intern-Atlas huggingface dataset:
MorpheuS: Neural Dynamic 360° Surface Reconstruction from Monocular RGB-D Video
Venue: arXiv cs.CV 2024
Year: 2024

Neural rendering has demonstrated remarkable success in dynamic scene reconstruction. Thanks to the expressiveness of neural representations, prior works can accurately capture the motion and achieve high-fidelity reconstruction of the target object. Despite this, real-world video scenarios often feature large unobserved regions where neural representations struggle to achieve realistic completion. To tackle this challenge, we introduce MorpheuS, a framework for dynamic 360° surface reconstruction from a casually captured RGB-D video. Our approach models the target scene as a canonical field that encodes its geometry and appearance, in conjunction with a deformation field that warps points from the current frame to the canonical space. We leverage a view-dependent diffusion prior and distill knowledge from it to achieve realistic completion of unobserved regions. Experimental results on various real-world and synthetic datasets show that our method can achieve high-fidelity 360° surface reconstruction of a deformable object from a monocular RGB-D video.
AV4EV: Open-Source Modular Autonomous Electric Vehicle Platform for Making Mobility Research Accessible
Venue: arXiv cs.RO 2024
Year: 2024

When academic researchers develop and validate autonomous driving algorithms, there is a challenge in balancing high-performance capabilities with the cost and complexity of the vehicle platform. Much of today's research on autonomous vehicles (AV) is limited to experimentation on expensive commercial vehicles that require large skilled teams to retrofit the vehicles and test them in dedicated facilities. On the other hand, 1/10th-1/16th scaled-down vehicle platforms are more affordable but have limited similitude in performance and drivability. To address this issue, we present the design of a one-third-scale autonomous electric go-kart platform with open-source mechatronics design along with fully functional autonomous driving software. The platform's multi-modal driving system is capable of manual, autonomous, and teleoperation driving modes. It also features a flexible sensing suite for the algorithm deployment across perception, localization, planning, and control. This development serves as a bridge between full-scale vehicles and reduced-scale cars while accelerating cost-effective algorithmic advancements. Our experimental results demonstrate the AV4EV platform's capabilities and ease of use for developing new AV algorithms. All materials are available at AV4EV.org to stimulate collaborative efforts within the AV and electric vehicle (EV) communities.
Harnessing Discrete Representations For Continual Reinforcement Learning
Venue: arXiv cs.LG 2024
Year: 2024

Reinforcement learning (RL) agents make decisions using nothing but observations from the environment, and consequently, heavily rely on the representations of those observations. Though some recent breakthroughs have used vector-based categorical representations of observations, often referred to as discrete representations, there is little work explicitly assessing the significance of such a choice. In this work, we provide a thorough empirical investigation of the advantages of representing observations as vectors of categorical values within the context of reinforcement learning. We perform evaluations on world-model learning, model-free RL, and ultimately continual RL problems, where the benefits best align with the needs of the problem setting. We find that, when compared to traditional continuous representations, world models learned over discrete representations accurately model more of the world with less capacity, and that agents trained with discrete representations learn better policies with less data. In the context of continual RL, these benefits translate into faster adapting agents. Additionally, our analysis suggests that the observed performance improvements can be attributed to the information contained within the latent vectors and potentially the encoding of the discrete representation itself.
MABViT -- Modified Attention Block Enhances Vision Transformers
Venue: arXiv cs.CV 2024
Year: 2024

Recent studies have demonstrated the effectiveness of Gated Linear Units (GLU) in enhancing transformer models, particularly in Large Language Models (LLMs). Additionally, utilizing a parallel configuration within each Transformer block rather than the conventional serialized method has been revealed to accelerate the training of LLMs without significantly impacting performance. However, when the MLP and attention block were run in parallel for the image classification task, we observed a noticeable decline in performance. We propose a novel transformer variant that integrates non-linearity within the attention block to tackle this problem. We implemented the GLU-based activation function on the Value tensor, and this new technique surpasses the current state-of-the-art S/16 variant of Vision Transformers by 0.6% on the ImageNet-1K dataset while utilizing fewer parameters. It also supersedes the B/16 variant while using only half the parameters. Furthermore, we provide results with the GELU activation function variant to confirm our assertions. Lastly, we showcase that the MABViT variants exhibit greater potential when utilized in deep transformers compared to the standard architecture.
Entropy and the Kullback-Leibler Divergence for Bayesian Networks: Computational Complexity and Efficient Implementation
Venue: arXiv cs.AI 2024
Year: 2024

Bayesian networks (BNs) are a foundational model in machine learning and causal inference. Their graphical structure can handle high-dimensional problems, divide them into a sparse collection of smaller ones, underlies Judea Pearl's causality, and determines their explainability and interpretability. Despite their popularity, there are almost no resources in the literature on how to compute Shannon's entropy and the Kullback-Leibler (KL) divergence for BNs under their most common distributional assumptions. In this paper, we provide computationally efficient algorithms for both by leveraging BNs' graphical structure, and we illustrate them with a complete set of numerical examples. In the process, we show it is possible to reduce the computational complexity of KL from cubic to quadratic for Gaussian BNs.
Torso-Based Control Interface for Standing Mobility-Assistive Devices
Venue: arXiv cs.RO 2024
Year: 2024

Wheelchairs and mobility devices have transformed our bodies into cybernic systems, enhancing our well-being by enabling individuals with reduced mobility to regain freedom. Notwithstanding, current interfaces of control primarily rely on hand operation, therefore constraining the user from performing functional activities of daily living. In this work, we propose a design of a torso-based control interface with compliant coupling support for standing mobility assistive devices. We consider the coupling between the human and robot in the interface design. The design includes a compliant support mechanism and mapping between the body movement space and the velocity space. We present experiments including multiple conditions, with a joystick for comparison with the proposed torso control interface. The results of a path-following experiment demonstrated that users could control the device naturally using the hands-free interface, and the performance was comparable with the joystick, with 10% more consumed time, an average cross error of 0.116 m and 4.9% less average acceleration. In an object-transferring experiment, the proposed interface demonstrated a clear advantage when users needed to manipulate objects during locomotion. Lastly, the torso control scored 15% less than the joystick on the system usability scale for the path-following task but 3.3% more for the object-transferring task.
How Many Validation Labels Do You Need? Exploring the Design Space of Label-Efficient Model Ranking
Venue: arXiv cs.LG 2024
Year: 2024

This paper presents LEMR (Label-Efficient Model Ranking) and introduces the MoraBench Benchmark. LEMR is a novel framework that minimizes the need for costly annotations in model selection by strategically annotating instances from an unlabeled validation set. To evaluate LEMR, we leverage the MoraBench Benchmark, a comprehensive collection of model outputs across diverse scenarios. Our extensive evaluation across 23 different NLP tasks in semi-supervised learning, weak supervision, and prompt selection tasks demonstrates LEMR's effectiveness in significantly reducing labeling costs. Key findings highlight the impact of suitable ensemble methods, uncertainty sampling strategies, and model committee selection in enhancing model ranking accuracy. LEMR, supported by the insights from MoraBench, provides a cost-effective and accurate solution for model selection, especially valuable in resource-constrained environments.
Effective Adapter for Face Recognition in the Wild
Venue: arXiv cs.CV 2024
Year: 2024

In this paper, we tackle the challenge of face recognition in the wild, where images often suffer from low quality and real-world distortions. Traditional heuristic approaches-either training models directly on these degraded images or their enhanced counterparts using face restoration techniques-have proven ineffective, primarily due to the degradation of facial features and the discrepancy in image domains. To overcome these issues, we propose an effective adapter for augmenting existing face recognition models trained on high-quality facial datasets. The key of our adapter is to process both the unrefined and enhanced images using two similar structures, one fixed and the other trainable. Such design can confer two benefits. First, the dual-input system minimizes the domain gap while providing varied perspectives for the face recognition model, where the enhanced image can be regarded as a complex non-linear transformation of the original one by the restoration model. Second, both two similar structures can be initialized by the pre-trained models without dropping the past knowledge. The extensive experiments in zero-shot settings show the effectiveness of our method by surpassing baselines of about 3%, 4%, and 7% in three datasets. Our code will be publicly available.
Minimizing Maximum Dissatisfaction in the Allocation of Indivisible Items under a Common Preference Graph
Venue: arXiv cs.DM 2024
Year: 2024

We consider the task of allocating indivisible items to agents, when the agents' preferences over the items are identical. The preferences are captured by means of a directed acyclic graph, with vertices representing items and an edge $(a,b)$, meaning that each of the agents prefers item $a$ over item $b$. The dissatisfaction of an agent is measured by the number of items that the agent does not receive and for which it also does not receive any more preferred item. The aim is to allocate the items to the agents in a fair way, i.e., to minimize the maximum dissatisfaction among the agents. We study the status of computational complexity of that problem and establish the following dichotomy: the problem is NP-hard for the case of at least three agents, even on fairly restricted graphs, but polynomially solvable for two agents. We also provide several polynomial-time results with respect to different underlying graph structures, such as graphs of width at most two and tree-like structures such as stars and matchings. These findings are complemented with fixed parameter tractability results related to path modules and independent set modules. Techniques employed in the paper include bottleneck assignment problem, greedy algorithm, dynamic programming, maximum network flow, and integer linear programming.
Speed and Performance of Parserless and Unsupervised Anomaly Detection Methods on Software Logs
Venue: arXiv cs.SE 2024
Year: 2024

Software log analysis can be laborious and time consuming. Time and labeled data are usually lacking in industrial settings. This paper studies unsupervised and time efficient methods for anomaly detection. We study two custom and two established models. The custom models are: an OOV (Out-Of-Vocabulary) detector, which counts the terms in the test data that are not present in the training data, and the Rarity Model (RM), which calculates a rarity score for terms based on their infrequency. The established models are KMeans and Isolation Forest. The models are evaluated on four public datasets (BGL, Thunderbird, Hadoop, HDFS) with three different representation techniques for the log messages (Words, character Trigrams, Parsed events). For training, we used both normal-only data, which is free of all anomalies, and unfiltered data, which contains both normal and anomalous instances. We used primarily the AUC-ROC metric for evaluation due to challenges in setting a threshold but we also include F1-scores for further insight. Different configurations are advised based on specific requirements. When training data is unfiltered, includes both normal and anomalous instances, the most effective combination is the Isolation Forest with event representation, achieving an AUC-ROC of 0.829. If it's possible to create a normal-only training dataset, combining the Out-Of-Vocabulary (OOV) detector with trigram representation yields the highest AUC-ROC of 0.846. For speed considerations, the OOV detector is optimal for filtered data, while the Rarity Model is the best choice for unfiltered data.
A Framework for Symmetric Self-Intersecting Surfaces
Venue: arXiv cs.CG 2024
Year: 2024

3D printing of surfaces has become an established method for prototyping and visualisation. However, surfaces often contain certain degenerations, such as self-intersecting faces or non-manifold parts, which pose problems in obtaining a 3D printable file. Therefore, it is necessary to examine these degenerations beforehand. Surfaces in three-dimensional space can be represented as embedded simplicial complexes describing a triangulation of the surface. We use this combinatorial description, and the notion of embedded simplicial surfaces (which can be understood as well-behaved surfaces) to give a framework for obtaining 3D printable files. This provides a new perspective on self-intersecting triangulated surfaces in three-dimensional space. Our method first retriangulates a surface using a minimal number of triangles, then computes its outer hull, and finally treats non-manifold parts. To this end, we prove an initialisation criterion for the computation of the outer hull. We also show how symmetry properties can be used to simplify computations. Implementations of the proposed algorithms are given in the computer algebra system GAP4. To verify our methods, we use a dataset of self-intersecting symmetric icosahedra. Exploiting the symmetry of the underlying embedded complex leads to a notable speed-up and enhanced numerical robustness when computing a retriangulation, compared to methods that do not take advantage of symmetry.
Wavefront Transformation-based Near-field Channel Prediction for Extremely Large Antenna Array with Mobility
Venue: arXiv cs.IT 2024
Year: 2024

This paper addresses the mobility problem in extremely large antenna array (ELAA) communication systems. In order to account for the performance loss caused by the spherical wavefront of ELAA in the mobility scenario, we propose a wavefront transformation-based matrix pencil (WTMP) channel prediction method. In particular, we design a matrix to transform the spherical wavefront into a new wavefront, which is closer to the plane wave. We also design a time-frequency projection matrix to capture the time-varying path delay. Furthermore, we adopt the matrix pencil (MP) method to estimate channel parameters. Our proposed WTMP method can mitigate the effect of near-field radiation when predicting future channels. Theoretical analysis shows that the designed matrix is asymptotically determined by the angles and distance between the base station (BS) antenna array and the scatterers or the user when the number of BS antennas is large enough. For an ELAA communication system in the mobility scenario, we prove that the prediction error converges to zero with the increasing number of BS antennas. Simulation results demonstrate that our designed transform matrix efficiently mitigates the near-field effect, and that our proposed WTMP method can overcome the ELAA mobility challenge and approach the performance in stationary setting.
InvertAvatar: Incremental GAN Inversion for Generalized Head Avatars
Venue: arXiv cs.CV 2024
Year: 2024

While high fidelity and efficiency are central to the creation of digital head avatars, recent methods relying on 2D or 3D generative models often experience limitations such as shape distortion, expression inaccuracy, and identity flickering. Additionally, existing one-shot inversion techniques fail to fully leverage multiple input images for detailed feature extraction. We propose a novel framework, \textbf{Incremental 3D GAN Inversion}, that enhances avatar reconstruction performance using an algorithm designed to increase the fidelity from multiple frames, resulting in improved reconstruction quality proportional to frame count. Our method introduces a unique animatable 3D GAN prior with two crucial modifications for enhanced expression controllability alongside an innovative neural texture encoder that categorizes texture feature spaces based on UV parameterization. Differentiating from traditional techniques, our architecture emphasizes pixel-aligned image-to-image translation, mitigating the need to learn correspondences between observation and canonical spaces. Furthermore, we incorporate ConvGRU-based recurrent networks for temporal data aggregation from multiple frames, boosting geometry and texture detail reconstruction. The proposed paradigm demonstrates state-of-the-art performance on one-shot and few-shot avatar animation tasks. Code will be available at https://github.com/XChenZ/invertAvatar.
Re-Nerfing: Improving Novel View Synthesis through Novel View Synthesis
Venue: arXiv cs.CV 2024
Year: 2024

Recent neural rendering and reconstruction techniques, such as NeRFs or Gaussian Splatting, have shown remarkable novel view synthesis capabilities but require hundreds of images of the scene from diverse viewpoints to render high-quality novel views. With fewer images available, these methods start to fail since they can no longer correctly triangulate the underlying 3D geometry and converge to a non-optimal solution. These failures can manifest as floaters or blurry renderings in sparsely observed areas of the scene. In this paper, we propose Re-Nerfing, a simple and general add-on approach that leverages novel view synthesis itself to tackle this problem. Using an already trained NVS method, we render novel views between existing ones and augment the training data to optimize a second model. This introduces additional multi-view constraints and allows the second model to converge to a better solution. With Re-Nerfing we achieve significant improvements upon multiple pipelines based on NeRF and Gaussian-Splatting in sparse view settings of the mip-NeRF 360 and LLFF datasets. Notably, Re-Nerfing does not require prior knowledge or extra supervision signals, making it a flexible and practical add-on.
FaultFormer: Pretraining Transformers for Adaptable Bearing Fault Classification
Venue: arXiv cs.LG 2024
Year: 2024

The growth of global consumption has motivated important applications of deep learning to smart manufacturing and machine health monitoring. In particular, analyzing vibration data offers great potential to extract meaningful insights into predictive maintenance by the detection of bearing faults. Deep learning can be a powerful method to predict these mechanical failures; however, they lack generalizability to new tasks or datasets and require expensive, labeled mechanical data. We address this by presenting a novel self-supervised pretraining and fine-tuning framework based on transformer models. In particular, we investigate different tokenization and data augmentation strategies to reach state-of-the-art accuracies using transformer models. Furthermore, we demonstrate self-supervised masked pretraining for vibration signals and its application to low-data regimes, task adaptation, and dataset adaptation. Pretraining is able to improve performance on scarce, unseen training samples, as well as when fine-tuning on fault classes outside of the pretraining distribution. Furthermore, pretrained transformers are shown to be able to generalize to a different dataset in a few-shot manner. This introduces a new paradigm where models can be pretrained on unlabeled data from different bearings, faults, and machinery and quickly deployed to new, data-scarce applications to suit specific manufacturing needs.
Flexible Communication for Optimal Distributed Learning over Unpredictable Networks
Venue: arXiv cs.DC 2024
Year: 2024

Gradient compression alleviates expensive communication in distributed deep learning by sending fewer values and its corresponding indices, typically via Allgather (AG). Training with high compression ratio (CR) achieves high accuracy like DenseSGD, but has lower parallel scaling due to high communication cost (i.e., parallel efficiency). Using lower CRs improves parallel efficiency by lowering synchronization cost, but degrades model accuracy as well (statistical efficiency). Further, speedup attained with different models and CRs also varies with network latency, effective bandwidth and collective op used for aggregation. In many cases, collectives like Allreduce (AR) have lower cost than AG to exchange the same amount of data. In this paper, we propose an AR-compatible Topk compressor that is bandwidth-optimal and thus performs better than AG in certain network configurations. We develop a flexible communication strategy that switches between AG and AR based on which collective is optimal in the current settings, and model the pareto-relationship between parallel and statistical efficiency as a multi-objective optimization (MOO) problem to dynamically adjust CR and accelerate training while still converging to high accuracy.
ULMA: Unified Language Model Alignment with Human Demonstration and Point-wise Preference
Venue: arXiv cs.LG 2024
Year: 2024

Aligning language models to human expectations, e.g., being helpful and harmless, has become a pressing challenge for large language models. A typical alignment procedure consists of supervised fine-tuning and preference learning. Most preference learning methods, such as RLHF and DPO, depend on pairwise preference data, which inadequately address scenarios where human feedback is point-wise, leading to potential information loss and suboptimal performance. Addressing this gap, we introduce Point-wise Direct Preference Optimization, a novel preference learning method designed to harness point-wise feedback effectively. Our work also uncovers a novel connection between supervised fine-tuning and point-wise preference learning, culminating in Unified Language Model Alignment, a single-step method that unifies the alignment with human demonstrations and point-wise preferences. Extensive experiments on point-wise preference datasets with binary or continuous labels validate the effectiveness of our methods. Our code and a new dataset with high-quality demonstration samples on harmlessness are released.