OpenRaiser/Intern-Atlas huggingface dataset:
Towards the Industrial Metaverse: A Game-Based VR Application for Fire Drill and Evacuation Training for Ships and Shipbuilding
Venue: arXiv cs.HC 2024
Year: 2024

This paper details the creation of a novel Virtual Reality-based application for the Industrial Metaverse aimed at shipboard fire emergency training for fire drill and evacuation, aligned with the Safety of Life at Sea (SOLAS) convention requirements. Specifically, the application includes gamified scenarios with different levels (e.g., varying fire intensities in engine rooms and galleys). The paper details comprehensively the VR development while providing a holistic overview and practical guidelines. Thus, it can guide practitioners, developers and future researchers to shape the next generation of Industrial Metaverse applications for the shipbuilding industry. Moreover, the paper includes the results of a preliminary user evaluation aimed at quantifying user decision-making and risk assessment skills. The presented results of the experiments provide insights into user performance and allow for pointing out at different future challenges.
Masked Autoencoders are Parameter-Efficient Federated Continual Learners
Venue: arXiv cs.CV 2024
Year: 2024

Federated learning is a specific distributed learning paradigm in which a central server aggregates updates from multiple clients' local models, thereby enabling the server to learn without requiring clients to upload their private data, maintaining data privacy. While existing federated learning methods are primarily designed for static data, real-world applications often require clients to learn new categories over time. This challenge necessitates the integration of continual learning techniques, leading to federated continual learning (FCL). To address both catastrophic forgetting and non-IID issues, we propose to use masked autoencoders (MAEs) as parameter-efficient federated continual learners, called pMAE. pMAE learns reconstructive prompt on the client side through image reconstruction using MAE. On the server side, it reconstructs the uploaded restore information to capture the data distribution across previous tasks and different clients, using these reconstructed images to fine-tune discriminative prompt and classifier parameters tailored for classification, thereby alleviating catastrophic forgetting and non-IID issues on a global scale. Experimental results demonstrate that pMAE achieves performance comparable to existing prompt-based methods and can enhance their effectiveness, particularly when using self-supervised pre-trained transformers as the backbone. Code is available at: https://github.com/ycheoo/pMAE.
On SMT Theory Design: The Case of Sequences
Venue: arXiv cs.LO 2024
Year: 2024

Choices in the semantics and the signature of a theory are integral in determining how the theory is used and how challenging it is to reason over it. Our interest in this paper lies in the SMT theory of sequences. Various versions of it exist in the literature and in state-of-the-art SMT solvers, but it has not yet been standardized in the SMT-LIB. We reflect on its existing variants, and we define a set of theory design criteria to help determine what makes one variant of a theory better than another. The criteria we define can be used to appraise theory proposals for other theories as well. Based on these criteria, we propose a series of changes to the SMT theory of sequences as a contribution to the discussion regarding its standardization.
UnSegMedGAT: Unsupervised Medical Image Segmentation using Graph Attention Networks Clustering
Venue: arXiv cs.CV 2024
Year: 2024

The data-intensive nature of supervised classification drives the interest of the researchers towards unsupervised approaches, especially for problems such as medical image segmentation, where labeled data is scarce. Building on the recent advancements of Vision transformers (ViT) in computer vision, we propose an unsupervised segmentation framework using a pre-trained Dino-ViT. In the proposed method, we leverage the inherent graph structure within the image to realize a significant performance gain for segmentation in medical images. For this, we introduce a modularity-based loss function coupled with a Graph Attention Network (GAT) to effectively capture the inherent graph topology within the image. Our method achieves state-of-the-art performance, even significantly surpassing or matching that of existing (semi)supervised technique such as MedSAM which is a Segment Anything Model in medical images. We demonstrate this using two challenging medical image datasets ISIC-2018 and CVC-ColonDB. This work underscores the potential of unsupervised approaches in advancing medical image analysis in scenarios where labeled data is scarce. The github repository of the code is available on [https://github.com/mudit-adityaja/UnSegMedGAT].
A new control- and management architecture for SDN-enabled quantum key distribution networks
Venue: arXiv cs.NI 2024
Year: 2024

This paper aims to address the challenge of designing secure and high performance Quantum Key Distribution Networks (QKDN), which are essential for encrypted communication in the era of quantum computing. Focusing on the control and management (CM) layer essential for monitoring and routing, the study emphasizes centrally managed software defined networks (SDN). We begin by analyzing QKDN routing characteristics needed for evaluating two existed architectures and the proposed, new CM layer implementation. Following the theoretical analysis, we conduct a discrete-event based simulation in which the proposed architecture is compared to an existent serving as performance-baseline. The results provide recommendations based on use cases for which different architectures show superiority and offer valuable insights into the development and evaluation of CM architectures for QKDNs.
Shortcut Learning in In-Context Learning: A Survey
Venue: arXiv cs.CL 2024
Year: 2024

Shortcut learning refers to the phenomenon where models employ simple, non-robust decision rules in practical tasks, which hinders their generalization and robustness. With the rapid development of large language models (LLMs) in recent years, an increasing number of studies have shown the impact of shortcut learning on LLMs. This paper provides a novel perspective to review relevant research on shortcut learning in In-Context Learning (ICL). It conducts a detailed exploration of the types of shortcuts in ICL tasks, their causes, available benchmarks, and strategies for mitigating shortcuts. Based on corresponding observations, it summarizes the unresolved issues in existing research and attempts to outline the future research landscape of shortcut learning.
Supervised Transfer Learning Framework for Fault Diagnosis in Wind Turbines
Venue: arXiv cs.LG 2024
Year: 2024

Common challenges in fault diagnosis include the lack of labeled data and the need to build models for each domain, resulting in many models that require supervision. Transfer learning can help tackle these challenges by learning cross-domain knowledge. Many approaches still require at least some labeled data in the target domain, and often provide unexplainable results. To this end, we propose a supervised transfer learning framework for fault diagnosis in wind turbines that operates in an Anomaly-Space. This space was created using SCADA data and vibration data and was built and provided to us by our research partner. Data within the Anomaly-Space can be interpreted as anomaly scores for each component in the wind turbine, making each value intuitive to understand. We conducted cross-domain evaluation on the train set using popular supervised classifiers like Random Forest, Light-Gradient-Boosting-Machines and Multilayer Perceptron as metamodels for the diagnosis of bearing and sensor faults. The Multilayer Perceptron achieved the highest classification performance. This model was then used for a final evaluation in our test set. The results show, that the proposed framework is able to detect cross-domain faults in the test set with a high degree of accuracy by using one single classifier, which is a significant asset to the diagnostic team.
Diffusion-based Virtual Fixtures
Venue: arXiv cs.RO 2024
Year: 2024

Virtual fixtures assist human operators in teleoperation settings by constraining their actions. This extended abstract introduces a novel virtual fixture formulation \emph{on surfaces} for tactile robotics tasks. Unlike existing methods, our approach constrains the behavior based on the position on the surface and generalizes it over the surface by considering the distance (metric) on the surface. Our method works directly on possibly noisy and partial point clouds collected via a camera. Given a set of regions on the surface together with their desired behaviors, our method diffuses the behaviors across the entire surface by taking into account the surface geometry. We demonstrate our method's ability in two simulated experiments (i) to regulate contact force magnitude or tangential speed based on surface position and (ii) to guide the robot to targets while avoiding restricted regions defined on the surface. All source codes, experimental data, and videos are available as open access at https://sites.google.com/view/diffusion-virtual-fixtures
Detect an Object At Once without Fine-tuning
Venue: arXiv cs.CV 2024
Year: 2024

When presented with one or a few photos of a previously unseen object, humans can instantly recognize it in different scenes. Although the human brain mechanism behind this phenomenon is still not fully understood, this work introduces a novel technical realization of this task. It consists of two phases: (1) generating a Similarity Density Map (SDM) by convolving the scene image with the given object image patch(es) so that the highlight areas in the SDM indicate the possible locations; (2) obtaining the object occupied areas in the scene through a Region Alignment Network (RAN). The RAN is constructed on a backbone of Deep Siamese Network (DSN), and different from the traditional DSNs, it aims to obtain the object accurate regions by regressing the location and area differences between the ground truths and the predicted ones indicated by the highlight areas in SDM. By pre-learning from labels annotated in traditional datasets, the SDM-RAN can detect previously unknown objects without fine-tuning. Experiments were conducted on the MS COCO, PASCAL VOC datasets. The results indicate that the proposed method outperforms state-of-the-art methods on the same task.
Improving Steering Vectors by Targeting Sparse Autoencoder Features
Venue: arXiv cs.LG 2024
Year: 2024

To control the behavior of language models, steering methods attempt to ensure that outputs of the model satisfy specific pre-defined properties. Adding steering vectors to the model is a promising method of model control that is easier than finetuning, and may be more robust than prompting. However, it can be difficult to anticipate the effects of steering vectors produced by methods such as CAA [Panickssery et al., 2024] or the direct use of SAE latents [Templeton et al., 2024]. In our work, we address this issue by using SAEs to measure the effects of steering vectors, giving us a method that can be used to understand the causal effect of any steering vector intervention. We use this method for measuring causal effects to develop an improved steering method, SAE-Targeted Steering (SAE-TS), which finds steering vectors to target specific SAE features while minimizing unintended side effects. We show that overall, SAE-TS balances steering effects with coherence better than CAA and SAE feature steering, when evaluated on a range of tasks.
Collective Model Intelligence Requires Compatible Specialization
Venue: arXiv cs.LG 2024
Year: 2024

In this work, we explore the limitations of combining models by averaging intermediate features, referred to as model merging, and propose a new direction for achieving collective model intelligence through what we call compatible specialization. Current methods for model merging, such as parameter and feature averaging, struggle to effectively combine specialized models due to representational divergence during fine-tuning. As models specialize to their individual domains, their internal feature representations become increasingly incompatible, leading to poor performance when attempting to merge them for new tasks. We analyze this phenomenon using centered kernel alignment (CKA) and show that as models specialize, the similarity in their feature space structure diminishes, hindering their capacity for collective use. To address these challenges, we investigate routing-based merging strategies, which offer more flexible methods for combining specialized models by dynamically routing across different layers. This allows us to improve on existing methods by combining features from multiple layers rather than relying on fixed, layer-wise combinations. However, we find that these approaches still face limitations when layers within models are representationally incompatible. Our findings highlight the importance of designing new approaches for model merging that operate on well-defined input and output spaces, similar to how humans communicate through language rather than intermediate neural activations.
The Role of DevOps in Enhancing Enterprise Software Delivery Success through R&D Efficiency and Source Code Management
Venue: arXiv cs.SE 2024
Year: 2024

This study examines the impact of DevOps practices on enterprise software delivery success, focusing on enhancing R&D efficiency and source code management (SCM). Using a qualitative methodology, data were collected from case studies of large-scale enterprises implementing DevOps to explore how these practices streamline software development processes. Findings reveal that DevOps significantly improves R&D productivity by fostering cross-functional collaboration, reducing development cycle times, and enhancing software quality through effective SCM practices, such as version control and continuous integration. Additionally, SCM tools within DevOps enable precise change tracking and reliable code maintenance, further supporting faster, more robust software delivery. However, the study identifies challenges, including cultural resistance and tool integration issues, that can hinder DevOps implementation. Additionally, This research contributes to the growing body of DevOps literature by highlighting the role of R&D efficiency and SCM as crucial factors for software delivery success. Future studies should investigate these factors across diverse industries to validate findings.
Parks: A Doubly Infinite Family of NP-Complete Puzzles and Generalizations of A002464
Venue: arXiv cs.CC 2024
Year: 2024

The Parks Puzzle is a paper-and-pencil puzzle game that is classically played on a square grid with different colored regions (the parks). The player needs to place a certain number of "trees" in each row, column, and park such that none are adjacent, even diagonally. We define a doubly-infinite family of such puzzles, the $(c, r)$-tree Parks puzzles, where there need be $c$ trees per column and $r$ per row. We then prove that for each $c$ and $r$ the set of $(c, r)$-tree puzzles is NP-complete. For each $c$ and $r$, there is a sequence of possible board sizes $m \times n$, and the number of possible puzzle solutions for these board sizes is a doubly-infinite generalization of OEIS sequence A002464, which itself describes the case $c = r = 1$. This connects the Parks puzzle to chess-based puzzle problems, as the sequence describes the number of ways to place non-attacking kings on a chessboard so that there is exactly one in each column and row (i.e. to place non-attacking dragon kings in shogi). These findings add yet another puzzle to the set of chess puzzles and expands the list of known NP-complete problems described.
Hunyuan-Large: An Open-Source MoE Model with 52 Billion Activated Parameters by Tencent
Venue: arXiv cs.CL 2024
Year: 2024

In this paper, we introduce Hunyuan-Large, which is currently the largest open-source Transformer-based mixture of experts model, with a total of 389 billion parameters and 52 billion activation parameters, capable of handling up to 256K tokens. We conduct a thorough evaluation of Hunyuan-Large's superior performance across various benchmarks including language understanding and generation, logical reasoning, mathematical problem-solving, coding, long-context, and aggregated tasks, where it outperforms LLama3.1-70B and exhibits comparable performance when compared to the significantly larger LLama3.1-405B model. Key practice of Hunyuan-Large include large-scale synthetic data that is orders larger than in previous literature, a mixed expert routing strategy, a key-value cache compression technique, and an expert-specific learning rate strategy. Additionally, we also investigate the scaling laws and learning rate schedule of mixture of experts models, providing valuable insights and guidances for future model development and optimization. The code and checkpoints of Hunyuan-Large are released to facilitate future innovations and applications. Codes: https://github.com/Tencent/Hunyuan-Large Models: https://huggingface.co/tencent/Tencent-Hunyuan-Large
Continuous Analysis: Evolution of Software Engineering and Reproducibility for Science
Venue: arXiv cs.SE 2024
Year: 2024

Reproducibility in research remains hindered by complex systems involving data, models, tools, and algorithms. Studies highlight a reproducibility crisis due to a lack of standardized reporting, code and data sharing, and rigorous evaluation. This paper introduces the concept of Continuous Analysis to address the reproducibility challenges in scientific research, extending the DevOps lifecycle. Continuous Analysis proposes solutions through version control, analysis orchestration, and feedback mechanisms, enhancing the reliability of scientific results. By adopting CA, the scientific community can ensure the validity and generalizability of research outcomes, fostering transparency and collaboration and ultimately advancing the field.
AutoVFX: Physically Realistic Video Editing from Natural Language Instructions
Venue: arXiv cs.CV 2024
Year: 2024

Modern visual effects (VFX) software has made it possible for skilled artists to create imagery of virtually anything. However, the creation process remains laborious, complex, and largely inaccessible to everyday users. In this work, we present AutoVFX, a framework that automatically creates realistic and dynamic VFX videos from a single video and natural language instructions. By carefully integrating neural scene modeling, LLM-based code generation, and physical simulation, AutoVFX is able to provide physically-grounded, photorealistic editing effects that can be controlled directly using natural language instructions. We conduct extensive experiments to validate AutoVFX's efficacy across a diverse spectrum of videos and instructions. Quantitative and qualitative results suggest that AutoVFX outperforms all competing methods by a large margin in generative quality, instruction alignment, editing versatility, and physical plausibility.
Automated Placement of Analog Integrated Circuits using Priority-based Constructive Heuristic
Venue: arXiv cs.NE 2024
Year: 2024

This paper presents a heuristic approach for solving the placement of Analog and Mixed-Signal Integrated Circuits. Placement is a crucial step in the physical design of integrated circuits. During this step, designers choose the position and variant of each circuit device. We focus on the specific class of analog placement, which requires so-called pockets, their possible merging, and parametrizable minimum distances between devices, which are features mostly omitted in recent research and literature. We formulate the problem using Integer Linear Programming and propose a priority-based constructive heuristic inspired by algorithms for the Facility Layout Problem. Our solution minimizes the perimeter of the circuit's bounding box and the approximated wire length. Multiple variants of the devices with different dimensions are considered. Furthermore, we model constraints crucial for the placement problem, such as symmetry groups and blockage areas. Our outlined improvements make the heuristic suitable to handle complex rules of placement. With a search guided either by a Genetic Algorithm or a Covariance Matrix Adaptation Evolution Strategy, we show the quality of the proposed method on both synthetically generated and real-life industrial instances accompanied by manually created designs. Furthermore, we apply reinforcement learning to control the hyper-parameters of the genetic algorithm. Synthetic instances with more than 200 devices demonstrate that our method can tackle problems more complex than typical industry examples. We also compare our method with results achieved by contemporary state-of-the-art methods on the MCNC and GSRC datasets.
Fairness Evaluation with Item Response Theory
Venue: arXiv cs.CY 2024
Year: 2024

Item Response Theory (IRT) has been widely used in educational psychometrics to assess student ability, as well as the difficulty and discrimination of test questions. In this context, discrimination specifically refers to how effectively a question distinguishes between students of different ability levels, and it does not carry any connotation related to fairness. In recent years, IRT has been successfully used to evaluate the predictive performance of Machine Learning (ML) models, but this paper marks its first application in fairness evaluation. In this paper, we propose a novel Fair-IRT framework to evaluate a set of predictive models on a set of individuals, while simultaneously eliciting specific parameters, namely, the ability to make fair predictions (a feature of predictive models), as well as the discrimination and difficulty of individuals that affect the prediction results. Furthermore, we conduct a series of experiments to comprehensively understand the implications of these parameters for fairness evaluation. Detailed explanations for item characteristic curves (ICCs) are provided for particular individuals. We propose the flatness of ICCs to disentangle the unfairness between individuals and predictive models. The experiments demonstrate the effectiveness of this framework as a fairness evaluation tool. Two real-world case studies illustrate its potential application in evaluating fairness in both classification and regression tasks. Our paper aligns well with the Responsible Web track by proposing a Fair-IRT framework to evaluate fairness in ML models, which directly contributes to the development of a more inclusive, equitable, and trustworthy AI.