OpenRaiser/Intern-Atlas huggingface dataset:
Recurrent Stochastic Configuration Networks with Hybrid Regularization for Nonlinear Dynamics Modelling
Venue: arXiv cs.LG 2024
Year: 2024

Recurrent stochastic configuration networks (RSCNs) have shown great potential in modelling nonlinear dynamic systems with uncertainties. This paper presents an RSCN with hybrid regularization to enhance both the learning capacity and generalization performance of the network. Given a set of temporal data, the well-known least absolute shrinkage and selection operator (LASSO) is employed to identify the significant order variables. Subsequently, an improved RSCN with L2 regularization is introduced to approximate the residuals between the output of the target plant and the LASSO model. The output weights are updated in real-time through a projection algorithm, facilitating a rapid response to dynamic changes within the system. A theoretical analysis of the universal approximation property is provided, contributing to the understanding of the network's effectiveness in representing various complex nonlinear functions. Experimental results from a nonlinear system identification problem and two industrial predictive tasks demonstrate that the proposed method outperforms other models across all testing datasets.
Flaws of ImageNet, Computer Vision's Favourite Dataset
Venue: arXiv cs.CV 2024
Year: 2024

Since its release, ImageNet-1k dataset has become a gold standard for evaluating model performance. It has served as the foundation for numerous other datasets and training tasks in computer vision. As models have improved in accuracy, issues related to label correctness have become increasingly apparent. In this blog post, we analyze the issues in the ImageNet-1k dataset, including incorrect labels, overlapping or ambiguous class definitions, training-evaluation domain shifts, and image duplicates. The solutions for some problems are straightforward. For others, we hope to start a broader conversation about refining this influential dataset to better serve future research.
Visual Error Patterns in Multi-Modal AI: A Statistical Approach
Venue: arXiv cs.LG 2024
Year: 2024

Multi-modal large language models (MLLMs), such as GPT-4o, excel at integrating text and visual data but face systematic challenges when interpreting ambiguous or incomplete visual stimuli. This study leverages statistical modeling to analyze the factors driving these errors, using a dataset of geometric stimuli characterized by features like 3D, rotation, and missing face/side. We applied parametric methods, non-parametric methods, and ensemble techniques to predict classification errors, with the non-linear gradient boosting model achieving the highest performance (AUC=0.85) during cross-validation. Feature importance analysis highlighted difficulties in depth perception and reconstructing incomplete structures as key contributors to misclassification. These findings demonstrate the effectiveness of statistical approaches for uncovering limitations in MLLMs and offer actionable insights for enhancing model architectures by integrating contextual reasoning mechanisms.
Graph Canvas for Controllable 3D Scene Generation
Venue: arXiv cs.CV 2024
Year: 2024

Spatial intelligence is foundational to AI systems that interact with the physical world, particularly in 3D scene generation and spatial comprehension. Current methodologies for 3D scene generation often rely heavily on predefined datasets, and struggle to adapt dynamically to changing spatial relationships. In this paper, we introduce GraphCanvas3D, a programmable, extensible, and adaptable framework for controllable 3D scene generation. Leveraging in-context learning, GraphCanvas3D enables dynamic adaptability without the need for retraining, supporting flexible and customizable scene creation. Our framework employs hierarchical, graph-driven scene descriptions, representing spatial elements as graph nodes and establishing coherent relationships among objects in 3D environments. Unlike conventional approaches, which are constrained in adaptability and often require predefined input masks or retraining for modifications, GraphCanvas3D allows for seamless object manipulation and scene adjustments on the fly. Additionally, GraphCanvas3D supports 4D scene generation, incorporating temporal dynamics to model changes over time. Experimental results and user studies demonstrate that GraphCanvas3D enhances usability, flexibility, and adaptability for scene generation. Our code and models are available on the project website: https://github.com/ILGLJ/Graph-Canvas.
Deep Neural Network-Based Prediction of B-Cell Epitopes for SARS-CoV and SARS-CoV-2: Enhancing Vaccine Design through Machine Learning
Venue: arXiv cs.LG 2024
Year: 2024

The accurate prediction of B-cell epitopes is critical for guiding vaccine development against infectious diseases, including SARS and COVID-19. This study explores the use of a deep neural network (DNN) model to predict B-cell epitopes for SARS-CoVandSARS-CoV-2,leveraging a dataset that incorporates essential protein and peptide features. Traditional sequence-based methods often struggle with large, complex datasets, but deep learning offers promising improvements in predictive accuracy. Our model employs regularization techniques, such as dropout and early stopping, to enhance generalization, while also analyzing key features, including isoelectric point and aromaticity, that influence epitope recognition. Results indicate an overall accuracy of 82% in predicting COVID-19 negative and positive cases, with room for improvement in detecting positive samples. This research demonstrates the applicability of deep learning in epitope mapping, suggesting that such approaches can enhance the speed and precision of vaccine design for emerging pathogens. Future work could incorporate structural data and diverse viral strains to further refine prediction capabilities.
Boundary Control Behaviors of Multiple Low-cost AUVs Using Acoustic Communication
Venue: arXiv cs.RO 2024
Year: 2024

This study presents acoustic-based methods for the control of multiple autonomous underwater vehicles (AUV). This study proposes two different models for implementing boundary and path control on low-cost AUVs using acoustic communication and a single central acoustic beacon. Two methods are presented: the Range Variation-Based (RVB) model completely relies on range data obtained by acoustic modems, whereas the Heading Estimation-Based (HEB) model uses ranges and range rates to estimate the position of the central boundary beacon and perform assigned behaviors. The models are tested on two boundary control behaviors: Fencing and Milling. Fencing behavior ensures AUVs return within predefined boundaries, while Milling enables the AUVs to move cyclically on a predefined path around the beacon. Models are validated by successfully performing the boundary control behaviors in simulations, pool tests, including artificial underwater currents, and field tests conducted in the ocean. All tests were performed with fully autonomous platforms, and no external input or sensor was provided to the AUVs during validation. Quantitative and qualitative analyses are presented in the study, focusing on the effect and application of a multi-robot system.
Semi-Supervised Neural Processes for Articulated Object Interactions
Venue: arXiv cs.RO 2024
Year: 2024

The scarcity of labeled action data poses a considerable challenge for developing machine learning algorithms for robotic object manipulation. It is expensive and often infeasible for a robot to interact with many objects. Conversely, visual data of objects, without interaction, is abundantly available and can be leveraged for pretraining and feature extraction. However, current methods that rely on image data for pretraining do not easily adapt to task-specific predictions, since the learned features are not guaranteed to be relevant. This paper introduces the Semi-Supervised Neural Process (SSNP): an adaptive reward-prediction model designed for scenarios in which only a small subset of objects have labeled interaction data. In addition to predicting reward labels, the latent-space of the SSNP is jointly trained with an autoencoding objective using passive data from a much larger set of objects. Jointly training with both types of data allows the model to focus more effectively on generalizable features and minimizes the need for extensive retraining, thereby reducing computational demands. The efficacy of SSNP is demonstrated through a door-opening task, leading to better performance than other semi-supervised methods, and only using a fraction of the data compared to other adaptive models.
AerialGo: Walking-through City View Generation from Aerial Perspectives
Venue: arXiv cs.CV 2024
Year: 2024

High-quality 3D urban reconstruction is essential for applications in urban planning, navigation, and AR/VR. However, capturing detailed ground-level data across cities is both labor-intensive and raises significant privacy concerns related to sensitive information, such as vehicle plates, faces, and other personal identifiers. To address these challenges, we propose AerialGo, a novel framework that generates realistic walking-through city views from aerial images, leveraging multi-view diffusion models to achieve scalable, photorealistic urban reconstructions without direct ground-level data collection. By conditioning ground-view synthesis on accessible aerial data, AerialGo bypasses the privacy risks inherent in ground-level imagery. To support the model training, we introduce AerialGo dataset, a large-scale dataset containing diverse aerial and ground-view images, paired with camera and depth information, designed to support generative urban reconstruction. Experiments show that AerialGo significantly enhances ground-level realism and structural coherence, providing a privacy-conscious, scalable solution for city-scale 3D modeling.
Modelling Networked Dynamical System by Temporal Graph Neural ODE with Irregularly Partial Observed Time-series Data
Venue: arXiv cs.LG 2024
Year: 2024

Modeling the evolution of system with time-series data is a challenging and critical task in a wide range of fields, especially when the time-series data is regularly sampled and partially observable. Some methods have been proposed to estimate the hidden dynamics between intervals like Neural ODE or Exponential decay dynamic function and combine with RNN to estimate the evolution. However, it is difficult for these methods to capture the spatial and temporal dependencies existing within graph-structured time-series data and take full advantage of the available relational information to impute missing data and predict the future states. Besides, traditional RNN-based methods leverage shared RNN cell to update the hidden state which does not capture the impact of various intervals and missing state information on the reliability of estimating the hidden state. To solve this problem, in this paper, we propose a method embedding Graph Neural ODE with reliability and time-aware mechanism which can capture the spatial and temporal dependencies in irregularly sampled and partially observable time-series data to reconstruct the dynamics. Also, a loss function is designed considering the reliability of the augment data from the above proposed method to make further prediction. The proposed method has been validated in experiments of different networked dynamical systems.
Towards the Ultimate Programming Language: Trust and Benevolence in the Age of Artificial Intelligence
Venue: arXiv cs.AI 2024
Year: 2024

This article explores the evolving role of programming languages in the context of artificial intelligence. It highlights the need for programming languages to ensure human understanding while eliminating unnecessary implementation details and suggests that future programs should be designed to recognize and actively support user interests. The vision includes a three-level process: using natural language for requirements, translating it into a precise system definition language, and finally optimizing the code for performance. The concept of an "Ultimate Programming Language" is introduced, emphasizing its role in maintaining human control over machines. Trust, reliability, and benevolence are identified as key elements that will enhance cooperation between humans and AI systems.
Enhanced LLM-Based Framework for Predicting Null Pointer Dereference in Source Code
Venue: arXiv cs.SE 2024
Year: 2024

Software security is crucial in any field where breaches can exploit sensitive data, and lead to financial losses. As a result, vulnerability detection becomes an essential part of the software development process. One of the key steps in maintaining software integrity is identifying vulnerabilities in the source code before deployment. A security breach like CWE-476, which stands for NULL pointer dereferences (NPD), is crucial because it can cause software crashes, unpredictable behavior, and security vulnerabilities. In this scientific era, there are several vulnerability checkers, where, previous tools often fall short in analyzing specific feature connections of the source code, which weakens the tools in real-world scenarios. In this study, we propose another novel approach using a fine-tuned Large Language Model (LLM) termed "DeLLNeuN". This model leverages the advantage of various layers to reduce both overfitting and non-linearity, enhancing its performance and reliability. Additionally, this method provides dropout and dimensionality reduction to help streamline the model, making it faster and more efficient. Our model showed 87% accuracy with 88% precision using the Draper VDISC dataset. As software becomes more complex and cyber threats continuously evolve, the need for proactive security measures will keep growing. In this particular case, the proposed model looks promising to use as an early vulnerability checker in software development.
Hybrid Spiking Neural Network -- Transformer Video Classification Model
Venue: arXiv cs.CV 2024
Year: 2024

In recent years, Spiking Neural Networks (SNNs) have gathered significant interest due to their temporal understanding capabilities. This work introduces, to the best of our knowledge, the first Cortical Column like hybrid architecture for the Time-Series Data Classification Task that leverages SNNs and is inspired by the brain structure, inspired from the previous hybrid models. We introduce several encoding methods to use with this model. Finally, we develop a procedure for training this network on the training dataset. As an effort to make using these models simpler, we make all the implementations available to the public.
Unlocking Diversity of Fast-Switched Optical Data Center Networks with Unified Routing
Venue: arXiv cs.NI 2024
Year: 2024

Optical data center networks (DCNs) are emerging as a promising solution for cloud infrastructure in the post-Moore's Law era, particularly with the advent of 'fast-switched' optical architectures capable of circuit reconfiguration at microsecond or even nanosecond scales. However, frequent reconfiguration of optical circuits introduces a unique challenge: in-flight packets risk loss during these transitions, hindering the deployment of many mature optical hardware designs due to the lack of suitable routing solutions. In this paper, we present Unified Routing for Optical networks (URO), a general routing framework designed to support fast-switched optical DCNs across various hardware architectures. URO combines theoretical modeling of this novel routing problem with practical implementation on programmable switches, enabling precise, time-based packet transmission. Our prototype on Intel Tofino2 switches achieves a minimum circuit duration of 2us, ensuring end-to-end, loss-free application performance. Large-scale simulations using production DCN traffic validate URO's generality across different hardware configurations, demonstrating its effectiveness and efficient system resource utilization.
Streamlining the review process: AI-generated annotations in research manuscripts
Venue: arXiv cs.AI 2024
Year: 2024

The increasing volume of research paper submissions poses a significant challenge to the traditional academic peer-review system, leading to an overwhelming workload for reviewers. This study explores the potential of integrating Large Language Models (LLMs) into the peer-review process to enhance efficiency without compromising effectiveness. We focus on manuscript annotations, particularly excerpt highlights, as a potential area for AI-human collaboration. While LLMs excel in certain tasks like aspect coverage and informativeness, they often lack high-level analysis and critical thinking, making them unsuitable for replacing human reviewers entirely. Our approach involves using LLMs to assist with specific aspects of the review process. This paper introduces AnnotateGPT, a platform that utilizes GPT-4 for manuscript review, aiming to improve reviewers' comprehension and focus. We evaluate AnnotateGPT using a Technology Acceptance Model (TAM) questionnaire with nine participants and generalize the findings. Our work highlights annotation as a viable middle ground for AI-human collaboration in academic review, offering insights into integrating LLMs into the review process and tuning traditional annotation tools for LLM incorporation.
Real-Time Metric-Semantic Mapping for Autonomous Navigation in Outdoor Environments
Venue: arXiv cs.RO 2024
Year: 2024

The creation of a metric-semantic map, which encodes human-prior knowledge, represents a high-level abstraction of environments. However, constructing such a map poses challenges related to the fusion of multi-modal sensor data, the attainment of real-time mapping performance, and the preservation of structural and semantic information consistency. In this paper, we introduce an online metric-semantic mapping system that utilizes LiDAR-Visual-Inertial sensing to generate a global metric-semantic mesh map of large-scale outdoor environments. Leveraging GPU acceleration, our mapping process achieves exceptional speed, with frame processing taking less than 7ms, regardless of scenario scale. Furthermore, we seamlessly integrate the resultant map into a real-world navigation system, enabling metric-semantic-based terrain assessment and autonomous point-to-point navigation within a campus environment. Through extensive experiments conducted on both publicly available and self-collected datasets comprising 24 sequences, we demonstrate the effectiveness of our mapping and navigation methodologies. Code has been publicly released: https://github.com/gogojjh/cobra
Adaptformer: Sequence models as adaptive iterative planners
Venue: arXiv cs.RO 2024
Year: 2024

Despite recent advances in learning-based behavioral planning for autonomous systems, decision-making in multi-task missions remains a challenging problem. For instance, a mission might require a robot to explore an unknown environment, locate the goals, and navigate to them, even if there are obstacles along the way. Such problems are difficult to solve due to: a) sparse rewards, meaning a reward signal is available only once all the tasks in a mission have been satisfied, and b) the agent having to perform tasks at run-time that are not covered in the training data, e.g., demonstrations only from an environment where all doors were unlocked. Consequently, state-of-the-art decision-making methods in such settings are limited to missions where the required tasks are well-represented in the training demonstrations and can be solved within a short planning horizon. To overcome these limitations, we propose Adaptformer, a stochastic and adaptive planner that utilizes sequence models for sample-efficient exploration and exploitation. This framework relies on learning an energy-based heuristic, which needs to be minimized over a sequence of high-level decisions. To generate successful action sequences for long-horizon missions, Adaptformer aims to achieve shorter sub-goals, which are proposed through an intrinsic sub-goal curriculum. Through these two key components, Adaptformer allows for generalization to out-of-distribution tasks and environments, i.e., missions that were not a part of the training data. Empirical results in multiple simulation environments demonstrate the effectiveness of our method. Notably, Adaptformer not only outperforms the state-of-the-art method by up to 25% in multi-goal maze reachability tasks but also successfully adapts to multi-task missions that the state-of-the-art method could not complete, leveraging demonstrations from single-goal-reaching tasks.
Raw Audio Classification with Cosine Convolutional Neural Network (CosCovNN)
Venue: arXiv cs.SD 2024
Year: 2024

This study explores the field of audio classification from raw waveform using Convolutional Neural Networks (CNNs), a method that eliminates the need for extracting specialised features in the pre-processing step. Unlike recent trends in literature, which often focuses on designing frontends or filters for only the initial layers of CNNs, our research introduces the Cosine Convolutional Neural Network (CosCovNN) replacing the traditional CNN filters with Cosine filters. The CosCovNN surpasses the accuracy of the equivalent CNN architectures with approximately $77\%$ less parameters. Our research further progresses with the development of an augmented CosCovNN named Vector Quantised Cosine Convolutional Neural Network with Memory (VQCCM), incorporating a memory and vector quantisation layer VQCCM achieves state-of-the-art (SOTA) performance across five different datasets in comparison with existing literature. Our findings show that cosine filters can greatly improve the efficiency and accuracy of CNNs in raw audio classification.
Myth: The loss of core developers is a critical issue for OSS communities
Venue: arXiv cs.SE 2024
Year: 2024

Throughout their lifetime, open-source software systems will naturally attract new contributors and lose existing contributors. Not all OSS contributors are equal, however, as some contributors within a project possess significant knowledge and expertise of the codebase (i.e., core developers). When investigating the ability of projects to attract new contributors and how often a project loses contributors, it is therefore important to take into account the expertise of the contributors. Since core developers are vital to the longevity of projects, we therefore aim to find out: can OSS projects attract new core developers and how often do OSS projects lose core developers? To investigate core developer contribution patterns, we calculate the truck factor (or bus factor) of over 36,000 OSS projects to investigate how often TF developers join or abandon OSS projects. We find that 89% of our studied projects have experienced losing their core development team at least once. Our results also show that in 70% of cases, this project abandonment happens within the first three years of the project life. We also find that most OSS projects rely on a single core developer to maintain development activities. Finally, we find that only 27% of projects that were abandoned were able to attract at least one new TF developer. Our analysis shows that it is not uncommon for OSS projects to lose their initial core development team. This is likely due to most OSS project relying on a single core developer to maintain development activities. The first year of development is critical for OSS projects since this is where they are most at risk of losing their core developer(s). Additionally, projects that lose their core developer(s) early seem less likely to survive this event than projects that lost their core developers later on during their life.