OpenRaiser/Intern-Atlas huggingface dataset:
SpikeGS: 3D Gaussian Splatting from Spike Streams with High-Speed Camera Motion
Venue: arXiv cs.CV 2024
Year: 2024

Novel View Synthesis plays a crucial role by generating new 2D renderings from multi-view images of 3D scenes. However, capturing high-speed scenes with conventional cameras often leads to motion blur, hindering the effectiveness of 3D reconstruction. To address this challenge, high-frame-rate dense 3D reconstruction emerges as a vital technique, enabling detailed and accurate modeling of real-world objects or scenes in various fields, including Virtual Reality or embodied AI. Spike cameras, a novel type of neuromorphic sensor, continuously record scenes with an ultra-high temporal resolution, showing potential for accurate 3D reconstruction. Despite their promise, existing approaches, such as applying Neural Radiance Fields (NeRF) to spike cameras, encounter challenges due to the time-consuming rendering process. To address this issue, we make the first attempt to introduce the 3D Gaussian Splatting (3DGS) into spike cameras in high-speed capture, providing 3DGS as dense and continuous clues of views, then constructing SpikeGS. Specifically, to train SpikeGS, we establish computational equations between the rendering process of 3DGS and the processes of instantaneous imaging and exposing-like imaging of the continuous spike stream. Besides, we build a very lightweight but effective mapping process from spikes to instant images to support training. Furthermore, we introduced a new spike-based 3D rendering dataset for validation. Extensive experiments have demonstrated our method possesses the high quality of novel view rendering, proving the tremendous potential of spike cameras in modeling 3D scenes.
Revolutionizing Bridge Operation and Maintenance with LLM-based Agents: An Overview of Applications and Insights
Venue: arXiv cs.MA 2024
Year: 2024

In various industrial fields of human social development, people have been exploring methods aimed at freeing human labor. Constructing LLM-based agents is considered to be one of the most effective tools to achieve this goal. Agent, as a kind of human-like intelligent entity with the ability of perception, planning, decision-making, and action, has created great production value in many fields. However, the bridge O&M field shows a relatively low level of intelligence compared to other industries. Nevertheless, the bridge O&M field has developed numerous intelligent inspection devices, machine learning algorithms, and autonomous evaluation and decision-making methods, which provide a feasible basis for breakthroughs in artificial intelligence in this field. The aim of this study is to explore the impact of AI bodies based on large-scale language models on the field of bridge O&M and to analyze the potential challenges and opportunities it brings to the core tasks of bridge O&M. Through in-depth research and analysis, this paper expects to provide a more comprehensive perspective for understanding the application of intelligentsia in this field.
Constraints on Meso-Scale Structure in Complex Networks
Venue: arXiv cs.SI 2024
Year: 2024

A key topic in network science is the detection of intermediate or meso-scale structures. Community, core-periphery, disassortative and other partitions allow us to understand the organisation and function of large networks. In this work we study under what conditions certain common meso-scale structures are detectable using the idea of block modularity. We find that the configuration model imposes strong restrictions on core-periphery and related structures in directed networks. We derive inequalities expressing when such structures can be detected under the configuration model. Nestedness is closely related to core-periphery and is similarly restricted to only be detectable under certain conditions. We show that these conditions are a generalisation of the resolution limit to structures other than assortative communities. We show how block modularity is related to the degree corrected Stochastic Block Model and that optimisation of one can be made equivalent to the other in general. Finally, we discuss these issues in inferential versus descriptive approaches to meso-scale structure detection.
A Self-Supervised Learning Pipeline for Demographically Fair Facial Attribute Classification
Venue: arXiv cs.CV 2024
Year: 2024

Published research highlights the presence of demographic bias in automated facial attribute classification. The proposed bias mitigation techniques are mostly based on supervised learning, which requires a large amount of labeled training data for generalizability and scalability. However, labeled data is limited, requires laborious annotation, poses privacy risks, and can perpetuate human bias. In contrast, self-supervised learning (SSL) capitalizes on freely available unlabeled data, rendering trained models more scalable and generalizable. However, these label-free SSL models may also introduce biases by sampling false negative pairs, especially at low-data regimes 200K images) under low compute settings. Further, SSL-based models may suffer from performance degradation due to a lack of quality assurance of the unlabeled data sourced from the web. This paper proposes a fully self-supervised pipeline for demographically fair facial attribute classifiers. Leveraging completely unlabeled data pseudolabeled via pre-trained encoders, diverse data curation techniques, and meta-learning-based weighted contrastive learning, our method significantly outperforms existing SSL approaches proposed for downstream image classification tasks. Extensive evaluations on the FairFace and CelebA datasets demonstrate the efficacy of our pipeline in obtaining fair performance over existing baselines. Thus, setting a new benchmark for SSL in the fairness of facial attribute classification.
TokenSHAP: Interpreting Large Language Models with Monte Carlo Shapley Value Estimation
Venue: arXiv cs.CL 2024
Year: 2024

As large language models (LLMs) become increasingly prevalent in critical applications, the need for interpretable AI has grown. We introduce TokenSHAP, a novel method for interpreting LLMs by attributing importance to individual tokens or substrings within input prompts. This approach adapts Shapley values from cooperative game theory to natural language processing, offering a rigorous framework for understanding how different parts of an input contribute to a model's response. TokenSHAP leverages Monte Carlo sampling for computational efficiency, providing interpretable, quantitative measures of token importance. We demonstrate its efficacy across diverse prompts and LLM architectures, showing consistent improvements over existing baselines in alignment with human judgments, faithfulness to model behavior, and consistency. Our method's ability to capture nuanced interactions between tokens provides valuable insights into LLM behavior, enhancing model transparency, improving prompt engineering, and aiding in the development of more reliable AI systems. TokenSHAP represents a significant step towards the necessary interpretability for responsible AI deployment, contributing to the broader goal of creating more transparent, accountable, and trustworthy AI systems.
A Bag of Tricks for Scaling CPU-based Deep FFMs to more than 300m Predictions per Second
Venue: arXiv cs.LG 2024
Year: 2024

Field-aware Factorization Machines (FFMs) have emerged as a powerful model for click-through rate prediction, particularly excelling in capturing complex feature interactions. In this work, we present an in-depth analysis of our in-house, Rust-based Deep FFM implementation, and detail its deployment on a CPU-only, multi-data-center scale. We overview key optimizations devised for both training and inference, demonstrated by previously unpublished benchmark results in efficient model search and online training. Further, we detail an in-house weight quantization that resulted in more than an order of magnitude reduction in bandwidth footprint related to weight transfers across data-centres. We disclose the engine and associated techniques under an open-source license to contribute to the broader machine learning community. This paper showcases one of the first successful CPU-only deployments of Deep FFMs at such scale, marking a significant stride in practical, low-footprint click-through rate prediction methodologies.
Explicit Commutative ROABPs from Partial Derivatives
Venue: arXiv cs.CC 2024
Year: 2024

The dimension of partial derivatives (Nisan and Wigderson, 1997) is a popular measure for proving lower bounds in algebraic complexity. It is used to give strong lower bounds on the Waring decomposition of polynomials (called Waring rank). This naturally leads to an interesting open question: does this measure essentially characterize the Waring rank of any polynomial? The well-studied model of Read-once Oblivious ABPs (ROABPs for short) lends itself to an interesting hierarchy of 'sub-models': Any-Order-ROABPs (ARO), Commutative ROABPs, and Diagonal ROABPs. It follows from previous works that for any polynomial, a bound on its Waring rank implies an analogous bound on its Diagonal ROABP complexity (called the duality trick), and a bound on its dimension of partial derivatives implies an analogous bound on its 'ARO complexity': ROABP complexity in any order (Nisan, 1991). Our work strengthens the latter connection by showing that a bound on the dimension of partial derivatives in fact implies a bound on the commutative ROABP complexity. Thus, we improve our understanding of partial derivatives and move a step closer towards answering the above question. Our proof builds on the work of Ramya and Tengse (2022) to show that the commutative-ROABP-width of any homogeneous polynomial is at most the dimension of its partial derivatives. The technique itself is a generalization of the proof of the duality trick due to Saxena (2008).
Multiple data sources and domain generalization learning method for road surface defect classification
Venue: arXiv cs.CV 2024
Year: 2024

Roads are an essential mode of transportation, and maintaining them is critical to economic growth and citizen well-being. With the continued advancement of AI, road surface inspection based on camera images has recently been extensively researched and can be performed automatically. However, because almost all of the deep learning methods for detecting road surface defects were optimized for a specific dataset, they are difficult to apply to a new, previously unseen dataset. Furthermore, there is a lack of research on training an efficient model using multiple data sources. In this paper, we propose a method for classifying road surface defects using camera images. In our method, we propose a scheme for dealing with the invariance of multiple data sources while training a model on multiple data sources. Furthermore, we present a domain generalization training algorithm for developing a generalized model that can work with new, completely unseen data sources without requiring model updates. We validate our method using an experiment with six data sources corresponding to six countries from the RDD2022 dataset. The results show that our method can efficiently classify road surface defects on previously unseen data.
Low Sensitivity Hopsets
Venue: arXiv cs.DS 2024
Year: 2024

Given a weighted graph $G$, a $(β,\varepsilon)$-hopset $H$ is an edge set such that for any $s,t \in V(G)$, where $s$ can reach $t$ in $G$, there is a path from $s$ to $t$ in $G \cup H$ which uses at most $β$ hops whose length is in the range $[dist_G(s,t), (1+\varepsilon)dist_G(s,t)]$. We break away from the traditional question that asks for a hopset that achieves small $|H|$ and instead study its sensitivity, a new quality measure which, informally, is the maximum number of times a vertex (or edge) is bypassed by an edge in $H$. The highlights of our results are: (i) $(\widetilde{O}(\sqrt{n}),0)$-hopsets on undirected graphs with $O(\log n)$ sensitivity, complemented with a lower bound showing that $\widetilde{O}(\sqrt{n})$ is tight up to polylogarithmic factors for any construction with polylogarithmic sensitivity. (ii) $(n^{o(1)},\varepsilon)$-hopsets on undirected graphs with $n^{o(1)}$ sensitivity for any $\varepsilon > 0$ that is at least inverse polylogarithmic, complemented with a lower bound on the tradeoff between $β, \varepsilon$, and the sensitivity. (iii) $\widetilde{O}(\sqrt{n})$-shortcut sets on directed graphs with $O(\log n)$ sensitivity, complemented with a lower bound showing that $β= \widetildeΩ(n^{1/3})$ for any construction with polylogarithmic sensitivity. We believe hopset sensitivity is a natural measure in and of itself, and could potentially find use in a diverse range of contexts. More concretely, the notion of hopset sensitivity is also directly motivated by the Differentially Private All Sets Range Queries problem. Our result for $O(\log n)$ sensitivity $(\widetilde{O}(\sqrt{n}),0)$-hopsets on undirected graphs immediately improves the current best-known upper bound on utility from $\widetilde{O}(n^{1/3})$ to $\widetilde{O}(n^{1/4})$ in the pure-DP setting, which is tight up to polylogarithmic factors.
Product and Ratio of Two $α-κ-μ$ Shadowed Random Variables and its Application to Wireless Communication
Venue: arXiv cs.IT 2024
Year: 2024

This work studies the product and ratio statistics of independent and non-identically distributed (i.n.i.d) $ α-κ- μ$ shadowed random variables. We derive the series expression for the probability density function (PDF), cumulative distribution function (CDF), and moment generating function (MGF) of the product and ratio of i.n.i.d $ α- κ- μ$ shadowed random variables. We then give the single integral representation for the derived PDF expressions. Further, as application examples, 1) outage probability has been derived for cascaded wireless systems, and 2) physical-layer security metrics like secrecy outage probability and strictly positive secrecy capacity are derived for the classic three-node model with $α-κ-μ$ shadowed fading. Next, we discuss an intelligent reflecting surface-assisted communication system over $α-κ-μ$ shadowed fading.
Improving On-Time Undergraduate Graduation Rate For Undergraduate Students Using Predictive Analytics
Venue: arXiv cs.CY 2024
Year: 2024

The on-time graduation rate among universities in Puerto Rico is significantly lower than in the mainland United States. This problem is noteworthy because it leads to substantial negative consequences for the student, both socially and economically, the educational institution and the local economy. This project aims to develop a predictive model that accurately detects students early in their academic pursuit at risk of not graduating on time. Various predictive models are developed to do this, and the best model, the one with the highest performance, is selected. Using a dataset containing information from 24432 undergraduate students at the University of Puerto Rico at Mayaguez, the predictive performance of the models is evaluated in two scenarios: Group I includes both the first year of college and pre-college factors, and Group II only considers pre-college factors. Overall, for both scenarios, the boosting model, trained on the oversampled dataset, is the most successful at predicting who will not graduate on time.
Towards An Online Incremental Approach to Predict Students Performance
Venue: arXiv cs.CY 2024
Year: 2024

Analytical models developed in offline settings with pre-prepared data are typically used to predict students' performance. However, when data are available over time, this learning method is not suitable anymore. Online learning is increasingly used to update the online models from stream data. A rehearsal technique is typically used, which entails re-training the model on a small training set that is updated each time new data is received. The main challenge in this regard is the construction of the training set with appropriate data samples to maintain good model performance. Typically, a random selection of samples is made, which can deteriorate the model's performance. In this paper, we propose a memory-based online incremental learning approach for updating an online classifier that predicts student performance using stream data. The approach is based on the use of the genetic algorithm heuristic while respecting the memory space constraints as well as the balance of class labels. In contrast to random selection, our approach improves the stability of the analytical model by promoting diversity when creating the training set. As a proof of concept, we applied it to the open dataset OULAD. Our approach achieves a notable improvement in model accuracy, with an enhancement of nearly 10% compared to the current state-of-the-art, while maintaining a relatively low standard deviation in accuracy, ranging from 1% to 2.1%.
Systematic analysis of the effectiveness of adding human mobility data to covid-19 case prediction linear models
Venue: arXiv cs.CY 2024
Year: 2024

Human mobility data has been extensively used in covid-19 case prediction models. Nevertheless, related work has questioned whether mobility data really helps that much. We present a systematic analysis across mobility datasets and prediction lookaheads and reveal that adding mobility data to predictive models improves model performance only for about two months at the onset of the testing period, and that performance improvements -- measured as predicted vs. actual correlation improvement over non-mobility baselines -- are at most 0.3.
Elements Of Legislation For Artificial Intelligence Systems
Venue: arXiv cs.CY 2024
Year: 2024

The significant part of the operational context for autonomous company management systems is the regulatory and legal environment in which corporations operate. In order to create a dedicated operational context for autonomous artificial intelligence systems, the wording of local regulatory documents can be simultaneously presented in two versions: for use by people and for use by autonomous systems. In this case, the artificial intelligence system will get a well-defined operational context that allows such a system to perform functions within the required standards. Local regulations that provide basis for the joint work of individuals and autonomous artificial intelligence systems can form the grounds for the relevant legislation governing the development and implementation of autonomous systems.
Sora and V-JEPA Have Not Learned The Complete Real World Model -- A Philosophical Analysis of Video AIs Through the Theory of Productive Imagination
Venue: arXiv cs.AI 2024
Year: 2024

Sora from Open AI has shown exceptional performance, yet it faces scrutiny over whether its technological prowess equates to an authentic comprehension of reality. Critics contend that it lacks a foundational grasp of the world, a deficiency V-JEPA from Meta aims to amend with its joint embedding approach. This debate is vital for steering the future direction of Artificial General Intelligence(AGI). We enrich this debate by developing a theory of productive imagination that generates a coherent world model based on Kantian philosophy. We identify three indispensable components of the coherent world model capable of genuine world understanding: representations of isolated objects, an a priori law of change across space and time, and Kantian categories. Our analysis reveals that Sora is limited because of its oversight of the a priori law of change and Kantian categories, flaws that are not rectifiable through scaling up the training. V-JEPA learns the context-dependent aspect of the a priori law of change. Yet it fails to fully comprehend Kantian categories and incorporate experience, leading us to conclude that neither system currently achieves a comprehensive world understanding. Nevertheless, each system has developed components essential to advancing an integrated AI productive imagination-understanding engine. Finally, we propose an innovative training framework for an AI productive imagination-understanding engine, centered around a joint embedding system designed to transform disordered perceptual input into a structured, coherent world model. Our philosophical analysis pinpoints critical challenges within contemporary video AI technologies and a pathway toward achieving an AI system capable of genuine world understanding, such that it can be applied for reasoning and planning in the future.
Ontology-driven Reinforcement Learning for Personalized Student Support
Venue: arXiv cs.CY 2024
Year: 2024

In the search for more effective education, there is a widespread effort to develop better approaches to personalize student education. Unassisted, educators often do not have time or resources to personally support every student in a given classroom. Motivated by this issue, and by recent advancements in artificial intelligence, this paper presents a general-purpose framework for personalized student support, applicable to any virtual educational system such as a serious game or an intelligent tutoring system. To fit any educational situation, we apply ontologies for their semantic organization, combining them with data collection considerations and multi-agent reinforcement learning. The result is a modular system that can be adapted to any virtual educational software to provide useful personalized assistance to students.
Addressing Class Imbalance and Data Limitations in Advanced Node Semiconductor Defect Inspection: A Generative Approach for SEM Images
Venue: arXiv cs.CV 2024
Year: 2024

Precision in identifying nanometer-scale device-killer defects is crucial in both semiconductor research and development as well as in production processes. The effectiveness of existing ML-based approaches in this context is largely limited by the scarcity of data, as the production of real semiconductor wafer data for training these models involves high financial and time costs. Moreover, the existing simulation methods fall short of replicating images with identical noise characteristics, surface roughness and stochastic variations at advanced nodes. We propose a method for generating synthetic semiconductor SEM images using a diffusion model within a limited data regime. In contrast to images generated through conventional simulation methods, SEM images generated through our proposed DL method closely resemble real SEM images, replicating their noise characteristics and surface roughness adaptively. Our main contributions, which are validated on three different real semiconductor datasets, are: i) proposing a patch-based generative framework utilizing DDPM to create SEM images with intended defect classes, addressing challenges related to class-imbalance and data insufficiency, ii) demonstrating generated synthetic images closely resemble real SEM images acquired from the tool, preserving all imaging conditions and metrology characteristics without any metadata supervision, iii) demonstrating a defect detector trained on generated defect dataset, either independently or combined with a limited real dataset, can achieve similar or improved performance on real wafer SEM images during validation/testing compared to exclusive training on a real defect dataset, iv) demonstrating the ability of the proposed approach to transfer defect types, critical dimensions, and imaging conditions from one specified CD/Pitch and metrology specifications to another, thereby highlighting its versatility.