OpenRaiser/Intern-Atlas huggingface dataset:
M4U: Evaluating Multilingual Understanding and Reasoning for Large Multimodal Models
Venue: arXiv cs.CV 2025
Year: 2025

Multilingual capability is an essential aspect for large multimodal models, since they are usually deployed across various countries and languages. However, most existing benchmarks for multilingual multimodal reasoning struggle to differentiate between models of varying performance; even language models without visual capabilities can easily achieve high scores. This leaves a comprehensive evaluation of leading multilingual multimodal models largely unexplored. In this work, we introduce M4U, a novel and challenging benchmark for assessing the capability of multi-discipline multilingual multimodal understanding and reasoning. M4U contains 10k samples covering 64 disciplines across 16 subfields in Science, Engineering, and Healthcare in six languages. Using M4U, we conduct extensive evaluations of leading Large Multimodal Models (LMMs) and Large Language Models (LLMs) with external tools. The evaluation results demonstrate that the state-of-the-art model, GPT-4o, achieves only 47.6% average accuracy on M4U. Additionally, we observe that the leading LMMs exhibit significant language preferences. Our in-depth analysis indicates that leading LMMs, including GPT-4o, struggle to perform reasoning using multilingual information present in both visual and textual context. Specifically, they suffer performance degradation when prompted with cross-lingual multimodal questions. Our code and dataset is public available.
A Logic of Knowledge and Justifications, with an Application to Computational Trust
Venue: arXiv cs.LO 2025
Year: 2025

We present a logical framework that enables us to define a formal theory of computational trust in which this notion is analysed in terms of epistemic attitudes towards the possible objects of trust and in relation to existing evidence in favour of the trustworthiness of these objects. The framework is based on a quantified epistemic and justification logic featuring a non-standard handling of identities. Thus, the theory is able to account for the hyperintensional nature of computational trust. We present a proof system and a frame semantics for the logic, we prove soundness and completeness results and we introduce the syntactical machinery required to define a theory of trust.
Scaling up the Banded Matrix Factorization Mechanism for Differentially Private ML
Venue: arXiv cs.LG 2025
Year: 2025

Correlated noise mechanisms such as DP Matrix Factorization (DP-MF) have proven to be effective alternatives to DP-SGD in large-epsilon few-epoch training regimes. Significant work has been done to find the best correlated noise strategies, and the current state-of-the-art approach is DP-BandMF, which optimally balances the benefits of privacy amplification and noise correlation. Despite it's utility advantages, severe scalability limitations prevent this mechanism from handling large-scale training scenarios where the number of training iterations may exceed $10^4$ and the number of model parameters may exceed $10^7$. In this work, we present techniques to scale up DP-BandMF along these two dimensions, significantly extending it's reach and enabling it to handle settings with virtually any number of model parameters and training iterations, with negligible utility degradation.
Diversifying Human Pose in Synthetic Data for Aerial-view Human Detection
Venue: arXiv cs.CV 2025
Year: 2025

Synthetic data generation has emerged as a promising solution to the data scarcity issue in aerial-view human detection. However, creating datasets that accurately reflect varying real-world human appearances, particularly diverse poses, remains challenging and labor-intensive. To address this, we propose SynPoseDiv, a novel framework that diversifies human poses within existing synthetic datasets. SynPoseDiv tackles two key challenges: generating realistic, diverse 3D human poses using a diffusion-based pose generator, and producing images of virtual characters in novel poses through a source-to-target image translator. The framework incrementally transitions characters into new poses using optimized pose sequences identified via Dijkstra's algorithm. Experiments demonstrate that SynPoseDiv significantly improves detection accuracy across multiple aerial-view human detection benchmarks, especially in low-shot scenarios, and remains effective regardless of the training approach or dataset size.
Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexity
Venue: arXiv cs.LG 2025
Year: 2025

Diffusion models have become a leading method for generative modeling of both image and scientific data. As these models are costly to train and \emph{evaluate}, reducing the inference cost for diffusion models remains a major goal. Inspired by the recent empirical success in accelerating diffusion models via the parallel sampling technique~\cite{shih2024parallel}, we propose to divide the sampling process into $\mathcal{O}(1)$ blocks with parallelizable Picard iterations within each block. Rigorous theoretical analysis reveals that our algorithm achieves $\widetilde{\mathcal{O}}(\mathrm{poly} \log d)$ overall time complexity, marking \emph{the first implementation with provable sub-linear complexity w.r.t. the data dimension $d$}. Our analysis is based on a generalized version of Girsanov's theorem and is compatible with both the SDE and probability flow ODE implementations. Our results shed light on the potential of fast and efficient sampling of high-dimensional data on fast-evolving modern large-memory GPU clusters.
Multi-Player Approaches for Dueling Bandits
Venue: arXiv cs.LG 2025
Year: 2025

Various approaches have emerged for multi-armed bandits in distributed systems. The multiplayer dueling bandit problem, common in scenarios with only preference-based information like human feedback, introduces challenges related to controlling collaborative exploration of non-informative arm pairs, but has received little attention. To fill this gap, we demonstrate that the direct use of a Follow Your Leader black-box approach matches the lower bound for this setting when utilizing known dueling bandit algorithms as a foundation. Additionally, we analyze a message-passing fully distributed approach with a novel Condorcet-winner recommendation protocol, resulting in expedited exploration in many cases. Our experimental comparisons reveal that our multiplayer algorithms surpass single-player benchmark algorithms, underscoring their efficacy in addressing the nuanced challenges of the multiplayer dueling bandit setting.
Exploring a Multimodal Fusion-based Deep Learning Network for Detecting Facial Palsy
Venue: arXiv cs.CV 2025
Year: 2025

Algorithmic detection of facial palsy offers the potential to improve current practices, which usually involve labor-intensive and subjective assessment by clinicians. In this paper, we present a multimodal fusion-based deep learning model that utilizes unstructured data (i.e. an image frame with facial line segments) and structured data (i.e. features of facial expressions) to detect facial palsy. We then contribute to a study to analyze the effect of different data modalities and the benefits of a multimodal fusion-based approach using videos of 21 facial palsy patients. Our experimental results show that among various data modalities (i.e. unstructured data - RGB images and images of facial line segments and structured data - coordinates of facial landmarks and features of facial expressions), the feed-forward neural network using features of facial expression achieved the highest precision of 76.22 while the ResNet-based model using images of facial line segments achieved the highest recall of 83.47. When we leveraged both images of facial line segments and features of facial expressions, our multimodal fusion-based deep learning model slightly improved the precision score to 77.05 at the expense of a decrease in the recall score.
On Sequential Maximum a Posteriori Inference for Continual Learning
Venue: arXiv cs.LG 2025
Year: 2025

We formulate sequential maximum a posteriori inference as a recursion of loss functions and reduce the problem of continual learning to approximating the previous loss function. We then propose two coreset-free methods: autodiff quadratic consolidation, which uses an accurate and full quadratic approximation, and neural consolidation, which uses a neural network approximation. These methods are not scalable with respect to the neural network size, and we study them for classification tasks in combination with a fixed pre-trained feature extractor. We also introduce simple but challenging classical task sequences based on Iris and Wine datasets. We find that neural consolidation performs well in the classical task sequences, where the input dimension is small, while autodiff quadratic consolidation performs consistently well in image task sequences with a fixed pre-trained feature extractor, achieving comparable performance to joint maximum a posteriori training in many cases.
Demystifying amortized causal discovery with transformers
Venue: arXiv cs.LG 2025
Year: 2025

Supervised learning approaches for causal discovery from observational data often achieve competitive performance despite seemingly avoiding explicit assumptions that traditional methods make for identifiability. In this work, we investigate CSIvA (Ke et al., 2023), a transformer-based model promising to train on synthetic data and transfer to real data. First, we bridge the gap with existing identifiability theory and show that constraints on the training data distribution implicitly define a prior on the test observations. Consistent with classical approaches, good performance is achieved when we have a good prior on the test data, and the underlying model is identifiable. At the same time, we find new trade-offs. Training on datasets generated from different classes of causal models, unambiguously identifiable in isolation, improves the test generalization. Performance is still guaranteed, as the ambiguous cases resulting from the mixture of identifiable causal models are unlikely to occur (which we formally prove). Overall, our study finds that amortized causal discovery still needs to obey identifiability theory, but it also differs from classical methods in how the assumptions are formulated, trading more reliance on assumptions on the noise type for fewer hypotheses on the mechanisms.
Delta-modular ILP Problems of Bounded Codimension, Discrepancy, and Convolution (new version)
Venue: arXiv cs.CC 2025
Year: 2025

For integers $k,n \geq 0$ and a cost vector $c \in Z^n$, we study two fundamental integer linear programming (ILP) problems: \[ \text{(Standard Form)} \quad \max\bigl\{c^\top x \colon Ax = b,\ x \in Z^n_{\geq 0}\bigr\} \text{ with } A \in Z^{k \times n}, \text{rank}(A) = k, b \in Z^k, \] \[ \text{(Canonical Form)} \quad \max\bigl\{c^\top x \colon Ax \leq b,\ x \in Z^n\bigr\} \text{ with } A \in Z^{(n+k) \times n}, \text{rank}(A) = n, b \in Z^{n+k}. \] We present improved algorithms for both problems and their feasibility versions, parameterized by $k$ and $Δ$, where $Δ$ denotes the maximum absolute value of $\text{rank}(A) \times \text{rank}(A)$ subdeterminants of $A$. Our main complexity results, stated in terms of required arithmetic operations, are: \[ \text{Optimization:}\quad O(\log k)^{2k} \cdot Δ^2 / 2^{Ω(\sqrt{\log Δ})} + 2^{O(k)} \cdot \text{poly}(φ), \] \[ \text{Feasibility:} \quad O(\log k)^k \cdot Δ\cdot (\log Δ)^3 + 2^{O(k)} \cdot \text{poly}(φ), \] where $φ$ represents the input size measured by the bit-encoding length of $(A,b,c)$. We also examine several special cases when $k \in \{0,1\}$, which have important applications in: expected computational complexity of ILP with varying right-hand side $b$, ILP problems with generic constraint matrices, ILP problems on simplices. Our results yield improved complexity bounds for these specific scenarios. As independent contributions, we present: An $n^2/2^{Ω(\sqrt{\log n})}$-time algorithm for the tropical convolution problem on sequences indexed by elements of a finite Abelian group of order $n$; A complete and self-contained error analysis of the generalized DFT over Abelian groups in the Word-RAM model.
The Poisson Midpoint Method for Langevin Dynamics: Provably Efficient Discretization for Diffusion Models
Venue: arXiv cs.LG 2025
Year: 2025

Langevin Dynamics is a Stochastic Differential Equation (SDE) central to sampling and generative modeling and is implemented via time discretization. Langevin Monte Carlo (LMC), based on the Euler-Maruyama discretization, is the simplest and most studied algorithm. LMC can suffer from slow convergence - requiring a large number of steps of small step-size to obtain good quality samples. This becomes stark in the case of diffusion models where a large number of steps gives the best samples, but the quality degrades rapidly with smaller number of steps. Randomized Midpoint Method has been recently proposed as a better discretization of Langevin dynamics for sampling from strongly log-concave distributions. However, important applications such as diffusion models involve non-log concave densities and contain time varying drift. We propose its variant, the Poisson Midpoint Method, which approximates a small step-size LMC with large step-sizes. We prove that this can obtain a quadratic speed up of LMC under very weak assumptions. We apply our method to diffusion models for image generation and show that it maintains the quality of DDPM with 1000 neural network calls with just 50-80 neural network calls and outperforms ODE based methods with similar compute.
A Comparative Study on Multi-task Uncertainty Quantification in Semantic Segmentation and Monocular Depth Estimation
Venue: arXiv cs.CV 2025
Year: 2025

Deep neural networks excel in perception tasks such as semantic segmentation and monocular depth estimation, making them indispensable in safety-critical applications like autonomous driving and industrial inspection. However, they often suffer from overconfidence and poor explainability, especially for out-of-domain data. While uncertainty quantification has emerged as a promising solution to these challenges, multi-task settings have yet to be explored. In an effort to shed light on this, we evaluate Monte Carlo Dropout, Deep Sub-Ensembles, and Deep Ensembles for joint semantic segmentation and monocular depth estimation. Thereby, we reveal that Deep Ensembles stand out as the preferred choice, particularly in out-of-domain scenarios, and show the potential benefit of multi-task learning with regard to the uncertainty quality in comparison to solving both tasks separately. Additionally, we highlight the impact of employing different uncertainty thresholds to classify pixels as certain or uncertain, with the median uncertainty emerging as a robust default.
Smoke and Mirrors in Causal Downstream Tasks
Venue: arXiv cs.LG 2025
Year: 2025

Machine Learning and AI have the potential to transform data-driven scientific discovery, enabling accurate predictions for several scientific phenomena. As many scientific questions are inherently causal, this paper looks at the causal inference task of treatment effect estimation, where the outcome of interest is recorded in high-dimensional observations in a Randomized Controlled Trial (RCT). Despite being the simplest possible causal setting and a perfect fit for deep learning, we theoretically find that many common choices in the literature may lead to biased estimates. To test the practical impact of these considerations, we recorded ISTAnt, the first real-world benchmark for causal inference downstream tasks on high-dimensional observations as an RCT studying how garden ants (Lasius neglectus) respond to microparticles applied onto their colony members by hygienic grooming. Comparing 6 480 models fine-tuned from state-of-the-art visual backbones, we find that the sampling and modeling choices significantly affect the accuracy of the causal estimate, and that classification accuracy is not a proxy thereof. We further validated the analysis, repeating it on a synthetically generated visual data set controlling the causal model. Our results suggest that future benchmarks should carefully consider real downstream scientific questions, especially causal ones. Further, we highlight guidelines for representation learning methods to help answer causal questions in the sciences.
How Do the Architecture and Optimizer Affect Representation Learning? On the Training Dynamics of Representations in Deep Neural Networks
Venue: arXiv cs.LG 2025
Year: 2025

In this paper, we elucidate how representations in deep neural networks (DNNs) evolve during training. Our focus is on overparameterized learning settings where the training continues much after the trained DNN starts to perfectly fit its training data. We examine the evolution of learned representations along the entire training process. We explore the representational similarity of DNN layers, each layer with respect to its own representations throughout the training process. For this, we use two similarity metrics: (1) The centered kernel alignment (CKA) similarity; (2) Similarity of decision regions of linear classifier probes that we train for the DNN layers. We visualize and analyze the decision regions of the DNN output and the layer probes during the DNN training to show how they geometrically evolve. Our extensive experiments discover training dynamics patterns that can emerge in layers depending on the relative layer-depth, architecture and optimizer. Among our findings: (i) The training phases, including those related to memorization, are more distinguishable in SGD training than in Adam training, and for Vision Transformer (ViT) than for ResNet; (ii) Unlike ResNet, the ViT layers have synchronized dynamics of representation learning.
Green AI in Action: Strategic Model Selection for Ensembles in Production
Venue: arXiv cs.LG 2025
Year: 2025

Integrating Artificial Intelligence (AI) into software systems has significantly enhanced their capabilities while escalating energy demands. Ensemble learning, combining predictions from multiple models to form a single prediction, intensifies this problem due to cumulative energy consumption. This paper presents a novel approach to model selection that addresses the challenge of balancing the accuracy of AI models with their energy consumption in a live AI ensemble system. We explore how reducing the number of models or improving the efficiency of model usage within an ensemble during inference can reduce energy demands without substantially sacrificing accuracy. This study introduces and evaluates two model selection strategies, Static and Dynamic, for optimizing ensemble learning systems performance while minimizing energy usage. Our results demonstrate that the Static strategy improves the F1 score beyond the baseline, reducing average energy usage from 100% from the full ensemble to 62%. The Dynamic strategy further enhances F1 scores, using on average 76% compared to 100% of the full ensemble. Moreover, we propose an approach that balances accuracy with resource consumption, significantly reducing energy usage without substantially impacting accuracy. This method decreased the average energy usage of the Static strategy from approximately 62% to 14%, and for the Dynamic strategy, from around 76% to 57%. Our field study of Green AI using an operational AI system developed by a large professional services provider shows the practical applicability of adopting energy-conscious model selection strategies in live production environments.
Joint Communication and Over-the-Air Computation for Semi-Federated Learning Towards Scalable AI in Computing-Heterogeneous IoT Systems
Venue: arXiv cs.NI 2025
Year: 2025

The proliferation of Internet of Things (IoT) systems demands scalable artificial intelligence (AI) solutions that can operate in computing-heterogeneous environments with diverse hardware capabilities and non-independent and identically distributed data. This paper proposes a semi-federated learning (SemiFL) framework that integrates centralized learning (CL) and federated learning (FL) to enable efficient model training across heterogeneous IoT devices. In SemiFL, only devices with sufficient computational resources are designated for local model training (referred to as FL users), while the remaining devices transmit raw data to a base station (BS) for remote computation (referred to as CL users). This collaborative computing framework enables all IoT devices to participate in global model training, regardless of their computational capabilities and data distributions. Furthermore, to alleviate radio resource scarcity of SemiFL, we propose a joint communication and over-the-air computation design that unifies wireless transmission and model aggregation. This approach reduces latency and enhances spectrum efficiency by allowing simultaneous communication and computation over the air. Furthermore, we design a transceiver architecture that integrates receive beamforming and successive interference cancellation to mitigate multi-user interference, ensuring reliable aggregation at the BS. Subsequently, two scenarios are examined to assess the efficacy of SemiFL within IoT systems. Simulation results demonstrate that the proposed next-generation multiple access (NGMA)-based SemiFL outperforms the fixed beamforming-based FL by 8% and the AirComp-based FL by 6.4%, demonstrating its superior learning efficiency and convergence performance.
Generalized Compressed Sensing for Image Reconstruction with Diffusion Probabilistic Models
Venue: arXiv cs.CV 2025
Year: 2025

We examine the problem of selecting a small set of linear measurements for reconstructing high-dimensional signals. Well-established methods for optimizing such measurements include principal component analysis (PCA), independent component analysis (ICA) and compressed sensing (CS) based on random projections, all of which rely on axis- or subspace-aligned statistical characterization of the signal source. However, many naturally occurring signals, including photographic images, contain richer statistical structure. To exploit such structure, we introduce a general method for obtaining an optimized set of linear measurements for efficient image reconstruction, where the signal statistics are expressed by the prior implicit in a neural network trained to perform denoising (known as a "diffusion model"). We demonstrate that the optimal measurements derived for two natural image datasets differ from those of PCA, ICA, or CS, and result in substantially lower mean squared reconstruction error. Interestingly, the marginal distributions of the measurement values are asymmetrical (skewed), substantially more so than those of previous methods. We also find that optimizing with respect to perceptual loss, as quantified by structural similarity (SSIM), leads to measurements different from those obtained when optimizing for MSE. Our results highlight the importance of incorporating the specific statistical regularities of natural signals when designing effective linear measurements.