Miracle-dz/UniFAR-FTUs huggingface dataset:
Peeking Behind the Curtains of Residual Learning

The utilization of residual learning has become widespread in deep and scalable neural nets. However, the fundamental principles that contribute to the success of residual learning remain elusive, thus hindering effective training of plain nets with depth scalability. In this paper, we peek behind the curtains of residual learning by uncovering the"dissipating inputs"phenomenon that leads to convergence failure in plain neural nets: the input is gradually compromised through plain layers due to non-linearities, resulting in challenges of learning feature representations. We theoretically demonstrate how plain neural nets degenerate the input to random noise and emphasize the significance of a residual connection that maintains a better lower bound of surviving neurons as a solution. With our theoretical discoveries, we propose"The Plain Neural Net Hypothesis"(PNNH) that identifies the internal path across non-linear layers as the most critical part in residual learning, and establishes a paradigm to support the training of deep plain neural nets devoid of residual connections. We thoroughly evaluate PNNH-enabled CNN architectures and Transformers on popular vision benchmarks, showing on-par accuracy, up to 0.3% higher training throughput, and 2x better parameter efficiency compared to ResNets and vision Transformers.
First Result for Dark Matter Search by WINERED

The identity of dark matter has been a mystery in astronomy, cosmology, and particle theory for about a century. Bessho, Ikeda, and Yin (2022), three of the current authors, proposed using the state-of-the-art infrared spectrographs, including WINERED at $6.5$m Magellan Clay telescope and NIRSpec at James Webb Space Telescope, as efficient detectors for the indirect detection of dark matter with the mass around eV by measuring the line photons from the dark matter two body decays. Applying this concept, we have performed spectrographic observations of dwarf spheroidal galaxies (dSphs) Leo V and Tucana II using WINERED by utilizing an object-sky-object nodding observation technique for background subtraction. We present the first result from this dark matter search. Employing zero consistent flux data after the sky subtraction, we have established one of the most stringent limits to date on dark matter decaying into line photons in the mass range of $1.8-2.7\,$eV. Our data can also be applied to constrain other spectra of photons from the dSphs.
Adjustment Identification Distance: A gadjid for Causal Structure Learning

Evaluating graphs learned by causal discovery algorithms is difficult: The number of edges that differ between two graphs does not reflect how the graphs differ with respect to the identifying formulas they suggest for causal effects. We introduce a framework for developing causal distances between graphs which includes the structural intervention distance for directed acyclic graphs as a special case. We use this framework to develop improved adjustment-based distances as well as extensions to completed partially directed acyclic graphs and causal orders. We develop polynomial-time reachability algorithms to compute the distances efficiently. In our package gadjid (open source at https://github.com/CausalDisco/gadjid), we provide implementations of our distances; they are orders of magnitude faster than the structural intervention distance and thereby provide a success metric for causal discovery that scales to graph sizes that were previously prohibitive.
Beyond the Mud: Datasets and Benchmarks for Computer Vision in Off-Road Racing

Despite significant progress in optical character recognition (OCR) and computer vision systems, robustly recognizing text and identifying people in images taken in unconstrained \emph{in-the-wild} environments remain an ongoing challenge. However, such obstacles must be overcome in practical applications of vision systems, such as identifying racers in photos taken during off-road racing events. To this end, we introduce two new challenging real-world datasets - the off-road motorcycle Racer Number Dataset (RND) and the Muddy Racer re-iDentification Dataset (MUDD) - to highlight the shortcomings of current methods and drive advances in OCR and person re-identification (ReID) under extreme conditions. These two datasets feature over 6,300 images taken during off-road competitions which exhibit a variety of factors that undermine even modern vision systems, namely mud, complex poses, and motion blur. We establish benchmark performance on both datasets using state-of-the-art models. Off-the-shelf models transfer poorly, reaching only 15% end-to-end (E2E) F1 score on text spotting, and 33% rank-1 accuracy on ReID. Fine-tuning yields major improvements, bringing model performance to 53% F1 score for E2E text spotting and 79% rank-1 accuracy on ReID, but still falls short of good performance. Our analysis exposes open problems in real-world OCR and ReID that necessitate domain-targeted techniques. With these datasets and analysis of model limitations, we aim to foster innovations in handling real-world conditions like mud and complex poses to drive progress in robust computer vision. All data was sourced from PerformancePhoto.co, a website used by professional motorsports photographers, racers, and fans. The top-performing text spotting and ReID models are deployed on this platform to power real-time race photo search.
Advancing Data-driven Weather Forecasting: Time-Sliding Data Augmentation of ERA5

Modern deep learning techniques, which mimic traditional numerical weather prediction (NWP) models and are derived from global atmospheric reanalysis data, have caused a significant revolution within a few years. In this new paradigm, our research introduces a novel strategy that deviates from the common dependence on high-resolution data, which is often constrained by computational resources, and instead utilizes low-resolution data (2.5 degrees) for global weather prediction and climate data analysis. Our main focus is evaluating data-driven weather prediction (DDWP) frameworks, specifically addressing sample size adequacy, structural improvements to the model, and the ability of climate data to represent current climatic trends. By using the Adaptive Fourier Neural Operator (AFNO) model via FourCastNet and a proposed time-sliding method to inflate the dataset of the ECMWF Reanalysis v5 (ERA5), this paper improves on conventional approaches by adding more variables and a novel approach to data augmentation and processing. Our findings reveal that despite the lower resolution, the proposed approach demonstrates considerable accuracy in predicting atmospheric conditions, effectively rivaling higher-resolution models. Furthermore, the study confirms the model's proficiency in reflecting current climate trends and its potential in predicting future climatic events, underscoring its utility in climate change strategies. This research marks a pivotal step in the realm of meteorological forecasting, showcasing the feasibility of lower-resolution data in producing reliable predictions and opening avenues for more accessible and inclusive climate modeling. The insights gleaned from this study not only contribute to the advancement of climate science but also lay the groundwork for future innovations in the field.
FESS Loss: Feature-Enhanced Spatial Segmentation Loss for Optimizing Medical Image Analysis

Medical image segmentation is a critical process in the field of medical imaging, playing a pivotal role in diagnosis, treatment, and research. It involves partitioning of an image into multiple regions, representing distinct anatomical or pathological structures. Conventional methods often grapple with the challenge of balancing spatial precision and comprehensive feature representation due to their reliance on traditional loss functions. To overcome this, we propose Feature-Enhanced Spatial Segmentation Loss (FESS Loss), that integrates the benefits of contrastive learning (which extracts intricate features, particularly in the nuanced domain of medical imaging) with the spatial accuracy inherent in the Dice loss. The objective is to augment both spatial precision and feature-based representation in the segmentation of medical images. FESS Loss signifies a notable advancement, offering a more accurate and refined segmentation process, ultimately contributing to heightened precision in the analysis of medical images. Further, FESS loss demonstrates superior performance in limited annotated data availability scenarios often present in the medical domain.
Confronting Discrimination in Classification: Smote Based on Marginalized Minorities in the Kernel Space for Imbalanced Data

Financial fraud detection poses a typical challenge characterized by class imbalance, where instances of fraud are extremely rare but can lead to unpredictable economic losses if misidentified. Precisely classifying these critical minority samples represents a challenging task within the classification. The primary difficulty arises from mainstream classifiers, which often exhibit"implicit discrimination"against minority samples in evaluation metrics, which results in frequent misclassifications, and the key to the problem lies in the overlap of feature spaces between majority and minority samples. To address these challenges, oversampling is a feasible solution, yet current classical oversampling methods often lack the necessary caution in sample selection, exacerbating feature space overlap. In response, we propose a novel classification oversampling approach based on the decision boundary and sample proximity relationships. This method carefully considers the distance between critical samples and the decision hyperplane, as well as the density of surrounding samples, resulting in an adaptive oversampling strategy in the kernel space. Finally, we test the proposed method on a classic financial fraud dataset, and the results show that our proposed method provides an effective and robust solution that can improve the classification accuracy of minorities.
Persistence diagrams for exploring the shape variability of abdominal aortic aneurysms

Abdominal Aortic Aneurysm consists of a permanent dilation in the abodminal portion of the aorta and, along with its associated pathologies like calcifications and intraluminal thrombi, is one of the most important pathologies of the circulatory system. The shape of the aorta is among the primary drivers for these health issues, with particular reference to all the characteristics which affects the hemodynamics. Starting from the computed tomography angiography of a patient, we propose to summarize such information using tools derived from Topological Data Analysis, obtaining persistence diagrams which describe the irregularities of the lumen of the aorta. We showcase the effectiveness of such shape-related descriptors with a series of supervised and unsupervised case studies.
Conditional Information Gain Trellis

Conditional computing processes an input using only part of the neural network's computational units. Learning to execute parts of a deep convolutional network by routing individual samples has several advantages: Reducing the computational burden is an obvious advantage. Furthermore, if similar classes are routed to the same path, that part of the network learns to discriminate between finer differences and better classification accuracies can be attained with fewer parameters. Recently, several papers have exploited this idea to take a particular child of a node in a tree-shaped network or to skip parts of a network. In this work, we follow a Trellis-based approach for generating specific execution paths in a deep convolutional neural network. We have designed routing mechanisms that use differentiable information gain-based cost functions to determine which subset of features in a convolutional layer will be executed. We call our method Conditional Information Gain Trellis (CIGT). We show that our conditional execution mechanism achieves comparable or better model performance compared to unconditional baselines, using only a fraction of the computational resources.
Uncertainty Quantification for Forward and Inverse Problems of PDEs via Latent Global Evolution

Deep learning-based surrogate models have demonstrated remarkable advantages over classical solvers in terms of speed, often achieving speedups of 10 to 1000 times over traditional partial differential equation (PDE) solvers. However, a significant challenge hindering their widespread adoption in both scientific and industrial domains is the lack of understanding about their prediction uncertainties, particularly in scenarios that involve critical decision making. To address this limitation, we propose a method that integrates efficient and precise uncertainty quantification into a deep learning-based surrogate model. Our method, termed Latent Evolution of PDEs with Uncertainty Quantification (LE-PDE-UQ), endows deep learning-based surrogate models with robust and efficient uncertainty quantification capabilities for both forward and inverse problems. LE-PDE-UQ leverages latent vectors within a latent space to evolve both the system's state and its corresponding uncertainty estimation. The latent vectors are decoded to provide predictions for the system's state as well as estimates of its uncertainty. In extensive experiments, we demonstrate the accurate uncertainty quantification performance of our approach, surpassing that of strong baselines including deep ensembles, Bayesian neural network layers, and dropout. Our method excels at propagating uncertainty over extended auto-regressive rollouts, making it suitable for scenarios involving long-term predictions. Our code is available at: https://github.com/AI4Science-WestlakeU/le-pde-uq.
Scribble-based fast weak-supervision and interactive corrections for segmenting whole slide images

This paper proposes a dynamic interactive and weakly supervised segmentation method with minimal user interactions to address two major challenges in the segmentation of whole slide histopathology images. First, the lack of hand-annotated datasets to train algorithms. Second, the lack of interactive paradigms to enable a dialogue between the pathologist and the machine, which can be a major obstacle for use in clinical routine. We therefore propose a fast and user oriented method to bridge this gap by giving the pathologist control over the final result while limiting the number of interactions needed to achieve a good result (over 90\% on all our metrics with only 4 correction scribbles).
A note on double Floquet-Bloch transforms and the far-field asymptotics of Green's functions tailored to periodic structures

We propose a general procedure to study double integrals arising when considering wave propagation in periodic structures. This method, based on a complex deformation of the integration surface to bypass the integrands' singularities, is particularly efficient to estimate the Green's functions of such structures in the far field. We provide several illustrative examples and explicit asymptotic formulae. Special attention is devoted to the pathological cases of degeneracies, such as Dirac conical points for instance.
Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast

A multimodal large language model (MLLM) agent can receive instructions, capture images, retrieve histories from memory, and decide which tools to use. Nonetheless, red-teaming efforts have revealed that adversarial images/prompts can jailbreak an MLLM and cause unaligned behaviors. In this work, we report an even more severe safety issue in multi-agent environments, referred to as infectious jailbreak. It entails the adversary simply jailbreaking a single agent, and without any further intervention from the adversary, (almost) all agents will become infected exponentially fast and exhibit harmful behaviors. To validate the feasibility of infectious jailbreak, we simulate multi-agent environments containing up to one million LLaVA-1.5 agents, and employ randomized pair-wise chat as a proof-of-concept instantiation for multi-agent interaction. Our results show that feeding an (infectious) adversarial image into the memory of any randomly chosen agent is sufficient to achieve infectious jailbreak. Finally, we derive a simple principle for determining whether a defense mechanism can provably restrain the spread of infectious jailbreak, but how to design a practical defense that meets this principle remains an open question to investigate. Our project page is available at https://sail-sg.github. io/Agent-Smith/.
Faster Repeated Evasion Attacks in Tree Ensembles

Tree ensembles are one of the most widely used model classes. However, these models are susceptible to adversarial examples, i.e., slightly perturbed examples that elicit a misprediction. There has been significant research on designing approaches to construct such examples for tree ensembles. But this is a computationally challenging problem that often must be solved a large number of times (e.g., for all examples in a training set). This is compounded by the fact that current approaches attempt to find such examples from scratch. In contrast, we exploit the fact that multiple similar problems are being solved. Specifically, our approach exploits the insight that adversarial examples for tree ensembles tend to perturb a consistent but relatively small set of features. We show that we can quickly identify this set of features and use this knowledge to speedup constructing adversarial examples.
A Systematic Review of Data-to-Text NLG

This systematic review undertakes a comprehensive analysis of current research on data-to-text generation, identifying gaps, challenges, and future directions within the field. Relevant literature in this field on datasets, evaluation metrics, application areas, multilingualism, language models, and hallucination mitigation methods is reviewed. Various methods for producing high-quality text are explored, addressing the challenge of hallucinations in data-to-text generation. These methods include re-ranking, traditional and neural pipeline architecture, planning architectures, data cleaning, controlled generation, and modification of models and training techniques. Their effectiveness and limitations are assessed, highlighting the need for universally applicable strategies to mitigate hallucinations. The review also examines the usage, popularity, and impact of datasets, alongside evaluation metrics, with an emphasis on both automatic and human assessment. Additionally, the evolution of data-to-text models, particularly the widespread adoption of transformer models, is discussed. Despite advancements in text quality, the review emphasizes the importance of research in low-resourced languages and the engineering of datasets in these languages to promote inclusivity. Finally, several application domains of data-to-text are highlighted, emphasizing their relevance in such domains. Overall, this review serves as a guiding framework for fostering innovation and advancing data-to-text generation.
Randomized Algorithms for Symmetric Nonnegative Matrix Factorization

Symmetric Nonnegative Matrix Factorization (SymNMF) is a technique in data analysis and machine learning that approximates a symmetric matrix with a product of a nonnegative, low-rank matrix and its transpose. To design faster and more scalable algorithms for SymNMF we develop two randomized algorithms for its computation. The first algorithm uses randomized matrix sketching to compute an initial low-rank input matrix and proceeds to use this input to rapidly compute a SymNMF. The second algorithm uses randomized leverage score sampling to approximately solve constrained least squares problems. Many successful methods for SymNMF rely on (approximately) solving sequences of constrained least squares problems. We prove theoretically that leverage score sampling can approximately solve nonnegative least squares problems to a chosen accuracy with high probability. Finally we demonstrate that both methods work well in practice by applying them to graph clustering tasks on large real world data sets. These experiments show that our methods approximately maintain solution quality and achieve significant speed ups for both large dense and large sparse problems.
On Frobenius liftability of surface singularities

We show that a plt surface singularity $(P\in X,B)$ is $F$-liftable if and only if it is $F$-pure and is not a rational double point of type $E_8^1$ in characteristic $p=5$. As a consequence, we prove the logarithmic extension theorem for $F$-pure surface pairs and Bogomolov-Sommese vanishing for globally $F$-split surface pairs. These results were previously known to hold in characteristic $p>5$.
A Generalized Approach to Online Convex Optimization

In this paper, we analyze the problem of online convex optimization in different settings. We show that any algorithm for online linear optimization with fully adaptive adversaries is an algorithm for online convex optimization. We also show that any such algorithm that requires full-information feedback may be transformed to an algorithm with semi-bandit feedback with comparable regret bound. We further show that algorithms that are designed for fully adaptive adversaries using deterministic semi-bandit feedback can obtain similar bounds using only stochastic semi-bandit feedback when facing oblivious adversaries. We use this to describe general meta-algorithms to convert first order algorithms to zeroth order algorithms with comparable regret bounds. Our framework allows us to analyze online optimization in various settings, such full-information feedback, bandit feedback, stochastic regret, adversarial regret and various forms of non-stationary regret. Using our analysis, we provide the first efficient projection-free online convex optimization algorithm using linear optimization oracles.
Migration to Microservices: A Comparative Study of Decomposition Strategies and Analysis Metrics

The microservices architectural style is widely favored for its scalability, reusability, and easy maintainability, prompting increased adoption by developers. However, transitioning from a monolithic to a microservices-based architecture is intricate and costly. In response, we present a novel method utilizing clustering to identify potential microservices in a given monolithic application. Our approach employs a density-based clustering algorithm considering static analysis, structural, and semantic relationships between classes, ensuring a functionally and contextually coherent partitioning. To assess the reliability of our microservice suggestion approach, we conducted an in-depth analysis of hyperparameter sensitivity and compared it with two established clustering algorithms. A comprehensive comparative analysis involved seven applications, evaluating against six baselines, utilizing a dataset of four open-source Java projects. Metrics assessed the quality of generated microservices. Furthermore, we meticulously compared our suggested microservices with manually identified ones in three microservices-based applications. This comparison provided a nuanced understanding of our approach's efficacy and reliability. Our methodology demonstrated promising outcomes, showcasing remarkable effectiveness and commendable stability.