Miracle-dz/UniFAR-FTUs huggingface dataset:
Robustly Learning a Single Neuron via Sharpness

We study the problem of learning a single neuron with respect to the $L_2^2$-loss in the presence of adversarial label noise. We give an efficient algorithm that, for a broad family of activations including ReLUs, approximates the optimal $L_2^2$-error within a constant factor. Our algorithm applies under much milder distributional assumptions compared to prior work. The key ingredient enabling our results is a novel connection to local error bounds from optimization theory.
Additive Multi-Index Gaussian process modeling, with application to multi-physics surrogate modeling of the quark-gluon plasma

The Quark-Gluon Plasma (QGP) is a unique phase of nuclear matter, theorized to have filled the Universe shortly after the Big Bang. A critical challenge in studying the QGP is that, to reconcile experimental observables with theoretical parameters, one requires many simulation runs of a complex physics model over a high-dimensional parameter space. Each run is computationally very expensive, requiring thousands of CPU hours, thus limiting physicists to only several hundred runs. Given limited training data for high-dimensional prediction, existing surrogate models often yield poor predictions with high predictive uncertainties, leading to imprecise scientific findings. To address this, we propose a new Additive Multi-Index Gaussian process (AdMIn-GP) model, which leverages a flexible additive structure on low-dimensional embeddings of the parameter space. This is guided by prior scientific knowledge that the QGP is dominated by multiple distinct physical phenomena (i.e., multiphysics), each involving a small number of latent parameters. The AdMIn-GP models for such embedded structures within a flexible Bayesian nonparametric framework, which facilitates efficient model fitting via a carefully constructed variational inference approach with inducing points. We show the effectiveness of the AdMIn-GP via a suite of numerical experiments and our QGP application, where we demonstrate considerably improved surrogate modeling performance over existing models.
Coordinated Dynamic Bidding in Repeated Second-Price Auctions with Budgets

In online ad markets, a rising number of advertisers are employing bidding agencies to participate in ad auctions. These agencies are specialized in designing online algorithms and bidding on behalf of their clients. Typically, an agency usually has information on multiple advertisers, so she can potentially coordinate bids to help her clients achieve higher utilities than those under independent bidding. In this paper, we study coordinated online bidding algorithms in repeated second-price auctions with budgets. We propose algorithms that guarantee every client a higher utility than the best she can get under independent bidding. We show that these algorithms achieve maximal coalition welfare and discuss bidders' incentives to misreport their budgets, in symmetric cases. Our proofs combine the techniques of online learning and equilibrium analysis, overcoming the difficulty of competing with a multi-dimensional benchmark. The performance of our algorithms is further evaluated by experiments on both synthetic and real data. To the best of our knowledge, we are the first to consider bidder coordination in online repeated auctions with constraints.
C-boundary links up to six crossings

An oriented link is called $\mathbb C$-boundary if it is realizable as $(\partial B,A\cap\partial B)$ where $A$ is an algebraic curve in $\mathbb C^2$ and $B$ is an embedded $4$-ball. This notion was introduced by Michel Boileau and Lee Rudolph in 1995. In a recent joint paper with N.G. Kruzhilin we gave a complete classification of $\mathbb C$-boundaries with at most 5 crossings. In the present paper a more regular method of construction of $\mathbb C$-boundaries is proposed and the classification is extended up to 6 crossings.
Non-visual Effects of Road Lighting CCT on Driver's Mood, Alertness, Fatigue and Reaction Time: A Comprehensive Neuroergonomic Evaluation Study

Good nighttime road lighting is critical for driving safety. To improve the quality of nighttime road lighting, this study used the triangulation method by fusing"EEG evaluation + subjective evaluation + behavioral evaluation"to qualitatively and quantitatively investigate the response characteristics of different correlated color temperature (CCT) (3500K, 4500K, 5500K, 6500K) on drivers' non-visual indicators (mood, alertness, fatigue and reaction time) under specific driving conditions (monotonous driving; waiting for red light and traffic jam; car-following task). The results showed that the CCT and Task interaction effect is mainly related to individual alertness and reaction time. Individual subjective emotional experience, subjective visual comfort and psychological security are more responsive to changes in CCT than individual mental fatigue and visual fatigue. The subjective and objective evaluation results demonstrated that the EEG evaluation indices used in this study could objectively reflect the response characteristics of various non-visual indicators. The findings also revealed that moderate CCT (4500K) appears to be the most beneficial to drivers in maintaining an ideal state of mind and body during nighttime driving, which is manifested as: good mood experience; it helps drivers maintain a relatively stable level of alterness and to respond quickly to external stimuli; both mental and visual fatigue were relatively low. This study extends nighttime road lighting design research from the perspective of non-visual effects by using comprehensive neuroergonomic evaluation methods, and it provides a theoretical and empirical basis for the future development of a humanized urban road lighting design evaluation system.
SqueezeLLM: Dense-and-Sparse Quantization

Generative Large Language Models (LLMs) have demonstrated remarkable results for a wide range of tasks. However, deploying these models for inference has been a significant challenge due to their unprecedented resource requirements. This has forced existing deployment frameworks to use multi-GPU inference pipelines, which are often complex and costly, or to use smaller and less performant models. In this work, we demonstrate that the main bottleneck for generative inference with LLMs is memory bandwidth, rather than compute, specifically for single batch inference. While quantization has emerged as a promising solution by representing model weights with reduced precision, previous efforts have often resulted in notable performance degradation. To address this, we introduce SqueezeLLM, a post-training quantization framework that not only enables lossless compression to ultra-low precisions of up to 3-bit, but also achieves higher quantization performance under the same memory constraint. Our framework incorporates two novel ideas: (i) sensitivity-based non-uniform quantization, which searches for the optimal bit precision assignment based on second-order information; and (ii) the Dense-and-Sparse decomposition that stores outliers and sensitive weight values in an efficient sparse format. When applied to the LLaMA models, our 3-bit quantization significantly reduces the perplexity gap from the FP16 baseline by up to 2.1x as compared to the state-of-the-art methods with the same memory requirement. Furthermore, when deployed on an A6000 GPU, our quantized models achieve up to 2.3x speedup compared to the baseline. Our code is open-sourced and available online.
Hyperbolic Graph Diffusion Model

Diffusion generative models (DMs) have achieved promising results in image and graph generation. However, real-world graphs, such as social networks, molecular graphs, and traffic graphs, generally share non-Euclidean topologies and hidden hierarchies. For example, the degree distributions of graphs are mostly power-law distributions. The current latent diffusion model embeds the hierarchical data in a Euclidean space, which leads to distortions and interferes with modeling the distribution. Instead, hyperbolic space has been found to be more suitable for capturing complex hierarchical structures due to its exponential growth property. In order to simultaneously utilize the data generation capabilities of diffusion models and the ability of hyperbolic embeddings to extract latent hierarchical distributions, we propose a novel graph generation method called, Hyperbolic Graph Diffusion Model (HGDM), which consists of an auto-encoder to encode nodes into successive hyperbolic embeddings, and a DM that operates in the hyperbolic latent space. HGDM captures the crucial graph structure distributions by constructing a hyperbolic potential node space that incorporates edge information. Extensive experiments show that HGDM achieves better performance in generic graph and molecule generation benchmarks, with a 48% improvement in the quality of graph generation with highly hierarchical structures.
Extending adjacency matrices to 3D with triangles

Social networks are the fabric of society and the subject of frequent visual analysis. Closed triads represent triangular relationships between three people in a social network and are significant for understanding inherent interconnections and influence within the network. The most common methods for representing social networks (node-link diagrams and adjacency matrices) are not optimal for understanding triangles. We propose extending the adjacency matrix form to 3D for better visualization of network triads. We design a 3D matrix reordering technique and implement an immersive interactive system to assist in visualizing and analyzing closed triads in social networks. A user study and usage scenarios demonstrate that our method provides substantial added value over node-link diagrams in improving the efficiency and accuracy of manipulating and understanding the social network triads.
Low-Resource White-Box Semantic Segmentation of Supporting Towers on 3D Point Clouds via Signature Shape Identification

Research in 3D semantic segmentation has been increasing performance metrics, like the IoU, by scaling model complexity and computational resources, leaving behind researchers and practitioners that (1) cannot access the necessary resources and (2) do need transparency on the model decision mechanisms. In this paper, we propose SCENE-Net, a low-resource white-box model for 3D point cloud semantic segmentation. SCENE-Net identifies signature shapes on the point cloud via group equivariant non-expansive operators (GENEOs), providing intrinsic geometric interpretability. Our training time on a laptop is 85~min, and our inference time is 20~ms. SCENE-Net has 11 trainable geometrical parameters and requires fewer data than black-box models. SCENE--Net offers robustness to noisy labeling and data imbalance and has comparable IoU to state-of-the-art methods. With this paper, we release a 40~000 Km labeled dataset of rural terrain point clouds and our code implementation.
A Theory of Unsupervised Speech Recognition

Unsupervised speech recognition ({pasted macro ‘ASRU’}/) is the problem of learning automatic speech recognition (ASR) systems from unpaired speech-only and text-only corpora. While various algorithms exist to solve this problem, a theoretical framework is missing to study their properties and address such issues as sensitivity to hyperparameters and training instability. In this paper, we proposed a general theoretical framework to study the properties of {pasted macro ‘ASRU’}/ systems based on random matrix theory and the theory of neural tangent kernels. Such a framework allows us to prove various learnability conditions and sample complexity bounds of {pasted macro ‘ASRU’}/. Extensive {pasted macro ‘ASRU’}/ experiments on synthetic languages with three classes of transition graphs provide strong empirical evidence for our theory (code available at https://github.com/cactuswiththoughts/UnsupASRTheory.gitcactuswiththoughts/UnsupASRTheory.git).
Multi-epoch machine learning 2: identifying physical drivers of galaxy properties in simulations

Using a novel machine learning method, we investigate the buildup of galaxy properties in different simulations, and in various environments within a single simulation. The aim of this work is to show the power of this approach at identifying the physical drivers of galaxy properties within simulations. We compare how the stellar mass is dependent on the value of other galaxy and halo properties at different points in time by examining the feature importance values of a machine learning model. By training the model on IllustrisTNG, we show that stars are produced at earlier times in higher density regions of the universe than they are in low density regions. We also apply the technique to the Illustris, EAGLE, and CAMELS simulations. We find that stellar mass is built up in a similar way in EAGLE and IllustrisTNG, but significantly differently in the original Illustris, suggesting that subgrid model physics is more important than the choice of hydrodynamics method. These differences are driven by the efficiency of supernova feedback. Applying principal component analysis to the CAMELS simulations allows us to identify a component associated with the importance of a halo’s gravitational potential and another component representing the time at which galaxies form. We discover that the speed of galactic winds is a more critical subgrid parameter than the total energy per unit star formation. Finally, we find that the Simba black hole feedback model has a larger effect on galaxy formation than the IllustrisTNG black hole feedback model.
Noise Sensitivity of the Minimum Spanning Tree of the Complete Graph

We study the noise sensitivity of the minimum spanning tree (MST) of the $n$-vertex complete graph when edges are assigned independent random weights. It is known that when the graph distance is rescaled by $n^{1/3}$ and vertices are given a uniform measure, the MST converges in distribution in the Gromov-Hausdorff-Prokhorov (GHP) topology. We prove that if the weight of each edge is resampled independently with probability $\varepsilon\gg n^{-1/3}$, then the pair of rescaled minimum spanning trees -- before and after the noise -- converges in distribution to independent random spaces. Conversely, if $\varepsilon\ll n^{-1/3}$, the GHP distance between the rescaled trees goes to $0$ in probability. This implies the noise sensitivity and stability for every property of the MST that corresponds to a continuity set of the random limit. The noise threshold of $n^{-1/3}$ coincides with the critical window of the Erd\H{o}s-R\'enyi random graphs. In fact, these results follow from an analog theorem we prove regarding the minimum spanning forest of critical random graphs.
Unlocking Foundation Models for Privacy-Enhancing Speech Understanding: An Early Study on Low Resource Speech Training Leveraging Label-guided Synthetic Speech Content

Automatic Speech Understanding (ASU) leverages the power of deep learning models for accurate interpretation of human speech, leading to a wide range of speech applications that enrich the human experience. However, training a robust ASU model requires the curation of a large number of speech samples, creating risks for privacy breaches. In this work, we investigate using foundation models to assist privacy-enhancing speech computing. Unlike conventional works focusing primarily on data perturbation or distributed algorithms, our work studies the possibilities of using pre-trained generative models to synthesize speech content as training data with just label guidance. We show that zero-shot learning with training label-guided synthetic speech content remains a challenging task. On the other hand, our results demonstrate that the model trained with synthetic speech samples provides an effective initialization point for low-resource ASU training. This result reveals the potential to enhance privacy by reducing user data collection but using label-guided synthetic speech content.
Malafide: a novel adversarial convolutive noise attack against deepfake and spoofing detection systems

We present Malafide, a universal adversarial attack against automatic speaker verification (ASV) spoofing countermeasures (CMs). By introducing convolutional noise using an optimised linear time-invariant filter, Malafide attacks can be used to compromise CM reliability while preserving other speech attributes such as quality and the speaker's voice. In contrast to other adversarial attacks proposed recently, Malafide filters are optimised independently of the input utterance and duration, are tuned instead to the underlying spoofing attack, and require the optimisation of only a small number of filter coefficients. Even so, they degrade CM performance estimates by an order of magnitude, even in black-box settings, and can also be configured to overcome integrated CM and ASV subsystems. Integrated solutions that use self-supervised learning CMs, however, are more robust, under both black-box and white-box settings.
Low-Complexity Soft-Decision Detection for Combating DFE Burst Errors in IM/DD Links

The deployment of non-binary pulse amplitude modulation (PAM) and soft decision (SD)-forward error correction (FEC) in future intensity-modulation (IM)/direct-detection (DD) links is inevitable. However, high-speed IM/DD links suffer from inter-symbol interference (ISI) due to bandwidth-limited hardware. Traditional approaches to mitigate the effects of ISI are filters and trellis-based algorithms targeting symbol-wise maximum a posteriori (MAP) detection. The former approach includes decision-feedback equalizer (DFE), and the latter includes Max-Log-MAP (MLM) and soft-output Viterbi algorithm (SOVA). Although DFE is easy to implement, it introduces error propagation (EP). Such burst errors distort the log-likelihood ratios (LLRs) required by SD-FEC, causing performance degradation. On the other hand, MLM and SOVA provide near-optimum performance, but their complexity is very high for high-order PAM. In this article, we consider a one-tap partial response channel model, which is relevant for high-speed IM/DD links. We propose to combine DFE with either MLM or SOVA in a low-complexity architecture. The key idea is to allow MLM or SOVA to detect only 3 typical DFE symbol errors, and use the detected error information to generate LLRs in a modified demapper. The proposed structure enables a tradeoff between complexity and performance: i) the complexity of MLM or SOVA is reduced and ii) the decoding penalty due to EP is mitigated. Compared to SOVA detection, the proposed scheme can achieve a significant complexity reduction of up to 94% for PAM-8 transmission. Simulation and experimental results show that the resulting SNR loss is roughly $0.3\! \sim\! 0.4$ dB for PAM-4, and becomes marginal 0.18 dB for PAM-8.
Marking anything: application of point cloud in extracting video target features

Extracting retrievable features from video is of great significance for structured video database construction, video copyright protection and fake video rumor refutation. Inspired by point cloud data processing, this paper proposes a method for marking anything (MA) in the video, which can extract the contour features of any target in the video and convert it into a feature vector with a length of 256 that can be retrieved. The algorithm uses YOLO-v8 algorithm, multi-object tracking algorithm and PointNet++ to extract contour of the video detection target to form spatial point cloud data. Then extract the point cloud feature vector and use it as the retrievable feature of the video detection target. In order to verify the effectiveness and robustness of contour feature, some datasets are crawled from Dou Yin and Kinetics-700 dataset as experimental data. For Dou Yin's homogenized videos, the proposed contour features achieve retrieval accuracy higher than 97% in Top1 return mode. For videos from Kinetics 700, the contour feature also showed good robustness for partial clip mode video tracing.
False discovery proportion envelopes with consistency

We provide new false discovery proportion (FDP) confidence envelopes in several multiple testing settings relevant to modern high dimensional-data methods. We revisit the scenarios considered in the recent work of \cite{katsevich2020simultaneous}(top-$k$, preordered -- including knockoffs -- , online) with a particular emphasis on obtaining FDP bounds that have both non-asymptotical coverage and asymptotical consistency, i.e. converge below the desired level $\alpha$ when applied to a classical $\alpha$-level false discovery rate (FDR) controlling procedure. This way, we derive new bounds that provide improvements over existing ones, both theoretically and practically, and are suitable for situations where at least a moderate number of rejections is expected. These improvements are illustrated with numerical experiments and real data examples. In particular, the improvement is significant in the knockoff setting, which shows the impact of the method for practical use. As side results, we introduce a new confidence envelope for the empirical cumulative distribution function of i.i. d. uniform variables and we provide new power results in sparse cases, both being of independent interest.
Layer Stripping Migration Velocity Analysis Using Accurate Survey Sinking

SUMMARY We present a survey sinking based framework for residual migration velocity analysis. By using a migration algorithm in the temporal frequency domain, our migration algorithm is efficient and allows us to pick velocity perturbations directly. Picking velocity perturbation directly removes the issues associated with estimating velocity perturbations from indirect picking parameters, such as time lags. Our methodology is based on a wave-equation migration method that combines high accuracy with computational efficiency. By relying on survey sinking migration, we avoid the problem of estimating the source wavelet, which is a rarely mentioned problem associated with RTM and shot-gather migration. Our survey sinking framework allows us to robustly estimate the velocity sequentially in relatively thin layers.
Mathematical Modeling of Consumer's Preferences Using Partial Differential Equations

The aim of this paper is to define consumer's preferences from the differentiable viewpoint in the sense of Debreu. In this framework given the marginal rates of substitution we can consider a vector field to represent consumer's preferences in the microeconomic theory. By definition the marginal rates of substitution satisfy a system of first order partial differential equations. For a continuously differentiable vector field that holds the integrability conditions we provide a general method to solve the system. In the special case of integrable preferences these conditions impose symmetry properties in the underlying preferences. Our results allow to characterize consumer's preferences in terms of the indifference map for the following classes: linear, quasi-linear, separable, homothetic, homothetic and separable. We show that this alternative approach is connected with the traditional formulation concerning the representability of preferences by utility functions. Moreover, we deduce even the general expression of utility functions that satisfy the integrability conditions in the context of ordinal utility.