Miracle-dz/UniFAR-FTUs huggingface dataset:
Stationary measures for integrable polymers on a strip

We prove that the stationary measures for the free-energy increment process for the geometric last passage percolation (LPP) and log-gamma polymer model on a diagonal strip is given by a marginal of a two-layer Gibbs measure with a simple and explicit description. This result is shown subject to certain restrictions on the parameters controlling the weights on the boundary of the strip. However, from this description and an analytic continuation argument we are able to access the stationary measure for all boundary parameters. Taking an intermediate disorder limit of the log-gamma polymer stationary measure in a strip we readily recover the conjectural description from arXiv:2105.15178 of the open KPZ stationary measure for all choices of boundary parameters $u,v\in \mathbb{R}$ (thus going beyond the restriction $u+v\geq 0$ from arXiv:2103.12253).
LexGPT 0.1: pre-trained GPT-J models with Pile of Law

This research aims to build generative language models specialized for the legal domain. The manuscript presents the development of LexGPT models based on GPT-J models and pre-trained with Pile of Law. The foundation model built in this manuscript is the initial step for the development of future applications in the legal domain, such as further training with reinforcement learning from human feedback. Another objective of this manuscript is to assist legal professionals in utilizing language models through the ``No Code'' approach. By fine-tuning models with specialized data and without modifying any source code, legal professionals can create custom language models for downstream tasks with minimum effort and technical knowledge. The downstream task in this manuscript is to turn a LexGPT model into a classifier, although the performance is notably lower than the state-of-the-art result. How to enhance downstream task performance without modifying the model or its source code is a research topic for future exploration.
Dynamic structure factor of two-dimensional Fermi superfluid with Rashba spin-orbit coupling

We theoretically calculate the dynamic structure factor of two-dimensional Rashba-type spinorbit coupled (SOC) Fermi superfluid with random phase approximation, and analyse the main characters of dynamical excitation sh own by both density and spin dynamic structure factor during a continuous phase transition between Bardeen-Cooper-Schrieffer superfluid and topological superfluid. Generally we find three different excitations, including collective phonon excitation, two-atom molecular and atomic excitations, and pair-breaking excitations due to two-branch structure of quasi-particle spectrum. It should be emphasized that collective phonon excitation is overlapped with a gapless DD type pair-breaking excitation at the critical Zeeman field hc, and is imparted a finite width to phonon peak when transferred momentum q is around Fermi vector kF. At a much larger transferred momentum (q = 4kF ), the pair-breaking excitation happens earlier than two-atom molecular excitation, which is different from the conventional Fermi superfluid without SOC effect.
Feature Selection on Sentinel-2 Multi-Spectral Imagery for Efficient Tree Cover Estimation

This paper proposes a multi-spectral random forest classifier with suitable feature selection and masking for tree cover estimation in urban areas. The key feature of the proposed classifier is filtering out the built-up region using spectral indices followed by random forest classification on the remaining mask with carefully selected features. Using Sentinel-2 satellite imagery, we evaluate the performance of the proposed technique on a specified area (approximately 82 acres) of Lahore University of Management Sciences (LUMS) and demonstrate that our method outperforms a conventional random forest classifier as well as state-of-the-art methods such as European Space Agency (ESA) WorldCover 10m 2020 product as well as a DeepLabv3 deep learning architecture.
Visually-Grounded Descriptions Improve Zero-Shot Image Classification

Language-vision models like CLIP have made significant progress in zero-shot vision tasks, such as zero-shot image classification (ZSIC). However, generating specific and expressive class descriptions remains a major challenge. Existing approaches suffer from granularity and label ambiguity issues. To tackle these challenges, we propose V-GLOSS: Visual Glosses, a novel method leveraging modern language models and semantic knowledge bases to produce visually-grounded class descriptions. We demonstrate V-GLOSS's effectiveness by achieving state-of-the-art results on benchmark ZSIC datasets including ImageNet and STL-10. In addition, we introduce a silver dataset with class descriptions generated by V-GLOSS, and show its usefulness for vision tasks. We make available our code and dataset.
Two Independent Teachers are Better Role Model

Recent deep learning models have attracted substantial attention in infant brain analysis. These models have performed state-of-the-art performance, such as semi-supervised techniques (e.g., Temporal Ensembling, mean teacher). However, these models depend on an encoder-decoder structure with stacked local operators to gather long-range information, and the local operators limit the efficiency and effectiveness. Besides, the $MRI$ data contain different tissue properties ($TPs$) such as $T1$ and $T2$. One major limitation of these models is that they use both data as inputs to the segment process, i.e., the models are trained on the dataset once, and it requires much computational and memory requirements during inference. In this work, we address the above limitations by designing a new deep-learning model, called 3D-DenseUNet, which works as adaptable global aggregation blocks in down-sampling to solve the issue of spatial information loss. The self-attention module connects the down-sampling blocks to up-sampling blocks, and integrates the feature maps in three dimensions of spatial and channel, effectively improving the representation potential and discriminating ability of the model. Additionally, we propose a new method called Two Independent Teachers ($2IT$), that summarizes the model weights instead of label predictions. Each teacher model is trained on different types of brain data, $T1$ and $T2$, respectively. Then, a fuse model is added to improve test accuracy and enable training with fewer parameters and labels compared to the Temporal Ensembling method without modifying the network architecture. Empirical results demonstrate the effectiveness of the proposed method. The code is available at https://github.com/AfifaKhaled/Two-Independent-Teachers-are-Better-Role-Model.
Virtual Node Tuning for Few-shot Node Classification

Few-shot Node Classification (FSNC) is a challenge in graph representation learning where only a few labeled nodes per class are available for training. To tackle this issue, meta-learning has been proposed to transfer structural knowledge from base classes with abundant labels to target novel classes. However, existing solutions become ineffective or inapplicable when base classes have no or limited labeled nodes. To address this challenge, we propose an innovative method dubbed Virtual Node Tuning (VNT). Our approach utilizes a pretrained graph transformer as the encoder and injects virtual nodes as soft prompts in the embedding space, which can be optimized with few-shot labels in novel classes to modulate node embeddings for each specific FSNC task. A unique feature of VNT is that, by incorporating a Graph-based Pseudo Prompt Evolution (GPPE) module, VNT-GPPE can handle scenarios with sparse labels in base classes. Experimental results on four datasets demonstrate the superiority of the proposed approach in addressing FSNC with unlabeled or sparsely labeled base classes, outperforming existing state-of-the-art methods and even fully supervised baselines.
Error Feedback Can Accurately Compress Preconditioners

Leveraging second-order information about the loss at the scale of deep networks is one of the main lines of approach for improving the performance of current optimizers for deep learning. Yet, existing approaches for accurate full-matrix preconditioning, such as Full-Matrix Adagrad (GGT) or Matrix-Free Approximate Curvature (M-FAC) suffer from massive storage costs when applied even to small-scale models, as they must store a sliding window of gradients, whose memory requirements are multiplicative in the model dimension. In this paper, we address this issue via a novel and efficient error-feedback technique that can be applied to compress preconditioners by up to two orders of magnitude in practice, without loss of convergence. Specifically, our approach compresses the gradient information via sparsification or low-rank compression \emph{before} it is fed into the preconditioner, feeding the compression error back into future iterations. Experiments on deep neural networks show that this approach can compress full-matrix preconditioners to up to 99\% sparsity without accuracy loss, effectively removing the memory overhead of full-matrix preconditioners such as GGT and M-FAC. Our code is available at \url{https://github.com/IST-DASLab/EFCP}.
BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrapping

Diffusion models have demonstrated excellent potential for generating diverse images. However, their performance often suffers from slow generation due to iterative denoising. Knowledge distillation has been recently proposed as a remedy that can reduce the number of inference steps to one or a few without significant quality degradation. However, existing distillation methods either require significant amounts of offline computation for generating synthetic training data from the teacher model or need to perform expensive online learning with the help of real data. In this work, we present a novel technique called BOOT, that overcomes these limitations with an efficient data-free distillation algorithm. The core idea is to learn a time-conditioned model that predicts the output of a pre-trained diffusion model teacher given any time step. Such a model can be efficiently trained based on bootstrapping from two consecutive sampled steps. Furthermore, our method can be easily adapted to large-scale text-to-image diffusion models, which are challenging for conventional methods given the fact that the training sets are often large and difficult to access. We demonstrate the effectiveness of our approach on several benchmark datasets in the DDIM setting, achieving comparable generation quality while being orders of magnitude faster than the diffusion teacher. The text-to-image results show that the proposed approach is able to handle highly complex distributions, shedding light on more efficient generative modeling.
Relationships of Measurement Error and Prediction Error in Observed-Score Regression.

The focus of this paper is assessing the impact of measurement errors on the prediction error of an observed-score regression. Measures are presented and described for decomposing the linear regression's prediction error variance into parts attributable to the true score variance and the error variances of the dependent variable and the predictor variable(s). These measures are demonstrated for regression situations reflecting a range of true score correlations and reliabilities and using one and two predictors. Simulation results also are presented which show that the measures of prediction error variance and its parts are generally well estimated for the considered ranges of true score correlations and reliabilities and for homoscedastic and heteroscedastic data. The final discussion considers how the decomposition might be useful for addressing additional questions about regression functions’ prediction error variances.
Improved coherence in optically defined niobium trilayer-junction qubits

Niobium offers the benefit of increased operating temperatures and frequencies for Josephson junctions, which are the core component of superconducting devices. However existing niobium processes are limited by more complicated fabrication methods and higher losses than now-standard aluminum junctions. Combining recent trilayer fabrication advancements, methods to remove lossy dielectrics and modern superconducting qubit design, we revisit niobium trilayer junctions and fabricate all-niobium transmons using only optical lithography. We characterize devices in the microwave domain, measuring coherence times up to $62~\mu$s and an average qubit quality factor above $10^5$: much closer to state-of-the-art aluminum-junction devices. We find the higher superconducting gap energy also results in reduced quasiparticle sensitivity above $0.16~$K, where aluminum junction performance deteriorates. Our junction process is readily applied to standard optical-based foundry processes, opening new avenues for direct integration and scalability, and paves the way for higher-temperature and higher-frequency quantum devices.
Adaptive Contextual Perception: How to Generalize to New Backgrounds and Ambiguous Objects

Biological vision systems make adaptive use of context to recognize objects in new settings with novel contexts as well as occluded or blurry objects in familiar settings. In this paper, we investigate how vision models adaptively use context for out-of-distribution (OOD) generalization and leverage our analysis results to improve model OOD generalization. First, we formulate two distinct OOD settings where the contexts are either irrelevant (Background-Invariance) or beneficial (Object-Disambiguation), reflecting the diverse contextual challenges faced in biological vision. We then analyze model performance in these two different OOD settings and demonstrate that models that excel in one setting tend to struggle in the other. Notably, prior works on learning causal features improve on one setting but hurt in the other. This underscores the importance of generalizing across both OOD settings, as this ability is crucial for both human cognition and robust AI systems. Next, to better understand the model properties contributing to OOD generalization, we use representational geometry analysis and our own probing methods to examine a population of models, and we discover that those with more factorized representations and appropriate feature weighting are more successful in handling Background-Invariance and Object-Disambiguation tests. We further validate these findings through causal intervention on representation factorization and feature weighting to demonstrate their causal effect on performance. Lastly, we propose new augmentation methods to enhance model generalization. These methods outperform strong baselines, yielding improvements in both in-distribution and OOD tests. In conclusion, to replicate the generalization abilities of biological vision, computer vision models must have factorized object vs. background representations and appropriately weight both kinds of features.
sMILES SSPs: a library of semi-empirical MILES stellar population models with variable [α/Fe] abundances

We present a new library of semi-empirical stellar population models that are based on the empirical MILES and semi-empirical sMILES stellar libraries. The models span a large range of age and metallicity, in addition to an [α/Fe] coverage from −0.2 to +0.6 dex, at MILES resolution (FWHM = $2.5\,$ Å) and wavelength coverage ($3540.5-7409.6\,$ Å). These models are aimed at exploring abundance ratios in the integrated light from stellar populations in star clusters and galaxies. Our approach is to build SSPs from semi-empirical stars at particular [α/Fe] values, thus producing new SSPs at a range of [α/Fe] values from sub-solar to super-solar. We compare these new SSPs with previously published and well-used models and find similar abundance pattern predictions, but with some differences in age indicators. We illustrate a potential application of our new SSPs, by fitting them to the high signal-to-noise data of stacked SDSS galaxy spectra. Age, metallicity, and [α/Fe] trends were measured for galaxy stacks with different stellar velocity dispersions and show systematic changes, in agreement with previous analyses of subsets of those data. These new SSPs are made publicly available.
Uncovering Hidden Massive Black Hole Companions with Tidal Disruption Events

Dynamical perturbations from supermassive black hole (SMBH) binaries can increase the rates of tidal disruption events (TDEs). However, most previous work focuses on TDEs from the heavier black hole in the SMBH binary (SMBHB) system. In this work, we focus on the lighter black holes in SMBHB systems and show that they can experience a similarly dramatic increase in their TDE rate due to perturbations from a more massive companion. While the increase in TDEs around the more massive black hole is mostly due to chaotic orbital perturbations, we find that, around the smaller black hole, the eccentric Kozai–Lidov mechanism is dominant and capable of producing a comparably large number of TDEs. In this scenario, the mass derived from the light curve and spectra of TDEs caused by the lighter SMBH companion is expected to be significantly smaller than the SMBH mass estimated from galaxy scaling relations, which are dominated by the more massive companion. This apparent inconsistency can help find SMBHB candidates that are not currently accreting as active galactic nuclei and that are at separations too small for them to be resolved as two distinct sources. In the most extreme cases, these TDEs provide us with the exciting opportunity to study SMBHBs in galaxies where the primary SMBH is too massive to disrupt Sun-like stars.
WSPAlign: Word Alignment Pre-training via Large-Scale Weakly Supervised Span Prediction

Most existing word alignment methods rely on manual alignment datasets or parallel corpora, which limits their usefulness. Here, to mitigate the dependence on manual data, we broaden the source of supervision by relaxing the requirement for correct, fully-aligned, and parallel sentences. Specifically, we make noisy, partially aligned, and non-parallel paragraphs in this paper. We then use such a large-scale weakly-supervised dataset for word alignment pre-training via span prediction. Extensive experiments with various settings empirically demonstrate that our approach, which is named WSPAlign, is an effective and scalable way to pre-train word aligners without manual data. When fine-tuned on standard benchmarks, WSPAlign has set a new state of the art by improving upon the best supervised baseline by 3.3 6.1 points in F1 and 1.5 6.1 points in AER. Furthermore, WSPAlign also achieves competitive performance compared with the corresponding baselines in few-shot, zero-shot and cross-lingual tests, which demonstrates that WSPAlign is potentially more practical for low-resource languages than existing methods.
Modulus of Concavity and Fundamental Gap Estimates on Surfaces

The fundamental gap of a domain is the difference between the first two eigenvalues of the Laplace operator. In a series of recent and celebrated works, it was shown that for convex domains in $\mathbb R^n$ and $\mathbb S^n$ with Dirichlet boundary condition the fundamental gap is at least $\frac{3 \pi^2}{D^2}$ where $D$ denotes the diameter of the domain. The key to these results is to establish a strong concavity estimate for the logarithm of the first eigenfunction. In this article, we prove corresponding log-concavity and fundamental gap estimates for surfaces with non-constant positive curvature via a two-point maximum principle. However, the curvature not being constant greatly increases the difficulty of the computation.
10 Security and Privacy Problems in Large Foundation Models

Foundation models--such as GPT, CLIP, and DINO--have achieved revolutionary progress in the past several years and are commonly believed to be a promising approach for general-purpose AI. In particular, self-supervised learning is adopted to pre-train a foundation model using a large amount of unlabeled data. A pre-trained foundation model is like an ``operating system'' of the AI ecosystem. Specifically, a foundation model can be used as a feature extractor for many downstream tasks with little or no labeled training data. Existing studies on foundation models mainly focused on pre-training a better foundation model to improve its performance on downstream tasks in non-adversarial settings, leaving its security and privacy in adversarial settings largely unexplored. A security or privacy issue of a pre-trained foundation model leads to a single point of failure for the AI ecosystem. In this book chapter, we discuss 10 basic security and privacy problems for the pre-trained foundation models, including six confidentiality problems, three integrity problems, and one availability problem. For each problem, we discuss potential opportunities and challenges. We hope our book chapter will inspire future research on the security and privacy of foundation models.
3D Point Cloud for Objects and Scenes Classification, Recognition, Segmentation, and Reconstruction: A Review

Three-dimensional (3D) point cloud analysis has become one of the attractive subjects in realistic imaging and machine visions due to its simplicity, flexibility and powerful capacity of visualization. Actually, the representation of scenes and buildings using 3D shapes and formats leveraged many applications among which automatic driving, scenes and objects reconstruction, etc. Nevertheless, working with this emerging type of data has been a challenging task for objects representation, scenes recognition, segmentation, and reconstruction. In this regard, a significant effort has recently been devoted to developing novel strategies, using different techniques such as deep learning models. To that end, we present in this paper a comprehensive review of existing tasks on 3D point cloud: a well-defined taxonomy of existing techniques is performed based on the nature of the adopted algorithms, application scenarios, and main objectives. Various tasks performed on 3D point could data are investigated, including objects and scenes detection, recognition, segmentation, and reconstruction. In addition, we introduce a list of used datasets, discuss respective evaluation metrics, and compare the performance of existing solutions to better inform the state-of-the-art and identify their limitations and strengths. Lastly, we elaborate on current challenges facing the subject of technology and future trends attracting considerable interest, which could be a starting point for upcoming research studies.
Comparison between simulations and lab results on the ASSIST test-bench

We present the latest comparison results between laboratory tests carried out on the ASSIST test bench and Octopus end-to end simulations. We simulated, as closely to the lab conditions as possible, the different AOF modes (Maintenance and commissioning mode (SCAO), GRAAL (GLAO in the near IR), Galacsi Wide Field mode (GLAO in the visible) and Galacsi narrow field mode (LTAO in the visible)). We then compared the simulation results to the ones obtained on the lab bench. Several aspects were investigated, like number of corrected modes, turbulence wind speeds, LGS photon flux etc. The agreement between simulations and lab is remarkably good for all investigated parameters, giving great confidence in both simulation tool and performance of the AO system in the lab.