Miracle-dz/UniFAR-FTUs huggingface dataset:
Learning Neural Eigenfunctions for Unsupervised Semantic Segmentation

Unsupervised semantic segmentation is a long-standing challenge in computer vision with great significance. Spectral clustering is a theoretically grounded solution to it where the spectral embeddings for pixels are computed to construct distinct clusters. Despite recent progress in enhancing spectral clustering with powerful pre-trained models, current approaches still suffer from inefficiencies in spectral decomposition and inflexibility in applying them to the test data. This work addresses these issues by casting spectral clustering as a parametric approach that employs neural network-based eigenfunctions to produce spectral embeddings. The outputs of the neural eigenfunctions are further restricted to discrete vectors that indicate clustering assignments directly. As a result, an end-to-end NN-based paradigm of spectral clustering emerges. In practice, the neural eigenfunctions are lightweight and take the features from pre-trained models as inputs, improving training efficiency and unleashing the potential of pre-trained models for dense prediction. We conduct extensive empirical studies to validate the effectiveness of our approach and observe significant performance gains over competitive baselines on Pascal Context, Cityscapes, and ADE20K benchmarks. The code is available at https://github.com/thudzj/NeuralEigenfunctionSegmentor.
Fundamental Gridding-Related Dispersion Effects in

The effect of electric and magnetic node arrangement on the dispersion characteristics of the multiresolution time-do- main (MRTD) technique is investigated in this paper. It is first noted that, by adopting multiresolution analysis principles, the dis- persion behavior of an arbitrary order MRTD scheme can be ex- tracted from the analysis of the corresponding S-MRTD scheme, which is based on scaling functions only. Namely, the introduction of one wavelet level is expected to bring about a refinement in the resolution of S-MRTD by a factor of two. However, this contra- dicts several dispersion analyses of MRTD schemes that have been recently presented in the literature. This conflict between theoret- ical predictions and numerical observations is resolved through the proof that the introduction of wavelets does not result in the ex- pected enhancement of resolution of an S-MRTD scheme, unless a certain arrangement of electric and magnetic field nodes is imple- mented. Nevertheless, dispersion analyses of several MRTD schemes have indicated that the addition of wavelets gives rise to phe- nomena that are not consistent with this expectation. In par- ticular, the dispersion analysis of the Battle-Lemarie wavelet- based W-MRTD scheme that was carried out in (4) showed an only incremental improvement in its accuracy, compared to the corresponding S-MRTD scheme. This was accompanied by the observation of nonphysical modes that were attributed to spurious side effects of the use of wavelets. Moreover, a Haar MRTD dispersion analysis, presented in (7), led to the conclu- sion that almost no accuracy improvement was produced by the introduction of zeroth-order Haar wavelets to the corresponding S-MRTD scheme, which in this case coincided with FDTD. This conclusion agreed with the results of the relevant numer- ical study of (8) and provided the additional information that the effect of boundary conditions, which were supposed to be
Krylov Methods are (nearly) Optimal for Low-Rank Approximation

We consider the problem of rank-1 low-rank approximation (LRA) in the matrix-vector product model under various Schatten norms:\begin{equation*}\min _{\|u\|_{2}=1}\left\|A\left(I-u u^{\top}\right)\right\|_{\mathcal{S}_{p}}\end{equation*}where $\|M\|_{\mathcal{S}_{p}}$ denotes the $\ell_{p}$ norm of the singular values of M. Given $\varepsilon\gt 0$, our goal is to output a unit vector v such that \begin{equation*}\left\|A\left(I-v v^{\top}\right)\right\|_{\mathcal{S}_{p}} \leqslant\left(1+\varepsilon\right) \min _{\|u\|_{2}=1}\left\|A\left(I-u u^{\top}\right)\right\|_{\mathcal{S}_{p}}\end{equation*}Our main result shows that Krylov methods (nearly) achieve the information-theoretically optimal1 number of matrix-vector products for Spectral $(p=\infty)$, Frobenius $(p=2)$ and Nuclear $(p=1)$ LRA. In particular, for Spectral LRA, we show that any algorithm requires $\Omega\left(\log (n) / \varepsilon^{1 / 2}\right)$ matrix-vector products, exactly matching the upper bound obtained by Krylov methods [40]. Our lower bound addresses Open Question 1 in [59], providing evidence for the lack of progress on algorithms for Spectral LRA and resolves Open Question 1.2 in [5]. Next, we show that for any fixed constant p, i.e. $1 \leqslant p=O(1)$, there is an upper bound of $O\left(\log (1 / \varepsilon) / \varepsilon^{1 / 3}\right)$ matrix-vector products, implying that the complexity does not grow as a function of input size. This improves the $O\left(\log (n / \varepsilon) / \varepsilon^{1 / 3}\right)$ bound recently obtained in [5], and matches their $\Omega\left(1 / \varepsilon^{1 / 3}\right)$ lower bound, to a $\log (1 / \varepsilon)$ factor.1For Spectral LRA, the upper and lower bounds match up to a fixed universal constant. For Frobenius and Nuclear LRA, they match up to a $\log (1 / \varepsilon)$ factor.
DeLiRa: Self-Supervised Depth, Light, and Radiance Fields

Differentiable volumetric rendering is a powerful paradigm for 3D reconstruction and novel view synthesis. However, standard volume rendering approaches struggle with degenerate geometries in the case of limited viewpoint diversity, a common scenario in robotics applications. In this work, we propose to use the multi-view photometric objective from the self-supervised depth estimation literature as a geometric regularizer for volumetric rendering, significantly improving novel view synthesis without requiring additional information. Building upon this insight, we explore the explicit modeling of scene geometry using a generalist Transformer, jointly learning a radiance field as well as depth and light fields with a set of shared latent codes. We demonstrate that sharing geometric information across tasks is mutually beneficial, leading to improvements over single-task learning without an increase in network complexity. Our DeLiRa architecture achieves state-of-the-art results on the ScanNet benchmark, enabling high quality volumetric rendering as well as real-time novel view and depth synthesis in the limited viewpoint diversity setting. Our project page is https://sites.google. com/view/tri-delira.
Diffusion MRI with free gradient waveforms on a high-performance gradient system: Probing restriction and exchange in the human brain

The dependence of the diffusion MRI signal on the diffusion time carries signatures of restricted diffusion and exchange. Here we seek to highlight these signatures in the human brain by performing experiments using free gradient waveforms that are selectively sensitive to the two effects. We examine six healthy volunteers using both strong and ultra-strong gradients (80, 200 and 300 mT/m). In an experiment featuring a large set of gradient waveforms with different sensitivities to restricted diffusion and exchange (150 samples), our results reveal unique time-dependence signatures in grey and white matter, where the former is characterised by both restricted diffusion and exchange and the latter predominantly exhibits restricted diffusion. Furthermore, we show that gradient waveforms with independently varying sensitivities to restricted diffusion and exchange can be used to map exchange in the human brain. We consistently find that exchange in grey matter is at least twice as fast as in white matter, across all subjects and all gradient strengths. The shortest exchange times observed in this study were in the cerebellar cortex (115 ms). We also assess the feasibility of future clinical applications of the method used in this work, where we find that the grey-white matter exchange contrast obtained with a 25-minute 300 mT/m protocol is preserved by a 4-minute 300 mT/m and a 10-minute 80 mT/m protocol. Our work underlines the utility of free waveforms for detecting time-dependence signatures due to restricted diffusion and exchange in vivo, which may potentially serve as a tool for studying diseased tissue.
Analog of atomic collapse for adatoms on rhombohedral multilayer graphene

We propose that a multi-graphene of ABC-type stacking yields virtual bound states lying within the Coulomb insulating gap of an Anderson-like adatom. Wondrously, a virtual state constitutes the counterpart of the atomic collapse phenomenon proposed in relativistic atomic Physics, while the second emerges as its particle-hole symmetric, analogous to a positron state. Thus, we introduce the effect as the adatomic collapse, which occurs due to a flat band with a dispersionless state and a divergent density of states $\sim|\varepsilon-\varepsilon_{F}|^{2/J-1}$ near the Fermi energy $\varepsilon_{F}$ for $J\geq3,$ where $J\pi$ is the Berry phase. We conclude this scenario based on the Kramers-Kronig transformation of the quasiparticle broadening, from where we observe that the aforementioned van Hove singularity induces virtual bound states. Counterintuitively, near the singularity, we find these states above and below the Fermi energy correlated to the existence of the bottom and top edges of the Coulomb insulating region, respectively. As such a behavior rises without a twist, the system is known as Moir\'eless and the phenomenon emerges also assisted by the adatom Coulomb correlations. Similarly to Science 340, 734 (2013) we find the effective critical atomic number $\mathcal{Z}_{c}\sim0.96$ in contrast to an ultra-heavy nucleus. Thus, we point out that multi-graphene is a proper playground for testing a predicted phenomenon of the relativistic atomic Physics in the domain of the condensed matter Physics.
On the Pareto Front of Multilingual Neural Machine Translation

In this work, we study how the performance of a given direction changes with its sampling ratio in Multilingual Neural Machine Translation (MNMT). By training over 200 multilingual models with various model sizes, data sizes, and language directions, we find it interesting that the performance of certain translation direction does not always improve with the increase of its weight in the multi-task optimization objective. Accordingly, scalarization method leads to a multitask trade-off front that deviates from the traditional Pareto front when there exists data imbalance in the training corpus, which poses a great challenge to improve the overall performance of all directions. Based on our observations, we propose the Double Power Law to predict the unique performance trade-off front in MNMT, which is robust across various languages, data adequacy, and the number of tasks. Finally, we formulate the sample ratio selection problem in MNMT as an optimization problem based on the Double Power Law. In our experiments, it achieves better performance than temperature searching and gradient manipulation methods with only 1/5 to 1/2 of the total training budget. We release the code at https://github.com/pkunlp-icler/ParetoMNMT for reproduction.
Online Adversarial Stabilization of Unknown Linear Time-Varying Systems

This paper studies the problem of online stabilization of an unknown discrete-time linear time-varying (LTV) system under bounded non-stochastic (potentially adversarial) disturbances. We propose a novel algorithm based on convex body chasing (CBC). Under the assumption of infrequently changing or slowly drifting dynamics, the algorithm guarantees bounded-input-bounded-output stability in the closed loop. Our approach avoids system identification and applies, with minimal disturbance assumptions, to a variety of LTV systems of practical importance. We demonstrate the algorithm numerically on examples of LTV systems including Markov linear jump systems with finitely many jumps.
Covert learning and disclosure

I study a model of information acquisition and transmission in which the sender's ability to misreport her findings is limited. In equilibrium, the sender only influences the receiver by choosing to remain selectively ignorant, rather than by deceiving her about the discoveries. Although deception does not occur, I highlight how deception possibilities determine what information the sender chooses to acquire and transmit. I then turn to comparative statics, characterizing in which sense the sender benefits from her claims being more verifiable, showing this is akin to increasing her commitment power. Finally, I characterize sender- and receiver-optimal falsification environments.
Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster

We study recent research advances that improve large language models through efficient pre-training and scaling, and open datasets and tools. We combine these advances to introduce Cerebras-GPT, a family of open compute-optimal language models scaled from 111M to 13B parameters. We train Cerebras-GPT models on the Eleuther Pile dataset following DeepMind Chinchilla scaling rules for efficient pre-training (highest accuracy for a given compute budget). We characterize the predictable power-law scaling and compare Cerebras-GPT with other publicly-available models to show all Cerebras-GPT models have state-of-the-art training efficiency on both pre-training and downstream objectives. We describe our learnings including how Maximal Update Parameterization ($\mu$P) can further improve large model scaling, improving accuracy and hyperparameter predictability at scale. We release our pre-trained models and code, making this paper the first open and reproducible work comparing compute-optimal model scaling to models trained on fixed dataset sizes. Cerebras-GPT models are available on HuggingFace: https://huggingface.co/cerebras.
A PHASE TRANSITION IN ZERO OVERCROWDING AND UNDERCROWDING PROBABILITIES FOR STATIONARY GAUSSIAN PROCESSES

We study the probability that a real stationary Gaussian process has at least ηT zeros in [0 , T ] (overcrowding), or at most this number (undercrowding). We show that if the spectral measure of the process is supported on ± [ B, A ], overcrowding probability transitions from exponential decay to Gaussian decay at η = Aπ , while undercrowding probability undergoes the reverse transition at η = Bπ .
Source-free Domain Adaptation Requires Penalized Diversity

While neural networks are capable of achieving human-like performance in many tasks such as image classification, the impressive performance of each model is limited to its own dataset. Source-free domain adaptation (SFDA) was introduced to address knowledge transfer between different domains in the absence of source data, thus, increasing data privacy. Diversity in representation space can be vital to a model`s adaptability in varied and difficult domains. In unsupervised SFDA, the diversity is limited to learning a single hypothesis on the source or learning multiple hypotheses with a shared feature extractor. Motivated by the improved predictive performance of ensembles, we propose a novel unsupervised SFDA algorithm that promotes representational diversity through the use of separate feature extractors with Distinct Backbone Architectures (DBA). Although diversity in feature space is increased, the unconstrained mutual information (MI) maximization may potentially introduce amplification of weak hypotheses. Thus we introduce the Weak Hypothesis Penalization (WHP) regularizer as a mitigation strategy. Our work proposes Penalized Diversity (PD) where the synergy of DBA and WHP is applied to unsupervised source-free domain adaptation for covariate shift. In addition, PD is augmented with a weighted MI maximization objective for label distribution shift. Empirical results on natural, synthetic, and medical domains demonstrate the effectiveness of PD under different distributional shifts.
Finite-size subthermal regime in disordered 
<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:mi>SU</mml:mi><mml:mo>(</mml:mo><mml:mi>N</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:math>
-symmetric Heisenberg chains

SU(N) symmetry is incompatible with the many-body localized (MBL) phase, even when strong disorder is present. However, recent studies have shown that finite-size SU(2) systems exhibit non-ergodic, subthermal behavior, characterized by the breakdown of the eigenstate thermalization hypothesis, and by the excited eigenstates entanglement entropy that is intermediate between area and volume law. In this work, we extend previous studies of the SU(2)-symmetric disordered Heisenberg model to larger systems, using the time-dependent density matrix renormalization group (tDMRG) method. We simulate quench dynamics from weakly entangled initial states up to long times, finding robust subthermal behavior at stronger disorder. Although we find an increased tendency towards thermalization at larger system sizes, the subthermal regime persists at intermediate time scales, nevertheless, and therefore should be accessible experimentally. At weaker disorder, we observe signatures of thermalization, however, entanglement entropy exhibits slow sublinear growth, in contrast to conventional thermalizing systems. Furthermore, we study dynamics of the SU(3)-symmetric disordered Heisenberg model. Similarly, strong disorder drives the system into subthermal regime, albeit thermalizing phase is broader compared to the SU(2) case. Our findings demonstrate the robustness of the subthermal regime in spin chains with non-Abelian continuous symmetry, and are consistent with eventual thermalization at large system sizes and long time scales, suggested by previous works.
Exploring the Utility of Self-Supervised Pretraining Strategies for the Detection of Absent Lung Sliding in M-Mode Lung Ultrasound

Self-supervised pretraining has been observed to improve performance in supervised learning tasks in medical imaging. This study investigates the utility of self-supervised pretraining prior to conducting supervised fine-tuning for the downstream task of lung sliding classification in M-mode lung ultrasound images. We propose a novel pairwise relationship that couples M-mode images constructed from the same B-mode image and investigate the utility of data augmentation procedure specific to M-mode lung ultrasound. The results indicate that self-supervised pretraining yields better performance than full supervision, most notably for feature extractors not initialized with ImageNet-pretrained weights. Moreover, we observe that including a vast volume of unlabelled data results in improved performance on external validation datasets, underscoring the value of self-supervision for improving generalizability in automatic ultrasound interpretation. To the authors’ best knowledge, this study is the first to characterize the influence of self-supervised pretraining for M-mode ultrasound.
GeoDE: a Geographically Diverse Evaluation Dataset for Object Recognition

Current dataset collection methods typically scrape large amounts of data from the web. While this technique is extremely scalable, data collected in this way tends to reinforce stereotypical biases, can contain personally identifiable information, and typically originates from Europe and North America. In this work, we rethink the dataset collection paradigm and introduce GeoDE, a geographically diverse dataset with 61,940 images from 40 classes and 6 world regions, and no personally identifiable information, collected through crowd-sourcing. We analyse GeoDE to understand differences in images collected in this manner compared to web-scraping. Despite the smaller size of this dataset, we demonstrate its use as both an evaluation and training dataset, highlight shortcomings in current models, as well as show improved performances when even small amounts of GeoDE (1000 - 2000 images per region) are added to a training dataset. We release the full dataset and code at https://geodiverse-data-collection.cs. princeton.edu/
Interpolation complexe d'un espace de Banach et de son antidual

Si on peut injecter de maniere dense un espace de Hilbert H dans un espace de Banach X, ou si X s'injecte de maniere dense dans H, alors on dispose d'un cadre pour interpoler X et son antidual X * , et on montre que dans ce cadre (X,X * ) 1/2 est isometrique a H.
A Multi-pass Sieve for Clinical Concept Normalization

Clinical concept normalization involves linking entity mentions in clinical narratives to their corresponding concepts in standardized medical terminologies. It can be used to determine the speciﬁc meaning of a mention, facilitating effective use and exchange of clinical information, and to support semantic cross-compatibility of texts. We present a rule-based multi-pass sieve approach incorporating both exact and approximate matching based on dictionaries, and experiment with back-translation as a means of data augmentation. The dictionaries are built from the UMLS Metathesaurus as well as MCN corpus training data. Additionally, we train a multi-class baseline based on BERT. Our multi-pass sieve approach achieves an accuracy of 82.0% on the MCN corpus , the highest for any rule-based method. A hybrid method combining these two achieves a slightly higher accuracy of 82.3%.
Sex-Determined Growth of Yearling Pacific Oysters (Crassostrea gigas)

ABSTRACT The sexes of sequentially hermaphroditic cupped oysters are both environmentally and genetically determined. Sex is also associated with growth or body size, females being larger than males of the same age on average. Whether sex determines size or size determines sex has not been resolved. To examine the relationship between growth and sex, 1,360 tagged individuals from six F2 families of Pacific oyster Crassostrea gigas were reared through their second summer in Thorndyke Bay, Washington; individual live weights were recorded for 5 consecutive months, starting in June. At the last weighing, microscopic examination of gonadal tissue revealed four “sexes” in the cohort, females (43% of cohort), males (29%), hermaphrodites (2%), and sex-undetermined individuals with no gametes (26%). Live weights were fit to various sigmoidal growth curves (logistic, Gompertz, and Richards) with fixed and random parameters to account for individual variation. The best fitting curves suggested that growth of the cohort was exponential until late August, when temperatures declined and more than 70% of final weight had been gained. Individual live weights, growth-curve parameters, and growth measures were fit to general linear models with family and sex classification variables. Family-by-sex interaction was significant in many such analyses. Females were larger than males on average and appeared to attain their size advantage from superior growth characteristics rather than from initial size. Males were strikingly either like females or like the smaller sex-undetermined oysters with respect to growth comparisons within family. Sex-undetermined oysters may have been “true” males that were too small to mature as yearlings, though a size-threshold for maturation was not apparent. Genetically determined sex and family genetics appeared to control growth.