Miracle-dz/UniFAR-FTUs huggingface dataset:
DAS: Densely-Anchored Sampling for Deep Metric Learning

Deep Metric Learning (DML) serves to learn an embedding function to project semantically similar data into nearby embedding space and plays a vital role in many applications, such as image retrieval and face recognition. However, the performance of DML methods often highly depends on sampling methods to choose effective data from the embedding space in the training. In practice, the embeddings in the embedding space are obtained by some deep models, where the embedding space is often with barren area due to the absence of training points, resulting in so called"missing embedding"issue. This issue may impair the sample quality, which leads to degenerated DML performance. In this work, we investigate how to alleviate the"missing embedding"issue to improve the sampling quality and achieve effective DML. To this end, we propose a Densely-Anchored Sampling (DAS) scheme that considers the embedding with corresponding data point as"anchor"and exploits the anchor's nearby embedding space to densely produce embeddings without data points. Specifically, we propose to exploit the embedding space around single anchor with Discriminative Feature Scaling (DFS) and multiple anchors with Memorized Transformation Shifting (MTS). In this way, by combing the embeddings with and without data points, we are able to provide more embeddings to facilitate the sampling process thus boosting the performance of DML. Our method is effortlessly integrated into existing DML frameworks and improves them without bells and whistles. Extensive experiments on three benchmark datasets demonstrate the superiority of our method.
Nonperturbatively Renormalized Nucleon Gluon Momentum Fraction in the Continuum Limit of $N_f=2+1+1$ Lattice QCD

We present the nonperturbatively renormalized nucleon gluon momentum fraction using ensembles with $2+1+1$ flavors of highly improved staggered quarks (HISQ), generated by MILC Collaboration. The calculation is done using clover fermions for the valence action with three pion masses, $220$, $310$ and $690$ MeV and three lattice spacings, 0.09, 0.12, and 0.15 fm. The renormalization is done using RI/MOM nonperturbative renormalization and using cluster-decomposition error reduction (CDER) to enhance the signal-to-noise ratio of the renormalization constant. We find the CDER technique is particularly important to improve the signal at the finer lattice ensembles where the lattice volume is larger. We extrapolate the gluon momentum fraction to the continuum-physical limit and obtain $\langle x \rangle_g = 0.492(52)_\text{stat.+NPR}(49)_\text{mixing}$ in the $\bar{\text{MS}}$ scheme at 2 GeV, where first error includes the statistical error and uncertainties in nonperturbative renormalization, while the latter systematic accounts for ignoring quark mixing. Our gluon momentum fraction is consistent with other recent lattice-QCD results at physical pion mass.
D3Former: Debiased Dual Distilled Transformer for Incremental Learning

In class incremental learning (CIL) setting, groups of classes are introduced to a model in each learning phase. The goal is to learn a unified model performant on all the classes observed so far. Given the recent popularity of Vision Transformers (ViTs) in conventional classification settings, an interesting question is to study their continual learning behaviour. In this work, we develop a Debiased Dual Distilled Transformer for CIL dubbed D3Former. The proposed model leverages a hybrid nested ViT design to ensure data efficiency and scalability to small as well as large datasets. In contrast to a recent ViT based CIL approach, our D3Former does not dynamically expand its architecture when new tasks are learned and remains suitable for a large number of incremental tasks. The improved CIL behaviour of D3Former owes to two fundamental changes to the ViT design. First, we treat the incremental learning as a long-tail classification problem where the majority samples from new classes vastly outnumber the limited exemplars available for old classes. To avoid the bias against the minority old classes, we propose to dynamically adjust logits to emphasize on retaining the representations relevant to old tasks. Second, we propose to preserve the configuration of spatial attention maps as the learning progresses across tasks. This helps in reducing catastrophic forgetting by constraining the model to retain the attention on the most discriminative regions. D3Former obtains favorable results on incremental versions of CIFAR-100, MNIST, SVHN, and ImageNet datasets. Code is available at https://tinyurl.com/d3former.
On periodic Takahashi manifolds

In thispaper we show thatperiodicTakahashi 3-manifolds are cycliccoverings of theconnected sum of two lens spaces(possibly cycliccoverings of S3), branched over knots. When thebase space is a 3-sphere,we prove that theassociatedbranching setisa two-bridge knot of genus one, and we determine itstype. Moreover, a geometric cyclicpresentation for the fundamental groups of these manifolds is obtained in severalinterestingcases,including the ones correspond- ing to the branched cycliccoverings of S3.
Generating Galton-Watson trees using random walks and percolation for the Gaussian free field

The study of Gaussian free field level sets on supercritical Galton-Watson trees has been initiated by Ab\"acherli and Sznitman in Ann. Inst. Henri Poincar\'{e} Probab. Stat., 54(1):173--201, 2018. By means of entirely different tools, we continue this investigation and generalize their main result on the positivity of the associated percolation critical parameter $h_*$ to the setting of arbitrary supercritical offspring distribution and random conductances. A fortiori, this provides a positive answer to the open question raised at the end of the aforementioned article. What is more, in our setting it also establishes a rigorous proof of the physics literature mantra that positive correlations facilitate percolation when compared to the independent case. Our proof proceeds by constructing the Galton-Watson tree through an exploration via finite random walk trajectories. This exploration of the tree progressively unveils an infinite connected component in the random interlacements set on the tree, which is stable under small quenched noise. Using a Dynkin-type isomorphism theorem, we then infer the strict positivity of the critical parameter $ h_* . $ As a byproduct of our proof we obtain the transience of the random interlacement set and the level sets of the Gaussian free field above small positive levels on such Galton-Watson trees.
Rare radiative Ξb−→ Ξ−γ decay in the relativistic quark model

Recently, the LHCb collaboration put the upper limit on the [Formula: see text] decay branching ratio [Formula: see text]. The measured value is below the light-cone sum rule prediction. In this paper, the rare radiative decay of the [Formula: see text] baryon is studied in the framework of the relativistic quark model based on the quasipotential approach and QCD. The decay form factors are calculated with the comprehensive account of the relativistic effects. The obtained result for the branching ratio is found to be below the upper limit set by LHCb and is consistent with theoretical predictions based on the SU(3) flavor-symmetry, light-front quark model and light-cone QCD sum rules in full theory within theoretical uncertainties.
Mismatching-Aware Unsupervised Translation Quality Estimation For Low-Resource Languages

Translation Quality Estimation (QE) is the task of predicting the quality of machine translation (MT) output without any reference. This task has gained increasing attention as an important component in the practical applications of MT. In this paper, we first propose XLMRScore, which is a cross-lingual counterpart of BERTScore computed via the XLM-RoBERTa (XLMR) model. This metric can be used as a simple unsupervised QE method, nevertheless facing two issues: firstly, the untranslated tokens leading to unexpectedly high translation scores, and secondly, the issue of mismatching errors between source and hypothesis tokens when applying the greedy matching in XLMRScore. To mitigate these issues, we suggest replacing untranslated words with the unknown token and the cross-lingual alignment of the pre-trained model to represent aligned words closer to each other, respectively. We evaluate the proposed method on four low-resource language pairs of the WMT21 QE shared task, as well as a new English$\rightarrow$Persian (En-Fa) test dataset introduced in this paper. Experiments show that our method could get comparable results with the supervised baseline for two zero-shot scenarios, i.e., with less than 0.01 difference in Pearson correlation, while outperforming unsupervised rivals in all the low-resource language pairs for above 8%, on average.
Self-supervised learning with rotation-invariant kernels

We introduce a regularization loss based on kernel mean embeddings with rotation-invariant kernels on the hypersphere (also known as dot-product kernels) for self-supervised learning of image representations. Besides being fully competitive with the state of the art, our method significantly reduces time and memory complexity for self-supervised training, making it implementable for very large embedding dimensions on existing devices and more easily adjustable than previous methods to settings with limited resources. Our work follows the major paradigm where the model learns to be invariant to some predefined image transformations (cropping, blurring, color jittering, etc.), while avoiding a degenerate solution by regularizing the embedding distribution. Our particular contribution is to propose a loss family promoting the embedding distribution to be close to the uniform distribution on the hypersphere, with respect to the maximum mean discrepancy pseudometric. We demonstrate that this family encompasses several regularizers of former methods, including uniformity-based and information-maximization methods, which are variants of our flexible regularization loss with different kernels. Beyond its practical consequences for state-of-the-art self-supervised learning with limited resources, the proposed generic regularization approach opens perspectives to leverage more widely the literature on kernel methods in order to improve self-supervised learning methods.
A Geochemical Evaluation of Metamorphic Rocks from Schirmacher Oasis East Antarctica: Implications of a MORB Source

We analyzed the Rare Earth Elements (REEs) and several trace elements of seven medium to high grade metamorphic rocks, being gneisses, a schist and a quartzite, from Schirmacher Oasis, East Antarctica. The petrographic study shows that the rock forming minerals were altered to secondary minerals. Relative to chondrite, the REEs demonstrate a fractionation with enrichment in light REEs, suggesting a mobilization of heavy REEs during crystallization of the potential protoliths. The presence of negative Eu anomalies of 0.3 to 0.86, support early fractionation of plagioclase. All samples indicate similar elemental patterns relative to chondrite leading to the conclusion that their affiliation was of the same source. However, based on patterns relative to ocean island basalt (OIB), three gneisses may be apparent to OIB. Based on the geochemical discrimination, we obtained a mid ocean ridge basalt (MORB) background as primary tectonic setting, possibly writing an alignment of a continental arc.
PASTA: A Dataset for Modeling PArticipant STAtes in Narratives

Abstract The events in a narrative are understood as a coherent whole via the underlying states of their participants. Often, these participant states are not explicitly mentioned, instead left to be inferred by the reader. A model that understands narratives should likewise infer these implicit states, and even reason about the impact of changes to these states on the narrative. To facilitate this goal, we introduce a new crowdsourced English-language, Participant States dataset, PASTA. This dataset contains inferable participant states; a counterfactual perturbation to each state; and the changes to the story that would be necessary if the counterfactual were true. We introduce three state-based reasoning tasks that test for the ability to infer when a state is entailed by a story, to revise a story conditioned on a counterfactual state, and to explain the most likely state change given a revised story. Experiments show that today’s LLMs can reason about states to some degree, but there is large room for improvement, especially in problems requiring access and ability to reason with diverse types of knowledge (e.g., physical, numerical, factual). 1
Scalable Distributed Optimization of Multi-Dimensional Functions Despite Byzantine Adversaries

The problem of distributed optimization requires a group of networked agents to compute a parameter that minimizes the average of their local cost functions. While there are a variety of distributed optimization algorithms that can solve this problem, they are typically vulnerable to"Byzantine"agents that do not follow the algorithm. Recent attempts to address this issue focus on single dimensional functions, or assume certain statistical properties of the functions at the agents. In this paper, we provide two resilient, scalable, distributed optimization algorithms for multi-dimensional functions. Our schemes involve two filters, (1) a distance-based filter and (2) a min-max filter, which each remove neighborhood states that are extreme (defined precisely in our algorithms) at each iteration. We show that these algorithms can mitigate the impact of up to $F$ (unknown) Byzantine agents in the neighborhood of each regular agent. In particular, we show that if the network topology satisfies certain conditions, all of the regular agents' states are guaranteed to converge to a bounded region that contains the minimizer of the average of the regular agents' functions.
White Spot Syndrome Virus Induces Metabolic Changes Resembling the Warburg Effect in Shrimp Hemocytes in the Early Stage of Infection (cid:1)

The Warburg effect is an abnormal glycolysis response that is associated with cancer cells. Here we present evidence that metabolic changes resembling the Warburg effect are induced by a nonmammalian virus. When shrimp were infected with white spot syndrome virus (WSSV), changes were induced in several metabolic pathways related to the mitochondria. At the viral genome replication stage (12 h postinfection [hpi]), glucose consumption and plasma lactate concentration were both increased in WSSV-infected shrimp, and the key enzyme of the pentose phosphate pathway, glucose-6-phosphate dehydrogenase (G6PDH), showed increased activity. We also found that at 12 hpi there was no alteration in the ADP/ATP ratio and that oxidative stress was lower than that in uninfected controls. All of these results are characteristic of the Warburg effect as it is present in mammals. There was also a signiﬁcant decrease in triglyceride concentration starting at 12 hpi. At the late stage of the infection cycle (24 hpi), hemocytes of WSSV-infected shrimp showed several changes associated with cell death. These included the induction of mitochondrial membrane permeabilization (MMP), increased oxidative stress, decreased glucose consumption, and disrupted energy production. A previous study showed that WSSV infection led to upregulation of the voltage-dependent anion channel (VDAC), which is known to be involved in both the Warburg effect and MMP. Here we show that double-stranded RNA (dsRNA) silencing of the VDAC reduces WSSV-induced mortality and virion copy number. For these results, we hypothesize a model depicting the metabolic changes in host cells at the early and late stages of WSSV infection.
Towards R-learner of conditional average treatment effects with a continuous treatment: T-identification, estimation, and inference

The R-learner has been popular in causal inference as a flexible and efficient meta-learning approach for heterogeneous treatment effect estimation. In this article, we show the identifiability transition of the generalized R-learning framework from a binary treatment to continuous treatment. To resolve the non-identification issue with continuous treatment, we propose a novel identification strategy named T-identification, acknowledging the use of Tikhonov regularization rooted in the nonlinear functional analysis. Following the new identification strategy, we introduce an `2-penalized R-learner framework to estimate the conditional average treatment effect with continuous treatment. The new R-learner framework accommodates modern, flexible machine learning algorithms for both nuisance function and target estimand estimation. Asymptotic properties are studied when the target estimand is approximated by sieve approximation, including general error bounds, asymptotic normality, and inference. Simulations illustrate the superior performance of our proposed estimator. An application of the new method to the medical information mart for intensive care data reveals the heterogeneous treatment effect of oxygen saturation on survival in sepsis patients.
Off-Policy Correction for Actor-Critic Algorithms in Deep Reinforcement Learning

Compared to on-policy policy gradient techniques, off-policy model-free deep reinforcement learning (RL) approaches that use previously gathered data can improve sampling efﬁciency. However, off-policy learning becomes challenging when the discrepancy between the distributions of the policy of interest and the policies that collected the data increases. Although the well-studied importance sampling and off-policy policy gradient techniques were proposed to compensate for this discrepancy, they usually require a collection of long trajectories that increases the computational complexity and induce additional problems such as vanishing or exploding gradients. Moreover, their generalization to continuous action domains is strictly limited as they require action probabilities, which is unsuitable for deterministic policies. To overcome these limitations, we introduce an alternative off-policy correction algorithm for continuous action spaces, Actor-Critic Off-Policy Correction (AC-Off-POC), to mitigate the potential drawbacks introduced by the previously collected data. Through a novel discrepancy measure computed by the agent’s most recent action decisions on the states of the randomly sampled batch of transitions, the approach does not require actual or estimated action probabilities for any policy and offers an adequate one-step importance sampling. Theoretical results show that the introduced approach can achieve a contraction mapping with a ﬁxed unique point, which allows a “safe" off-policy learning. Our empirical results suggest that AC-Off-POC consistently improves the state-of-the-art and attains higher returns in fewer steps than the competing methods by efﬁciently scheduling the learning rate in Q-learning and policy optimization.
Automatically Discovering Novel Visual Categories With Adaptive Prototype Learning

This article targets the task of novel category discovery (NCD), which aims to discover unknown categories when a certain number of classes are already known. The NCD task is challenging due to its closeness to real-world scenarios, where we have only encountered some partial classes and corresponding images. Unlike previous approaches to NCD, we propose a novel adaptive prototype learning method that leverages prototypes to emphasize category discrimination and alleviate the issue of missing annotations for novel classes. Concretely, the proposed method consists of two main stages: prototypical representation learning and prototypical self-training. In the first stage, we develop a robust feature extractor that could effectively handle images from both base and novel categories. This ability of instance and category discrimination of the feature extractor is boosted by self-supervised learning and adaptive prototypes. In the second stage, we utilize the prototypes again to rectify offline pseudo labels and train a final parametric classifier for category clustering. We conduct extensive experiments on four benchmark datasets, demonstrating our method’s effectiveness and robustness with state-of-the-art performance.
On the stunning abundance of super-early, luminous galaxies revealed by JWST

The earliest JWST observations have revealed an unexpected abundance of super-early (z > 10), massive ($M_*\, \approx 10^9 {\rm M}_{\odot }$ ) galaxies at the bright-end (MUV ≈ −21) of the ultraviolet luminosity function (UV LF). We present a minimal physical model that explains the observed galaxy abundance at z = 10–14. The model primarily combines (i) the halo mass function, with (ii) an obscured star formation fraction prescription that is consistent with findings of the ALMA REBELS dusty galaxy survey. It has been successfully tested on well-known UV LFs up to z = 7. We argue that the weak evolution from z = 7 to z ≈ 14 of the LF bright-end can arise from a conspiracy between a decreasing dust attenuation, making galaxies brighter, that almost exactly compensates for the increasing shortage of their host halos. Our minimal model naturally reproduces the z = 10–14 LF if galaxies at $z\lower.5ex\rm{\,\, \buildrel\gt \over \sim \,\,}11$ contain a negligible amounts of dust. We speculate that dust could have been efficiently ejected during the very first phases of galaxy build-up.
Visual correspondence-based explanations improve AI robustness and human-AI team accuracy

Explaining artificial intelligence (AI) predictions is increasingly important and even imperative in many high-stakes applications where humans are the ultimate decision-makers. In this work, we propose two novel architectures of self-interpretable image classifiers that first explain, and then predict (as opposed to post-hoc explanations) by harnessing the visual correspondences between a query image and exemplars. Our models consistently improve (by 1 to 4 points) on out-of-distribution (OOD) datasets while performing marginally worse (by 1 to 2 points) on in-distribution tests than ResNet-50 and a $k$-nearest neighbor classifier (kNN). Via a large-scale, human study on ImageNet and CUB, our correspondence-based explanations are found to be more useful to users than kNN explanations. Our explanations help users more accurately reject AI's wrong decisions than all other tested methods. Interestingly, for the first time, we show that it is possible to achieve complementary human-AI team accuracy (i.e., that is higher than either AI-alone or human-alone), in ImageNet and CUB image classification tasks.