Miracle-dz/UniFAR-FTUs huggingface dataset:
No More Shortcuts: Realizing the Potential of Temporal Self-Supervision

Self-supervised approaches for video have shown impressive results in video understanding tasks. However, unlike early works that leverage temporal self-supervision, current state-of-the-art methods primarily rely on tasks from the image domain (e.g., contrastive learning) that do not explicitly promote the learning of temporal features. We identify two factors that limit existing temporal self-supervision: 1) tasks are too simple, resulting in saturated training performance, and 2) we uncover shortcuts based on local appearance statistics that hinder the learning of high-level features. To address these issues, we propose 1) a more challenging reformulation of temporal self-supervision as frame-level (rather than clip-level) recognition tasks and 2) an effective augmentation strategy to mitigate shortcuts. Our model extends a representation of single video frames, pre-trained through contrastive learning, with a transformer that we train through temporal self-supervision. We demonstrate experimentally that our more challenging frame-level task formulations and the removal of shortcuts drastically improve the quality of features learned through temporal self-supervision. Our extensive experiments show state-of-the-art performance across 10 video understanding datasets, illustrating the generalization ability and robustness of our learned video representations. Project Page: https://daveishan.github. io/nms-webpage.
BSL: Understanding and Improving Softmax Loss for Recommendation

Loss functions steer the optimization direction of recommendation models and are critical to model performance, but have received relatively little attention in recent recommendation research. Among various losses, we find Softmax loss (SL) stands out for not only achieving remarkable accuracy but also better robustness and fairness. Nevertheless, the current literature lacks a comprehensive explanation for the efficacy of SL. Toward addressing this research gap, we conduct theoretical analyses on SL and uncover three insights: 1) Optimizing SL is equivalent to performing Distributionally Robust Optimization (DRO) on the negative data, thereby learning against perturbations on the negative distribution and yielding robustness to noisy negatives. 2) Comparing with other loss functions, SL implicitly penalizes the prediction variance, resulting in a smaller gap between predicted values and and thus producing fairer results. Building on these insights, we further propose a novel loss function Bilateral SoftMax Loss (BSL) that extends the advantage of SL to both positive and negative sides. BSL augments SL by applying the same Log-Expectation-Exp structure to positive examples as is used for negatives, making the model robust to the noisy positives as well. Remarkably, BSL is simple and easy-to-implement -- requiring just one additional line of code compared to SL. Experiments on four real-world datasets and three representative backbones demonstrate the effectiveness of our proposal. The code is available at https://github.com/junkangwu/BSL
Logarithmic prismatic cohomology, motivic sheaves, and comparison theorems

We prove that (logarithmic) prismatic and (logarithmic) syntomic cohomology are representable in the category of logarithmic motives. As an application, we obtain Gysin maps for prismatic and syntomic cohomology, and we explicitly identify their cofibers. We also prove a smooth blow-up formula and we compute prismatic and syntomic cohomology of Grassmannians. In the second part of the paper, we develop a descent technique inspired by the work of Nizio\l~ on log $K$-theory. Using the resulting \emph{saturated descent}, we prove de Rham and crystalline comparison theorems for log prismatic cohomology, and the existence of Gysin maps for $A_{\inf}$-cohomology.
Quantum Observables of Quantized Fluxes

While it has become widely appreciated that defining (higher) gauge theories requires, in addition to ordinary phase space data, also"flux quantization"laws in generalized differential cohomology, there has been little discussion of the general rules, if any, for lifting Poisson-brackets of (flux-)observables and their quantization from traditional phase spaces to the resulting higher moduli stacks of flux-quantized gauge fields. In this short note, we present a systematic analysis of (i) the canonical quantization of flux observables in Yang-Mills theory and (ii) of valid flux quantization laws in abelian Yang-Mills, observing (iii) that the resulting topological quantum observables form the homology Pontrjagin algebra of the loop space of the moduli space of flux-quantized gauge fields. This is remarkable because the homology Ponrjagin algebra on loops of moduli makes immediate sense in broad generality for higher and non-abelian (non-linearly coupled) gauge fields, such as for the C-field in 11d supergravity, where it recovers the quantum effects previously discussed in the context of"Hypothesis H".
Non-linear photoconductivity of strongly driven graphene

We present the non-linear DC photoconductivity of graphene under strong infra-red (IR) radiation. The photoconductivity is obtained as the response to a strong DC electric field, with field strengths outside of the linear-response regime, while the IR radiation is described by a strong AC electric field. The conductivity displays two distinct regimes in which either the DC or the AC field dominates. We explore these regimes and associate them with the dynamics of driven Landau-Zener quenches in the case of a large DC field. In the limit of large AC field, we describe the conductivity in a Floquet picture and compare the results to the closely related Tien-Gordon effect. We present analytical calculations for the non-linear differential photoconductivity, for both regimes based on the corresponding mechanisms. As part of this discussion of the non-equilibrium state of graphene, we present analytical estimates of the conductivity of undriven graphene as a function of temperature and DC bias field strength that show very good agreement with our simulations.
Dynamical Stability and Instability of Poincar\'e--Einstein Manifolds

We prove dynamical stability and instability theorems for Poincar\'{e}-Einstein metrics under the Ricci flow. Our key tool is a variant of the expander entropy for asymptotically hyperbolic manifolds, which Dahl, McCormick and the first author established in a recent article. It allows us to characterize stability and instability in terms of a local positive mass theorem and in terms of volume comparison for nearby metrics.
Efficient Verification-Based Face Identification

We study the problem of performing face verification with an efficient neural model $f$. The efficiency of $f$ stems from simplifying the face verification problem from an embedding nearest neighbor search into a binary problem; each user has its own neural network $f$. To allow information sharing between different individuals in the training set, we do not train $f$ directly but instead generate the model weights using a hypernetwork $h$. This leads to the generation of a compact personalized model for face identification that can be deployed on edge devices. Key to the method's success is a novel way of generating hard negatives and carefully scheduling the training objectives. Our model leads to a substantially small $f$ requiring only 23k parameters and 5M floating point operations (FLOPS). We use six face verification datasets to demonstrate that our method is on par or better than state-of-the-art models, with a significantly reduced number of parameters and computational burden. Furthermore, we perform an extensive ablation study to demonstrate the importance of each element in our method.
The entanglement membrane in exactly solvable lattice models

Entanglement membrane theory is an effective coarse-grained description of entanglement dynamics and operator growth in chaotic quantum many-body systems. The fundamental quantity characterizing the membrane is the entanglement line tension. However, determining the entanglement line tension for microscopic models is in general exponentially difficult. We compute the entanglement line tension in a recently introduced class of exactly solvable yet chaotic unitary circuits, so-called generalized dual-unitary circuits, obtaining a non-trivial form that gives rise to a hierarchy of velocity scales with $v_E<v_B$. We find that these circuits saturate certain bounds on entanglement growth that are also saturated in holographic models. Furthermore, we relate the entanglement line tension to temporal entanglement and correlation functions. We also find new methods of constructing generalized dual-unitary gates beyond qubits that display behavior unique to local dimension $\geq3$. Our results shed light on entanglement membrane theory in microscopic Floquet lattice models and enable us to perform non-trivial checks on the validity of its predictions by comparison to exact and numerical calculations.
Topology, magnetism and charge order in twisted MoTe2 at higher integer hole fillings

Twisted homobilayer transition metal dichalcogenide (TMD) attracts an expanding experimental interest recently for exhibiting a variety of topological and magnetic states even at zero magnetic field. Most of the studies right now focus on hole filling nu_h<1, while a rich phase diagram at higher hole filling calls for more investigation. We perform a thorough survey of possible interaction-driven phases at higher integer hole fillings. We first construct the continuum model from a first-principles calculation, and then perform a self-consistent Hartree-Fock study of the interacting ground states. We identify various valley polarized (VP) states at odd integer fillings and intervalley coherent (IVC) states at even integer fillings and discuss the energetics competition among them. We also discuss the origin and the experimental implications of the curious Chern insulator at nu_h = 2.
SoK: A Broad Comparative Evaluation of Software Debloating Tools

Software debloating tools seek to improve the program security and performance by removing unnecessary code, called bloat. While many techniques have been proposed, several barriers to their adoption have emerged. Namely, debloating tools are highly specialized, making it difficult for adopters to find the right type of tool for their needs. This is further hindered by a lack of established metrics and comparative evaluations between tools. To close this gap, we surveyed of 10 years of debloating literature and several tools currently under commercial development to systematize the debloating ecosystem's knowledge. We then conducted a broad comparative evaluation of 10 debloating tools to determine their relative strengths and weaknesses. Our evaluation, conducted on a diverse set of 20 benchmark programs, measures tools across 16 performance, security, correctness, and usability metrics. Our evaluation surfaces several concerning findings that contradict the prevailing narrative in debloating literature. First, debloating tools lack the required maturity to be used on real-world software, evidenced by a slim 21% overall success rate for creating passable debloated versions of medium- and high-complexity benchmarks. Second, debloating tools struggle to produce sound and robust programs. Using our novel differential fuzzing tool, DIFFER, we discovered that only 13% of our debloating attempts produced a sound and robust debloated program. Finally, our results indicate that debloating tools typically do not improve the performance or security posture of debloated programs by a significant degree. We believe that our contributions in this paper will help potential adopters better understand the landscape of tools and will motivate future research and development of more capable debloating tools. To this end, we have made our benchmark set, data, and custom tools publicly available.
Development and application of an inverse model technique of sources and sinks of atmospheric CO_2.

On the base of a 2-dimensional global atmospheric chemical transport model, a simplified 2-dimensional dispersion and transport model was developed for research of the inversion of sources and sinks of CO2. After verification of the model effectiveness, it was applied to study the distribution of sources and sinks of atmospheric CO2. The atmospheric CO2 sources were concentrated mainly in equatorial area and mid-high latitude of Northern Hemisphere (HNH). Ocean and terrestrial ecosystems (destruction of tropical rainforest) was the main carbon sources of the equatorial area; the terrestrial ecosystems in HNH were important carbon sinks; while the mid-high latitude of Sourthern Hemisphere displayed weak CO2 sinks. The carbon sources and sinks distribution state of 1982 (year with El Nino event) compared with that in other years was different markedly. As there was no significant variation in the state of fossil fuel combustion emission, this unusual change of carbon flux was supposed to originate from the influence of EI Nino event on the ocean and terrestrial carbon cycles.
Categorical, Ratio, and Professorial Data: The Case for Reciprocal Rank

Search engine results pages are usually abstracted as binary relevance vectors and hence are categorical data, meaning that only a limited set of operations is permitted, most notably tabulation of occurrence frequencies, with determination of medians and averages not possible. To compare retrieval systems it is thus usual to make use of a categorical-to-numeric effectiveness mapping. A previous paper has argued that any desired categorical-to-numeric mapping may be used, provided only that there is an argued connection between each category of SERP and the score that is assigned to that category by the mapping. Further, once that plausible connection has been established, then the mapped values can be treated as real-valued observations on a ratio scale, allowing the computation of averages. This article is written in support of that point of view, and to respond to ongoing claims that SERP scores may only be averaged if very restrictive conditions are imposed on the effectiveness mapping.
Excitations and phase ordering of the spin-stripe phase of a binary dipolar condensate

We consider the ground states, excitations and dynamics of a quasi-two-dimensional binary dipolar Bose-Einstein condensate. Our focus is on the transition to a spin-stripe ground state in which the translational invariance is spontaneously broken by a striped immiscible pattern of the alternating components. We develop a ground state phase diagram showing the parameter regime where the spin-stripe state occurs. Using Bogoliubov theory we calculate the excitation spectrum and structure factors. We identify a balanced regime where the system has a $\mathbb{Z}_2$ symmetry, and in the spin-stripe state this yields a nonsymmorphic symmetry. We consider the evolution of the system following a quench from the uniform to spin-stripe state, revealing novel ordering dynamics involving defects of the stripe order. Using an order parameter to characterize the orientational order of the stripes, we show that the phase ordering exhibits dynamic scaling.
Learning to Reweight for Graph Neural Network

Graph Neural Networks (GNNs) show promising results for graph tasks. However, existing GNNs' generalization ability will degrade when there exist distribution shifts between testing and training graph data. The cardinal impetus underlying the severe degeneration is that the GNNs are architected predicated upon the I.I. D assumptions. In such a setting, GNNs are inclined to leverage imperceptible statistical correlations subsisting in the training set to predict, albeit it is a spurious correlation. In this paper, we study the problem of the generalization ability of GNNs in Out-Of-Distribution (OOD) settings. To solve this problem, we propose the Learning to Reweight for Generalizable Graph Neural Network (L2R-GNN) to enhance the generalization ability for achieving satisfactory performance on unseen testing graphs that have different distributions with training graphs. We propose a novel nonlinear graph decorrelation method, which can substantially improve the out-of-distribution generalization ability and compares favorably to previous methods in restraining the over-reduced sample size. The variables of the graph representation are clustered based on the stability of the correlation, and the graph decorrelation method learns weights to remove correlations between the variables of different clusters rather than any two variables. Besides, we interpose an efficacious stochastic algorithm upon bi-level optimization for the L2R-GNN framework, which facilitates simultaneously learning the optimal weights and GNN parameters, and avoids the overfitting problem. Experimental results show that L2R-GNN greatly outperforms baselines on various graph prediction benchmarks under distribution shifts.
Bandit Sequential Posted Pricing via Half-Concavity

Sequential posted pricing auctions are popular because of their simplicity in practice and their tractability in theory. A usual assumption in their study is that the Bayesian prior distributions of the buyers are known to the seller, while in reality these priors can only be accessed from historical data. To overcome this assumption, we study sequential posted pricing in the bandit learning model, where the seller interacts with $n$ buyers over $T$ rounds: In each round the seller posts $n$ prices for the $n$ buyers and the first buyer with a valuation higher than the price takes the item. The only feedback that the seller receives in each round is the revenue. Our main results obtain nearly-optimal regret bounds for single-item sequential posted pricing in the bandit learning model. In particular, we achieve an $\tilde{O}(\mathsf{poly}(n)\sqrt{T})$ regret for buyers with (Myerson's) regular distributions and an $\tilde{O}(\mathsf{poly}(n)T^{{2}/{3}})$ regret for buyers with general distributions, both of which are tight in the number of rounds $T$. Our result for regular distributions was previously not known even for the single-buyer setting and relies on a new half-concavity property of the revenue function in the value space. For $n$ sequential buyers, our technique is to run a generalized single-buyer algorithm for all the buyers and to carefully bound the regret from the sub-optimal pricing of the suffix buyers.
Vertical structure anomalies of oceanic eddies in the Kuroshio Extension region

Using collocated altimetry sea surface height anomalies (SSHA) and Argo profiles within detected eddies, we investigated structures of temperature, salinity, potential density, geostrophic current, mixed layer depth (MLD), potential vorticity (PV), and buoyancy frequency (N) in the Kuroshio Extension (KE) region under the influences of oceanic eddies. We identified 54,302 oceanic eddies (snapshots) in the KE region during the period of 1999-2013. The composite analysis showed that changes in physical parameters modulated by the climatological composite eddies (hereinafter referred as composite eddies) were mainly confined in the upper 800 m. At the eddy core, the maximum cooling in the composite cyclonic eddy (CE) reaches 2.00°C at ∼360 m, with maximum salinity change of −0.13 psu at ∼260 m and maximum potential density change of +0.27 kg/m3 at ∼310 m. In contrast, the maximum warming in the composite anticyclonic eddy (AE) reaches 1.78°C at ∼410 m of the eddy core, with maximum salinity change of 0.12 psu at ∼260 m and maximum potential density change of −0.22 kg/m3 at ∼410 m. There were obvious anti-clockwise and clockwise geostrophic current anomalies associated with the composite CE and AE, respectively. The seasonal mean eddy-modulated MLD anomaly had significant seasonal variations. Besides, they could modulate opposite PV changes, the magnitude of which varied with depth. This article is protected by copyright. All rights reserved.
SEASONAL NUTRIENT DYNAMICS OF TUNDRA VEGETATION AT BARROW, ALASKA

Seasonal patterns of nutrient concentration and standing stock were documented in aboveground plant biomass of wet meadow tundra vegetation at Barrow, Alaska. Mobile elements, such as nitrogen, phosphorus, and potassium, were present in higher concentrations in tundra than in comparable temperate species. These elements reached peak concentration within 10 days of snowmelt and decreased to about half their maximum concentration in the course of the growing season. The total standing stock of these elements increased more rapidly aboveground than did biomass itself. Nearly half of the total nitrogen, phosphorus, and potassium complement of aboveground material was retranslocated belowground before the end of the growing season. That portion of these nutrients remaining in standing dead material at the end of the growing season was leached at rates reflecting their respective solubilities in plant tissue. Calcium and magnesium increased in concentration and in total content in the course of the growing season. Dicots had higher concentrations of phosphorus and of cations than did monocots. This is discussed with respect to their differences in mycorrhizal association and distribution. The implications for herbivores of seasonal and species differences in nutrient concentration are discussed. Fertilization with nitrogen, phosphorus,
TrkA-mediated sensory innervation of injured mouse tendon supports tendon sheath progenitor cell expansion and tendon repair

Peripheral neurons terminate at the surface of tendons partly to relay nociceptive pain signals; however, the role of peripheral nerves in tendon injury and repair remains unclear. Here, we show that after Achilles tendon injury in mice, there is new nerve growth near tendon cells that express nerve growth factor (NGF). Conditional deletion of the Ngf gene in either myeloid or mesenchymal mouse cells limited both innervation and tendon repair. Similarly, inhibition of the NGF receptor tropomyosin receptor kinase A (TrkA) abrogated tendon healing in mouse tendon injury. Sural nerve transection blocked the postinjury increase in tendon sensory innervation and the expansion of tendon sheath progenitor cells (TSPCs) expressing tubulin polymerization promoting protein family member 3. Single cell and spatial transcriptomics revealed that disruption of sensory innervation resulted in dysregulated inflammatory signaling and transforming growth factor–β (TGFβ) signaling in injured mouse tendon. Culture of mouse TSPCs with conditioned medium from dorsal root ganglia neuron further supported a role for neuronal mediators and TGFβ signaling in TSPC proliferation. Transcriptomic and histologic analyses of injured human tendon biopsy samples supported a role for innervation and TGFβ signaling in human tendon regeneration. Last, treating mice after tendon injury systemically with a small-molecule partial agonist of TrkA increased neurovascular response, TGFβ signaling, TSPC expansion, and tendon tissue repair. Although further studies should investigate the potential effects of denervation on mechanical loading of tendon, our results suggest that peripheral innervation is critical for the regenerative response after acute tendon injury. Sensory nerves support tendon repair through NGF-TrkA signaling in mice. Editor’s summary Tendons have peripheral nerve inputs for sensory feedback; however, the role of these nerves in tendon injury and repair remains unclear. Here, Cherief and colleagues demonstrated that an increase in peripheral nerve growth occurs after Achilles tendon injury in mice. Experimental blockade of nerve growth resulted in decreased tendon healing and reduced expansion of tendon sheath progenitor cells. Human tendon tissue imaging and transcriptomics corroborated the relationship between nerve growth and tendon healing. Last, activating the nerve growth factor receptor TrkA enhanced nerve responses to tendon injury, boosted progenitor cell expansion, and improved metrics of tendon repair. These findings implicate TrkA as a potential translational target to enhance tendon repair. —Molly Ogle