Miracle-dz/UniFAR-FTUs huggingface dataset:
Global existence of strong solutions to the Landau-Lifshitz-Slonczewski equation

In this paper, we focus on the existence of strong solutions for the Cauchy problem of the three-dimensional Landau-Lifshitz-Slonczewski equation. We construct a new combination of Bourgain space and Lebesgue space where linear and nonlinear estimates can be closed by applying frequency decomposition and energy methods. Finally, we establish the existence and uniqueness of the global strong solution provided that the initial data belongs to Besov space $\dot{B}^{\frac{n}{2}}_{\Omega}$.
TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environments

Although the estimation of 3D human pose and shape (HPS) is rapidly progressing, current methods still cannot reliably estimate moving humans in global coordinates, which is critical for many applications. This is particularly challenging when the camera is also moving, entangling human and camera motion. To address these issues, we adopt a novel 5D representation (space, time, and identity) that enables end-to-end reasoning about people in scenes. Our method, called TRACE, introduces several novel architectural components. Most importantly, it uses two new “maps” to reason about the 3D trajectory of people over time in camera, and world, coordinates. An additional memory unit enables persistent tracking of people even during long occlusions. TRACE is the first one-stage method to jointly recover and track 3D humans in global coordinates from dynamic cameras. By training it end-to-end, and using full image information, TRACE achieves state-of-the-art performance on tracking and HPS benchmarks. The code11https://www.yusun. work/TRACE/TRACE.html and dataset22https://github.com/Arthur151/DynaCam are released for research purposes.
MultiLegalPile: A 689GB Multilingual Legal Corpus

Large, high-quality datasets are crucial for training Large Language Models (LLMs). However, so far, there are few datasets available for specialized critical domains such as law and the available ones are often only for the English language. We curate and release MultiLegalPile, a 689GB corpus in 24 languages from 17 jurisdictions. The MultiLegalPile corpus, which includes diverse legal data sources with varying licenses, allows for pretraining NLP models under fair use, with more permissive licenses for the Eurlex Resources and Legal mC4 subsets. We pretrain two RoBERTa models and one Longformer multilingually, and 24 monolingual models on each of the language-specific subsets and evaluate them on LEXTREME. Additionally, we evaluate the English and multilingual models on LexGLUE. Our multilingual models set a new SotA on LEXTREME and our English models on LexGLUE. We release the dataset, the trained models, and all of the code under the most open possible licenses.
Calibrated Stackelberg Games: Learning Optimal Commitments Against Calibrated Agents

In this paper, we introduce a generalization of the standard Stackelberg Games (SGs) framework: Calibrated Stackelberg Games (CSGs). In CSGs, a principal repeatedly interacts with an agent who (contrary to standard SGs) does not have direct access to the principal's action but instead best-responds to calibrated forecasts about it. CSG is a powerful modeling tool that goes beyond assuming that agents use ad hoc and highly specified algorithms for interacting in strategic settings and thus more robustly addresses real-life applications that SGs were originally intended to capture. Along with CSGs, we also introduce a stronger notion of calibration, termed adaptive calibration, that provides fine-grained any-time calibration guarantees against adversarial sequences. We give a general approach for obtaining adaptive calibration algorithms and specialize them for finite CSGs. In our main technical result, we show that in CSGs, the principal can achieve utility that converges to the optimum Stackelberg value of the game both in finite and continuous settings, and that no higher utility is achievable. Two prominent and immediate applications of our results are the settings of learning in Stackelberg Security Games and strategic classification, both against calibrated agents.
Restricted shtukas and $\Psi$-factorizable sheaves

We show that Lusztig's theories of two-sided cells and non-unipotent representations of a reductive group over a finite field are compatible with the V. Lafforgue's automorphic-to-galois direction of the Langlands correspondence. To do this, we extend cases where nearby cycles commutes with pushforward from sheaves on the moduli space of shtukas to a product of curves to include certain depth $0$ cases. More generally, we introduce the notion of $\Psi$-factorizability to study nearby cycles over general bases, whereby a sheaf is $\Psi$-factorizable if its nearby cycles are the same as iterated nearby cycles with respect to arbitrary compositions of specializations on the base. The Satake sheaves on Beilinson-Drinfeld grassmannians and their cohomology sheaves on curves are nontrivial examples of $\Psi$-factorizable sheaves. This notion allows us to adapt arguments from Xue. As an application, for certain automorphic forms in depth zero attached to a Langlands parameter, we characterize the image of the tame generator of this parameter in terms of semisimple orbits and two-sided cells attached to representations, extending ideas of Lusztig-Yun and Bezrukavnikov-Finkelberg-Ostrik.
A Simple yet Effective Self-Debiasing Framework for Transformer Models

Current Transformer-based natural language understanding (NLU) models heavily rely on dataset biases, while failing to handle real-world out-of-distribution (OOD) instances. Many methods have been proposed to deal with this issue, but they ignore the fact that the features learned in different layers of Transformer-based NLU models are different. In this paper, we first conduct preliminary studies to obtain two conclusions: 1) both low- and high-layer sentence representations encode common biased features during training; 2) the low-layer sentence representations encode fewer unbiased features than the highlayer ones. Based on these conclusions, we propose a simple yet effective self-debiasing framework for Transformer-based NLU models. Concretely, we first stack a classifier on a selected low layer. Then, we introduce a residual connection that feeds the low-layer sentence representation to the top-layer classifier. In this way, the top-layer sentence representation will be trained to ignore the common biased features encoded by the low-layer sentence representation and focus on task-relevant unbiased features. During inference, we remove the residual connection and directly use the top-layer sentence representation to make predictions. Extensive experiments and indepth analyses on NLU tasks show that our framework performs better than several competitive baselines, achieving a new SOTA on all OOD test sets.
Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalization

The out-of-distribution (OOD) problem generally arises when neural networks encounter data that significantly deviates from the training data distribution, i.e., in-distribution (InD). In this paper, we study the OOD problem from a neuron activation view. We first formulate neuron activation states by considering both the neuron output and its influence on model decisions. Then, to characterize the relationship between neurons and OOD issues, we introduce the \textit{neuron activation coverage} (NAC) -- a simple measure for neuron behaviors under InD data. Leveraging our NAC, we show that 1) InD and OOD inputs can be largely separated based on the neuron behavior, which significantly eases the OOD detection problem and beats the 21 previous methods over three benchmarks (CIFAR-10, CIFAR-100, and ImageNet-1K). 2) a positive correlation between NAC and model generalization ability consistently holds across architectures and datasets, which enables a NAC-based criterion for evaluating model robustness. Compared to prevalent InD validation criteria, we show that NAC not only can select more robust models, but also has a stronger correlation with OOD test performance.
The disruption index is biased by citation inflation

A recent analysis of scientific publication and patent citation networks by Park et al. (Nature, 2023) suggests that publications and patents are becoming less disruptive over time. Here we show that the reported decrease in disruptiveness is an artifact of systematic shifts in the structure of citation networks unrelated to innovation system capacity. Instead, the decline is attributable to 'citation inflation', an unavoidable characteristic of real citation networks that manifests as a systematic time-dependent bias and renders cross-temporal analysis challenging. One driver of citation inflation is the ever-increasing lengths of reference lists over time, which in turn increases the density of links in citation networks, and causes the disruption index to converge to 0. A second driver is attributable to shifts in the construction of reference lists, which is increasingly impacted by self-citations that increase in the rate of triadic closure in citation networks, and thus confounds efforts to measure disruption, which is itself a measure of triadic closure. Combined, these two systematic shifts render the disruption index temporally biased, and unsuitable for cross-temporal analysis. The impact of this systematic bias further stymies efforts to correlate disruption to other measures that are also time-dependent, such as team size and citation counts. In order to demonstrate this fundamental measurement problem, we present three complementary lines of critique (deductive, empirical and computational modeling), and also make available an ensemble of synthetic citation networks that can be used to test alternative citation-based indices for systematic bias.
JADES Initial Data Release for the Hubble Ultra Deep Field: Revealing the Faint Infrared Sky with Deep JWST NIRCam Imaging

JWST has revolutionized the field of extragalactic astronomy with its sensitive and high-resolution infrared view of the distant Universe. Adding to the new legacy of JWST observations, we present the first NIRCam imaging data release from the JWST Advanced Deep Extragalactic Survey (JADES), providing nine filters of infrared imaging of ∼25 arcmin2 covering the Hubble Ultra Deep Field and portions of Great Observatories Origins Deep Survey South. Utilizing 87 on-sky dual-filter hours of exposure time, these images reveal the deepest ever near-infrared view of this iconic field. We supply carefully constructed nine-band mosaics of the JADES bands, as well as matching reductions of five additional bands from the JWST Extragalactic Medium-band Survey. Combining with existing Hubble Space Telescope imaging, we provide 23-band space-based photometric catalogs and photometric redshifts for ≈47,500 sources. To promote broad engagement with JADES, we have created an interactive FitsMap website to provide an interface for professional researchers and the public to experience these JWST data sets. Combined with the first JADES NIRSpec data release, these public JADES imaging and spectroscopic data sets provide a new foundation for discoveries of the infrared Universe by the worldwide scientific community.
Evaluating GPT's Programming Capability through CodeWars' Katas

In the burgeoning field of artificial intelligence (AI), understanding the capabilities and limitations of programming-oriented models is crucial. This paper presents a novel evaluation of the programming proficiency of Generative Pretrained Transformer (GPT) models, specifically GPT-3.5 and GPT-4, against coding problems of varying difficulty levels drawn from Codewars. The experiments reveal a distinct boundary at the 3kyu level, beyond which these GPT models struggle to provide solutions. These findings led to the proposal of a measure for coding problem complexity that incorporates both problem difficulty and the time required for solution. The research emphasizes the need for validation and creative thinking capabilities in AI models to better emulate human problem-solving techniques. Future work aims to refine this proposed complexity measure, enhance AI models with these suggested capabilities, and develop an objective measure for programming problem difficulty. The results of this research offer invaluable insights for improving AI programming capabilities and advancing the frontier of AI problem-solving abilities.
Binary and Ternary Natural Language Generation

Ternary and binary neural networks enable multiplication-free computation and promise multiple orders of magnitude efficiency gains over full-precision networks if implemented on specialized hardware. However, since both the parameter and the output space are highly discretized, such networks have proven very difficult to optimize. The difficulties are compounded for the class of transformer text generation models due to the sensitivity of the attention operation to quantization and the noise-compounding effects of autoregressive decoding in the high-cardinality output space. We approach the problem with a mix of statistics-based quantization for the weights and elastic quantization of the activations and demonstrate the first ternary and binary transformer models on the downstream tasks of summarization and machine translation. Our ternary BART base achieves an R1 score of 41 on the CNN/DailyMail benchmark, which is merely 3.9 points behind the full model while being 16x more efficient. Our binary model, while less accurate, achieves a highly non-trivial score of 35.6. For machine translation, we achieved BLEU scores of 21.7 and 17.6 on the WMT16 En-Ro benchmark, compared with a full precision mBART model score of 26.8. We also compare our approach in the 8-bit activation setting, where our ternary and even binary weight models can match or outperform the best existing 8-bit weight models in the literature. Our code and models are available at: https://github.com/facebookresearch/Ternary_Binary_Transformer.
Neuralangelo: High-Fidelity Neural Surface Reconstruction

Neural surface reconstruction has been shown to be powerful for recovering dense 3D surfaces via image-based neural rendering. However, current methods struggle to recover detailed structures of real-world scenes. To address the issue, we present Neuralangelo, which combines the representation power of multiresolution 3D hash grids with neural surface rendering. Two key ingredients enable our approach: (1) numerical gradients for computing higher-order derivatives as a smoothing operation and (2) coarse-to-fine optimization on the hash grids controlling different levels of details. Even without auxiliary inputs such as depth, Neuralangelo can effectively recover dense 3D surface structures from multiview images with fidelity significantly surpassing previous methods, enabling detailed large-scale scene reconstruction from RGB video captures.
Measuring the X-ray luminosities of DESI groups from eROSITA Final Equatorial-Depth Survey – I. X-ray luminosity–halo mass scaling relation

We use the eROSITA Final Equatorial-Depth Survey (eFEDS) to measure the rest-frame 0.1–2.4 keV band X-ray luminosities of ∼600 000 DESI groups using two different algorithms in the overlap region of the two observations. These groups span a large redshift range of 0.0 ≤ zg ≤ 1.0 and group mass range of $10^{10.76}\, h^{-1}\, \mathrm{M}_{\odot } \le M_h \le 10^{15.0}\, h^{-1}\, \mathrm{M}_{\odot }$. (1) Using the blind detection pipeline of eFEDS, we find that 10932 X-ray emission peaks can be cross-matched with our groups, ∼38 per cent of which have a signal-to-noise ratio $\rm {S}/\rm {N} \ge 3$ in X-ray detection. Comparing to the numbers reported in previous studies, this matched sample size is a factor of ∼6 larger. (2) By stacking X-ray maps around groups with similar masses and redshifts, we measure the average X-ray luminosity of groups as a function of halo mass in five redshift bins. We find that in a wide halo mass range, the X-ray luminosity, LX, is roughly linearly proportional to Mh and quite independent to the redshift of the groups. (3) We use a Poisson distribution to model the X-ray luminosities obtained using two different algorithms and obtain the best-fit $L_{\rm X}=10^{28.46\pm 0.03}M_{\rm h}^{1.024\pm 0.002}$ and $L_{\rm X}=10^{26.73 \pm 0.04}M_{\rm h}^{1.140 \pm 0.003}$ scaling relations, respectively. The best-fit slopes are flatter than the results previously obtained but closer to a self-similar prediction.
Reassembling Broken Objects using Breaking Curves

Reassembling 3D broken objects is a challenging task. A robust solution that generalizes well must deal with diverse patterns associated with different types of broken objects. We propose a method that tackles the pairwise assembly of 3D point clouds, that is agnostic on the type of object, and that relies solely on their geometrical information, without any prior information on the shape of the reconstructed object. The method receives two point clouds as input and segments them into regions using detected closed boundary contours, known as breaking curves. Possible alignment combinations of the regions of each broken object are evaluated and the best one is selected as the final alignment. Experiments were carried out both on available 3D scanned objects and on a recent benchmark for synthetic broken objects. Results show that our solution performs well in reassembling different kinds of broken objects.
Boundary criticality via gauging finite subgroups: a case study on the clock model

Gauging a finite Abelian normal subgroup $\Gamma$ of a nonanomalous 0-form symmetry $G$ of a theory in $(d+1)$D spacetime can yield an unconventional critical point if the original theory has a continuous transition where $\Gamma$ is completely spontaneously broken and if $G$ is a nontrivial extension of $G/\Gamma$ by $\Gamma$. The gauged theory has symmetry $G/\Gamma \times \hat{\Gamma}^{(d-1)}$, where $\hat{\Gamma}^{(d-1)}$ is the $(d-1)$-form dual symmetry of $\Gamma$, and a 't Hooft anomaly between them. Thus it can be viewed as a boundary of a topological phase protected by $G/\Gamma \times \hat{\Gamma}^{(d-1)}$. The ordinary critical point, upon gauging, is mapped to a deconfined quantum critical point between two ordinary symmetry-breaking phases ($d =1$) or an unconventional quantum critical point between an ordinary symmetry-breaking phase and a topologically ordered phase ($d\ge 2$) associated with $G/\Gamma$ and $\hat{\Gamma}^{(d-1)}$, respectively. Order parameters and disorder parameters, before and after gauging, can be directly related. As a concrete example, we gauge the $\mathbb{Z}_2$ subgroup of $\mathbb{Z}_4$ symmetry of a 4-state clock model on a 1D lattice and a 2D square lattice. Since the symmetry of the clock model contains $D_8$, the dihedral group of order 8, we also analyze the anomaly structure which is similar to that in the compactified $SU(2)$ gauge theory with $\theta =\pi$ in $(3+1)$D and its mixed gauge theory. The general case is also discussed.
Alternating parity weak sequencing

A subset of a group is ‐weakly sequenceable if there is an ordering of its elements such that the partial sums , given by and for , satisfy whenever and . By Costa et al., it was proved that if the order of a group is then all sufficiently large subsets of the nonidentity elements are ‐weakly sequenceable when is prime, and . Inspired by this result, we show that, if is the semidirect product of and and the subset is balanced, then admits, regardless of its size, an alternating parity ‐weak sequencing whenever is prime and . A subset of is balanced if it contains the same number of even elements and odd elements and an alternating parity ordering alternates even and odd elements. Then using a hybrid approach that combines both Ramsey theory and the probabilistic method we also prove, for groups that are semidirect products of a generic (nonnecessarily abelian) group and , that all sufficiently large balanced subsets of the nonidentity elements admit an alternating parity ‐weak sequencing. The same procedure works also for studying the weak sequenceability for generic sufficiently large (not necessarily balanced) sets. Here we have been able to prove that, if the size of a subset of a group is large enough and if does not contain 0, then is ‐weakly sequenceable.
Comparison of two coefficients of variation: a new Bayesian approach

The coefficient of variation is a useful indicator for comparing the spread of values between dataset with different units or widely different means. In this paper we address the problem of investigating the equality of the coefficients of variation from two independent populations. In order to do this we rely on the Bayesian Discrepancy Measure recently introduced in the literature. Computing this Bayesian measure of evidence is straightforward when the coefficient of variation is a function of a single parameter of the distribution. In contrast, it becomes difficult when it is a function of more parameters, often requiring the use of MCMC methods. We calculate the Bayesian Discrepancy Measure by considering a variety of distributions whose coefficients of variation depend on more than one parameter. We consider also applications to real data. As far as we know, some of the examined problems have not yet been covered in the literature.
Technical Report on Token Position Bias in Transformers

Language Models (LMs) have shown state-of-the-art performance in Natural Language Processing (NLP) tasks. Downstream tasks such as Named Entity Recognition (NER) or Part-of-Speech (POS) tagging are known to suffer from data imbalance issues, specifically in terms of the ratio of positive to negative examples, and class imbalance. In this paper, we investigate an additional specific issue for language models, namely the position bias of positive examples in token classification tasks. Therefore, we conduct an in-depth evaluation of the impact of position bias on the performance of LMs when fine-tuned on Token Classification benchmarks. Our study includes CoNLL03 and OntoNote5.0 for NER, English Tree Bank UD_en and TweeBank for POS tagging. We propose an evaluation approach to investigate position bias in Transformer models. We show that encoders like BERT, ERNIE, ELECTRA, and decoders such as GPT2 and BLOOM can suffer from this bias with an average drop of 3\% and 9\% in their performance. To mitigate this effect, we propose two methods: Random Position Shifting and Context Perturbation, that we apply on batches during the training process. The results show an improvement of $\approx$ 2\% in the performance of the model on CoNLL03, UD_en, and TweeBank.