Miracle-dz/UniFAR-FTUs huggingface dataset:
Universal NER: A Gold-Standard Multilingual Named Entity Recognition Benchmark

We introduce Universal NER (UNER), an open, community-driven project to develop gold-standard NER benchmarks in many languages. The overarching goal of UNER is to provide high-quality, cross-lingually consistent annotations to facilitate and standardize multilingual NER research. UNER v1 contains 18 datasets annotated with named entities in a cross-lingual consistent schema across 12 diverse languages. In this paper, we detail the dataset creation and composition of UNER; we also provide initial modeling baselines on both in-language and cross-lingual learning settings. We release the data, code, and fitted models to the public.
Flow reconstruction and particle characterization from inertial Lagrangian tracks

This text describes a method to simultaneously reconstruct flow states and determine particle properties from Lagrangian particle tracking (LPT) data. LPT is a popular measurement strategy for fluids in which particles in a flow are illuminated, imaged (typically with multiple cameras), localized in 3D, and then tracked across a series of frames. The resultant"tracks"are spatially sparse, and a reconstruction algorithm is commonly employed to determine dense Eulerian velocity and pressure fields that are consistent with the data as well as the equations governing fluid dynamics. Existing LPT reconstruction algorithms presume that the particles perfectly follow the flow, but this assumption breaks down for inertial particles, which can exhibit lag or ballistic motion and may impart significant momentum to the surrounding fluid. We report an LPT reconstruction strategy that incorporates the transport physics of both the carrier fluid and particle phases, which may be parameterized to account for unknown particle properties like size and density. Our method enables the reconstruction of unsteady flow states and determination of particle properties from LPT data and the coupled governing equations for both phases. We use a neural solver to represent flow states and data-constrained polynomials to represent the tracks (though we note that our technique is compatible with a variety of solvers). Numerical tests are performed to demonstrate the reconstruction of forced isotropic turbulence and a cone-cylinder shock structure from inertial tracks that exhibit significant lag, streamline crossing, and preferential sampling.
Language Semantic Graph Guided Data-Efficient Learning

Developing generalizable models that can effectively learn from limited data and with minimal reliance on human supervision is a significant objective within the machine learning community, particularly in the era of deep neural networks. Therefore, to achieve data-efficient learning, researchers typically explore approaches that can leverage more related or unlabeled data without necessitating additional manual labeling efforts, such as Semi-Supervised Learning (SSL), Transfer Learning (TL), and Data Augmentation (DA). SSL leverages unlabeled data in the training process, while TL enables the transfer of expertise from related data distributions. DA broadens the dataset by synthesizing new data from existing examples. However, the significance of additional knowledge contained within labels has been largely overlooked in research. In this paper, we propose a novel perspective on data efficiency that involves exploiting the semantic information contained in the labels of the available data. Specifically, we introduce a Language Semantic Graph (LSG) which is constructed from labels manifest as natural language descriptions. Upon this graph, an auxiliary graph neural network is trained to extract high-level semantic relations and then used to guide the training of the primary model, enabling more adequate utilization of label knowledge. Across image, video, and audio modalities, we utilize the LSG method in both TL and SSL scenarios and illustrate its versatility in significantly enhancing performance compared to other data-efficient learning approaches. Additionally, our in-depth analysis shows that the LSG method also expedites the training process.
Data Similarity is Not Enough to Explain Language Model Performance

Large language models achieve high performance on many but not all downstream tasks. The interaction between pretraining data and task data is commonly assumed to determine this variance: a task with data that is more similar to a model's pretraining data is assumed to be easier for that model. We test whether distributional and example-specific similarity measures (embedding-, token- and model-based) correlate with language model performance through a large-scale comparison of the Pile and C4 pretraining datasets with downstream benchmarks. Similarity correlates with performance for multilingual datasets, but in other benchmarks, we surprisingly find that similarity metrics are not correlated with accuracy or even each other. This suggests that the relationship between pretraining data and downstream tasks is more complex than often assumed.
Klein tunneling degradation and enhanced Fabry-Pérot interference in graphene/h-BN moiré-superlattice devices

Hexagonal boron-nitride (h-BN) provides an ideal substrate for supporting graphene devices to achieve fascinating transport properties, such as Klein tunneling, electron optics and other novel quantum transport phenomena. However, depositing graphene on h-BN creates moiré superlattices, whose electronic properties can be significantly manipulated by controlling the lattice alignment between layers. In this work, the effects of these moiré structures on the transport properties of graphene are investigated using atomistic simulations. At large misalignment angles (leading to small moiré cells), the transport properties (most remarkably, Klein tunneling) of pristine graphene devices are conserved. On the other hand, in the nearly aligned cases, the moiré interaction induces stronger effects, significantly affecting electron transport in graphene. In particular, Klein tunneling is significantly degraded. In contrast, strong Fabry-Pérot interference (accordingly, strong quantum confinement) effects and non-linear I-V characteristics are observed. P-N interface smoothness engineering is also considered, suggesting as a potential way to improve these transport features in graphene/h-BN devices.
OFA: A Framework of Initializing Unseen Subword Embeddings for Efficient Large-scale Multilingual Continued Pretraining

Instead of pretraining multilingual language models from scratch, a more efficient method is to adapt existing pretrained language models (PLMs) to new languages via vocabulary extension and continued pretraining. However, this method usually randomly initializes the embeddings of new subwords and introduces substantially more embedding parameters to the model, thus weakening the efficiency. To address these issues, we propose a novel framework: $\textbf{O}$ne $\textbf{F}$or $\textbf{A}$ll ($\textbf{OFA}$), which wisely initializes the embeddings of unseen subwords and thus can adapt a PLM to multiple languages efficiently and effectively. OFA takes advantage of external well-aligned multilingual static word vectors and injects the alignment knowledge into the subword embeddings. In addition, OFA applies matrix factorization and replaces the cumbersome embeddings with two lower-dimensional matrices, which largely reduces the number of parameters. We show OFA accelerates the convergence of continued pretraining, which is environmentally friendly as much fewer carbon footprints are generated. Through extensive experiments, we demonstrate OFA can achieve competitive or better performance than default continued pretraining baselines on a wide range of crosslingual downstream tasks. We make our code and models publicly available.
Corrections to "Universal Densities Exist for Every Finite Reference Measure"

In the article"Universal Densities Exist for Every Finite Reference Measure"(IEEE Trans. Inform. Theory, vol. 69, no. 8, pp. 5277--5288, 2023) we neglected to mention relevant contributions of Boris Ryabko. We cited a source by him that contains a construction of the universal density that we claimed to be our own invention without checking the source after drafting the article. Our article was motivated by a preprint by Feutrill and Roughan, about which we had learned when reviewing the PhD thesis by Andrew Feutrill. Whereas we were not allowed to contact Feutrill and Roughan besides the review form, we developed some ideas of theirs further, ignoring that we stepped into the area previously researched by Ryabko. Our published results exceed those by Ryabko but the article should have been refocused to report Ryabko's contributions. In this note, we detail our citation mistakes.
MAP's not dead yet: Uncovering true language model modes by conditioning away degeneracy

It has been widely observed that exact or approximate MAP (mode-seeking) decoding from natural language generation (NLG) models consistently leads to degenerate outputs (Stahlberg and Byrne, 2019, Holtzman et al., 2019). This has generally been attributed to either a fundamental inadequacy of modes in models or weaknesses in language modeling. Contrastingly in this work, we emphasize that degenerate modes can even occur in the absence of any model error, due to contamination of the training data. Specifically, we show that mixing even a tiny amount of low-entropy noise with a population text distribution can cause the data distribution's mode to become degenerate, implying that any models trained on it will be as well. As the unconditional mode of NLG models will often be degenerate, we therefore propose to apply MAP decoding to the model's distribution conditional on avoiding specific degeneracies. Using exact-search, we empirically verify that the length-conditional modes of machine translation models and language models are indeed more fluent and topical than their unconditional modes. For the first time, we also share many examples of exact modal sequences from these models, and from several variants of the LLaMA-7B model. Notably, the modes of the LLaMA models are still degenerate, showing that improvements in modeling have not fixed this issue. Because of the cost of exact mode finding algorithms, we develop an approximate mode finding approach, ACBS, which finds sequences that are both high-likelihood and high-quality. We apply this approach to LLaMA-7B, a model which was not trained for instruction following, and find that we are able to elicit reasonable outputs without any finetuning.
ConvNet vs Transformer, Supervised vs CLIP: Beyond ImageNet Accuracy

Modern computer vision offers a great variety of models to practitioners, and selecting a model from multiple options for specific applications can be challenging. Conventionally, competing model architectures and training protocols are compared by their classification accuracy on ImageNet. However, this single metric does not fully capture performance nuances critical for specialized tasks. In this work, we conduct an in-depth comparative analysis of model behaviors beyond ImageNet accuracy, for both ConvNet and Vision Transformer architectures, each across supervised and CLIP training paradigms. Although our selected models have similar ImageNet accuracies and compute requirements, we find that they differ in many other aspects: types of mistakes, output calibration, transferability, and feature invariance, among others. This diversity in model characteristics, not captured by traditional metrics, highlights the need for more nuanced analysis when choosing among different models. Our code is available at https://github.com/kirill-vish/Beyond-INet.
Interannual variability of cirrus clouds in the tropics in El Niño Southern Oscillation (ENSO) regions based on International Satellite Cloud Climatology Project (ISCCP) satellite data

The percentage of the interannual variance of cirrus clouds, explained by the variability of the El Niño Southern Oscillation (ENSO), the Quasi-biennial Oscillation (QBO) and solar activity over the tropics, is presented in this article. Analysis is focused on the eastern and western tropical Pacific Ocean, which is strongly affected by the ENSO. It is shown that over the eastern tropical Pacific Ocean, the amplitude of the ENSO in cirrus-cloud cover (CCC) is about 8.0%. The amplitude of the annual cycle is about 0.8% and the amplitudes of the QBO, solar cycle and long-term trends are the order of 1.0%, 0.1% and 0.3%, respectively. Using as an index of convective activity in the upper troposphere the vertical velocities at 300 hPa, we have calculated a vertical velocity related to cloud component. It is shown that the total contribution, from all related cloud components examined explains about 65% of the variance in cirrus clouds over the eastern Pacific and about 50% of the variance in cirrus clouds over its western part.
Abelian gauge-like groups of $L_\infty$-algebras

Given a finite type nilpotent $L_\infty$-algebra, we construct an abelian group that acts on the set of Maurer-Cartan elements of the given $L_\infty$-algebra so that the quotient by this action becomes the moduli space of equivalence classes of Maurer-Cartan elements.
Image complexity based fMRI-BOLD visual network categorization across visual datasets using topological descriptors and deep-hybrid learning

This study proposes a new approach that investigates differences in topological characteristics of visual networks, which are constructed using fMRI BOLD time-series corresponding to visual datasets of COCO, ImageNet, and SUN. A publicly available BOLD5000 dataset is utilized that contains fMRI scans while viewing 5254 images of diverse complexities. The objective of this study is to examine how network topology differs in response to distinct visual stimuli from these visual datasets. To achieve this, 0- and 1-dimensional persistence diagrams are computed for each visual network representing COCO, ImageNet, and SUN. For extracting suitable features from topological persistence diagrams, K-means clustering is executed. The extracted K-means cluster features are fed to a novel deep-hybrid model that yields accuracy in the range of 90%-95% in classifying these visual networks. To understand vision, this type of visual network categorization across visual datasets is important as it captures differences in BOLD signals while perceiving images with different contexts and complexities. Furthermore, distinctive topological patterns of visual network associated with each dataset, as revealed from this study, could potentially lead to the development of future neuroimaging biomarkers for diagnosing visual processing disorders like visual agnosia or prosopagnosia, and tracking changes in visual cognition over time.
Fast Certification of Vision-Language Models Using Incremental Randomized Smoothing

A key benefit of deep vision-language models such as CLIP is that they enable zero-shot open vocabulary classification; the user has the ability to define novel class labels via natural language prompts at inference time. However, while CLIP-based zero-shot classifiers have demonstrated competitive performance across a range of domain shifts, they remain highly vulnerable to adversarial attacks. Therefore, ensuring the robustness of such models is crucial for their reliable deployment in the wild. In this work, we introduce Open Vocabulary Certification (OVC), a fast certification method designed for open-vocabulary models like CLIP via randomized smoothing techniques. Given a base"training"set of prompts and their corresponding certified CLIP classifiers, OVC relies on the observation that a classifier with a novel prompt can be viewed as a perturbed version of nearby classifiers in the base training set. Therefore, OVC can rapidly certify the novel classifier using a variation of incremental randomized smoothing. By using a caching trick, we achieve approximately two orders of magnitude acceleration in the certification process for novel prompts. To achieve further (heuristic) speedups, OVC approximates the embedding space at a given input using a multivariate normal distribution bypassing the need for sampling via forward passes through the vision backbone. We demonstrate the effectiveness of OVC on through experimental evaluation using multiple vision-language backbones on the CIFAR-10 and ImageNet test datasets.
Comparative effects of alternate partial root-zone drying and conventional deficit irrigation on growth and yield of field grown maize (Zea mays L.) hybrid

A field experiment was conducted to study the effects of partial root-zone drying on growth and yield of maize ( Zea mays L.), grown under drip irrigation system. Experimental treatments included alternate partial root-zone drying (APRD), fixed partial root-zone drying (FPRD), deficit irrigation (DI) and full irrigation (FI). Soil moisture in root-zone of plants under FI treatment was kept close to the field capacity while plants under other treatments were irrigated with 50% of the water applied to FI throughout the growing season. Water deficit under APRD, FPRD and DI treatments caused significant inhibition of plant growth by affecting plant height, leaf area, and produced biomass. Number of leaves per plant remained at par with each other for plants under FI and APRD treatment, however, significant reduction in plant leaves was observed in FPRD and DI treatments. Water deficit in APRD, FPRD and DI treatment decreased grain yield per plant by 14.44%, 27.52% and 29.29% respectively. The difference between FPRD and DI was not significant either in the vegetative growth or grain yield. Data indicated maximum water use efficiency, higher grain yield, and better vegetative growth in plants under APRD irrigation scheme when compared to FPRD and DI.
Black Hole Mass and Eddington-ratio Distributions of Less-luminous Quasars at z ∼ 4 in the Subaru Hyper Suprime-Cam Wide Field

We investigate the black hole mass function (BHMF) and Eddington-ratio distribution function (ERDF) of broad-line active galactic nuclei (AGNs) at z = 4, based on a sample of 52 quasars with i < 23.2 at 3.50 ≤ z ≤ 4.25 from the Hyper Suprime-Cam Subaru Strategic Program S16A-Wide2 data set, and 1462 quasars with i < 20.2 in the same redshift range from the Sloan Digital Sky Survey data release 7 quasar catalog. Virial black hole (BH) masses of quasars are estimated using the width of the CIV 1549 Å line and the continuum luminosity at 1350 Å. To obtain the intrinsic broad-line AGN BHMF and ERDF, we correct for the incompleteness in the low-mass and/or low-Eddington-ratio ranges caused by the flux-limited selection. The resulting BHMF is constrained down to log M BH / M ⊙ ∼ 7.5 . In comparison with broad-line AGN BHMFs at z ∼ 2 in the literature, we find that the number density of massive SMBHs peaks at higher redshifts, consistent with the downsizing evolutionary scenario. Additionally, the resulting ERDF shows a negative dependence on BH mass, suggesting more massive SMBHs tend to accrete at lower-Eddington ratios at z = 4. With the derived intrinsic broad-line AGN BHMF, we also evaluate the active fraction of broad-line AGNs among the entire SMBH population at z = 4. The resulting active fraction may suggest a positive dependence on BH mass. Finally, we examine the time evolution of broad-line AGN BHMF between z = 4 and 6 through solving the continuity equation. The results suggest that the broad-line AGN BHMFs at z = 4–6 only show evolution in their normalization, but with no significant changes in their shape.
A Diffusion Model Based Quality Enhancement Method for HEVC Compressed Video

Video post-processing methods can improve the quality of compressed videos at the decoder side. Most of the existing methods need to train corresponding models for compressed videos with different quantization parameters to improve the quality of compressed videos. However, in most cases, the quantization parameters of the decoded video are unknown. This makes existing methods have their limitations in improving video quality. To tackle this problem, this work proposes a diffusion model based post-processing method for compressed videos. The proposed method first estimates the feature vectors of the compressed video and then uses the estimated feature vectors as the prior information for the quality enhancement model to adaptively enhance the quality of compressed video with different quantization parameters. Experimental results show that the quality enhancement results of our proposed method on mixed datasets are superior to existing methods.
Unsupervised approaches based on optimal transport and convex analysis for inverse problems in imaging

Unsupervised deep learning approaches have recently become one of the crucial research areas in imaging owing to their ability to learn expressive and powerful reconstruction operators even when paired high-quality training data is scarcely available. In this chapter, we review theoretically principled unsupervised learning schemes for solving imaging inverse problems, with a particular focus on methods rooted in optimal transport and convex analysis. We begin by reviewing the optimal transport-based unsupervised approaches such as the cycle-consistency-based models and learned adversarial regularization methods, which have clear probabilistic interpretations. Subsequently, we give an overview of a recent line of works on provably convergent learned optimization algorithms applied to accelerate the solution of imaging inverse problems, alongside their dedicated unsupervised training schemes. We also survey a number of provably convergent plug-and-play algorithms (based on gradient-step deep denoisers), which are among the most important and widely applied unsupervised approaches for imaging problems. At the end of this survey, we provide an overview of a few related unsupervised learning frameworks that complement our focused schemes. Together with a detailed survey, we provide an overview of the key mathematical results that underlie the methods reviewed in the chapter to keep our discussion self-contained.
From plactic monoids to hypoplactic monoids

The plactic monoids can be obtained from the tensor product of crystals. Similarly, the hypoplactic monoids can be obtained from the quasi-tensor product of quasi-crystals. In this paper, we present a unified approach to these constructions by expressing them in the context of quasi-crystals. We provide a sufficient condition to obtain a quasi-crystal monoid for the quasi-tensor product from a quasi-crystal monoid for the tensor product. We also establish a sufficient condition for a hypoplactic monoid to be a quotient of the plactic monoid associated to the same seminormal quasi-crystal.
Growth of orbital resonances around a black hole surrounded by matter

This work studies the dynamics of geodesic motion within a curved spacetime around a Schwarzschild black hole, perturbed by a gravitational field of a far axisymmetric distribution of mass enclosing the system. This spacetime can serve as a versatile model for a diverse range of astrophysical scenarios and, in particular, for extreme mass ratio inspirals as in our work. We show that the system is non-integrable by employing Poincar\'e surface of section and rotation numbers. By utilising the rotation numbers, the widths of resonances are calculated, which are then used in establishing the relation between the underlying perturbation parameter driving the system from integrability and the quadrupole parameter characterising the perturbed metric. This relation allows us to estimate the phase shift caused by the resonance during an inspiral.