Miracle-dz/UniFAR-FTUs huggingface dataset:
AlignMiF: Geometry-Aligned Multimodal Implicit Field for LiDAR-Camera Joint Synthesis

Neural implicit fields have been a de facto standard in novel view synthesis. Recently, there exist some methods exploring fusing multiple modalities within a single field, aiming to share implicit features from different modalities to enhance reconstruction performance. However, these modalities often exhibit misaligned behaviors: optimizing for one modality, such as LiDAR, can adversely affect another, like camera performance, and vice versa. In this work, we conduct comprehensive analyses on the multimodal implicit field of LiDAR-camera joint synthesis, revealing the underlying issue lies in the misalignment of different sensors. Furthermore, we introduce AlignMiF, a geometrically aligned multimodal implicit field with two proposed modules: Geometry-Aware Alignment (GAA) and Shared Geometry Initialization (SGI). These modules effectively align the coarse geometry across different modalities, significantly enhancing the fusion process between LiDAR and camera data. Through extensive experiments across various datasets and scenes, we demonstrate the effectiveness of our approach in facilitating better interaction between LiDAR and camera modalities within a unified neural field. Specifically, our proposed AlignMiF, achieves remarkable improvement over recent implicit fusion methods (+2.01 and +3.11 image PSNR on the KITTI-360 and Waymo datasets) and consistently surpasses single modality performance (13.8% and 14.2% reduction in LiDAR Chamfer Distance on the respective datasets).
CorpusBrain++: A Continual Generative Pre-Training Framework for Knowledge-Intensive Language Tasks

Knowledge-intensive language tasks (KILTs) typically require retrieving relevant documents from trustworthy corpora, e.g., Wikipedia, to produce specific answers. Very recently, a pre-trained generative retrieval model for KILTs, named CorpusBrain, was proposed and reached new state-of-the-art retrieval performance. However, most existing research on KILTs, including CorpusBrain, has predominantly focused on a static document collection, overlooking the dynamic nature of real-world scenarios, where new documents are continuously being incorporated into the source corpus. To address this gap, it is crucial to explore the capability of retrieval models to effectively handle the dynamic retrieval scenario inherent in KILTs. In this work, we first introduce the continual document learning (CDL) task for KILTs and build a novel benchmark dataset named KILT++ based on the original KILT dataset for evaluation. Then, we conduct a comprehensive study over the use of pre-trained CorpusBrain on KILT++. Unlike the promising results in the stationary scenario, CorpusBrain is prone to catastrophic forgetting in the dynamic scenario, hence hampering the retrieval performance. To alleviate this issue, we propose CorpusBrain++, a continual generative pre-training framework. Empirical results demonstrate the significant effectiveness and remarkable efficiency of CorpusBrain++ in comparison to both traditional and generative IR methods.
Deep Learning Based Named Entity Recognition Models for Recipes

Food touches our lives through various endeavors, including flavor, nourishment, health, and sustainability. Recipes are cultural capsules transmitted across generations via unstructured text. Automated protocols for recognizing named entities, the building blocks of recipe text, are of immense value for various applications ranging from information extraction to novel recipe generation. Named entity recognition is a technique for extracting information from unstructured or semi-structured data with known labels. Starting with manually-annotated data of 6,611 ingredient phrases, we created an augmented dataset of 26,445 phrases cumulatively. Simultaneously, we systematically cleaned and analyzed ingredient phrases from RecipeDB, the gold-standard recipe data repository, and annotated them using the Stanford NER. Based on the analysis, we sampled a subset of 88,526 phrases using a clustering-based approach while preserving the diversity to create the machine-annotated dataset. A thorough investigation of NER approaches on these three datasets involving statistical, fine-tuning of deep learning-based language models and few-shot prompting on large language models (LLMs) provides deep insights. We conclude that few-shot prompting on LLMs has abysmal performance, whereas the fine-tuned spaCy-transformer emerges as the best model with macro-F1 scores of 95.9%, 96.04%, and 95.71% for the manually-annotated, augmented, and machine-annotated datasets, respectively.
SongComposer: A Large Language Model for Lyric and Melody Composition in Song Generation

We present SongComposer, an innovative LLM designed for song composition. It could understand and generate melodies and lyrics in symbolic song representations, by leveraging the capability of LLM. Existing music-related LLM treated the music as quantized audio signals, while such implicit encoding leads to inefficient encoding and poor flexibility. In contrast, we resort to symbolic song representation, the mature and efficient way humans designed for music, and enable LLM to explicitly compose songs like humans. In practice, we design a novel tuple design to format lyric and three note attributes (pitch, duration, and rest duration) in the melody, which guarantees the correct LLM understanding of musical symbols and realizes precise alignment between lyrics and melody. To impart basic music understanding to LLM, we carefully collected SongCompose-PT, a large-scale song pretraining dataset that includes lyrics, melodies, and paired lyrics-melodies in either Chinese or English. After adequate pre-training, 10K carefully crafted QA pairs are used to empower the LLM with the instruction-following capability and solve diverse tasks. With extensive experiments, SongComposer demonstrates superior performance in lyric-to-melody generation, melody-to-lyric generation, song continuation, and text-to-song creation, outperforming advanced LLMs like GPT-4.
Generalized Ces\`aro operator acting on Hilbert spaces of analytic functions

Let $\mathbb{D}$ denote the unit disc in $\mathbb{C}$. We define the generalized Ces\`aro operator as follows $$ C_{\omega}(f)(z)=\int_0^1 f(tz)\left(\frac{1}{z}\int_0^z B^{\omega}_t(u)\,du\right)\,\omega(t)dt,$$ where $\{B^{\omega}_\zeta\}_{\zeta\in\mathbb{D}}$ are the reproducing kernels of the Bergman space $A^2_\omega$ induced by a radial weight $\omega$ in the unit disc $\mathbb{D}$. We study the action of the operator $C_{\omega}$ on weighted Hardy spaces of analytic functions $\mathcal{H}_{\gamma}$, $\gamma>0$ and on general weighted Bergman spaces $A^2_{\mu}$.
Uncertainty Quantification in Anomaly Detection with Cross-Conformal p-Values

Given the growing significance of reliable, trustworthy, and explainable machine learning, the requirement of uncertainty quantification for anomaly detection systems has become increasingly important. In this context, effectively controlling Type I error rates ($\alpha$) without compromising the statistical power ($1-\beta$) of these systems can build trust and reduce costs related to false discoveries, particularly when follow-up procedures are expensive. Leveraging the principles of conformal prediction emerges as a promising approach for providing respective statistical guarantees by calibrating a model's uncertainty. This work introduces a novel framework for anomaly detection, termed cross-conformal anomaly detection, building upon well-known cross-conformal methods designed for prediction tasks. With that, it addresses a natural research gap by extending previous works in the context of inductive conformal anomaly detection, relying on the split-conformal approach for model calibration. Drawing on insights from conformal prediction, we demonstrate that the derived methods for calculating cross-conformal $p$-values strike a practical compromise between statistical efficiency (full-conformal) and computational efficiency (split-conformal) for uncertainty-quantified anomaly detection on benchmark datasets.
Multi-Task Contrastive Learning for 8192-Token Bilingual Text Embeddings

We introduce a novel suite of state-of-the-art bilingual text embedding models that are designed to support English and another target language. These models are capable of processing lengthy text inputs with up to 8192 tokens, making them highly versatile for a range of natural language processing tasks such as text retrieval, clustering, and semantic textual similarity (STS) calculations. By focusing on bilingual models and introducing a unique multi-task learning objective, we have significantly improved the model performance on STS tasks, which outperforms the capabilities of existing multilingual models in both target language understanding and cross-lingual evaluation tasks. Moreover, our bilingual models are more efficient, requiring fewer parameters and less memory due to their smaller vocabulary needs. Furthermore, we have expanded the Massive Text Embedding Benchmark (MTEB) to include benchmarks for German and Spanish embedding models. This integration aims to stimulate further research and advancement in text embedding technologies for these languages.
Tight Lower Bounds for Block-Structured Integer Programs

We study fundamental block-structured integer programs called tree-fold and multi-stage IPs. Tree-fold IPs admit a constraint matrix with independent blocks linked together by few constraints in a recursive pattern; and transposing their constraint matrix yields multi-stage IPs. The state-of-the-art algorithms to solve these IPs have an exponential gap in their running times, making it natural to ask whether this gap is inherent. We answer this question affirmative. Assuming the Exponential Time Hypothesis, we prove lower bounds showing that the exponential difference is necessary, and that the known algorithms are near optimal. Moreover, we prove unconditional lower bounds on the norms of the Graver basis, a fundamental building block of all known algorithms to solve these IPs. This shows that none of the current approaches can be improved beyond this bound.
Spatial Distribution of Inertial Particles in Turbulent Taylor-Couette Flow

This study investigates the spatial distribution of inertial particles in turbulent Taylor-Couette flow. Direct numerical simulations are performed using a one-way coupled Eulerian-Lagrangian approach, with a fixed inner wall Reynolds number of 2500 for the carrier flow, while the particle Stokes number varies from 0.034 to 1 for the dispersed phase. We first examine the issue of preferential concentration of particles near the outer wall region. Employing two-dimensional (2D) Voronoi analysis, we observe a pronounced particle clustering with increasing $St$, particularly evident in regions of low fluid velocity. Additionally, we investigate the concentration balance equation, inspired by the work of johnson et al.(2020), to examine particle radial distribution. We discern the predominant sources of influence, namely biased sampling, turbophoresis, and centrifugal effects. Across all cases, centrifugal force emerges as the primary driver, causing particle migration towards the outer wall. Biased sampling predominantly affects smaller inertial particles, driving them towards the inner wall due to sampling within Taylor rolls with inward radial velocity. Conversely, turbophoresis primarily impacts larger inertial particles, inducing migration towards both walls where turbulent intensity is weaker compared to the bulk. With the revealed physics, our work provides a basis for predicting and controlling particle movement and distribution in industrial applications.
One-Shot Graph Representation Learning Using Hyperdimensional Computing

We present a novel, simple, fast, and efficient approach for semi-supervised learning on graphs. The proposed approach takes advantage of hyper-dimensional computing which encodes data samples using random projections into a high dimensional space (HD space for short). Specifically, we propose a Hyper-dimensional Graph Learning (HDGL) algorithm that leverages the injectivity property of the node representations of a family of graph neural networks. HDGL maps node features to the HD space and then uses HD operators such as bundling and binding to aggregate information from the local neighborhood of each node. Results of experiments with widely used benchmark data sets show that HDGL achieves predictive performance that is competitive with the state-of-the-art deep learning methods, without the need for computationally expensive training.
Matrix weighted modulation spaces

Given a matrix-weight $W$ in the Muckenhoupt class $\mathbf{A}_p(\mathbb{R}^n)$, $1\leq p<\infty$, we introduce corresponding vector-valued continuous and discrete $\alpha$-modulation spaces $M^{s,\alpha}_{p,q}(W)$ and $m^{s,\alpha}_{p,q}(W)$ and prove their equivalence through the use of adapted tight frames. Compatible notions of molecules and almost diagonal matrices are also introduced, and an application to the study of pseudo-differential operators on vector valued spaces is given.
Scribble Hides Class: Promoting Scribble-Based Weakly-Supervised Semantic Segmentation with Its Class Label

Scribble-based weakly-supervised semantic segmentation using sparse scribble supervision is gaining traction as it reduces annotation costs when compared to fully annotated alternatives. Existing methods primarily generate pseudo-labels by diffusing labeled pixels to unlabeled ones with local cues for supervision. However, this diffusion process fails to exploit global semantics and class-specific cues, which are important for semantic segmentation. In this study, we propose a class-driven scribble promotion network, which utilizes both scribble annotations and pseudo-labels informed by image-level classes and global semantics for supervision. Directly adopting pseudo-labels might misguide the segmentation model, thus we design a localization rectification module to correct foreground representations in the feature space. To further combine the advantages of both supervisions, we also introduce a distance entropy loss for uncertainty reduction, which adapts per-pixel confidence weights according to the reliable region determined by the scribble and pseudo-label's boundary. Experiments on the ScribbleSup dataset with different qualities of scribble annotations outperform all the previous methods, demonstrating the superiority and robustness of our method. The code is available at https://github.com/Zxl19990529/Class-driven-Scribble-Promotion-Network.
Domain Embeddings for Generating Complex Descriptions of Concepts in Italian Language

In this work, we propose a Distributional Semantic resource enriched with linguistic and lexical information extracted from electronic dictionaries, designed to address the challenge of bridging the gap between the continuous semantic values represented by distributional vectors and the discrete descriptions offered by general semantics theory. Recently, many researchers have concentrated on the nexus between embeddings and a comprehensive theory of semantics and meaning. This often involves decoding the representation of word meanings in Distributional Models into a set of discrete, manually constructed properties such as semantic primitives or features, using neural decoding techniques. Our approach introduces an alternative strategy grounded in linguistic data. We have developed a collection of domain-specific co-occurrence matrices, derived from two sources: a classification of Italian nouns categorized into 4 semantic traits and 20 concrete noun sub-categories, and a list of Italian verbs classified according to their semantic classes. In these matrices, the co-occurrence values for each word are calculated exclusively with a defined set of words pertinent to a particular lexical domain. The resource comprises 21 domain-specific matrices, one comprehensive matrix, and a Graphical User Interface. Our model facilitates the generation of reasoned semantic descriptions of concepts by selecting matrices directly associated with concrete conceptual knowledge, such as a matrix based on location nouns and the concept of animal habitats. We assessed the utility of the resource through two experiments, achieving promising outcomes in both: the automatic classification of animal nouns and the extraction of animal features.
A Computational Tsirelson's Theorem for the Value of Compiled XOR Games

Nonlocal games are a foundational tool for understanding entanglement and constructing quantum protocols in settings with multiple spatially separated quantum devices. In this work, we continue the study initiated by Kalai et al. (STOC '23) of compiled nonlocal games, played between a classical verifier and a single cryptographically limited quantum device. Our main result is that the compiler proposed by Kalai et al. is sound for any two-player XOR game. A celebrated theorem of Tsirelson shows that for XOR games, the quantum value is exactly given by a semidefinite program, and we obtain our result by showing that the SDP upper bound holds for the compiled game up to a negligible error arising from the compilation. This answers a question raised by Natarajan and Zhang (FOCS '23), who showed soundness for the specific case of the CHSH game. Using our techniques, we obtain several additional results, including (1) tight bounds on the compiled value of parallel-repeated XOR games, (2) operator self-testing statements for any compiled XOR game, and (3) a ``nice"sum-of-squares certificate for any XOR game, from which operator rigidity is manifest.
Spin Effect induced Momentum Spiral and Asymmetry Degree in Pair Production

Spin effect on the pair production under circularly polarized fields are investigated. Significantly different from what momentum spirals caused by two counter-rotating fields with a time delay, we find for the first time that the spirals can also be induced due to the particles spin effect even if in a single field. We further examine the bichromatic combinational fields, the inhomogeneous spiral structures can be observed in the momentum spectrum, in particular, the spiral not only does exist in two cases of spin but also is about two orders of magnitude amplifier than that in the single field. Meanwhile, the spin asymmetry degree on the momentum distributions is investigated and found that there exist the effect of spin flip with increasing time delay between two fields. The spin asymmetry degree on the number density can reach to $98\%$ in a certain of condition. These results indicate that the signatures of created particles, especially the spiral structures are strongly associated with the information of laser field as well as the created particle spin, which can deepen the understanding of vacuum pair production.
A smooth Birman-Hilden theory for hyperk\"ahler manifolds

This article grew out of an effort to understand the smooth mapping class groups of certain 4-manifolds in a geometric manner. We prove a smooth analog of the Birman-Hilden theorem for manifolds that admit a hyperk\"ahler structure. This allows us to probe the smooth mapping class groups associated to certain manifolds with nontrivial fundamental groups. Along the way, and of independent interest, we prove a global Torelli theorem for generalized Enriques manifolds. The techniques used are analogous to Teichm\"uller-theoretic methods in the classical theory of mapping class groups. We apply this hyperk\"ahler Birman-Hilden theorem to obtain results regarding smooth, metric, and complex Nielsen realization on Enriques surfaces.
What Do Language Models Hear? Probing for Auditory Representations in Language Models

This work explores whether language models encode meaningfully grounded representations of sounds of objects. We learn a linear probe that retrieves the correct text representation of an object given a snippet of audio related to that object, where the sound representation is given by a pretrained audio model. This probe is trained via a contrastive loss that pushes the language representations and sound representations of an object to be close to one another. After training, the probe is tested on its ability to generalize to objects that were not seen during training. Across different language models and audio models, we find that the probe generalization is above chance in many cases, indicating that despite being trained only on raw text, language models encode grounded knowledge of sounds for some objects.
Spatial modelling of rainfall trends using satellite datasets and geographic information system

ABSTRACT This study developed a standard methodology for identifying spatial trends using satellite-based raster datasets. It involves the novelty of exploring the capabilities of a geographic information system in implementing the procedures of three trend tests, the Spearman rank order correlation (SROC) test, the Kendall rank correlation (KRC) test and the Mann-Kendall (MK) test, on raster datasets of the Tropical Rainfall Measuring Mission at 0.25° × 0.25° resolution. Comparative evaluation of the three tests revealed fair agreement of a major part of the test results for pre-, post- and non-monsoon and one-day maximum rainfall. Also, similar results from KRC and MK tests were obtained over a considerable area for annual, monsoon and monthly maximum rainfall. These findings suggest the importance of selecting the appropriate test depending on rainfall magnitudes at the chosen time scale and emphasize the robustness of the KRC and MK tests.
Lane2Seq: Towards Unified Lane Detection via Sequence Generation

In this paper, we present a novel sequence generation-based framework for lane detection, called Lane2Seq. It unifies various lane detection formats by casting lane detection as a sequence generation task. This is different from previous lane detection methods, which depend on well-designed task-specific head networks and corresponding loss functions. Lane2Seq only adopts a plain transformer-based encoder-decoder architecture with a simple cross-entropy loss. Additionally, we propose a new multi-format model tuning based on reinforcement learning to incorporate the task-specific knowledge into Lane2Seq. Experimental results demonstrate that such a simple sequence generation paradigm not only unifies lane detection but also achieves competitive performance on benchmarks. For example, Lane2Seq gets 97.95\% and 97.42\% F1 score on Tusimple and LLAMAS datasets, establishing a new state-of-the-art result for two benchmarks.
Covert Communication Over Additive-Noise Channels

We study the fundamental limits of covert communications over general memoryless additive-noise channels. Under mild integrability assumptions, we find a general upper bound on the square-root scaling constant, which only involves the variance of the logarithm of the probability density function of the noise. Furthermore, we show that under some additional assumptions, this upper bound is tight. We also provide upper bounds on the length of the secret key required to achieve the optimal scaling.