Miracle-dz/UniFAR-FTUs huggingface dataset:
Judging LLM-as-a-judge with MT-Bench and Chatbot Arena

Evaluating large language model (LLM) based chat assistants is challenging due to their broad capabilities and the inadequacy of existing benchmarks in measuring human preferences. To address this, we explore using strong LLMs as judges to evaluate these models on more open-ended questions. We examine the usage and limitations of LLM-as-a-judge, including position, verbosity, and self-enhancement biases, as well as limited reasoning ability, and propose solutions to mitigate some of them. We then verify the agreement between LLM judges and human preferences by introducing two benchmarks: MT-bench, a multi-turn question set; and Chatbot Arena, a crowdsourced battle platform. Our results reveal that strong LLM judges like GPT-4 can match both controlled and crowdsourced human preferences well, achieving over 80% agreement, the same level of agreement between humans. Hence, LLM-as-a-judge is a scalable and explainable way to approximate human preferences, which are otherwise very expensive to obtain. Additionally, we show our benchmark and traditional benchmarks complement each other by evaluating several variants of LLaMA and Vicuna. The MT-bench questions, 3K expert votes, and 30K conversations with human preferences are publicly available at https://github.com/lm-sys/FastChat/tree/main/fastchat/llm_judge.
The ultracompact regions G40.54+2.59 and G34.13+0.47: A new detection of compact radio sources

We report the detection of three compact ($<0.001$ pc) radio sources (CRSs) at K$_{a}$-band (0.9 cm) in the \uchiirs G040.54+2.59 (two CRSs) and G034.13+0.47 (one CRS). These CRSs have weak flux densities and are located at the center of their respective \uchii regions. We found no clear association between massive ionizing stars and CRSs but some radiative influence on the latter, as suggested by their large emission measures (>$10^7 \mathrm{cm}^{-6}\mathrm{pc}$), typical of photo evaporating neutral objects close to or associated with massive stars. Our modelling of G40.54+2.59 shows that their CRSs supply enough ionized material to shape its morphology while significantly extending its observable lifetime. On the other hand, despite the possible relation of the CRS with the large-scale outflow signatures observed in G034.13+0.47, the influence of this CRS on the evolution of the \uchii region is unlikely. Our results show that the presence of CRSs can alleviate the so-called lifetime problem of UCHII regions. Still, to address their dynamical evolution adequately, the scenario must include additional mechanisms like ambient confinement, or the role of the kinematics of their associated stellar objects.
Motion-DVAE: Unsupervised learning for fast human motion denoising

Pose and motion priors are crucial for recovering realistic and accurate human motion from noisy observations. Substantial progress has been made on pose and shape estimation from images, and recent works showed impressive results using priors to refine frame-wise predictions. However, a lot of motion priors only model transitions between consecutive poses and are used in time-consuming optimization procedures, which is problematic for many applications requiring real-time motion capture. We introduce Motion-DVAE, a motion prior to capture the short-term dependencies of human motion. As part of the dynamical variational autoencoder (DVAE) models family, Motion-DVAE combines the generative capability of VAE models and the temporal modeling of recurrent architectures. Together with Motion-DVAE, we introduce an unsupervised learned denoising method unifying regression- and optimization-based approaches in a single framework for real-time 3D human pose estimation. Experiments show that the proposed approach reaches competitive performance with state-of-the-art methods while being much faster.
DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text

With the rapid progress of large language models (LLMs) and the huge amount of text they generated, it becomes more and more impractical to manually distinguish whether a text is machine-generated. Given the growing use of LLMs in social media and education, it prompts us to develop methods to detect machine-generated text, preventing malicious usage such as plagiarism, misinformation, and propaganda. Previous work has studied several zero-shot methods, which require no training data. These methods achieve good performance, but there is still a lot of room for improvement. In this paper, we introduce two novel zero-shot methods for detecting machine-generated text by leveraging the log rank information. One is called DetectLLM-LRR, which is fast and efficient, and the other is called DetectLLM-NPR, which is more accurate, but slower due to the need for perturbations. Our experiments on three datasets and seven language models show that our proposed methods improve over the state of the art by 3.9 and 1.75 AUROC points absolute. Moreover, DetectLLM-NPR needs fewer perturbations than previous work to achieve the same level of performance, which makes it more practical for real-world use. We also investigate the efficiency--performance trade-off based on users preference on these two measures and we provide intuition for using them in practice effectively. We release the data and the code of both methods in https://github.com/mbzuai-nlp/DetectLLM
Static and dynamical properties of the spin-5/2 nearly ideal triangular lattice antiferromagnet Ba3MnSb2O9

We study the ground state and spin excitations in Ba3MnSb2O9, an easy-plane S = 5/2 triangular lattice antiferromagnet. By combining single-crystal neutron scattering, electric spin resonance (ESR), and spin wave calculations, we determine the frustrated quasi-two-dimensional spin Hamiltonian parameters describing the material. While the material has a slight monoclinic structural distortion, which could allow for isosceles-triangular exchanges and biaxial anisotropy by symmetry, we observe no deviation from the behavior expected for spin waves in the in-plane 120o state. Even the easy-plane anisotropy is so small that it can only be detected by ESR in our study. In conjunction with the quasi-two-dimensionality, our study establishes that Ba3MnSb2O9 is a nearly ideal triangular lattice antiferromagnet with the quasi-classical spin S = 5/2, which suggests that it has the potential for an experimental study of Z- or Z2-vortex excitations.
Differentially Private Image Classification by Learning Priors from Random Processes

In privacy-preserving machine learning, differentially private stochastic gradient descent (DP-SGD) performs worse than SGD due to per-sample gradient clipping and noise addition. A recent focus in private learning research is improving the performance of DP-SGD on private data by incorporating priors that are learned on real-world public data. In this work, we explore how we can improve the privacy-utility tradeoff of DP-SGD by learning priors from images generated by random processes and transferring these priors to private data. We propose DP-RandP, a three-phase approach. We attain new state-of-the-art accuracy when training from scratch on CIFAR10, CIFAR100, MedMNIST and ImageNet for a range of privacy budgets $\varepsilon \in [1, 8]$. In particular, we improve the previous best reported accuracy on CIFAR10 from $60.6 \%$ to $72.3 \%$ for $\varepsilon=1$.
On the Importance of Exploration for Generalization in Reinforcement Learning

Existing approaches for improving generalization in deep reinforcement learning (RL) have mostly focused on representation learning, neglecting RL-specific aspects such as exploration. We hypothesize that the agent's exploration strategy plays a key role in its ability to generalize to new environments. Through a series of experiments in a tabular contextual MDP, we show that exploration is helpful not only for efficiently finding the optimal policy for the training environments but also for acquiring knowledge that helps decision making in unseen environments. Based on these observations, we propose EDE: Exploration via Distributional Ensemble, a method that encourages exploration of states with high epistemic uncertainty through an ensemble of Q-value distributions. Our algorithm is the first value-based approach to achieve state-of-the-art on both Procgen and Crafter, two benchmarks for generalization in RL with high-dimensional observations. The open-sourced implementation can be found at https://github.com/facebookresearch/ede .
Trapping LLM Hallucinations Using Tagged Context Prompts

Recent advances in large language models (LLMs), such as ChatGPT, have led to highly sophisticated conversation agents. However, these models suffer from"hallucinations,"where the model generates false or fabricated information. Addressing this challenge is crucial, particularly with AI-driven platforms being adopted across various sectors. In this paper, we propose a novel method to recognize and flag instances when LLMs perform outside their domain knowledge, and ensuring users receive accurate information. We find that the use of context combined with embedded tags can successfully combat hallucinations within generative language models. To do this, we baseline hallucination frequency in no-context prompt-response pairs using generated URLs as easily-tested indicators of fabricated data. We observed a significant reduction in overall hallucination when context was supplied along with question prompts for tested generative engines. Lastly, we evaluated how placing tags within contexts impacted model responses and were able to eliminate hallucinations in responses with 98.88% effectiveness.
Space-time Trade-offs for the LCP Array of Wheeler DFAs

Recently, Conte et al. generalized the longest-common prefix (LCP) array from strings to Wheeler DFAs, and they showed that it can be used to efficiently determine matching statistics on a Wheeler DFA [DCC 2023]. However, storing the LCP array requires $ O(n \log n) $ bits, $ n $ being the number of states, while the compact representation of Wheeler DFAs often requires much less space. In particular, the BOSS representation of a de Bruijn graph only requires a linear number of bits, if the size of alphabet is constant. In this paper, we propose a sampling technique that allows to access an entry of the LCP array in logarithmic time by only storing a linear number of bits. We use our technique to provide a space-time trade-off to compute matching statistics on a Wheeler DFA. In addition, we show that by augmenting the BOSS representation of a $ k $-th order de Bruijn graph with a linear number of bits we can navigate the underlying variable-order de Bruijn graph in time logarithmic in $ k $, thus improving a previous bound by Boucher et al. which was linear in $ k $ [DCC 2015].
Exploring Effective Mask Sampling Modeling for Neural Image Compression

Image compression aims to reduce the information redundancy in images. Most existing neural image compression methods rely on side information from hyperprior or context models to eliminate spatial redundancy, but rarely address the channel redundancy. Inspired by the mask sampling modeling in recent self-supervised learning methods for natural language processing and high-level vision, we propose a novel pretraining strategy for neural image compression. Specifically, Cube Mask Sampling Module (CMSM) is proposed to apply both spatial and channel mask sampling modeling to image compression in the pre-training stage. Moreover, to further reduce channel redundancy, we propose the Learnable Channel Mask Module (LCMM) and the Learnable Channel Completion Module (LCCM). Our plug-and-play CMSM, LCMM, LCCM modules can apply to both CNN-based and Transformer-based architectures, significantly reduce the computational cost, and improve the quality of images. Experiments on the public Kodak and Tecnick datasets demonstrate that our method achieves competitive performance with lower computational complexity compared to state-of-the-art image compression methods.
Posterior predictive checking for gravitational-wave detection with pulsar timing arrays. II. Posterior predictive distributions and pseudo-Bayes factors

The detection of nanoHertz gravitational waves through pulsar timing arrays hinges on identifying a common stochastic process affecting all pulsars in a correlated way across the sky. In the presence of other deterministic and stochastic processes affecting the time-of-arrival of pulses, a detection claim must be accompanied by a detailed assessment of the various physical or phenomenological models used to describe the data. In this study, we propose posterior predictive checks as a model-checking tool that relies on the predictive performance of the models with regards to new data. We derive and study predictive checks based on different components of the models, namely the Fourier coefficients of the stochastic process, the correlation pattern, and the timing residuals. We assess the ability of our checks to identify model misspecification in simulated datasets. We find that they can accurately flag a stochastic process spectral shape that deviates from the common power-law model as well as a stochastic process that does not display the expected angular correlation pattern. Posterior predictive likelihoods derived under different assumptions about the correlation pattern can further be used to establish detection significance. In the era of nanoHertz gravitational wave detection from different pulsar-timing datasets, such tests represent an essential tool in assessing data consistency and supporting astrophysical inference.
Weight Freezing: A Regularization Approach for Fully Connected Layers with an Application in EEG Classification

In the realm of EEG decoding, enhancing the performance of artificial neural networks (ANNs) carries significant potential. This study introduces a novel approach, termed"weight freezing", that is anchored on the principles of ANN regularization and neuroscience prior knowledge. The concept of weight freezing revolves around the idea of reducing certain neurons' influence on the decision-making process for a specific EEG task by freezing specific weights in the fully connected layer during the backpropagation process. This is actualized through the use of a mask matrix and a threshold to determine the proportion of weights to be frozen during backpropagation. Moreover, by setting the masked weights to zero, weight freezing can not only realize sparse connections in networks with a fully connected layer as the classifier but also function as an efficacious regularization method for fully connected layers. Through experiments involving three distinct ANN architectures and three widely recognized EEG datasets, we validate the potency of weight freezing. Our method significantly surpasses previous peak performances in classification accuracy across all examined datasets. Supplementary control experiments offer insights into performance differences pre and post weight freezing implementation and scrutinize the influence of the threshold in the weight freezing process. Our study underscores the superior efficacy of weight freezing compared to traditional fully connected networks for EEG feature classification tasks. With its proven effectiveness, this innovative approach holds substantial promise for contributing to future strides in EEG decoding research.
Tropical Cyclone Structure (TCS08) Field Experiment Science Basis, Observational Platforms, and Strategy

The Tropical Cyclone Structure (TCS08) in cooperation with the THORPEX Pacific Asian Regional Campaign (T-PARC) during August and September 2008 is the most significant tropical cyclone-related field experiment in the western North Pacific since 1990. This article will summarize the science basis, observational platforms, and strategy for the TCS08. The tropical cyclone formation component of TCS08 will address the multi-scale aspects, but the primary focus will be on a ventilation hypothesis and a marsupial paradigm on the synoptic scale, and a top-down and a bottom-up hypothesis on the mesoscale. A number of numerical modeling, adjoint sensitivity and targeting, and observational studies are supporting the TCS08 formation component. The primary foci of the structure change component of TCS08 are changes associated with secondary eyewall formation, air-sea interaction impacts via boundary layer rolls and the ocean response and feedback, and understanding of superintensity. A major objective of the TCS08 structure change component is to make in situ observations of the intensity and structure and their changes for validating satellite-based techniques. Numerical modeling and data assimilation studies will also support the structure change component. The long-range lightning network in the western North Pacific is being augmented with at least five new stations and these observations will be correlated with intensity changes in conjunction with modeling studies. An open data policy applies to both TCS08 and T-PARC, and researchers are encouraged to collaborate in the field experiments and in the analyses, numerical modeling, and interpretations of the TCS08 data sets.
Investigation of discharge in a railgun by means of Rogowski coils

A series of experimental methods and experimental results are presented which have in common the Rogowski induction coil as a primary sensor. Having improved during some years the technique of making measurements in plasmas by means of Rogowski coils, the authors consider the railgun as a device for obtaining a discharge with a quasi-stationary structure. Problems related to the existence of a quasi-stationary structure and associated experimental possibilities are the focus of the investigations described. We would like to draw the attention of our colleagues to the results of the following measurements: one-dimensional spatial structure of the discharge; two-dimensional structure of the discharge; kinematic measurements in the railgun; pressure at the discharge/accelerated body boundary and the mass of discharge plasma; current distribution in rails; -investigation of the leading discharge effect on the railgun processes.
Automated Labeling of German Chest X-Ray Radiology Reports using Deep Learning

Radiologists are in short supply globally, and deep learning models offer a promising solution to address this shortage as part of clinical decision-support systems. However, training such models often requires expensive and time-consuming manual labeling of large datasets. Automatic label extraction from radiology reports can reduce the time required to obtain labeled datasets, but this task is challenging due to semantically similar words and missing annotated data. In this work, we explore the potential of weak supervision of a deep learning-based label prediction model, using a rule-based labeler. We propose a deep learning-based CheXpert label prediction model, pre-trained on reports labeled by a rule-based German CheXpert model and fine-tuned on a small dataset of manually labeled reports. Our results demonstrate the effectiveness of our approach, which significantly outperformed the rule-based model on all three tasks. Our findings highlight the benefits of employing deep learning-based models even in scenarios with sparse data and the use of the rule-based labeler as a tool for weak supervision.
SMEFT probes in future precision DIS experiments

We analyze the potential of future high-energy deep-inelastic scattering (DIS) experiments to probe new physics within the framework of the Standard Model Effective Field Theory (SMEFT). We perform a detailed study of SMEFT probes at a future Large Hadron-electron Collider (LHeC) and a Future Circular lepton-hadron Collider (FCC-eh) machine, and extend previous simulations of the potential of a Electron-Ion Collider (EIC) to include Z-boson vertex corrections. Precision Z-pole constraints on vertex corrections suffer from numerous degeneracies in the Wilson-coefficient parameter space. We find that both the LHeC and the FCC-eh can help remove these degeneracies present in the existing global fits of precision Z-pole observables and LHC data. The FCC-eh and LHeC will in many cases improve upon the existing precision electroweak bounds on the SMEFT parameter space. This highlights the important role of precision DIS measurements for new physics studies.
Investigating the Impact of Metallicity on Star Formation in the Outer Galaxy. I. VLT/KMOS Survey of Young Stellar Objects in Canis Major

The effects of metallicity on the evolution of protoplanetary disks may be studied in the outer Galaxy where the metallicity is lower than in the solar neighborhood. We present the VLT/KMOS integral field spectroscopy in the near-infrared of ∼120 candidate young stellar objects (YSOs) in the CMa-ℓ224 star-forming region located at a Galactocentric distance of 9.1 kpc. We characterize the YSO accretion luminosities and accretion rates using the hydrogen Brγ emission and find a median accretion luminosity of log(Lacc)=−0.82−0.82+0.80L⊙ . Based on the measured accretion luminosities, we investigate the hypothesis of star formation history in the CMa-ℓ224. Their median values suggest that Cluster C, where most of YSO candidates have been identified, might be the most evolved part of the region. The accretion luminosities are similar to those observed toward low-mass YSOs in the Perseus and Orion molecular clouds, and they do not reveal the impact of lower metallicity. Similar studies in other outer Galaxy clouds covering a wide range of metallicities are critical to gain a complete picture of star formation in the Galaxy.
How Can Recommender Systems Benefit from Large Language Models: A Survey

With the rapid development of online services, recommender systems (RS) have become increasingly indispensable for mitigating information overload. Despite remarkable progress, conventional recommendation models (CRM) still have some limitations, e.g., lacking open-world knowledge, and difficulties in comprehending users' underlying preferences and motivations. Meanwhile, large language models (LLM) have shown impressive general intelligence and human-like capabilities, which mainly stem from their extensive open-world knowledge, reasoning ability, as well as their comprehension of human culture and society. Consequently, the emergence of LLM is inspiring the design of recommender systems and pointing out a promising research direction, i.e., whether we can incorporate LLM and benefit from their knowledge and capabilities to compensate for the limitations of CRM. In this paper, we conduct a comprehensive survey on this research direction from the perspective of the whole pipeline in real-world recommender systems. Specifically, we summarize existing works from two orthogonal aspects: where and how to adapt LLM to RS. For the WHERE question, we discuss the roles that LLM could play in different stages of the recommendation pipeline, i.e., feature engineering, feature encoder, scoring/ranking function, user interaction, and pipeline controller. For the HOW question, we investigate the training and inference strategies, resulting in two fine-grained taxonomy criteria, i.e., whether to tune LLM or not, and whether to involve conventional recommendation models for inference. Then, we highlight key challenges in adapting LLM to RS from three aspects, i.e., efficiency, effectiveness, and ethics. Finally, we summarize the survey and discuss the future prospects. We actively maintain a GitHub repository for papers and other related resources: https://github.com/CHIANGEL/Awesome-LLM-for-RecSys/.