Miracle-dz/UniFAR-FTUs huggingface dataset:
Efficient Detection of LLM-generated Texts with a Bayesian Surrogate Model

The detection of machine-generated text, especially from large language models (LLMs), is crucial in preventing serious social problems resulting from their misuse. Some methods train dedicated detectors on specific datasets but fall short in generalizing to unseen test data, while other zero-shot ones often yield suboptimal performance. Although the recent DetectGPT has shown promising detection performance, it suffers from significant inefficiency issues, as detecting a single candidate requires scoring hundreds of its perturbations with the source LLM. This paper aims to bridge this gap. Technically, we propose to incorporate a Bayesian surrogate model, which allows us to select typical samples based on Bayesian uncertainty and interpolate scores from typical samples to other ones, to improve query efficiency. Our empirical results demonstrate that our method significantly outperforms existing approaches under a low query budget. Notably, our method achieves similar performance with up to 2 times fewer queries than DetectGPT and 3.7% higher AUROC at a query number of 5.
Which Features are Learnt by Contrastive Learning? On the Role of Simplicity Bias in Class Collapse and Feature Suppression

Contrastive learning (CL) has emerged as a powerful technique for representation learning, with or without label supervision. However, supervised CL is prone to collapsing representations of subclasses within a class by not capturing all their features, and unsupervised CL may suppress harder class-relevant features by focusing on learning easy class-irrelevant features; both significantly compromise representation quality. Yet, there is no theoretical understanding of \textit{class collapse} or \textit{feature suppression} at \textit{test} time. We provide the first unified theoretically rigorous framework to determine \textit{which} features are learnt by CL. Our analysis indicate that, perhaps surprisingly, bias of (stochastic) gradient descent towards finding simpler solutions is a key factor in collapsing subclass representations and suppressing harder class-relevant features. Moreover, we present increasing embedding dimensionality and improving the quality of data augmentations as two theoretically motivated solutions to {feature suppression}. We also provide the first theoretical explanation for why employing supervised and unsupervised CL together yields higher-quality representations, even when using commonly-used stochastic gradient methods.
2-bit Conformer quantization for automatic speech recognition

Large speech models are rapidly gaining traction in research community. As a result, model compression has become an important topic, so that these models can fit in memory and be served with reduced cost. Practical approaches for compressing automatic speech recognition (ASR) model use int8 or int4 weight quantization. In this study, we propose to develop 2-bit ASR models. We explore the impact of symmetric and asymmetric quantization combined with sub-channel quantization and clipping on both LibriSpeech dataset and large-scale training data. We obtain a lossless 2-bit Conformer model with 32% model size reduction when compared to state of the art 4-bit Conformer model for LibriSpeech. With the large-scale training data, we obtain a 2-bit Conformer model with over 40% model size reduction against the 4-bit version at the cost of 17% relative word error rate degradation
Set-based Neural Network Encoding

We propose an approach to neural network weight encoding for generalization performance prediction that utilizes set-to-set and set-to-vector functions to efficiently encode neural network parameters. Our approach is capable of encoding neural networks in a modelzoo of mixed architecture and different parameter sizes as opposed to previous approaches that require custom encoding models for different architectures. Furthermore, our \textbf{S}et-based \textbf{N}eural network \textbf{E}ncoder (SNE) takes into consideration the hierarchical computational structure of neural networks by utilizing a layer-wise encoding scheme that culminates to encoding all layer-wise encodings to obtain the neural network encoding vector. Additionally, we introduce a \textit{pad-chunk-encode} pipeline to efficiently encode neural network layers that is adjustable to computational and memory constraints. We also introduce two new tasks for neural network generalization performance prediction: cross-dataset and cross-architecture. In cross-dataset performance prediction, we evaluate how well performance predictors generalize across modelzoos trained on different datasets but of the same architecture. In cross-architecture performance prediction, we evaluate how well generalization performance predictors transfer to modelzoos of different architecture. Experimentally, we show that SNE outperforms the relevant baselines on the cross-dataset task and provide the first set of results on the cross-architecture task.
Leveraging Domain Knowledge for Inclusive and Bias-aware Humanitarian Response Entry Classification

Accurate and rapid situation analysis during humanitarian crises is critical to delivering humanitarian aid efficiently and is fundamental to humanitarian imperatives and the Leave No One Behind (LNOB) principle. This data analysis can highly benefit from language processing systems, e.g., by classifying the text data according to a humanitarian ontology. However, approaching this by simply fine-tuning a generic large language model (LLM) involves considerable practical and ethical issues, particularly the lack of effectiveness on data-sparse and complex subdomains, and the encoding of societal biases and unwanted associations. In this work, we aim to provide an effective and ethically-aware system for humanitarian data analysis. We approach this by (1) introducing a novel architecture adjusted to the humanitarian analysis framework, (2) creating and releasing a novel humanitarian-specific LLM called HumBert, and (3) proposing a systematic way to measure and mitigate biases. Our experiments' results show the better performance of our approach on zero-shot and full-training settings in comparison with strong baseline models, while also revealing the existence of biases in the resulting LLMs. Utilizing a targeted counterfactual data augmentation approach, we significantly reduce these biases without compromising performance.
Uncertain Pose Estimation during Contact Tasks using Differentiable Contact Features

For many robotic manipulation and contact tasks, it is crucial to accurately estimate uncertain object poses, for which certain geometry and sensor information are fused in some optimal fashion. Previous results for this problem primarily adopt sampling-based or end-to-end learning methods, which yet often suffer from the issues of efficiency and generalizability. In this paper, we propose a novel differentiable framework for this uncertain pose estimation during contact, so that it can be solved in an efficient and accurate manner with gradient-based solver. To achieve this, we introduce a new geometric definition that is highly adaptable and capable of providing differentiable contact features. Then we approach the problem from a bi-level perspective and utilize the gradient of these contact features along with differentiable optimization to efficiently solve for the uncertain pose. Several scenarios are implemented to demonstrate how the proposed framework can improve existing methods.
IMBERT: Making BERT Immune to Insertion-based Backdoor Attacks

Backdoor attacks are an insidious security threat against machine learning models. Adversaries can manipulate the predictions of compromised models by inserting triggers into the training phase. Various backdoor attacks have been devised which can achieve nearly perfect attack success without affecting model predictions for clean inputs. Means of mitigating such vulnerabilities are underdeveloped, especially in natural language processing. To fill this gap, we introduce IMBERT, which uses either gradients or self-attention scores derived from victim models to self-defend against backdoor attacks at inference time. Our empirical studies demonstrate that IMBERT can effectively identify up to 98.5% of inserted triggers. Thus, it significantly reduces the attack success rate while attaining competitive accuracy on the clean dataset across widespread insertion-based attacks compared to two baselines. Finally, we show that our approach is model-agnostic, and can be easily ported to several pre-trained transformer models.
Learning Preconditioner for Conjugate Gradient PDE Solvers

Efficient numerical solvers for partial differential equations empower science and engineering. One of the commonly employed numerical solvers is the preconditioned conjugate gradient (PCG) algorithm which can solve large systems to a given precision level. One challenge in PCG solvers is the selection of preconditioners, as different problem-dependent systems can benefit from different preconditioners. We present a new method to introduce \emph{inductive bias} in preconditioning conjugate gradient algorithm. Given a system matrix and a set of solution vectors arise from an underlying distribution, we train a graph neural network to obtain an approximate decomposition to the system matrix to be used as a preconditioner in the context of PCG solvers. We conduct extensive experiments to demonstrate the efficacy and generalizability of our proposed approach in solving various 2D and 3D linear second-order PDEs.
Aluminium-26 from Massive Binary Stars. III. Binary Stars up to Core Collapse and Their Impact on the Early Solar System

Many of the short-lived radioactive nuclei that were present in the early solar system can be produced in massive stars. In the first paper in this series, we focused on the production of 26Al in massive binaries. In our second paper, we considered rotating single stars; two more short-lived radioactive nuclei, 36Cl and 41Ca; and the comparison to the early solar system data. In this work, we update our previous conclusions by further considering the impact of binary interactions. We used the MESA stellar evolution code with an extended nuclear network to compute massive (10–80 M ⊙), binary stars at various initial periods and solar metallicity (Z = 0.014), up to the onset of core collapse. The early solar system abundances of 26Al and 41Ca can be matched self-consistently by models with initial masses ≥25 M ⊙, while models with initial primary masses ≥35 M ⊙ can also match 36Cl. Almost none of the models provide positive net yields for 19F, while for 22Ne the net yields are positive from 30 M ⊙ and higher. This leads to an increase by a factor of approximately 4 in the amount of 22Ne produced by a stellar population of binary stars, relative to single stars. In addition, besides the impact on the stellar yields, our 10 M ⊙ primary star undergoing Case A mass transfer ends its life as a white dwarf instead of as a core-collapse supernova. This demonstrates that binary interactions can also strongly impact the evolution of stars close to the supernova boundary.
ZeroAvatar: Zero-shot 3D Avatar Generation from a Single Image

Recent advancements in text-to-image generation have enabled significant progress in zero-shot 3D shape generation. This is achieved by score distillation, a methodology that uses pre-trained text-to-image diffusion models to optimize the parameters of a 3D neural presentation, e.g. Neural Radiance Field (NeRF). While showing promising results, existing methods are often not able to preserve the geometry of complex shapes, such as human bodies. To address this challenge, we present ZeroAvatar, a method that introduces the explicit 3D human body prior to the optimization process. Specifically, we first estimate and refine the parameters of a parametric human body from a single image. Then during optimization, we use the posed parametric body as additional geometry constraint to regularize the diffusion model as well as the underlying density field. Lastly, we propose a UV-guided texture regularization term to further guide the completion of texture on invisible body parts. We show that ZeroAvatar significantly enhances the robustness and 3D consistency of optimization-based image-to-3D avatar generation, outperforming existing zero-shot image-to-3D methods.
Continual Learning through Human-Robot Interaction - Human Perceptions of a Continual Learning Robot in Repeated Interactions

For long-term deployment in dynamic real-world environments, assistive robots must continue to learn and adapt to their environments. Researchers have developed various computational models for continual learning (CL) that can allow robots to continually learn from limited training data, and avoid forgetting previous knowledge. While these CL models can mitigate forgetting on static, systematically collected datasets, it is unclear how human users might perceive a robot that continually learns over multiple interactions with them. In this paper, we developed a system that integrates CL models for object recognition with a Fetch mobile manipulator robot and allows human participants to directly teach and test the robot over multiple sessions. We conducted an in-person study with 60 participants who interacted with our system in 300 sessions (5 sessions per participant). We conducted a between-participant study with three different CL models (3 experimental conditions) to understand human perceptions of continual learning robots over multiple sessions. Our results suggest that participants' perceptions of trust, competence, and usability of a continual learning robot significantly decrease over multiple sessions if the robot forgets previously learned objects. However, the perceived task load on participants for teaching and testing the robot remains the same over multiple sessions even if the robot forgets previously learned objects. Our results also indicate that state-of-the-art CL models might perform unreliably when applied to robots interacting with human participants. Further, continual learning robots are not perceived as very trustworthy or competent by human participants, regardless of the underlying continual learning model or the session number.
Channel Measurement, Modeling, and Simulation for 6G: A Survey and Tutorial

The sixth generation (6G) mobile communications have attracted substantial attention in the global research community of information and communication technologies (ICT). 6G systems are expected to support not only extended 5G usage scenarios, but also new usage scenarios, such as integrated sensing and communication (ISAC), integrated artificial intelligence (AI) and communication, and communication and ubiquitous connectivity. To realize this goal, channel characteristics must be comprehensively studied and properly exploited, so as to promote the design, standardization, and optimization of 6G systems. In this paper, we first summarize the requirements and challenges in 6G channel research. Our focus is on channels for five promising technologies enabling 6G, including terahertz (THz), extreme MIMO (E-MIMO), ISAC, reconfigurable intelligent surface (RIS), and space-air-ground integrated network (SAGIN). Then, a survey of the progress of the 6G channel research regarding the above five promising technologies is presented in terms of the latest measurement campaigns, new characteristics, modeling methods, and research prospects. Moreover, a tutorial on the 6G channel simulations is presented. We introduce the BUPTCMG- 6G, a 6G link-level channel simulator, developed based on the ITU/3GPP 3D geometry-based stochastic model (GBSM) methodology. The simulator supports the channel simulation of the aforementioned 6G potential technologies. To facilitate the use of the simulator, the tutorial encompasses the design framework, user guidelines, and application examples. This paper offers in-depth, hands-on insights into the best practices of channel measurements, modeling, and simulations for the evaluation of 6G technologies, the development of 6G standards, and the implementation and optimization of 6G systems.
Large Language Models Are Partially Primed in Pronoun Interpretation

While a large body of literature suggests that large language models (LLMs) acquire rich linguistic representations, little is known about whether they adapt to linguistic biases in a human-like way. The present study probes this question by asking whether LLMs display human-like referential biases using stimuli and procedures from real psycholinguistic experiments. Recent psycholinguistic studies suggest that humans adapt their referential biases with recent exposure to referential patterns; closely replicating three relevant psycholinguistic experiments from Johnson&Arnold (2022) in an in-context learning (ICL) framework, we found that InstructGPT adapts its pronominal interpretations in response to the frequency of referential patterns in the local discourse, though in a limited fashion: adaptation was only observed relative to syntactic but not semantic biases. By contrast, FLAN-UL2 fails to generate meaningful patterns. Our results provide further evidence that contemporary LLMs discourse representations are sensitive to syntactic patterns in the local context but less so to semantic patterns. Our data and code are available at \url{https://github.com/zkx06111/llm_priming}.
Use of medicinal plants as a remedy against lymphatic filariasis: Current status and future prospect

Despite the successes achieved so far with the Global Programme to Eliminate Lymphatic Filariasis, there is still an appreciable number of lymphatic filarial patients who need alternative treatment and morbidity management strategies. The unresponsiveness of some cohorts to the drugs used in the mass drug administration program is currently raising a lot of questions and this needs urgent attention. Natural medicinal plants have a long‐standing history of being effective against most disease conditions. Countries such as India have been able to integrate their natural plant remedies into the treatment of lymphatic filarial conditions, and the results are overwhelmingly positive. Components of Azadirachta indica A. Juss, Parkia biglobosa, Adansonia digitata, and Ocimum spp have been shown to have anti‐inflammatory, anticancerous, and antimicrobial activities in animal models. Therefore, this review calls for attention toward the use of natural plant components as an alternate treatment against lymphatic filariasis to help reduce the World Health Organization's burden of providing drugs for people in need of treatment every year.
Targeting the NLRP3 inflammasome reduces inflammation in hidradenitis suppurativa skin.

BACKGROUND Treatment for the debilitating disease hidradenitis suppurativa (HS) is inadequate in many patients. Despite an incidence of approximately 1%, HS is often underrecognized and underdiagnosed and is associated with a high morbidity and poor quality of life. OBJECTIVES A better understanding of its pathogenesis is required to design new therapeutic strategies. METHODS In this study we employed single cell RNA sequencing to analyse gene expression in immune cells isolated from involved HS skin compared with healthy skin. Flow cytometry was used to quantify the absolute numbers of the main immune populations. Secretion of inflammatory mediators from skin explant cultures was measured using multiplex assays and ELISA. RESULTS Single cell RNA sequencing analysis identified a significant enrichment in the frequency of plasma cells, Th17 cells and dendritic cell subsets in HS skin, and the immune transcriptome was distinct and much more heterogenous when compared with healthy skin. Flow cytometry revealed significantly increased numbers of T cells, B cells, neutrophils, dermal macrophages, and dendritic cells into involved HS skin. Genes and pathways associated with Th17 cells, IL-17, IL-1β, and the NLRP3 inflammasome were enhanced in HS skin, particularly in samples with a high inflammatory load. Inflammasome constituent genes principally mapped to Langerhans cells and a subpopulation of dendritic cells. The secretome of HS skin explants contained significantly increased concentrations of inflammatory mediators including IL-1β and IL-17A, and culture with an NLRP3 inflammasome inhibitor significantly reduced the secretion of these as well as other key mediators of inflammation. CONCLUSIONS These data provide a rationale for targeting the NLRP3 inflammasome in HS using small molecule inhibitors which are currently being tested for other indications.
Does music intervention relieve depression or anxiety in people living with dementia? A systematic review and meta-analysis

Abstract Background This study aimed to summarize the available evidence on music intervention alleviating depression or anxiety in dementia. Methods A comprehensive literature search was performed to analyze the effects of music intervention on depression or anxiety. Subgroups were created to explore the effect of intervention period, duration, and frequency on efficacy. The effect size was reported as a mean standardized difference (SMD) with 95% confidence interval (CI). Results The analysis included 19 articles involving 614 samples. Thirteen studies for relieving depression revealed that, with an increase in intervention period, the efficacy decreased and then increased, whereas with an increase of intervention duration, the effect became better. A weekly intervention is ideal. Seven studies verifying the impact on anxiety relief revealed that the effect of intervention within 12 wk is significant; with an increase of intervention duration, the effect became better. A weekly intervention is ideal. Collaborative analysis showed that long low-frequency interventions are more efficient than short high-frequency interventions. Conclusions Music intervention can relieve depression or anxiety in people living with dementia. Weekly short interventions of more than 45 min are effective for emotional regulation. Future research should concentrate on severe dementia and follow-up impact.
Characterizations of convex approximate subdifferential calculus in Banach spaces

We establish subdifferential calculus rules for the sum of convex functions defined on normed spaces. This is achieved by means of a condition relying on the continuity behaviour of the inf-convolution of their corresponding conjugates, with respect to any given topology intermediate between the norm and the weak* topologies on the dual space. Such a condition turns out to also be necessary in Banach spaces. These results extend both the classical formulas by Hiriart-Urruty and Phelps and by Thibault.
Synaptic devices based on purely electronic memristors

Memristive devices have been widely employed to emulate biological synaptic behavior. In these cases, the memristive switching generally originates from electrical field induced ion migration or Joule heating induced phase change. In this letter, the Ti/ZnO/Pt structure was found to show memristive switching ascribed to a carrier trapping/detrapping of the trap sites (e.g., oxygen vacancies or zinc interstitials) in ZnO. The carrier trapping/detrapping level can be controllably adjusted by regulating the current compliance level or voltage amplitude. Multi-level conductance states can, therefore, be realized in such memristive device. The spike-timing-dependent plasticity, an important Hebbian learning rule, has been implemented in this type of synaptic device. Compared with filamentary-type memristive devices, purely electronic memristors have potential to reduce their energy consumption and work more stably and reliably, since no structural distortion occurs.