Miracle-dz/UniFAR-FTUs huggingface dataset:
Data Quality Matters: Suicide Intention Detection on Social Media Posts Using a RoBERTa-CNN Model

Suicide remains a global health concern for the field of health, which urgently needs innovative approaches for early detection and intervention. In this paper, we focus on identifying suicidal intentions in SuicideWatch Reddit posts and present a novel approach to suicide detection using the cutting-edge RoBERTa-CNN model, a variant of RoBERTa (Robustly optimized BERT approach). RoBERTa is used for various Natural Language Processing (NLP) tasks, including text classification and sentiment analysis. The effectiveness of the RoBERTa lies in its ability to capture textual information and form semantic relationships within texts. By adding the Convolution Neural Network (CNN) layer to the original model, the RoBERTa enhances its ability to capture important patterns from heavy datasets. To evaluate the RoBERTa-CNN, we experimented on the Suicide and Depression Detection dataset and obtained solid results. For example, RoBERTa-CNN achieves 98% mean accuracy with the standard deviation (STD) of 0.0009. It also reaches over 97.5% mean AUC value with an STD of 0.0013. In the meanwhile, RoBERTa-CNN outperforms competitive methods, demonstrating the robustness and ability to capture nuanced linguistic patterns for suicidal intentions. Therefore, RoBERTa-CNN can detect suicide intention on text data very well.
Quantization Aware Attack: Enhancing Transferable Adversarial Attacks by Model Quantization

Quantized neural networks (QNNs) have received increasing attention in resource-constrained scenarios due to their exceptional generalizability. However, their robustness against realistic black-box adversarial attacks has not been extensively studied. In this scenario, adversarial transferability is pursued across QNNs with different quantization bitwidths, which particularly involve unknown architectures and defense methods. Previous studies claim that transferability is difficult to achieve across QNNs with different bitwidths on the condition that they share the same architecture. However, we discover that under different architectures, transferability can be largely improved by using a QNN quantized with an extremely low bitwidth as the substitute model. We further improve the attack transferability by proposing quantization aware attack (QAA), which fine-tunes a QNN substitute model with a multiple-bitwidth training objective. In particular, we demonstrate that QAA addresses the two issues that are commonly known to hinder transferability: 1) quantization shifts and 2) gradient misalignments. Extensive experimental results validate the high transferability of the QAA to diverse target models. For instance, when adopting the ResNet-34 substitute model on ImageNet, QAA outperforms the current best attack in attacking standardly trained DNNs, adversarially trained DNNs, and QNNs with varied bitwidths by 4.3% ~ 20.9%, 8.7% ~ 15.5%, and 2.6% ~ 31.1% (absolute), respectively. In addition, QAA is efficient since it only takes one epoch for fine-tuning. In the end, we empirically explain the effectiveness of QAA from the view of the loss landscape. Our code is available at https://github.com/yyl-github-1896/QAA/.
Fine-tuning Reinforcement Learning Models is Secretly a Forgetting Mitigation Problem

Fine-tuning is a widespread technique that allows practitioners to transfer pre-trained capabilities, as recently showcased by the successful applications of foundation models. However, fine-tuning reinforcement learning (RL) models remains a challenge. This work conceptualizes one specific cause of poor transfer, accentuated in the RL setting by the interplay between actions and observations: forgetting of pre-trained capabilities. Namely, a model deteriorates on the state subspace of the downstream task not visited in the initial phase of fine-tuning, on which the model behaved well due to pre-training. This way, we lose the anticipated transfer benefits. We identify conditions when this problem occurs, showing that it is common and, in many cases, catastrophic. Through a detailed empirical analysis of the challenging NetHack and Montezuma's Revenge environments, we show that standard knowledge retention techniques mitigate the problem and thus allow us to take full advantage of the pre-trained capabilities. In particular, in NetHack, we achieve a new state-of-the-art for neural models, improving the previous best score from $5$K to over $10$K points in the Human Monk scenario.
NOAH: Learning Pairwise Object Category Attentions for Image Classification

A modern deep neural network (DNN) for image classification tasks typically consists of two parts: a backbone for feature extraction, and a head for feature encoding and class predication. We observe that the head structures of mainstream DNNs adopt a similar feature encoding pipeline, exploiting global feature dependencies while disregarding local ones. In this paper, we revisit the feature encoding problem, and propose Non-glObal Attentive Head (NOAH) that relies on a new form of dot-product attention called pairwise object category attention (POCA), efficiently exploiting spatially dense category-specific attentions to augment classification performance. NOAH introduces a neat combination of feature split, transform and merge operations to learn POCAs at local to global scales. As a drop-in design, NOAH can be easily used to replace existing heads of various types of DNNs, improving classification performance while maintaining similar model efficiency. We validate the effectiveness of NOAH on ImageNet classification benchmark with 25 DNN architectures spanning convolutional neural networks, vision transformers and multi-layer perceptrons. In general, NOAH is able to significantly improve the performance of lightweight DNNs, e.g., showing 3.14\%|5.3\%|1.9\% top-1 accuracy improvement to MobileNetV2 (0.5x)|Deit-Tiny (0.5x)|gMLP-Tiny (0.5x). NOAH also generalizes well when applied to medium-size and large-size DNNs. We further show that NOAH retains its efficacy on other popular multi-class and multi-label image classification benchmarks as well as in different training regimes, e.g., showing 3.6\%|1.1\% mAP improvement to large ResNet101|ViT-Large on MS-COCO dataset. Project page: https://github.com/OSVAI/NOAH.
The Political Preferences of LLMs

We report here a comprehensive analysis about the political preferences embedded in Large Language Models (LLMs). Namely, we administer 11 political orientation tests, designed to identify the political preferences of the test taker, to 24 state-of-the-art conversational LLMs, both close and open source. The results indicate that when probed with questions/statements with political connotations most conversational LLMs tend to generate responses that are diagnosed by most political test instruments as manifesting preferences for left-of-center viewpoints. We note that this is not the case for base (i.e. foundation) models upon which LLMs optimized for conversation with humans are built. However, base models' suboptimal performance at coherently answering questions suggests caution when interpreting their classification by political orientation tests. Though not conclusive, our results provide preliminary evidence for the intriguing hypothesis that the embedding of political preferences into LLMs might be happening mostly post-pretraining. Namely, during the supervised fine-tuning (SFT) and/or Reinforcement Learning (RL) stages of the conversational LLMs training pipeline. We provide further support for this hypothesis by showing that LLMs are easily steerable into target locations of the political spectrum via SFT requiring only modest compute and custom data, illustrating the ability of SFT to imprint political preferences onto LLMs. As LLMs have started to displace more traditional information sources such as search engines or Wikipedia, the implications of political biases embedded in LLMs has important societal ramifications.
On Least Squares Estimation in Softmax Gating Mixture of Experts

Mixture of experts (MoE) model is a statistical machine learning design that aggregates multiple expert networks using a softmax gating function in order to form a more intricate and expressive model. Despite being commonly used in several applications owing to their scalability, the mathematical and statistical properties of MoE models are complex and difficult to analyze. As a result, previous theoretical works have primarily focused on probabilistic MoE models by imposing the impractical assumption that the data are generated from a Gaussian MoE model. In this work, we investigate the performance of the least squares estimators (LSE) under a deterministic MoE model where the data are sampled according to a regression model, a setting that has remained largely unexplored. We establish a condition called strong identifiability to characterize the convergence behavior of various types of expert functions. We demonstrate that the rates for estimating strongly identifiable experts, namely the widely used feed forward networks with activation functions $\mathrm{sigmoid}(\cdot)$ and $\tanh(\cdot)$, are substantially faster than those of polynomial experts, which we show to exhibit a surprising slow estimation rate. Our findings have important practical implications for expert selection.
Improved Performances and Motivation in Intelligent Tutoring Systems: Combining Machine Learning and Learner Choice

Large class sizes pose challenges to personalized learning in schools, which educational technologies, especially intelligent tutoring systems (ITS), aim to address. In this context, the ZPDES algorithm, based on the Learning Progress Hypothesis (LPH) and multi-armed bandit machine learning techniques, sequences exercises that maximize learning progress (LP). This algorithm was previously shown in field studies to boost learning performances for a wider diversity of students compared to a hand-designed curriculum. However, its motivational impact was not assessed. Also, ZPDES did not allow students to express choices. This limitation in agency is at odds with the LPH theory concerned with modeling curiosity-driven learning. We here study how the introduction of such choice possibilities impact both learning efficiency and motivation. The given choice concerns dimensions that are orthogonal to exercise difficulty, acting as a playful feature. In an extensive field study (265 7-8 years old children, RCT design), we compare systems based either on ZPDES or a hand-designed curriculum, both with and without self-choice. We first show that ZPDES improves learning performance and produces a positive and motivating learning experience. We then show that the addition of choice triggers intrinsic motivation and reinforces the learning effectiveness of the LP-based personalization. In doing so, it strengthens the links between intrinsic motivation and performance progress during the serious game. Conversely, deleterious effects of the playful feature are observed for hand-designed linear paths. Thus, the intrinsic motivation elicited by a playful feature is beneficial only if the curriculum personalization is effective for the learner. Such a result deserves great attention due to increased use of playful features in non adaptive educational technologies.
Robust support vector machines via conic optimization

We consider the problem of learning support vector machines robust to uncertainty. It has been established in the literature that typical loss functions, including the hinge loss, are sensible to data perturbations and outliers, thus performing poorly in the setting considered. In contrast, using the 0-1 loss or a suitable non-convex approximation results in robust estimators, at the expense of large computational costs. In this paper we use mixed-integer optimization techniques to derive a new loss function that better approximates the 0-1 loss compared with existing alternatives, while preserving the convexity of the learning problem. In our computational results, we show that the proposed estimator is competitive with the standard SVMs with the hinge loss in outlier-free regimes and better in the presence of outliers.
Farber's conjecture and beyond

We survey two decades of work on the (sequential) topological complexity of configuration spaces of graphs (ordered and unordered), aiming to give an account that is unifying, elementary, and self-contained. We discuss the traditional approach through cohomology, with its limitations, and the more modern approach through asphericity and the fundamental group, explaining how they are in fact variations on the same core ideas. We close with a list of open problems in the field.
Hints of the $J^{PC}=0^{--}$ and $1^{--}$ $K^*\bar K_1(1270)$ Molecules in the $J/\psi\to\phi\eta\eta'$ Decay

The primary objective of this study is to investigate hadronic molecules of $K^*\bar K_1(1270)$ using a one-boson-exchange model, which incorporates exchanges of vector and pseudoscalar mesons in the $t$-channel, as well as the pion exchange in the $u$-channel. Additionally, careful consideration is given to the three-body effects resulting from the on-shell pion originating from $K_1(1270)\to K^*\pi$. Then the BESIII data of the $J/\psi\to\phi\eta\eta'$ process is fitted using the $K^*\bar K_1(1270)$ scattering amplitude with $J^{PC}=0^{--}$ or $1^{--}$. The analysis reveals that both the $J^{PC}=0^{--}$ and $1^{--}$ assumptions for $K^*\bar K_1(1270)$ scattering provide good descriptions of the data, with similar fit qualities. Notably, the parameters obtained from the best fits indicate the existence of $K^*\bar K_1(1270)$ bound states, denoted by $\phi(2100)$ and $\phi_0(2100)$ for the $1^{--}$ and $0^{--}$ states, respectively. The current experimental data, including the $\eta$ polar angular distribution, cannot distinguish which $K^*\bar K_1(1270)$ bound state contributes to the $J/\psi\to\phi\eta\eta'$ process, or if both are involved. Therefore, we propose further explorations of this process, as well as other processes, in upcoming experiments with many more $J/\psi$ events to disentangle the different possibilities.
Agronomic performance and susceptibility of selected spring wheat lines to fusarium head blight and leaf rust in Eastern Canada

Twenty-six spring wheat (Triticum aestivum L.) germplasm lines, including eight known cultivars were evaluated for their agronomic performance, namely yield and quality traits as well as their resistance to fusarium head blight and leaf rust during the 2012 and 2013 growing seasons in Eastern Canada. Six of the lines, namely FL62R1, ‘Fuzion’, ‘Helios’, ‘Megantic’, ‘Nass’ and ‘Quantum’ were tested only for fusarium head blight and leaf rust and were not tested in the field because the seeds were unavailable. The cultivar ‘Morocco’ was added as a check because of its susceptibility to leaf rust. Lines AW774, AW775 and HY162-HRF had the highest yield, followed by AW759 and CRGB-08C.159. The least lodging was observed for ACS11614, CM20809 and HY162-HRF. Lines AW769, AW759 and AW765 originated from Eastern breeding program ranked among the tallest compared to CM20809, ‘AC Carberry’, ACS10604, ACS11614 and ACS12642 which had the shortest plant height. Days to heading for the tested lines ranged from 51.8 for ‘AC Carberry’ to 60.5 for AW765. The most FHB-infected lines were ‘Quantum’ followed by CM20809 and HY162-HRF and the most resistant line was FL62R1, which showed the fewest fusarium head blight symptoms and lowest deoxynivalenol content in grains followed by AW765. A negative correlation was also observed between height and fusarium head blight (-0.69) and deoxynivalenol. The susceptible check cultivar ‘Morocco’ scored as moderately to very susceptible to the five leaf rust races, whereas ‘AC Carberry’, ACS10604, ACS11614 and CM9019 showed the highest resistance to all races.
Cusped hyperbolic Lagrangians as mirrors to lines in three-space

We construct a Lagrangian in the cotangent bundle of a 3-torus whose projection to the fiber is a neighborhood of a tropical curve with a single 4-valent vertex. This Lagrangian has an isolated conical singular point, and its smooth locus is diffeomorphic to the minimally-twisted five component chain link complement, a cusped hyperbolic 3-manifold. From this singular Lagrangian, we construct an immersed Lagrangian, and determine when it is unobstructed in the wrapped Fukaya category. We show, using quilted Floer theory, that for a generic line in projective 3-space, there is a local system on this immersed Lagrangian such that the resulting object of the wrapped Fukaya category is homologically mirror to an object of the derived category supported on the line.
A Fast Method for Lasso and Logistic Lasso

We propose a fast method for solving compressed sensing, Lasso regression, and Logistic Lasso regression problems that iteratively runs an appropriate solver using an active set approach. We design a strategy to update the active set that achieves a large speedup over a single call of several solvers, including gradient projection for sparse reconstruction (GPSR), lassoglm of Matlab, and glmnet. For compressed sensing, the hybrid of our method and GPSR is 31.41 times faster than GPSR on average for Gaussian ensembles and 25.64 faster on average for binary ensembles. For Lasso regression, the hybrid of our method and GPSR achieves a 30.67-fold average speedup in our experiments. In our experiments on Logistic Lasso regression, the hybrid of our method and lassoglm gives an 11.95-fold average speedup, and the hybrid of our method and glmnet gives a 1.40-fold average speedup.
Image-Caption Encoding for Improving Zero-Shot Generalization

Recent advances in vision-language models have combined contrastive approaches with generative methods to achieve state-of-the-art (SOTA) on downstream inference tasks like zero-shot image classification. However, a persistent issue of these models for image classification is their out-of-distribution (OOD) generalization capabilities. We first show that when an OOD data point is misclassified, the correct class can be typically found in the Top-K predicted classes. In order to steer the model prediction toward the correct class within the top predicted classes, we propose the Image-Caption Encoding (ICE) method, a straightforward approach that directly enforces consistency between the image-conditioned and caption-conditioned predictions at evaluation time only. Intuitively, we take advantage of unique properties of the generated captions to guide our local search for the correct class label within the Top-K predicted classes. We show that our method can be easily combined with other SOTA methods to enhance Top-1 OOD accuracies by 0.5% on average and up to 3% on challenging datasets. Our code: https://github.com/Chris210634/ice
Probing Planck scale physics with Belle II and LHCb

With the advent of Belle II and the LHCb upgrade, the precision measurements of various B-Physics observables are on cards. This holds significant potential for delving into physics beyond the standard model of electroweak interactions. These measurements can also serve as means to establish limits on phenomena occurring at much finer length scales, such as quantum decoherence, which may arise due to potential discreteness in space-time or non-trivial topological effects. In this work, we set up the formalism to investigate the impact of quantum decoherence on several potential observables in $B$ meson systems. The approach employs the trace-preserving Kraus operator formalism, extending unitary evolution to non-unitary dynamics while maintaining complete positivity. In this formalism, the decoherence effects are parametrized in terms of a single parameter. Through the analysis of purely leptonic, semileptonic, and non-leptonic decays of $B$ mesons, we identify observables that could, in principle, be influenced by decoherence. The theoretical expressions are provided without neglecting the impact of decay width difference ($\Delta \Gamma$) and $CP$ violation in mixing. Considering that many of these observables can be measured with high precision using the abundant data collected by LHCb and Belle II, our formalism can be applied to establish constraints on the decoherence parameter through multiple decay channels. This offers an alternative set-up for such studies, which, at present, are predominantly conducted in the neutrino sector.
Neur2BiLO: Neural Bilevel Optimization

Bilevel optimization deals with nested problems in which a leader takes the first decision to minimize their objective function while accounting for a follower's best-response reaction. Constrained bilevel problems with integer variables are particularly notorious for their hardness. While exact solvers have been proposed for mixed-integer linear bilevel optimization, they tend to scale poorly with problem size and are hard to generalize to the non-linear case. On the other hand, problem-specific algorithms (exact and heuristic) are limited in scope. Under a data-driven setting in which similar instances of a bilevel problem are solved routinely, our proposed framework, Neur2BiLO, embeds a neural network approximation of the leader's or follower's value function, trained via supervised regression, into an easy-to-solve mixed-integer program. Neur2BiLO serves as a heuristic that produces high-quality solutions extremely fast for the bilevel knapsack interdiction problem, the"critical node game"from network security, a donor-recipient healthcare problem, and discrete network design from transportation planning. These problems are diverse in that they have linear or non-linear objectives/constraints and integer or mixed-integer variables, making Neur2BiLO unique in its versatility.
Noisy group testing via spatial coupling

We study the problem of identifying a small set $k\sim n^\theta$, $0<\theta<1$, of infected individuals within a large population of size $n$ by testing groups of individuals simultaneously. All tests are conducted concurrently. The goal is to minimise the total number of tests required. In this paper we make the (realistic) assumption that tests are noisy, i.e.\ that a group that contains an infected individual may return a negative test result or one that does not contain an infected individual may return a positive test results with a certain probability. The noise need not be symmetric. We develop an algorithm called SPARC that correctly identifies the set of infected individuals up to $o(k)$ errors with high probability with the asymptotically minimum number of tests. Additionally, we develop an algorithm called SPEX that exactly identifies the set of infected individuals w.h. p. with a number of tests that matches the information-theoretic lower bound for the constant column design, a powerful and well-studied test design.
Integrating AI in Educational Measurement: ChatGPT's Efficacy in Item Response Theory Data Generation

This paper explores the efficacy of ChatGPT in generating data for Item Response Theory (IRT) using the R programming language. Focusing on the 2 Parameter Logistic Model (2PLM), it evaluates datasets produced by ChatGPT against several IRT assumptions like unidimensionality and local independence. The study compares these datasets with those generated by researchers, assessing compliance with simulation conditions, bias, and RMSE values. The results indicate that while ChatGPT algorithms successfully generate data adhering to IRT assumptions, they exhibit more issues with item parameter compliance compared to researcher-generated algorithms. This study highlights ChatGPT's potential in data generation, but also underscores the importance of human expertise in guiding its outputs for scientific research.