Miracle-dz/UniFAR-FTUs huggingface dataset:
Breaking Character: Are Subwords Good Enough for MRLs After All?

Large pretrained language models (PLMs) typically tokenize the input string into contiguous subwords before any pretraining or inference. However, previous studies have claimed that this form of subword tokenization is inadequate for processing morphologically-rich languages (MRLs). We revisit this hypothesis by pretraining a BERT-style masked language model over character sequences instead of word-pieces. We compare the resulting model, dubbed TavBERT, against contemporary PLMs based on subwords for three highly complex and ambiguous MRLs (Hebrew, Turkish, and Arabic), testing them on both morphological and semantic tasks. Our results show, for all tested languages, that while TavBERT obtains mild improvements on surface-level tasks \`a la POS tagging and full morphological disambiguation, subword-based PLMs achieve significantly higher performance on semantic tasks, such as named entity recognition and extractive question answering. These results showcase and (re)confirm the potential of subword tokenization as a reasonable modeling assumption for many languages, including MRLs.
Small-Gain Theorem for Safety Verification under High-Relative-Degree Constraints

This paper develops a small-gain technique for the safety analysis and verification of interconnected systems with high-relative-degree safety constraints. In this technique, input-to-state safety (ISSf) is used to characterize how the safety of a subsystem is influenced by the external input, and ISSf-barrier functions (ISSf-BFs) with high relative degree are employed to capture the safety of subsystems. With a coordination transform, the relationship between ISSf-BFs and the existing high-relative-degree (or high-order) barrier functions is established in order to simplify the ISSf analysis. With the help of high-relative-degree ISSf-BFs, a small-gain theorem is proposed for safety verification. It is shown that, under the small-gain condition, i) the interconnection of ISSf subsystems is still ISSf; and ii) the overall interconnected system is input-to-state stable (ISS) with respect to the compositional safe set. The effectiveness of the proposed small-gain theorem is illustrated on the output-constrained decentralized control of two inverted pendulums connected by a spring mounted on two carts.
Metaethical Perspectives on 'Benchmarking' AI Ethics

Benchmarks are seen as the cornerstone for measuring technical progress in Artificial Intelligence (AI) research and have been developed for a variety of tasks ranging from question answering to facial recognition. An increasingly prominent research area in AI is ethics, which currently has no set of benchmarks nor commonly accepted way for measuring the 'ethicality' of an AI system. In this paper, drawing upon research in moral philosophy and metaethics, we argue that it is impossible to develop such a benchmark. As such, alternative mechanisms are necessary for evaluating whether an AI system is 'ethical'. This is especially pressing in light of the prevalence of applied, industrial AI research. We argue that it makes more sense to talk about 'values' (and 'value alignment') rather than 'ethics' when considering the possible actions of present and future AI systems. We further highlight that, because values are unambiguously relative, focusing on values forces us to consider explicitly what the values are and whose values they are. Shifting the emphasis from ethics to values therefore gives rise to several new ways of understanding how researchers might advance research programmes for robustly safe or beneficial AI. We conclude by highlighting a number of possible ways forward for the field as a whole, and we advocate for different approaches towards more value-aligned AI research.
“That Is a Suspicious Reaction!”: Interpreting Logits Variation to Detect NLP Adversarial Attacks

Adversarial attacks are a major challenge faced by current machine learning research. These purposely crafted inputs fool even the most advanced models, precluding their deployment in safety-critical applications. Extensive research in computer vision has been carried to develop reliable defense strategies. However, the same issue remains less explored in natural language processing. Our work presents a model-agnostic detector of adversarial text examples. The approach identifies patterns in the logits of the target classifier when perturbing the input text. The proposed detector improves the current state-of-the-art performance in recognizing adversarial inputs and exhibits strong generalization capabilities across different NLP models, datasets, and word-level attacks.
Pushing on Personality Detection from Verbal Behavior: A Transformer Meets Text Contours of Psycholinguistic Features

Research at the intersection of personality psychology, computer science, and linguistics has recently focused increasingly on modeling and predicting personality from language use. We report two major improvements in predicting personality traits from text data: (1) to our knowledge, the most comprehensive set of theory-based psycholinguistic features and (2) hybrid models that integrate a pre-trained Transformer Language Model BERT and Bidirectional Long Short-Term Memory (BLSTM) networks trained on within-text distributions (‘text contours’) of psycholinguistic features. We experiment with BLSTM models (with and without Attention) and with two techniques for applying pre-trained language representations from the transformer model - ‘feature-based’ and ‘fine-tuning’. We evaluate the performance of the models we built on two benchmark datasets that target the two dominant theoretical models of personality: the Big Five Essay dataset (Pennebaker and King, 1999) and the MBTI Kaggle dataset (Li et al., 2018). Our results are encouraging as our models outperform existing work on the same datasets. More specifically, our models achieve improvement in classification accuracy by 2.9% on the Essay dataset and 8.28% on the Kaggle MBTI dataset. In addition, we perform ablation experiments to quantify the impact of different categories of psycholinguistic features in the respective personality prediction models.
Single-Turn Debate Does Not Help Humans Answer Hard Reading-Comprehension Questions

Current QA systems can generate reasonable-sounding yet false answers without explanation or evidence for the generated answer, which is especially problematic when humans cannot readily check the model’s answers. This presents a challenge for building trust in machine learning systems. We take inspiration from real-world situations where difficult questions are answered by considering opposing sides (see Irving et al., 2018). For multiple-choice QA examples, we build a dataset of single arguments for both a correct and incorrect answer option in a debate-style set-up as an initial step in training models to produce explanations for two candidate answers. We use long contexts—humans familiar with the context write convincing explanations for pre-selected correct and incorrect answers, and we test if those explanations allow humans who have not read the full context to more accurately determine the correct answer. We do not find that explanations in our set-up improve human accuracy, but a baseline condition shows that providing human-selected text snippets does improve accuracy. We use these findings to suggest ways of improving the debate set up for future data collection efforts.
Exploring and Validating Exoplanet Atmospheric Retrievals with Solar System Analog Observations

Solar system observations that serve as analogs for exoplanet remote sensing data can provide important opportunities to validate ideas and models related to exoplanet environments. Critically, and unlike true exoplanet observations, solar system analog data benefit from available high-quality ground- or orbiter-derived “truth” constraints that enable strong validations of exoplanet data interpretation tools. In this work, we first present a versatile atmospheric retrieval suite, capable of application to reflected light, thermal emission, and transmission observations spanning a broad range of wavelengths and thermochemical conditions. The tool—dubbed rfast—is designed, in part, to enable exoplanet mission concept feasibility studies. Following model validation, the retrieval tool is applied to a range of solar system analog observations for exoplanet environments. Retrieval studies using Earth reflected light observations from NASA’s EPOXI mission provide a key proof of concept for exo-Earth direct imaging concept missions under development. Inverse modeling applied to an infrared spectrum of Earth from the Mars Global Surveyor Thermal Emission Spectrometer achieves good constraints on atmospheric gases, including many biosignature gases. Finally, retrieval analysis applied to a transit spectrum of Titan derived from the Cassini Visual and Infrared Mapping Spectrometer provides a proof of concept for interpreting more feature-rich transiting exoplanet observations from NASA’s James Webb Space Telescope. In the future, solar system analog observations for exoplanets could be used to verify exoplanet models and parameterizations, and future exoplanet analog observations of any solar system worlds from planetary science missions should be encouraged.
Efficient Extraction of Pathologies from C-Spine Radiology Reports using Multi-Task Learning

Pretrained Transformer based models ﬁnetuned on domain speciﬁc corpora have changed the landscape of NLP. Generally, if one has multiple tasks on a given dataset, one may ﬁnetune diﬀerent models or use task speciﬁc adapters. In this work, we show that a multi-task model can beat or achieve the performance of multiple BERT-based models ﬁnetuned on various tasks and various task speciﬁc adapter augmented BERT-based models. We validate our method on our internal radiologist’s report dataset on cervical spine. We hypothesize that the tasks are semantically close and related and thus multitask learners are powerful classiﬁers. Our work opens the scope of using our method to radiologist’s reports on various body parts.
Divergence-aware Federated Self-Supervised Learning

Self-supervised learning (SSL) is capable of learning remarkable representations from centrally available data. Recent works further implement federated learning with SSL to learn from rapidly growing decentralized unlabeled images (e.g., from cameras and phones), often resulted from privacy constraints. Extensive attention has been paid to SSL approaches based on Siamese networks. However, such an effort has not yet revealed deep insights into various fundamental building blocks for the federated self-supervised learning (FedSSL) architecture. We aim to fill in this gap via in-depth empirical study and propose a new method to tackle the non-independently and identically distributed (non-IID) data problem of decentralized data. Firstly, we introduce a generalized FedSSL framework that embraces existing SSL methods based on Siamese networks and presents flexibility catering to future methods. In this framework, a server coordinates multiple clients to conduct SSL training and periodically updates local models of clients with the aggregated global model. Using the framework, our study uncovers unique insights of FedSSL: 1) stop-gradient operation, previously reported to be essential, is not always necessary in FedSSL; 2) retaining local knowledge of clients in FedSSL is particularly beneficial for non-IID data. Inspired by the insights, we then propose a new approach for model update, Federated Divergence-aware Exponential Moving Average update (FedEMA). FedEMA updates local models of clients adaptively using EMA of the global model, where the decay rate is dynamically measured by model divergence. Extensive experiments demonstrate that FedEMA outperforms existing methods by 3-4% on linear evaluation. We hope that this work will provide useful insights for future research.
Snowmass White Paper: Gravitational Waves and Scattering Amplitudes

We review recent progress and future prospects for harnessing powerful tools from theoretical high-energy physics, such as scattering amplitudes and effective field theory, to develop a precise and systematically improvable framework for calculating gravitational-wave signals from binary systems composed of black holes and/or neutron stars. This effort aims to provide state-of-the-art predictions that will enable high-precision measurements at future gravitational-wave detectors. In turn, applying the tools of quantum field theory in this new arena will uncover theoretical structures that can transform our understanding of basic phenomena and lead to new tools that will further the cycle of innovation. While still in a nascent stage, this research direction has already derived new analytic results in general relativity, and promises to advance the development of highly accurate waveform models for ever more sensitive detectors.
Image Harmonization by Matching Regional References

To achieve visual consistency in composite images, recent image harmonization methods typically summarize the appearance pattern of global background and apply it to the global foreground without location discrepancy. However, for a real image, the appearances (illumination, color temperature, saturation, hue, texture, etc) of different regions can vary significantly. So previous methods, which transfer the appearance globally, are not optimal. Trying to solve this issue, we firstly match the contents between the foreground and background and then adaptively adjust every foreground location according to the appearance of its content-related background regions. Further, we design a residual reconstruction strategy, that uses the predicted residual to adjust the appearance, and the composite foreground to reserve the image details. Extensive experiments demonstrate the effectiveness of our method. The source code will be available publicly.
Impact of LHC probes of SUSY and recent measurement of $(g-2)_{\mu}$ on $\mathbb{Z}_3$-NMSSM

It is well known that excessively heavy supersymmetric particles (sparticles) are disfavored to explain the $(g-2)_\mu$ anomaly, but some people overlook that moderately light sparticles are also disfavored by the LHC probes of supersymmetry. We take the Next-to-Minimal Supersymmetric Standard Model as an example to emphasize the latter point. It is found that, if the theory is required to explain the anomaly at $2\sigma$ level and meanwhile keep consistent with the LHC results, the following lower bounds may be set: $\tan \beta \gtrsim 20$, $|M_1| \gtrsim 275~{\rm GeV}$, $M_2 \gtrsim 300~{\rm GeV}$, $\mu \gtrsim 460~{\rm GeV}$, $m_{\tilde{\mu}_L} \gtrsim 310~{\rm GeV}$, and $m_{\tilde{\mu}_R} \gtrsim 350~{\rm GeV}$, where $M_1$ and $M_2$ denote gaugino masses, $\mu$ represents the Higgsino mass, and $m_{\tilde{\mu}_L}$ and $m_{\tilde{\mu}_R}$ are the mass of Smuons with $L$ and $R$ denoting their dominant chiral component. This observation has significant impacts on dark matter (DM) physics, e.g., the popular $Z$- and Higgs-funnel regions have been excluded, and the Bino-dominated neutralino DM has to co-annihilate with the Wino-dominated electroweakinos (in most cases) and/or Smuons (in few cases) to obtain the correct density. It is also inferred that these conclusions should apply to the Minimal Supersymmetric Standard Model since the underlying physics for the bounds are the same.
Exploiting Hidden Representations from a DNN-based Speech Recogniser for Speech Intelligibility Prediction in Hearing-impaired Listeners

An accurate objective speech intelligibility prediction algorithms is of great interest for many applications such as speech enhancement for hearing aids. Most algorithms measures the signal-to-noise ratios or correlations between the acoustic features of clean reference signals and degraded signals. However, these hand-picked acoustic features are usually not explicitly correlated with recognition. Meanwhile, deep neural network (DNN) based automatic speech recogniser (ASR) is approaching human performance in some speech recognition tasks. This work leverages the hidden representations from DNN-based ASR as features for speech intelligibility prediction in hearing-impaired listeners. The experiments based on a hearing aid intelligibility database show that the proposed method could make better prediction than a widely used short-time objective intelligibility (STOI) based binaural measure.
Benchmarking for Public Health Surveillance tasks on Social Media with a Domain-Specific Pretrained Language Model

A user-generated text on social media enables health workers to keep track of information, identify possible outbreaks, forecast disease trends, monitor emergency cases, and ascertain disease awareness and response to official health correspondence. This exchange of health information on social media has been regarded as an attempt to enhance public health surveillance (PHS). Despite its potential, the technology is still in its early stages and is not ready for widespread application. Advancements in pretrained language models (PLMs) have facilitated the development of several domain-specific PLMs and a variety of downstream applications. However, there are no PLMs for social media tasks involving PHS. We present and release PHS-BERT, a transformer-based PLM, to identify tasks related to public health surveillance on social media. We compared and benchmarked the performance of PHS-BERT on 25 datasets from different social medial platforms related to 7 different PHS tasks. Compared with existing PLMs that are mainly evaluated on limited tasks, PHS-BERT achieved state-of-the-art performance on all 25 tested datasets, showing that our PLM is robust and generalizable in the common PHS tasks. By making PHS-BERT available, we aim to facilitate the community to reduce the computational cost and introduce new baselines for future works across various PHS-related tasks.
Revenue Management Under the Markov Chain Choice Model with Joint Price and Assortment Decisions

Finding the optimal product prices and product assortment are two fundamental problems in revenue management. Usually, a seller needs to jointly determine the prices and assortment while managing a network of resources with limited capacity. However, there is not yet a tractable method to eﬃciently solve such a problem. Existing papers studying static joint optimization of price and assortment cannot incorporate resource constraints. Then we study the revenue management problem with resource constraints and price bounds, where the prices and the product assortments need to be jointly determined over time. We showed that under the Markov chain (MC) choice model (which subsumes the multinomial logit (MNL) model), we could reformulate the choice-based joint optimization problem as a tractable convex conic optimization problem. We also proved that an optimal solution with a constant price vector exists even with constraints on resources. In addition, a solution with both constant assortment and price vector can be optimal when there is no resource constraint.
NeuS: Neutral Multi-News Summarization for Mitigating Framing Bias

Media news framing bias can increase political polarization and undermine civil society. The need for automatic mitigation methods is therefore growing. We propose a new task, a neutral summary generation from multiple news articles of the varying political leaningsto facilitate balanced and unbiased news reading.In this paper, we first collect a new dataset, illustrate insights about framing bias through a case study, and propose a new effective metric and model (NeuS-Title) for the task. Based on our discovery that title provides a good signal for framing bias, we present NeuS-Title that learns to neutralize news content in hierarchical order from title to article. Our hierarchical multi-task learning is achieved by formatting our hierarchical data pair (title, article) sequentially with identifier-tokens (“TITLE=>”, “ARTICLE=>”) and fine-tuning the auto-regressive decoder with the standard negative log-likelihood objective.We then analyze and point out the remaining challenges and future directions. One of the most interesting observations is that neural NLG models can hallucinate not only factually inaccurate or unverifiable content but also politically biased content.
WO-type Wolf–Rayet Stars: The Last Hurrah of Massive Star Evolution

Are WO-type Wolf–Rayet (WR) stars in the final stage of massive star evolution before core-collapse? Although WC- and WO-type WRs have very similar spectra, WOs show a much stronger O vi λλ3811,34 emission-line feature. This has usually been interpreted to mean that WOs are more oxygen rich than WCs, and thus further evolved. However, previous studies have failed to model this line, leaving the relative abundances uncertain, and the relationship between the two types unresolved. To answer this fundamental question, we modeled six WCs and two WOs in the LMC using UV, optical, and NIR spectra with the radiative transfer code cmfgen in order to determine their physical properties. We find that WOs are not richer in oxygen; rather, the O vi feature is insensitive to the abundance. However, the WOs have a significantly higher carbon and lower helium content than the WCs, and hence are further evolved. A comparison of our results with single-star Geneva and binary BPASS evolutionary models show that, while many properties match, there is more carbon and less oxygen in the WOs than either set of evolutionary model predicts. This discrepancy may be due to the large uncertainty in the 12C+4He → 16O nuclear reaction rate; we show that if the Kunz et al. rate is decreased by a factor of 25%–50%, then there would be a good match with the observations. It would also help explain the LIGO/VIRGO detection of black holes whose masses are in the theoretical upper mass gap.
A general framework for penalized mixed-effects multitask learning with applications on DNA methylation surrogate biomarkers creation

Recent evidence highlights the usefulness of DNA methylation (DNAm) biomarkers as surrogates for exposure to risk factors for non-communicable diseases in epidemiological studies and randomized trials. DNAm variability has been demonstrated to be tightly related to lifestyle behavior and exposure to environmental risk factors, ultimately providing an unbiased proxy of an individual state of health. At present, the creation of DNAm surrogates relies on univariate penalized regression models, with elastic-net regularizer being the gold standard when accomplishing the task. Nonetheless, more advanced modeling procedures are required in the presence of multivariate outcomes with a structured dependence pattern among the study samples. In this work we propose a general framework for mixed-effects multitask learning in presence of high-dimensional predictors to develop a multivariate DNAm biomarker from a multi-center study. A penalized estimation scheme based on an expectation-maximization algorithm is devised, in which any penalty criteria for fixed-effects models can be conveniently incorporated in the fitting process. We apply the proposed methodology to create novel DNAm surrogate biomarkers for multiple correlated risk factors for cardiovascular diseases and comorbidities. We show that the proposed approach, modeling multiple outcomes together, outperforms state-of-the-art alternatives, both in predictive power and bio-molecular interpretation of the results.