Miracle-dz/UniFAR-FTUs huggingface dataset:
Spinning black holes with axion hair

In this work we construct and analyze non-perturbative stationary and axially-symmetric black hole solutions in general relativity coupled to an electromagnetic and an axion field. The axion field is coupled to the electromagnetic field, which leads to hairy solutions in the presence of an electric charge and rotation. We investigate the existence and characteristics of these solutions for different values of the spin, charge and coupling constant. Our analysis shows the presence of violations of the Kerr–Newman bound, solutions with large positive and negative values of the gyromagnetic ratio, and the existence of multiple branches of solutions with distinct properties, demonstrating that black hole uniqueness does not hold in this scenario. The code used in this study is publicly available, providing a valuable tool for further research on this model.
Towards Adaptable and Interactive Image Captioning with Data Augmentation and Episodic Memory

Interactive machine learning (IML) is a beneficial learning paradigm in cases of limited data availability, as human feedback is incrementally integrated into the training process. In this paper, we present an IML pipeline for image captioning which allows us to incrementally adapt a pre-trained image captioning model to a new data distribution based on user input. In order to incorporate user input into the model, we explore the use of a combination of simple data augmentation methods to obtain larger data batches for each newly annotated data instance and implement continual learning methods to prevent catastrophic forgetting from repeated updates. For our experiments, we split a domain-specific image captioning dataset, namely VizWiz, into non-overlapping parts to simulate an incremental input flow for continually adapting the model to new data. We find that, while data augmentation worsens results, even when relatively small amounts of data are available, episodic memory is an effective strategy to retain knowledge from previously seen clusters.
Conformal anomaly and gravitational pair production

We argue that the rate density of particle pair production $\Gamma$ in background fields in conformal field theories is determined by the conformal anomaly and related to anomalous trace of the energy-momentum tensor as $\Gamma = (\pi/2) \langle T^\mu_{\ \mu}\rangle$ if the trace is positive (and $\Gamma = 0$ otherwise). This formula perfectly reproduces (presumably, non-Hawking) radiation generated by static gravitational fields in the absence of an event horizon via a new evaporation mechanism suggested recently. Our relation also correctly describes the one-loop Schwinger pair creation in massless (scalar and spinor) quantum electrodynamics. It also accurately points to the Savvidi instability of the gluonic vacuum towards the formation of the chromomagnetic condensate. Photon and neutrino pair production are also discussed.
Lumos in the Night Sky: AI-enabled Visual Tool for Exploring Night-Time Light Patterns

We introduce NightPulse, an interactive tool for Night-time light (NTL) data visualization and analytics, which enables researchers and stakeholders to explore and analyze NTL data with a user-friendly platform. Powered by efficient system architecture, NightPulse supports image segmentation, clustering, and change pattern detection to identify urban development and sprawl patterns. It captures temporal trends of NTL and semantics of cities, answering questions about demographic factors, city boundaries, and unusual differences.
On the Difference of BERT-style and CLIP-style Text Encoders

Masked language modeling (MLM) has been one of the most popular pretraining recipes in natural language processing, e.g., BERT, one of the representative models. Recently, contrastive language-image pretraining (CLIP) has also attracted attention, especially its vision models that achieve excellent performance on a broad range of vision tasks. However, few studies are dedicated to studying the text encoders learned by CLIP. In this paper, we analyze the difference between BERT-style and CLIP-style text encoders from three experiments: (i) general text understanding, (ii) vision-centric text understanding, and (iii) text-to-image generation. Experimental analyses show that although CLIP-style text encoders underperform BERT-style ones for general text understanding tasks, they are equipped with a unique ability, i.e., synesthesia, for the cross-modal association, which is more similar to the senses of humans.
Hertz-linewidth and frequency-agile photonic integrated extended-DBR lasers

Recent advances in the development of ultra-low loss silicon nitride (Si3N4)-based photonic integrated circuits have allowed integrated lasers to achieve a coherence exceeding those of fiber lasers and enabled unprecedentedly fast (Megahertz bandwidth) tuning using monolithically integrated piezoelectrical actuators. While this marks the first time that fiber laser coherence is achieved using photonic integrated circuits, in conjunction with frequency agility that exceeds those of legacy bulk lasers, the approach is presently compounded by the high cost of manufacturing DFB, as required for self-injection locking, as well as the precise control over the laser current and temperature to sustain a low noise locked operation. Reflective semiconductor optical amplifiers (RSOA) provide a cost-effective alternative solution but have not yet achieved similar performance in coherence or frequency agility, as required for frequency modulated continuous wave (FMCW) LiDAR, laser locking in frequency metrology or wavelength modulation spectroscopy for gas sensing. Here, we overcome this challenge and demonstrate an RSOA-based and frequency agile integrated laser tuned with high speed, good linearity, high optical output power, and turn-key operability while maintaining a small footprint. This is achieved using a tunable extended distributed Bragg reflector (E-DBR) in an ultra-low loss 200 nm thin Si3N4 platform with monolithically integrated piezoelectric actuators. We co-integrate the DBR with a compact ultra-low loss spiral resonator to further reduce the intrinsic optical linewidth of the laser to the Hertz level -- on par with the noise of a fiber laser -- via self-injection locking.
Exploring Hybrid Linguistic Features for Turkish Text Readability

This paper presents the first comprehensive study on automatic readability assessment of Turkish texts. We combine state-of-the-art neural network models with linguistic features at lexical, morphosyntactic, syntactic and discourse levels to develop an advanced readability tool. We evaluate the effectiveness of traditional readability formulas compared to modern automated methods and identify key linguistic features that determine the readability of Turkish texts.
ωTest: WebView-Oriented Testing for Android Applications

WebView is a UI widget that helps integrate web applications into the native context of Android apps. It provides powerful mechanisms for bi-directional interactions between the native-end (Java) and the web-end (JavaScript) of an Android app. However, these interaction mechanisms are complicated and have induced various types of bugs. To mitigate the problem, various techniques have been proposed to detect WebView-induced bugs via dynamic analysis, which heavily relies on executing tests to explore WebView behaviors. Unfortunately, these techniques either require manual effort or adopt random test generation approaches, which are not able to effectively explore diverse WebView behaviors. In this paper, we study the problem of test generation for WebViews in Android apps. Effective test generation for WebViews requires identifying the essential program properties to be covered by the generated tests. To this end, we propose WebView-specific properties to characterize WebView behaviors, and devise a cross-language dynamic analysis method to identify these properties. We develop ωTest, a test generation technique that searches for event sequences covering the identified WebView-specific properties. An evaluation on 74 real-world open-/closed-source Android apps shows that ωTest can cover diverse WebView behaviors and detect WebView-induced bugs effectively. ωTest detected 36 previously-unknown bugs. From the 22 bugs that we have reported to the app developers, 13 bugs were confirmed, 9 of which were fixed.
Binary Classification with Instance and Label Dependent Label Noise

Learning with label dependent label noise has been extensively explored in both theory and practice; however, dealing with instance (i.e., feature) and label dependent label noise continues to be a challenging task. The difficulty arises from the fact that the noise rate varies for each instance, making it challenging to estimate accurately. The question of whether it is possible to learn a reliable model using only noisy samples remains unresolved. We answer this question with a theoretical analysis that provides matching upper and lower bounds. Surprisingly, our results show that, without any additional assumptions, empirical risk minimization achieves the optimal excess risk bound. Specifically, we derive a novel excess risk bound proportional to the noise level, which holds in very general settings, by comparing the empirical risk minimizers obtained from clean samples and noisy samples. Second, we show that the minimax lower bound for the 0-1 loss is a constant proportional to the average noise rate. Our findings suggest that learning solely with noisy samples is impossible without access to clean samples or strong assumptions on the distribution of the data.
Microscopic resolution of superconducting electrons in ultrahigh-pressed hydrogen sulfide

We investigate the electronic and phonon properties of hydrogen sulfide (SH$_3$) under ultrahigh pressure to elucidate the origin of its high-T$_c$ superconductivity. Contrary to the prevailing belief that the metalized S-H $\sigma$ bond is responsible, our analysis, based on the anisotropic Migdal-Eliashberg equation and the crystal orbital Hamilton population (COHP) calculation, reveals that the H-H $\sigma$-antibonding states play a dominant role in the large electron-phonon coupling that leads to the superconducting pairing in SH$_3$. Furthermore, by partially restricting the vibration of S atoms, we demonstrate that the S-H bonds provide subsidiary contributions to the pairing interaction. These findings shed light on the importance of the previously overlooked H-H $\sigma^*$ bonds in driving high-T$_c$ superconductivity in SH$_3$ and offer insights into the relationship between metallic H-H covalent antibonding and high-T$_c$ superconductivity in other hydrogen-rich materials under high pressure.
I'm Afraid I Can't Do That: Predicting Prompt Refusal in Black-Box Generative Language Models

Since the release of OpenAI's ChatGPT, generative language models have attracted extensive public attention. The increased usage has highlighted generative models' broad utility, but also revealed several forms of embedded bias. Some is induced by the pre-training corpus; but additional bias specific to generative models arises from the use of subjective fine-tuning to avoid generating harmful content. Fine-tuning bias may come from individual engineers and company policies, and affects which prompts the model chooses to refuse. In this experiment, we characterize ChatGPT's refusal behavior using a black-box attack. We first query ChatGPT with a variety of offensive and benign prompts (n=1,706), then manually label each response as compliance or refusal. Manual examination of responses reveals that refusal is not cleanly binary, and lies on a continuum; as such, we map several different kinds of responses to a binary of compliance or refusal. The small manually-labeled dataset is used to train a refusal classifier, which achieves an accuracy of 96%. Second, we use this refusal classifier to bootstrap a larger (n=10,000) dataset adapted from the Quora Insincere Questions dataset. With this machine-labeled data, we train a prompt classifier to predict whether ChatGPT will refuse a given question, without seeing ChatGPT's response. This prompt classifier achieves 76% accuracy on a test set of manually labeled questions (n=985). We examine our classifiers and the prompt n-grams that are most predictive of either compliance or refusal. Our datasets and code are available at https://github.com/maxwellreuter/chatgpt-refusals.
Efficient Anomaly Detection with Budget Annotation Using Semi-Supervised Residual Transformer

Anomaly Detection is challenging as usually only the normal samples are seen during training and the detector needs to discover anomalies on-the-fly. The recently proposed deep-learning-based approaches could somehow alleviate the problem but there is still a long way to go in obtaining an industrial-class anomaly detector for real-world applications. On the other hand, in some particular AD tasks, a few anomalous samples are labeled manually for achieving higher accuracy. However, this performance gain is at the cost of considerable annotation efforts, which can be intractable in many practical scenarios. In this work, the above two problems are addressed in a unified framework. Firstly, inspired by the success of the patch-matching-based AD algorithms, we train a sliding vision transformer over the residuals generated by a novel position-constrained patch-matching. Secondly, the conventional pixel-wise segmentation problem is cast into a block-wise classification problem. Thus the sliding transformer can attain even higher accuracy with much less annotation labor. Thirdly, to further reduce the labeling cost, we propose to label the anomalous regions using only bounding boxes. The unlabeled regions caused by the weak labels are effectively exploited using a highly-customized semi-supervised learning scheme equipped with two novel data augmentation methods. The proposed method outperforms all the state-of-the-art approaches using all the evaluation metrics in both the unsupervised and supervised scenarios. On the popular MVTec-AD dataset, our SemiREST algorithm obtains the Average Precision (AP) of 81.2% in the unsupervised condition and 84.4% AP for supervised anomaly detection. Surprisingly, with the bounding-box-based semi-supervisions, SemiREST still outperforms the SOTA methods with full supervision (83.8% AP) on MVTec-AD.
Geographic mobility and treatment outcomes among people in care for tuberculosis in the Lake Victoria region of East Africa: A multi-site prospective cohort study

Geographic mobility may disrupt continuity of care and contribute to poor clinical outcomes among people receiving treatment for tuberculosis (TB). This may occur especially where health services are not well coordinated across international borders, particularly in lower and middle income country settings. In this work, we describe mobility and the relationship between mobility and unfavorable TB treatment outcomes (i.e., death, loss to follow-up, or treatment failure) among a cohort of adults who initiated TB treatment at one of 12 health facilities near Lake Victoria. We abstracted data from health facility records for all 776 adults initiating TB treatment during a 6-month period at the selected facilities in Kenya, Tanzania, and Uganda. We interviewed 301 cohort members to assess overnight travel outside one’s residential district/sub-county. In our analyses, we estimated the proportion of cohort members traveling in 2 and 6 months following initiation of TB treatment, explored correlates of mobility, and examined the association between mobility and an unfavorable TB treatment outcome. We estimated that 40.7% (95% CI: 33.3%, 49.6%) of people on treatment for TB traveled overnight at least once in the 6 months following treatment initiation. Mobility was more common among people who worked in the fishing industry and among those with extra-pulmonary TB. Mobility was not strongly associated with other characteristics examined, however, suggesting that efforts to improve TB care for mobile populations should be broad ranging. We found that in this cohort, people who were mobile were not at increased risk of an unfavorable TB treatment outcome. Findings from this study can help inform development and implementation of mobility-competent health services for people with TB in East Africa.
Hospital length of stay prediction for general surgery and total knee arthroplasty admissions: Systematic review and meta-analysis of published prediction models

Objective Systematic review of length of stay (LOS) prediction models to assess the study methods (including prediction variables), study quality, and performance of predictive models (using area under receiver operating curve (AUROC)) for general surgery populations and total knee arthroplasty (TKA). Method LOS prediction models published since 2010 were identified in five major research databases. The main outcomes were model performance metrics including AUROC, prediction variables, and level of validation. Risk of bias was assessed using the PROBAST checklist. Results Five general surgery studies (15 models) and 10 TKA studies (24 models) were identified. All general surgery and 20 TKA models used statistical approaches; 4 TKA models used machine learning approaches. Risk scores, diagnosis, and procedure types were predominant predictors used. Risk of bias was ranked as moderate in 3/15 and high in 12/15 studies. Discrimination measures were reported in 14/15 and calibration measures in 3/15 studies, with only 4/39 externally validated models (3 general surgery and 1 TKA). Meta-analysis of externally validated models (3 general surgery) suggested the AUROC 95% prediction interval is excellent and ranges between 0.803 and 0.970. Conclusion This is the first systematic review assessing quality of risk prediction models for prolonged LOS in general surgery and TKA groups. We showed that these risk prediction models were infrequently externally validated with poor study quality, typically related to poor reporting. Both machine learning and statistical modelling methods, plus the meta-analysis, showed acceptable to good predictive performance, which are encouraging. Moving forward, a focus on quality methods and external validation is needed before clinical application.
Selection principle for the Fleming-Viot process with drift $-1$

We consider the Fleming-Viot particle system consisting of $N$ identical particles evolving in $\mathbb{R}_{>0}$ as Brownian motions with constant drift $-1$. Whenever a particle hits $0$, it jumps onto another particle in the interior. It is known that this particle system has a hydrodynamic limit as $N\rightarrow\infty$ given by Brownian motion with drift $-1$ conditioned not to hit $0$. This killed Brownian motion has an infinite family of quasi-stationary distributions (QSDs), with a Yaglom limit given by the unique QSD minimising the survival probability. On the other hand, for fixed $N<\infty$, this particle system converges to a unique stationary distribution as time $t\rightarrow\infty$. We prove the following selection principle: the empirical measure of the $N$-particle stationary distribution converges to the aforedescribed Yaglom limit as $N\rightarrow\infty$. The selection problem for this particular Fleming-Viot process is closely connected to the microscopic selection problem in front propagation, in particular for the $N$-branching Brownian motion. The proof requires neither fine estimates on the particle system nor the use of Lyapunov functions.
CrystalGPT: Enhancing system-to-system transferability in crystallization prediction and control using time-series-transformers

For prediction and real-time control tasks, machine-learning (ML)-based digital twins are frequently employed. However, while these models are typically accurate, they are custom-designed for individual systems, making system-to-system (S2S) transferability difficult. This occurs even when substantial similarities exist in the process dynamics across different chemical systems. To address this challenge, we developed a novel time-series-transformer (TST) framework that exploits the powerful transfer learning capabilities inherent in transformer algorithms. This was demonstrated using readily available process data obtained from different crystallizers operating under various operational scenarios. Using this extensive dataset, we trained a TST model (CrystalGPT) to exhibit remarkable S2S transferability not only across all pre-established systems, but also to an unencountered system. CrystalGPT achieved a cumulative error across all systems, which is eight times superior to that of existing ML models. Additionally, we coupled CrystalGPT with a predictive controller to reduce the variance in setpoint tracking to just 1%.
Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere

Fourier Neural Operators (FNOs) have proven to be an efficient and effective method for resolution-independent operator learning in a broad variety of application areas across scientific machine learning. A key reason for their success is their ability to accurately model long-range dependencies in spatio-temporal data by learning global convolutions in a computationally efficient manner. To this end, FNOs rely on the discrete Fourier transform (DFT), however, DFTs cause visual and spectral artifacts as well as pronounced dissipation when learning operators in spherical coordinates since they incorrectly assume a flat geometry. To overcome this limitation, we generalize FNOs on the sphere, introducing Spherical FNOs (SFNOs) for learning operators on spherical geometries. We apply SFNOs to forecasting atmospheric dynamics, and demonstrate stable auto\-regressive rollouts for a year of simulated time (1,460 steps), while retaining physically plausible dynamics. The SFNO has important implications for machine learning-based simulation of climate dynamics that could eventually help accelerate our response to climate change.
"Medium"LMs of Code in the Era of LLMs: Lessons From StackOverflow

Large pre-trained neural language models have brought immense progress to both NLP and software engineering. Models in OpenAI's GPT series now dwarf Google's BERT and Meta's RoBERTa, which previously set new benchmarks on a wide range of NLP applications. These models are trained on massive corpora of heterogeneous data from web crawls, which enables them to learn general language patterns and semantic relationships. However, the largest models are both expensive to train and deploy and are often closed-source, so we lack access to their data and design decisions. We argue that this trend towards large, general-purpose models should be complemented with single-purpose, more modestly sized pre-trained models. In this work, we take StackOverflow (SO) as a domain example in which large volumes of rich aligned code and text data is available. We adopt standard practices for pre-training large language models, including using a very large context size (2,048 tokens), batch size (0.5M tokens) and training set (27B tokens), coupled with a powerful toolkit (Megatron-LM), to train two models: SOBertBase, with 109M parameters, and SOBertLarge with 762M parameters, at a budget of just $\$187$ and $\$800$ each. We compare the performance of our models with both the previous SOTA model trained on SO data exclusively as well general-purpose BERT models and OpenAI's ChatGPT on four SO-specific downstream tasks - question quality prediction, closed question prediction, named entity recognition and obsoletion prediction (a new task we introduce). Not only do our models consistently outperform all baselines, the smaller model is often sufficient for strong results. Both models are released to the public. These results demonstrate that pre-training both extensively and properly on in-domain data can yield a powerful and affordable alternative to leveraging closed-source general-purpose models.