Miracle-dz/UniFAR-FTUs huggingface dataset:
Catalyst Property Prediction with CatBERTa: Unveiling Feature Exploration Strategies through Large Language Models

Efficient catalyst screening necessitates predictive models for adsorption energy, a key property of reactivity. However, prevailing methods, notably graph neural networks (GNNs), demand precise atomic coordinates for constructing graph representations, while integrating observable attributes remains challenging. This research introduces CatBERTa, an energy prediction Transformer model using textual inputs. Built on a pretrained Transformer encoder, CatBERTa processes human-interpretable text, incorporating target features. Attention score analysis reveals CatBERTa's focus on tokens related to adsorbates, bulk composition, and their interacting atoms. Moreover, interacting atoms emerge as effective descriptors for adsorption configurations, while factors such as bond length and atomic properties of these atoms offer limited predictive contributions. By predicting adsorption energy from the textual representation of initial structures, CatBERTa achieves a mean absolute error (MAE) of 0.75 eV-comparable to vanilla Graph Neural Networks (GNNs). Furthermore, the subtraction of the CatBERTa-predicted energies effectively cancels out their systematic errors by as much as 19.3% for chemically similar systems, surpassing the error reduction observed in GNNs. This outcome highlights its potential to enhance the accuracy of energy difference predictions. This research establishes a fundamental framework for text-based catalyst property prediction, without relying on graph representations, while also unveiling intricate feature-property relationships.
Approximate Bayesian computation for Markovian binary trees in phylogenetics

Phylogenetic trees describe the relationships between species in the evolutionary process, and provide information about the rates of diversification. To understand the mechanisms behind macroevolution, we consider a class of multitype branching processes called Markovian binary trees (MBTs). MBTs allow for trait-based variation in diversification rates, and provide a flexible and realistic probabilistic model for phylogenetic trees. We use an approximate Bayesian computation (ABC) scheme to infer the rates of MBTs by exploiting information in the shapes of phylogenetic trees. We evaluate the accuracy of this inference method using simulation studies, and find that our method is accurate and able to detect variation in the diversification rates. In an application to a real-life phylogeny of squamata, we reinforce conclusions drawn from earlier studies, in particular supporting the existence of ovi-/viviparity transitions in both directions. This demonstrates the potential for more complex models of evolution to be employed in phylogenetic inference.
New general dependence measures: construction, estimation and application to high-frequency stock returns

We propose a set of dependence measures that are non-linear, local, invariant to a wide range of transformations on the marginals, can show tail and risk asymmetries, are always well-defined, are easy to estimate and can be used on any dataset. We propose a nonparametric estimator and prove its consistency and asymptotic normality. Thereby we significantly improve on existing (extreme) dependence measures used in asset pricing and statistics. To show practical utility, we use these measures on high-frequency stock return data around market distress events such as the 2010 Flash Crash and during the GFC. Contrary to ubiquitously used correlations we find that our measures clearly show tail asymmetry, non-linearity, lack of diversification and endogenous buildup of risks present during these distress events. Additionally, our measures anticipate large (joint) losses during the Flash Crash while also anticipating the bounce back and flagging the subsequent market fragility. Our findings have implications for risk management, portfolio construction and hedging at any frequency.
Baseline Defenses for Adversarial Attacks Against Aligned Language Models

As Large Language Models quickly become ubiquitous, it becomes critical to understand their security vulnerabilities. Recent work shows that text optimizers can produce jailbreaking prompts that bypass moderation and alignment. Drawing from the rich body of work on adversarial machine learning, we approach these attacks with three questions: What threat models are practically useful in this domain? How do baseline defense techniques perform in this new domain? How does LLM security differ from computer vision? We evaluate several baseline defense strategies against leading adversarial attacks on LLMs, discussing the various settings in which each is feasible and effective. Particularly, we look at three types of defenses: detection (perplexity based), input preprocessing (paraphrase and retokenization), and adversarial training. We discuss white-box and gray-box settings and discuss the robustness-performance trade-off for each of the defenses considered. We find that the weakness of existing discrete optimizers for text, combined with the relatively high costs of optimization, makes standard adaptive attacks more challenging for LLMs. Future research will be needed to uncover whether more powerful optimizers can be developed, or whether the strength of filtering and preprocessing defenses is greater in the LLMs domain than it has been in computer vision.
Thermoacoustic Instability Suppression and Heat-Release Forcing of a Laminar Flame Using Ionic Wind

Advancements in combustion technologies are often impeded by complex combustion dynamics. Active control has proven effective at mitigating these dynamics in the lab, but mass adoption requires more affordable, lightweight, and reliable actuators. Here, a new actuator concept is presented which utilizes sub-breakdown electric fields, the inherent plasma nature of flames, and the electrohydrodynamic effect to create flame stabilization points. These electrically controlled stabilization points allow variable distortion of a laminar flame and bidirectional forcing of the flame heat release. The electric field-based actuator is combined with a simple feedback controller to demonstrate suppression of a thermoacoustic instability. The instability sound pressure level was reduced by 27 dB and in less than 60 ms upon enabling the controller. The use of a sub breakdown electric field requires a mere 40 mW to stabilize a 3.4 kW thermal power flame. The absence of any moving parts and low electrical power required make this a promising actuator concept for many combustion applications.
Satisfiability Checking of Multi-Variable TPTL with Unilateral Intervals Is PSPACE-Complete

We investigate the decidability of the ${0,\infty}$ fragment of Timed Propositional Temporal Logic (TPTL). We show that the satisfiability checking of TPTL$^{0,\infty}$ is PSPACE-complete. Moreover, even its 1-variable fragment (1-TPTL$^{0,\infty}$) is strictly more expressive than Metric Interval Temporal Logic (MITL) for which satisfiability checking is EXPSPACE complete. Hence, we have a strictly more expressive logic with computationally easier satisfiability checking. To the best of our knowledge, TPTL$^{0,\infty}$ is the first multi-variable fragment of TPTL for which satisfiability checking is decidable without imposing any bounds/restrictions on the timed words (e.g. bounded variability, bounded time, etc.). The membership in PSPACE is obtained by a reduction to the emptiness checking problem for a new"non-punctual"subclass of Alternating Timed Automata with multiple clocks called Unilateral Very Weak Alternating Timed Automata (VWATA$^{0,\infty}$) which we prove to be in PSPACE. We show this by constructing a simulation equivalent non-deterministic timed automata whose number of clocks is polynomial in the size of the given VWATA$^{0,\infty}$.
Iterative, Small-Signal L2 Stability Analysis of Nonlinear Constrained Systems

This paper provides a method to analyze the small-signal L2 gain of control-affine nonlinear systems on compact sets via iterative semi-definite programs. First, a continuous piecewise affine storage function and the corresponding upper bound on the L2 gain are found on a bounded, compact set's triangulation. Then, to ensure that the state does not escape this set, a barrier function is found that is robust to small-signal inputs. Small-signal L2 stability then holds inside each sublevel set of the barrier function inside the set where the storage function was found. The bound on the inputs is also found while searching for a barrier function. The method's effectiveness is shown in a numerical example.
Self-supervised Semantic Segmentation: Consistency over Transformation

Accurate medical image segmentation is of utmost importance for enabling automated clinical decision procedures. However, prevailing supervised deep learning approaches for medical image segmentation encounter significant challenges due to their heavy dependence on extensive labeled training data. To tackle this issue, we propose a novel self-supervised algorithm, S3-Net, which integrates a robust framework based on the proposed Inception Large Kernel Attention (I-LKA) modules. This architectural enhancement makes it possible to comprehensively capture contextual information while preserving local intricacies, thereby enabling precise semantic segmentation. Furthermore, considering that lesions in medical images often exhibit deformations, we leverage deformable convolution as an integral component to effectively capture and delineate lesion deformations for superior object boundary definition. Additionally, our self-supervised strategy emphasizes the acquisition of invariance to affine transformations, which is commonly encountered in medical scenarios. This emphasis on robustness with respect to geometric distortions significantly enhances the model’s ability to accurately model and handle such distortions. To enforce spatial consistency and promote the grouping of spatially connected image pixels with similar feature representations, we introduce a spatial consistency loss term. This aids the network in effectively capturing the relationships among neighboring pixels and enhancing the overall segmentation quality. The S3-Net approach iteratively learns pixel-level feature representations for image content clustering in an end-to-end manner. Our experimental results on skin lesion and lung organ segmentation tasks show the superior performance of our method compared to the SOTA approaches. Github.
Regular $(2+1)$-dimensional spatially homogeneous $\alpha'$-corrected BTZ-like black hole in string theory

We consider a $(2+1)$-dimensional spacetime whose two-dimensional space part is Weyl-related to a surface of arbitrary negative constant Gaussian curvature with symmetries of two-dimensional Lie algebra. It is shown that the geometry is a Lobachevsky-type geometry described by deformed hyperbolic function. At leading order string effective action with the source given by dilaton and antisymmetric $B$-field in the presence of central charge deficit term $\Lambda$, we obtained a solution whose line element is Weyl-related to this homogeneous spacetime with arbitrary negative Gaussian curvature. The solution can be transformed to the BTZ-like black hole by coordinate redefinition, while the BTZ black hole can be recovered by choosing a special set of parameters. The solutions appear to be in the high curvature limit $R\alpha'\gtrsim1$, with emphasis on including the higher order $\alpha'$ corrections. Considering the two-loop (first order $\alpha'$) $\beta$-function equations of $\sigma$-model, we also present the $\alpha'$-corrected black hole solutions.
FactLLaMA: Optimizing Instruction-Following Language Models with External Knowledge for Automated Fact-Checking

Automatic fact-checking plays a crucial role in combating the spread of misinformation. Large Language Models (LLMs) and Instruction-Following variants, such as InstructGPT and Alpaca, have shown remarkable performance in various natural language processing tasks. However, their knowledge may not always be up-to-date or sufficient, potentially leading to inaccuracies in fact-checking. To address this limitation, we propose combining the power of instruction-following language models with external evidence retrieval to enhance fact-checking performance. Our approach involves leveraging search engines to retrieve relevant evidence for a given input claim. This external evidence serves as valuable supplementary information to augment the knowledge of the pretrained language model. Then, we instruct-tune an open-sourced language model, called LLaMA, using this evidence, enabling it to predict the veracity of the input claim more accurately. To evaluate our method, we conducted experiments on two widely used fact-checking datasets: RAWFC and LIAR. The results demonstrate that our approach achieves state-of-the-art performance in fact-checking tasks. By integrating external evidence, we bridge the gap between the model's knowledge and the most up-to-date and sufficient context available, leading to improved fact-checking outcomes. Our findings have implications for combating misinformation and promoting the dissemination of accurate information on online platforms. Our released materials are accessible at: https://thcheung.github. io/factllama.
Optimal lifting of Levi-degenerate hypersurfaces and applications to the Cauchy--Szeg\"o projection

We consider a family of Levi-degenerate finite type hypersurfaces in $\mathbb C^2$, where in general there is no group structure. We lift these domains to stratified Lie groups via a constructive proof, which optimizes the well-known lifting procedure to free Lie groups of general manifolds defined by Rothschild and Stein. This yields an explicit version of the Taylor expansion with respect to the horizontal vector fields induced by the sub-Riemannian structure on these hypersurfaces. Hence, as an application, we establish the Schatten class estimates for the commutator of the Cauchy--Szeg\"{o} projection with respect to a suitable quasi-metric defined on the hypersurface.
Multichannel quantum defect theory with a frame transformation for ultracold molecular collisions in magnetic fields

We extend the powerful formalism of multichannel quantum defect theory combined with a frame transformation (MQDT-FT) to ultracold molecular collisions in magnetic fields. By solving the coupled-channel equations with hyperfine and Zeeman interactions omitted at short range, MQDT-FT enables a drastically simplified description of the intricate quantum dynamics of ultracold molecular collisions in terms of a small number of short-range parameters. We apply the formalism to ultracold Mg + NH collisions in a magnetic field, achieving a 10$^4$-fold reduction in computational effort.
FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning

LLMs have demonstrated great capabilities in various NLP tasks. Different entities can further improve the performance of those LLMs on their specific downstream tasks by fine-tuning LLMs. When several entities have similar interested tasks, but their data cannot be shared because of privacy concerns regulations, federated learning (FL) is a mainstream solution to leverage the data of different entities. However, fine-tuning LLMs in federated learning settings still lacks adequate support from existing FL frameworks because it has to deal with optimizing the consumption of significant communication and computational resources, data preparation for different tasks, and distinct information protection demands. This paper first discusses these challenges of federated fine-tuning LLMs, and introduces our package FS-LLM as a main contribution, which consists of the following components: (1) we build an end-to-end benchmarking pipeline, automizing the processes of dataset preprocessing, federated fine-tuning execution, and performance evaluation on federated LLM fine-tuning; (2) we provide comprehensive federated parameter-efficient fine-tuning algorithm implementations and versatile programming interfaces for future extension in FL scenarios with low communication and computation costs, even without accessing the full model; (3) we adopt several accelerating and resource-efficient operators for fine-tuning LLMs with limited resources and the flexible pluggable sub-routines for interdisciplinary study. We conduct extensive experiments to validate the effectiveness of FS-LLM and benchmark advanced LLMs with state-of-the-art parameter-efficient fine-tuning algorithms in FL settings, which also yields valuable insights into federated fine-tuning LLMs for the research community. To facilitate further research and adoption, we release FS-LLM at https://github.com/alibaba/FederatedScope/tree/llm.
Illuminating the Dark Side of Cosmic Star Formation. II. A Second Date with RS-NIRdark Galaxies in COSMOS

About 12 billion years ago, the Universe was first experiencing light again after the dark ages, and galaxies filled the environment with stars, metals, and dust. How efficient was this process? How fast did these primordial galaxies form stars and dust? We can answer these questions by tracing the star formation rate density (SFRD) back to its widely unknown high-redshift tail, traditionally observed in the near-infrared (NIR), optical, and UV bands. Thus, objects with a large amount of dust were missing. We aim to fill this knowledge gap by studying radio-selected NIR-dark (RS-NIRdark) sources, i.e., sources not having a counterpart at UV-to-NIR wavelengths. We widen the sample of Talia et al. from 197 to 272 objects in the Cosmic Evolution Survey (COSMOS) field, including also photometrically contaminated sources, which were previously excluded. Another important step forward consists in the visual inspection of each source in the bands from u* to MIPS 24 μm. According to their “environment” in the different bands, we are able to highlight different cases of study and calibrate an appropriate photometric procedure for the objects affected by confusion issues. We estimate that the contribution of RS-NIRdark sources to the cosmic SFRD at 3 < z < 5 is ∼10%–25% of that based on UV-selected galaxies.
Communication-Efficient Distributed Deep Learning: A Comprehensive Survey

Distributed deep learning (DL) has become prevalent in recent years to reduce training time by leveraging multiple computing devices (e.g., GPUs/TPUs) due to larger models and datasets. However, system scalability is limited by communication becoming the performance bottleneck. Addressing this communication issue has become a prominent research topic. In this paper, we provide a comprehensive survey of the communication-efficient distributed training algorithms, focusing on both system-level and algorithmic-level optimizations. We first propose a taxonomy of data-parallel distributed training algorithms that incorporates four primary dimensions: communication synchronization, system architectures, compression techniques, and parallelism of communication and computing tasks. We then investigate state-of-the-art studies that address problems in these four dimensions. We also compare the convergence rates of different algorithms to understand their convergence speed. Additionally, we conduct extensive experiments to empirically compare the convergence performance of various mainstream distributed training algorithms. Based on our system-level communication cost analysis, theoretical and experimental convergence speed comparison, we provide readers with an understanding of which algorithms are more efficient under specific distributed environments. Our research also extrapolates potential directions for further optimizations.
The insider problem in the trinomial model : a discrete-time jump process approach

In an incomplete market underpinned by the trinomial model, we consider two investors: an ordinary agent whose decisions are driven by public information and an insider who possesses from the beginning a surplus of information encoded through a random variable for which he or she knows the outcome. Through the definition of an auxiliary model based on a marked binomial process, we handle the trinomial model as a volatility one, and use the stochastic analysis and Malliavin calculus toolboxes available in that context. In particular, we connect the information drift, i.e. the drift to eliminate in order to preserve the martingale property within an initial enlargement of filtration in terms of Malliavin’s derivative. We solve explicitly the agent and the insider expected logarithmic utility maximization problems and provide a Ocone-Karatzas type formula for replicable claims. We identify insider’s expected additional utility with the Shannon entropy of the extra information, and examine then the existence of arbitrage opportunities for the insider.
LiDAR-Based Landslide Inventory and Susceptibility Mapping, and Differential LiDAR Analysis for the Panther Creek Watershed, Coast Range, Oregon

i Acknowledgements . iii List of Tables . vii List of Figures . viii Chapter 1: Introduction . 1 Chapter 2: Background . 4 2.1 Study Area . 4 2.2 Climate . 4 2.3 Vegetation . 6 2.4 Geology . 7 2.5 Pedology (Soils) . 10 2.6 Land Use . 11 2.7 Landslides in the Coast Range . 12 Chapter 3: Methods . 14 3.1 LiDAR . 14 3.2 Landslide Inventory Mapping . 14 3.3 Landslide Geodatabase . 16 3.4 Field Data Collection for Landslide Inventory . 20 3.5 Additional Landslide Mapping with Aerial Photography . 21 3.6 Differential LiDAR: Mapping Active Landslides . 23 3.7 Shallow-Seated Landslide Susceptibility Mapping . 25 3.7. 1 Creation of the Factor of Safety (FOS) Map . 26 3.7. 2 Buffers . 29 3.7. 3 Creation of the Final Shallow-Seated Susceptibility map . 30 3.8 Deep-Seated Landslide Susceptibility Mapping . 31 3.8. 1 Head Scarp Buffer . 31 3.8. 2 Susceptible Geologic Units and Slope Angles . 32 3.8. 3 Creation of the Final Deep-Seated Susceptibility Map . 32 Chapter 4: Results . 33 vi 4.1 Landslide Inventory . 33 4.1. 1 Landslide Attributes . 33 4.1. 2 Landslide Inventory Compared to Previous Studies . 35 4.2 Mapping Active Landslides with Differential LiDAR . 35 4.3 Shallow-Seated Susceptibility Map . 44 4.4 Deep-Seated Susceptibility Map . 44 Chapter 5: Discussion . 47 5.1 Accuracy of Landslide Mapping on LiDAR . 47 5.2 Factors Controlling Landslide Distribution . 47 5.3 Factors Controlling Recent Landslide Distribution . 48 5.4 False landslide identification with differential LiDAR . 50 5.5 Comparing Rainfall Data and the Differential Datasets . 52 5.6 Accuracy of the Landslide Susceptibility Maps . 54 Chapter 6: Conclusions . 58 Chapter 7: Future Work . 62 References . 64 Appendix A: Panther Creek Landslide Geodatabase . 68 Appendix B: Panther Creek Landslide Geodatabase Shapefiles.Supplementary File Appendix C: Panther Creek Landslide Inventory Map. Supplementary File