Miracle-dz/UniFAR-FTUs huggingface dataset:
Can Orbital-Selective N\'eel Transitions Survive Strong Nonlocal Electronic Correlations?

Spin- or orbital-selective behaviours in correlated electron materials offer rich promise for spintronics or orbitronics phenomena and applications deriving from them. Strong local electronic Coulomb correlations might lead to an orbital-selective Mott state, characterised by the coexistence of localized electrons in some orbitals with itinerant electrons in others. Nonlocal electronic fluctuations are much more entangled in orbital space than the local ones. For this reason, finding orbital-selective phenomena related to nonlocal correlations, such as orbital-selective magnetic transitions, is a challenge. In this work we investigate possibilities to realize an orbital-selective N\'eel transition (OSNT). We illustrate that stabilising this state requires a decoupling of magnetic fluctuations in different orbitals, which can only be realized in the absence of Hund's exchange coupling. On the basis of two-orbital calculations for a Hubbard model with different bandwidths we show that the proposed OSNT can be found all the way from the weak to the strong coupling regime. In the weak coupling regime the transition is governed by a Slater mechanism and thus occurs first for the narrow orbital. At strong coupling a Heisenberg mechanism of the OSNT sets in, and the transition occurs first for the wide orbital. Remarkably, at intermediate values of the interaction we find a non-trivial regime of the OSNT, where the Slater mechanism leads to a N\'eel transition occurring first for the wide orbital. Our work suggests strategies for searching for orbital-selective N\'eel ordering in real materials, in view of possible spin-orbitronics applications.
Optimizations for a Current-Controlled Memristor- Based Neuromorphic Synapse Design

The synapse is a key element of neuromorphic computing in terms of efficiency and accuracy. In this paper, an optimized current-controlled memristive synapse circuit is proposed. Our proposed synapse demonstrates reliability in the face of process variation and the inherent stochastic behavior of memristors. Up to an 82% energy optimization can be seen during the SET operation over prior work. In addition, the READ process shows up to 54% energy savings. Our current-controlled approach also provides more reliable programming over traditional programming methods. This design is demonstrated with a 4-bit memory precision configuration. Using a spiking neural network (SNN), a neuromorphic application analysis was performed with this precision configuration. Our optimized design showed up to a 82% improvement in control applications and a 2.7x improvement in classification applications compared with other design cases.
An All-Sky, Three-Flavor Search for Neutrinos from Gamma-Ray Bursts with the IceCube Neutrino Observatory

Title of dissertation: AN ALL-SKY, THREE-FLAVOR SEARCH FOR NEUTRINOS FROM GAMMA-RAY BURSTS WITH THE ICECUBE NEUTRINO OBSERVATORY Robert E. Hellauer III, Doctor of Philosophy, 2015 Dissertation directed by: Professor Gregory Sullivan Department of Physics Ultra high energy cosmic rays (UHECRs), defined by energy greater than 10 eV, have been observed for decades, but their sources remain unknown. Protons and heavy ions, which comprise cosmic rays, interact with galactic and intergalactic magnetic fields and, consequently, do not point back to their sources upon measurement. Neutrinos, which are inevitably produced in photohadronic interactions, travel unimpeded through the universe and disclose the directions of their sources. Among the most plausible candidates for the origins of UHECRs is a class of astrophysical phenomena known as gamma-ray bursts (GRBs). GRBs are the most violent and energetic events witnessed in the observable universe. The IceCube Neutrino Observatory, located in the glacial ice 1450 m to 2450 m below the South Pole surface, is the largest neutrino detector in operation. IceCube detects charged particles, such as those emitted in high energy neutrino interactions in the ice, by the Cherenkov light radiated by these particles. The measurement of neutrinos of 100 TeV energy or greater in IceCube correlated with gamma-ray photons from GRBs, measured by spacecraft detectors, would provide evidence of hadronic interaction in these powerful phenomena and confirm their role in ultra high energy cosmic ray production. This work presents the first IceCube GRB-neutrino coincidence search optimized for charged-current interactions of electron and tau neutrinos as well as neutral-current interactions of all neutrino flavors, which produce nearly spherical Cherenkov light showers in the ice. These results for three years of data are combined with the results of previous searches over four years of data optimized for charged-current muon neutrino interactions, which produce extended Cherenkov light tracks. Several low significance events correlated with GRBs were detected, but are consistent with the background expectation from atmospheric muons and neutrinos. The combined results produce limits that place the strongest constraints thus far on models of neutrino and UHECR production in GRB fireballs. AN ALL-SKY, THREE-FLAVOR SEARCH FOR NEUTRINOS FROM GAMMA-RAY BURSTS WITH THE ICECUBE NEUTRINO OBSERVATORY by Robert Eugene Hellauer III Dissertation submitted to the Faculty of the Graduate School of the University of Maryland, College Park in partial fulfillment of the requirements for the degree of Doctor of Philosophy 2015 Advisory Committee: Dr. Gregory Sullivan, Chair/Advisor Dr. Erik Blaufuss Dr. Kara Hoffman Dr. Peter Shawhan Dr. Cole Miller c © Copyright by Robert Eugene Hellauer III 2015
Randomized Positional Encodings Boost Length Generalization of Transformers

Transformers have impressive generalization capabilities on tasks with a fixed context length. However, they fail to generalize to sequences of arbitrary length, even for seemingly simple tasks such as duplicating a string. Moreover, simply training on longer sequences is inefficient due to the quadratic computation complexity of the global attention mechanism. In this work, we demonstrate that this failure mode is linked to positional encodings being out-of-distribution for longer sequences (even for relative encodings) and introduce a novel family of positional encodings that can overcome this problem. Concretely, our randomized positional encoding scheme simulates the positions of longer sequences and randomly selects an ordered subset to fit the sequence’s length. Our large-scale empirical evaluation of 6000 models across 15 algorithmic reasoning tasks shows that our method allows Transformers to generalize to sequences of unseen length (increasing test accuracy by 12.0% on average).
Neural Architecture Search for Parameter-Efficient Fine-tuning of Large Pre-trained Language Models

Parameter-efficient tuning (PET) methods fit pre-trained language models (PLMs) to downstream tasks by either computing a small compressed update for a subset of model parameters, or appending and fine-tuning a small number of new model parameters to the pre-trained network. Hand-designed PET architectures from the literature perform well in practice, but have the potential to be improved via automated neural architecture search (NAS). We propose an efficient NAS method for learning PET architectures via structured and unstructured pruning. We present experiments on GLUE demonstrating the effectiveness of our algorithm and discuss how PET architectural design choices affect performance in practice.
Complete Multiparty Session Type Projection with Automata

Multiparty session types (MSTs) are a type-based approach to verifying communication protocols. Central to MSTs is a projection operator: a partial function that maps protocols represented as global types to correct-by-construction implementations for each participant, represented as a communicating state machine. Existing projection operators are syntactic in nature, and trade efficiency for completeness. We present the first projection operator that is sound, complete, and efficient. Our projection separates synthesis from checking implementability. For synthesis, we use a simple automata-theoretic construction; for checking implementability, we present succinct conditions that summarize insights into the property of implementability. We use these conditions to show that MST implementability is in PSPACE. This improves upon a previous decision procedure that is in EXPSPACE and applies to a smaller class of MSTs. We demonstrate the effectiveness of our approach using a prototype implementation, which handles global types not supported by previous work without sacrificing performance.
Future-conditioned Unsupervised Pretraining for Decision Transformer

Recent research in offline reinforcement learning (RL) has demonstrated that return-conditioned supervised learning is a powerful paradigm for decision-making problems. While promising, return conditioning is limited to training data labeled with rewards and therefore faces challenges in learning from unsupervised data. In this work, we aim to utilize generalized future conditioning to enable efficient unsupervised pretraining from reward-free and sub-optimal offline data. We propose Pretrained Decision Transformer (PDT), a conceptually simple approach for unsupervised RL pretraining. PDT leverages future trajectory information as a privileged context to predict actions during training. The ability to make decisions based on both present and future factors enhances PDT's capability for generalization. Besides, this feature can be easily incorporated into a return-conditioned framework for online finetuning, by assigning return values to possible futures and sampling future embeddings based on their respective values. Empirically, PDT outperforms or performs on par with its supervised pretraining counterpart, especially when dealing with sub-optimal data. Further analysis reveals that PDT can extract diverse behaviors from offline data and controllably sample high-return behaviors by online finetuning. Code is available at here.
Evaluating Open-Domain Dialogues in Latent Space with Next Sentence Prediction and Mutual Information

The long-standing one-to-many issue of the open-domain dialogues poses significant challenges for automatic evaluation methods, i.e., there may be multiple suitable responses which differ in semantics for a given conversational context.To tackle this challenge, we propose a novel learning-based automatic evaluation metric (CMN), which can robustly evaluate open-domain dialogues by augmenting Conditional Variational Autoencoders (CVAEs) with a Next Sentence Prediction (NSP) objective and employing Mutual Information (MI) to model the semantic similarity of text in the latent space. Experimental results on two open-domain dialogue datasets demonstrate the superiority of our method compared with a wide range of baselines, especially in handling responses which are distant to the “golden” reference responses in semantics.
Coherent Soft Imitation Learning

Imitation learning methods seek to learn from an expert either through behavioral cloning (BC) of the policy or inverse reinforcement learning (IRL) of the reward. Such methods enable agents to learn complex tasks from humans that are difficult to capture with hand-designed reward functions. Choosing BC or IRL for imitation depends on the quality and state-action coverage of the demonstrations, as well as additional access to the Markov decision process. Hybrid strategies that combine BC and IRL are not common, as initial policy optimization against inaccurate rewards diminishes the benefit of pretraining the policy with BC. This work derives an imitation method that captures the strengths of both BC and IRL. In the entropy-regularized ('soft') reinforcement learning setting, we show that the behaviour-cloned policy can be used as both a shaped reward and a critic hypothesis space by inverting the regularized policy update. This coherency facilitates fine-tuning cloned policies using the reward estimate and additional interactions with the environment. This approach conveniently achieves imitation learning through initial behaviour cloning, followed by refinement via RL with online or offline data sources. The simplicity of the approach enables graceful scaling to high-dimensional and vision-based tasks, with stable learning and minimal hyperparameter tuning, in contrast to adversarial approaches. For the open-source implementation and simulation results, see https://joemwatson.github. io/csil/.
Oscillatory flows of a slightly rarefied gas: a kinetic theory investigation

Classical continuum theory provides a rigorous framework for studying a diverse range of flow phenomena, with solutions of the Navier-Stokes equations subject to the usual no-slip boundary condition at a solid wall being widely reported. Flows exhibited by miniaturised devices, such as as those found in nanoelectromechanical systems (NEMS), can occur in a regime that obviates the use of the Navier-Stokes equations. More advanced approaches that rigorously describe the kinetic nature of gas flows are thus required to contend with the intrinsic gas rarefaction. The Boltzmann equation describes the evolution of the mass distribution function of a dilute gas resulting from interparticle collisions and phase space advection. This enables the study of gas flows at arbitrary degrees of rarefaction. Practical use of this formalism in the study of non-equilibrium flows is complicated by the nature of the collision integral in the Boltzmann equation, which typically exhibits a quadratic nonlinearity in the mass distribution function. In addition, several useful models for the interparticle force, such as the hard sphere model and the variable hard sphere model, yield particle collision frequencies that are dependent on the relative velocity of colliding particles. This restricts the scope for direct analysis. For this reason, model equations for the collision integral have been proposed. In particular, the relaxation approximation that was independently proposed by Bhatnagar, Gross and Krook (1954) and Welander (1954), has been widely applied to model dilute gas flows. This is commonly referred to as the BGK relaxation time approximation, and dramatically simplifies the Boltzmann equation. While not an exact formulation, this model displays many of the characteristics of real gas flows, and has been solved both
Impossible Distillation: from Low-Quality Model to High-Quality Dataset & Model for Summarization and Paraphrasing

We present Impossible Distillation, a novel framework for paraphrasing and sentence summarization, that distills a high-quality dataset and model from a low-quality teacher that itself cannot perform these tasks. Unlike prior works that rely on an extreme-scale teacher model (e.g., GPT3) or task-specific architecture, we hypothesize and verify the paraphrastic proximity intrinsic to pre-trained LMs (e.g., GPT2), where paraphrases occupy a proximal subspace in the LM distribution. By identifying and distilling generations from these subspaces, Impossible Distillation produces a high-quality dataset and model even from GPT2-scale LMs. We evaluate our method on multiple benchmarks spanning unconstrained / syntax-controlled paraphrase generation and sentence summarization. Our model with 770M parameters consistently outperforms strong baselines, including models distilled from ChatGPT, and sometimes, even ChatGPT itself. Also, we find that our distilled dataset from 1.5B LMs exhibits higher diversity and fidelity than up to 13 times larger datasets.
Fast and Order-invariant Inference in Bayesian VARs with Non-Parametric Shocks

The shocks which hit macroeconomic models such as Vector Autoregressions (VARs) have the potential to be non-Gaussian, exhibiting asymmetries and fat tails. This consideration motivates the VAR developed in this paper which uses a Dirichlet process mixture (DPM) to model the shocks. However, we do not follow the obvious strategy of simply modeling the VAR errors with a DPM since this would lead to computationally infeasible Bayesian inference in larger VARs and potentially a sensitivity to the way the variables are ordered in the VAR. Instead we develop a particular additive error structure inspired by Bayesian nonparametric treatments of random effects in panel data models. We show that this leads to a model which allows for computationally fast and order-invariant inference in large VARs with nonparametric shocks. Our empirical results with nonparametric VARs of various dimensions shows that nonparametric treatment of the VAR errors is particularly useful in periods such as the financial crisis and the pandemic.
Neural Machine Translation for Mathematical Formulae

We tackle the problem of neural machine translation of mathematical formulae between ambiguous presentation languages and unambiguous content languages. Compared to neural machine translation on natural language, mathematical formulae have a much smaller vocabulary and much longer sequences of symbols, while their translation requires extreme precision to satisfy mathematical information needs. In this work, we perform the tasks of translating from LaTeX to Mathematica as well as from LaTeX to semantic LaTeX. While recurrent, recursive, and transformer networks struggle with preserving all contained information, we find that convolutional sequence-to-sequence networks achieve 95.1% and 90.7% exact matches, respectively.
Surface Protection and Activation of Mid-IR Plasmonic Waveguides for Spectroscopy of Liquids

Liquid spectroscopy in the mid-infrared spectral range is a very powerful, yet premature technique for selective and sensitive molecule detection. Due to the lack of suitable concepts and materials for versatile miniaturized sensors, it is often still limited to bulky systems and offline analytics. Mid-infrared plasmonics is a promising field of current research for such compact and surface-sensitive structures, enabling new pathways for much-needed photonic integrated sensors. In this work, we focus on extending the concept of Ge/Au-based mid-infrared plasmonic waveguides to enable broadband liquid detection. Through the implementation of high-quality dielectric passivation layers deposited by atomic layer deposition (ALD), we cover the weak and water-soluble Ge native oxide. We show that approximately 10 nm of e.g. Al<inline-formula><tex-math notation="LaTeX">$_{2}$</tex-math></inline-formula>O<inline-formula><tex-math notation="LaTeX">$_{3}$</tex-math></inline-formula> or ZrO<inline-formula><tex-math notation="LaTeX">$_{2}$</tex-math></inline-formula> can already protect the plasmonic waveguides for up to 90 min of direct water exposure. This unlocks integrated sensing schemes for broadband molecule detection based on mid-infrared plasmonics. In a proof-of-concept experiment, we further demonstrate that the ZrO<inline-formula><tex-math notation="LaTeX">$_{2}$</tex-math></inline-formula> coated waveguides can be activated by surface functionalization, allowing the direct measurement of diethyl ether at a wavelength of 9.38 <inline-formula><tex-math notation="LaTeX">$\mu$</tex-math></inline-formula>m.
Too Few Bug Reports? Exploring Data Augmentation for Improved Changeset-based Bug Localization

Modern Deep Learning (DL) architectures based on transformers (e.g., BERT, RoBERTa) are exhibiting performance improvements across a number of natural language tasks. While such DL models have shown tremendous potential for use in software engineering applications, they are often hampered by insufficient training data. Particularly constrained are applications that require project-specific data, such as bug localization, which aims at recommending code to fix a newly submitted bug report. Deep learning models for bug localization require a substantial training set of fixed bug reports, which are at a limited quantity even in popular and actively developed software projects. In this paper, we examine the effect of using synthetic training data on transformer-based DL models that perform a more complex variant of bug localization, which has the goal of retrieving bug-inducing changesets for each bug report. To generate high-quality synthetic data, we propose novel data augmentation operators that act on different constituent components of bug reports. We also describe a data balancing strategy that aims to create a corpus of augmented bug reports that better reflects the entire source code base, because existing bug reports used as training data usually reference a small part of the code base.
ParaAMR: A Large-Scale Syntactically Diverse Paraphrase Dataset by AMR Back-Translation

Paraphrase generation is a long-standing task in natural language processing (NLP). Supervised paraphrase generation models, which rely on human-annotated paraphrase pairs, are cost-inefficient and hard to scale up. On the other hand, automatically annotated paraphrase pairs (e.g., by machine back-translation), usually suffer from the lack of syntactic diversity – the generated paraphrase sentences are very similar to the source sentences in terms of syntax. In this work, we present ParaAMR, a large-scale syntactically diverse paraphrase dataset created by abstract meaning representation back-translation. Our quantitative analysis, qualitative examples, and human evaluation demonstrate that the paraphrases of ParaAMR are syntactically more diverse compared to existing large-scale paraphrase datasets while preserving good semantic similarity. In addition, we show that ParaAMR can be used to improve on three NLP tasks: learning sentence embeddings, syntactically controlled paraphrase generation, and data augmentation for few-shot learning. Our results thus showcase the potential of ParaAMR for improving various NLP applications.
Explaining Deep Learning for ECG Analysis: Building Blocks for Auditing and Knowledge Discovery

Deep neural networks have become increasingly popular for analyzing ECG data because of their ability to accurately identify cardiac conditions and hidden clinical factors. However, the lack of transparency due to the black box nature of these models is a common concern. To address this issue, explainable AI (XAI) methods can be employed. In this study, we present a comprehensive analysis of post-hoc XAI methods, investigating the local (attributions per sample) and global (based on domain expert concepts) perspectives. We have established a set of sanity checks to identify sensible attribution methods, and we provide quantitative evidence in accordance with expert rules. This dataset-wide analysis goes beyond anecdotal evidence by aggregating data across patient subgroups. Furthermore, we demonstrate how these XAI techniques can be utilized for knowledge discovery, such as identifying subtypes of myocardial infarction. We believe that these proposed methods can serve as building blocks for a complementary assessment of the internal validity during a certification process, as well as for knowledge discovery in the field of ECG analysis.
Asymptotic tail properties of Poisson mixture distributions

Count data are omnipresent in many applied fields, often with overdispersion. With mixtures of Poisson distributions representing an elegant and appealing modelling strategy, we focus here on how the tail behaviour of the mixing distribution is related to the tail of the resulting Poisson mixture. We define five sets of mixing distributions, and we identify for each case whenever the Poisson mixture is in, close to or far from a domain of attraction of maxima. We also characterize how the Poisson mixture behaves similarly to a standard Poisson distribution when the mixing distribution has a finite support. Finally, we study, both analytically and numerically, how goodness‐of‐fit can be assessed with the inspection of tail behaviour.