Miracle-dz/UniFAR-FTUs huggingface dataset:
Cross-Task Attention Network: Improving Multi-Task Learning for Medical Imaging Applications

Multi-task learning (MTL) is a powerful approach in deep learning that leverages the information from multiple tasks during training to improve model performance. In medical imaging, MTL has shown great potential to solve various tasks. However, existing MTL architectures in medical imaging are limited in sharing information across tasks, reducing the potential performance improvements of MTL. In this study, we introduce a novel attention-based MTL framework to better leverage inter-task interactions for various tasks from pixel-level to image-level predictions. Specifically, we propose a Cross-Task Attention Network (CTAN) which utilizes cross-task attention mechanisms to incorporate information by interacting across tasks. We validated CTAN on four medical imaging datasets that span different domains and tasks including: radiation treatment planning prediction using planning CT images of two different target cancers (Prostate, OpenKBP); pigmented skin lesion segmentation and diagnosis using dermatoscopic images (HAM10000); and COVID-19 diagnosis and severity prediction using chest CT scans (STOIC). Our study demonstrates the effectiveness of CTAN in improving the accuracy of medical imaging tasks. Compared to standard single-task learning (STL), CTAN demonstrated a 4.67% improvement in performance and outperformed both widely used MTL baselines: hard parameter sharing (HPS) with an average performance improvement of 3.22%; and multi-task attention network (MTAN) with a relative decrease of 5.38%. These findings highlight the significance of our proposed MTL framework in solving medical imaging tasks and its potential to improve their accuracy across domains.
Stroke-based Neural Painting and Stylization with Dynamically Predicted Painting Region

Stroke-based rendering aims to recreate an image with a set of strokes. Most existing methods render complex images using an uniform-block-dividing strategy, which leads to boundary inconsistency artifacts. To solve the problem, we propose Compositional Neural Painter, a novel stroke-based rendering framework which dynamically predicts the next painting region based on the current canvas, instead of dividing the image plane uniformly into painting regions. We start from an empty canvas and divide the painting process into several steps. At each step, a compositor network trained with a phasic RL strategy first predicts the next painting region, then a painter network trained with a WGAN discriminator predicts stroke parameters, and a stroke renderer paints the strokes onto the painting region of the current canvas. Moreover, we extend our method to stroke-based style transfer with a novel differentiable distance transform loss, which helps preserve the structure of the input image during stroke-based stylization. Extensive experiments show our model outperforms the existing models in both stroke-based neural painting and stroke-based stylization.
The Effect of Branchless Collisions and Population Control on Correlations in Monte Carlo Power Iteration

The investigation of correlations in Monte Carlo power iteration has been long dominated by the question of generational correlations and their effects on the estimation of statistical uncertainties. More recently, there has been a growing interest in spatial correlations, prompted by the discovery of neutron clustering. Despite several attempts, a comprehensive framework concerning how Monte Carlo sampling strategies, population control and variance reduction methods affect the strength of such correlations is still lacking. In this work, we propose a set of global and local (i.e., space-dependent) tallies that can be used to characterize the impact of correlations. These tallies encompass the Shannon entropy, the pair distance, the normalized variance and the Feynman moment. In order to have a clean, yet fully meaningful setting, we carry out our analysis in a few homogeneous and heterogeneous benchmark problems of varying dominance ratio. Several classes of collision sampling strategies, population control and variance reduction techniques are tested, and their relative advantages and drawbacks are assessed with respect to the proposed tallies. The major finding of our study is that branchless collisions, which suppress the emergence of branches in neutron histories, also considerably reduce the effects of correlations in most of the explored configurations.
XGen-7B Technical Report

Large Language Models (LLMs) have become ubiquitous across various domains, transforming the way we interact with information and conduct research. However, most high-performing LLMs remain confined behind proprietary walls, hindering scientific progress. Most open-source LLMs, on the other hand, are limited in their ability to support longer sequence lengths, which is a key requirement for many tasks that require inference over an input context. To address this, we have trained XGen, a series of 7B parameter models on up to 8K sequence length for up to 1.5T tokens. We have also finetuned the XGen models on public-domain instructional data, creating their instruction-tuned counterparts (XGen-Inst). We open-source our models for both research advancements and commercial applications. Our evaluation on standard benchmarks shows that XGen models achieve comparable or better results when compared with state-of-the-art open-source LLMs. Our targeted evaluation on long sequence modeling tasks shows the benefits of our 8K-sequence models over 2K-sequence open-source LLMs.
Exploring the capabilities of Gibbs sampling in pulsar timing arrays

We explore the use of Gibbs sampling in estimating the noise properties of individual pulsars and illustrate its effectiveness using the NANOGrav 11-year data set. We find that Gibbs sampling noise modeling (GM) is more efficient than the current standard Bayesian techniques (SM) for single pulsar analyses by yielding model parameter posteriors with average effective-sample-size ratio (GM/SM) of 6 across all parameters and pulsars. Furthermore, the output of GM contains posteriors for the Fourier coefficients that can be used to characterize the underlying red noise process of any pulsar's timing residuals, which are absent in current implementations of SM. Through simulations, we demonstrate the potential for such coefficients to measure the spatial cross-correlations between pulsar pairs produced by a gravitational wave background.
New Cost-Effective Method for Long-Term Groundwater Monitoring Programs

Abstract : This project involved basic research on an alternative groundwater sampling approach--vapor-phase groundwater monitoring--that relies on a different set of physical processes and analytical instruments to provide the Department of Defense (DoD) with reliable and accurate long-term monitoring for volatile organic compounds (VOCs). The overall goal of this research project is to evaluate the utility of on-site vapor-phase analysis of samples from a groundwater monitoring well as an alternative to off-site analysis of groundwater samples. Current approaches for long-term groundwater monitoring programs rely on water sampling and analysis using traditional decades-old protocols that are time-consuming and costly. Complying with the requirements of these monitoring programs comprise a significant portion of life-cycle remediation costs the for Department of Defense (DoD). There is an opportunity to use existing vapor-phase based technologies as part of a new approach that generates monitoring data more rapidly at a lower overall cost.
Arsenic in the Soils of Northwest Oregon

i ACKNOWLEDGEMENTS . iii LIST OF TABLES . viii LIST OF FIGURES . ix CHAPTER 1: INTRODUCTION . 1 CHAPTER 2: BACKGROUND . 6 2.1: ARSENIC AND HUMAN HEALTH . 6 2.2: DEFAULT BACKGROUND CONCENTRATIONS AND RISK BASED . 6 CONCENTRATIONS . 6 2.3: BACKGROUND STUDIES OF ARSENIC IN NORTHWEST OREGON SOILS . 8 2.4: ARSENIC IN THE ENVIRONMENT . 10 2.4.1. ORIGIN OF ENVIRONMENTAL ARSENIC. 11 2.4.2. ARSENIC BEHAVIOR IN SOILS . 14 CHAPTER 3: SITE ENVIRONMENT . 20 3.1: GENERAL GEOLOGY . 20 3.1. 1: GEOLOGIC HISTORY OF OREGON . 20 3.1. 2: THE COAST RANGE . 23 3.1. 3: THE WILLAMETTE VALLEY . 27 3.1. 4: THE CASCADE RANGE . 30 3.2: SITE CHARACTERIZATIONS . 33 3.2.1: VEGETATION . 33 3.2.2: SOIL CHARACTERIZATION. 33 CHAPTER 4: METHODS . 35 4.1: USING GEOGRAPHIC INFORMATION SYSYSTEMS (GIS) TO DETERMINE UNSAMPLED GEOLOGIC MAP UNITS . 35 4.2: FIELDWORK . 37 4.3: LABORATORY ANALYSIS . 38 4.4: GROUPINGS FOR STATISTIC ANALYSIS . 41
Your Battery Is a Blast! Safeguarding Against Counterfeit Batteries with Authentication

Lithium-ion (Li-ion) batteries are the primary power source in various applications due to their high energy and power density. Their market was estimated to be up to 48 billion U.S. dollars in 2022. However, the widespread adoption of Li-ion batteries has resulted in counterfeit cell production, which can pose safety hazards to users. Counterfeit cells can cause explosions or fires, and their prevalence in the market makes it difficult for users to detect fake cells. Indeed, current battery authentication methods can be susceptible to advanced counterfeiting techniques and are often not adaptable to various cells and systems. In this paper, we improve the state of the art on battery authentication by proposing two novel methodologies, DCAuth and EISthentication, which leverage the internal characteristics of each cell through Machine Learning models. Our methods automatically authenticate lithium-ion battery models and architectures using data from their regular usage without the need for any external device. They are also resilient to the most common and critical counterfeit practices and can scale to several batteries and devices. To evaluate the effectiveness of our proposed methodologies, we analyze time-series data from a total of 20 datasets that we have processed to extract meaningful features for our analysis. Our methods achieve high accuracy in battery authentication for both architectures (up to 0.99) and models (up to 0.96). Moreover, our methods offer comparable identification performances. By using our proposed methodologies, manufacturers can ensure that devices only use legitimate batteries, guaranteeing the operational state of any system and safety measures for the users.
Privately-Owned versus Shared Automated Vehicle: The Roles of Utilitarian and Hedonic Beliefs

Transportation systems will be likely transformed by the emergence of automated vehicles (AVs) promising for safe, convenient, and efficient mobility, especially if used in shared systems (shared AV or SAV). However, the potential tendency is observed towards owning AV as a private asset rather than using SAV. This calls for a research on investigating individuals' attitude towards AV in comparison with SAV to recognize the barriers to the public's tendency towards SAV. To do so, the present study proposes a modeling framework based on the theories in behavioral psychology to explain individuals' preference for owning AV over using SAV, built as a latent (subjective) psychometric construct, by three groups of explanatory latent constructs including: (i) desire for searching for benefits, i.e., extrinsic motive manifested in utilitarian beliefs; (ii) tendency towards seeking pleasure and joy, i.e., intrinsic motive reflected in hedonic beliefs; and (iii) attitude towards three configurations of shared mobility, i.e., experience with car and ridesharing, bikesharing, and public transit. Estimated on a sample dataset from the State of California, the findings can shed initial lights on the psychological determinants of the public's attitude towards owning AV versus using SAV, which can furthermore provide policy implications intriguing for policy makers and stakeholders. Of note, the findings reveal the strongest influential factor on preference for AV over SAV as hedonic beliefs reflected in perceived enjoyment. This preference is next affected by utilitarian beliefs, particularly perceived benefit and trust of stranger, followed by attitude towards car and ride sharing.
Natural Example-Based Explainability: a Survey

Explainable Artificial Intelligence (XAI) has become increasingly significant for improving the interpretability and trustworthiness of machine learning models. While saliency maps have stolen the show for the last few years in the XAI field, their ability to reflect models' internal processes has been questioned. Although less in the spotlight, example-based XAI methods have continued to improve. It encompasses methods that use examples as explanations for a machine learning model's predictions. This aligns with the psychological mechanisms of human reasoning and makes example-based explanations natural and intuitive for users to understand. Indeed, humans learn and reason by forming mental representations of concepts based on examples. This paper provides an overview of the state-of-the-art in natural example-based XAI, describing the pros and cons of each approach. A"natural"example simply means that it is directly drawn from the training data without involving any generative process. The exclusion of methods that require generating examples is justified by the need for plausibility which is in some regards required to gain a user's trust. Consequently, this paper will explore the following family of methods: similar examples, counterfactual and semi-factual, influential instances, prototypes, and concepts. In particular, it will compare their semantic definition, their cognitive impact, and added values. We hope it will encourage and facilitate future work on natural example-based XAI.
Experimental calibration of the aluminum-in-hornblende geobarometer with application

A new geobarometer based on the Al content of igneous hornblendes in equilibrium with melt, fluid, biotite, quartz, sanidine, plagioclase, sphene, and magnetite or ilmenite has been calibrated experimentally. The calibration was performed by equilibrating the required phase assemblage over the pressure range 2-8 kbar at 740-780C, and then analyzing euhedral hornblendes in equilibrium with glass (melt). Experiments were performed on natural samples of both volcanic and plutonic rocks. Earlier empirical calibrations of this geobarometer relied on analyzing natural hornblendes from plutons with the required phase assemblage and inferring pressure from nearby metamorphic country rocks. The experimental calibration differs from the empirical calibrations, especially above 5 kbar, and shows that the Al content of hornblendes in equilibrium with the required phase assemblage is greater for a given total pressure than previously thought. The geobarometers uncertainty is dramatically reduced. The derived equation is P ({plus minus}0.5 kbar) = 3 {minus}3.46 ({plus minus}0.24) + 4.23 ({plus minus}0.13) (Al{sup T}). The geobarometer is applied to post-Bishop Tuff volcanic rocks from Long Valley caldera, California, and reveals that most rhyodacites in this complex erupted from depths of about 6 km. These eruptions occurred over 500,000 yr, suggesting that the rhyodacitic magma reservoir beneath Longmore » Valley had reached a steady P (depth)-T state. « less
Scalable Learning of Intrusion Responses through Recursive Decomposition

We study automated intrusion response for an IT infrastructure and formulate the interaction between an attacker and a defender as a partially observed stochastic game. To solve the game we follow an approach where attack and defense strategies co-evolve through reinforcement learning and self-play toward an equilibrium. Solutions proposed in previous work prove the feasibility of this approach for small infrastructures but do not scale to realistic scenarios due to the exponential growth in computational complexity with the infrastructure size. We address this problem by introducing a method that recursively decomposes the game into subgames which can be solved in parallel. Applying optimal stopping theory we show that the best response strategies in these subgames exhibit threshold structures, which allows us to compute them efficiently. To solve the decomposed game we introduce an algorithm called Decompositional Fictitious Self-Play (DFSP), which learns Nash equilibria through stochastic approximation. We evaluate the learned strategies in an emulation environment where real intrusions and response actions can be executed. The results show that the learned strategies approximate an equilibrium and that DFSP significantly outperforms a state-of-the-art algorithm for a realistic infrastructure configuration.
A closer look at the binary content of NGC 1850

Studies of young clusters have shown that a large fraction of O-/early B-type stars are in binary systems, where the binary fraction increases with mass. These massive stars are present in clusters of a few Myr, but gradually disappear for older clusters. The lack of detailed studies of intermediate-age clusters has meant that almost no information is available on the multiplicity properties of stars with $M\lt 4\, {\rm M}_{\odot }$. In this study we present the first characterization of the binary content of NGC 1850, a 100 Myr-old massive star cluster in the Large Magellanic Cloud, relying on a VLT/MUSE multi-epoch spectroscopic campaign. By sampling stars down to M = 2.5 M⊙, we derive a close binary fraction of 24 ± 5 per cent in NGC 1850, in good agreement with the multiplicity frequency predicted for stars of this mass range. We also find a trend with stellar mass (magnitude), with higher mass (brighter) stars having higher binary fractions. We modelled the radial velocity curves of individual binaries using the joker and constrained the orbital properties of 27 systems, ∼17 per cent of all binaries with reliable radial velocities in NGC 1850. This study has brought to light a number of interesting objects, such as four binaries showing mass functions f(M) > 1.25 M⊙. One of these, star #47, has a peculiar spectrum, explainable with the presence of two discs in the system, around the visible star and the dark companion, which is a black hole candidate. These results confirm the importance and urgency of studying the binary content of clusters of any age.
Empirical Risk Minimization for Losses without Variance

This paper considers an empirical risk minimization problem under heavy-tailed settings, where data does not have finite variance, but only has $p$-th moment with $p \in (1,2)$. Instead of using estimation procedure based on truncated observed data, we choose the optimizer by minimizing the risk value. Those risk values can be robustly estimated via using the remarkable Catoni's method (Catoni, 2012). Thanks to the structure of Catoni-type influence functions, we are able to establish excess risk upper bounds via using generalized generic chaining methods. Moreover, we take computational issues into consideration. We especially theoretically investigate two types of optimization methods, robust gradient descent algorithm and empirical risk-based methods. With an extensive numerical study, we find that the optimizer based on empirical risks via Catoni-style estimation indeed shows better performance than other baselines. It indicates that estimation directly based on truncated data may lead to unsatisfactory results.
Automated Characterization of a Double Quantum Dot using the Hubbard Model

Semiconductor quantum dots are favorable candidates for quantum information processing due to their long coherence time and potential scalability. However, the calibration and characterization of interconnected quantum dot arrays have proven to be challenging tasks. One method to characterize the configuration of such an array involves using the Hubbard model. In this paper, we present an automated characterization algorithm that efficiently extracts the Hubbard model parameters, including tunnel coupling and capacitive coupling energy, from experimental stability diagrams. Leveraging the dual annealing optimizer, we determine the set of Hubbard parameters that best characterize the experimental data. We compare our method with an alternate, well-established measure of the tunnel coupling and find good agreement within the investigated regime. Our extracted tunnel couplings range from 69 to 517 $\mu$eV, and we discuss the limiting factors of our method.
Fast FixMatch: Faster Semi-Supervised Learning with Curriculum Batch Size

Advances in Semi-Supervised Learning (SSL) have almost entirely closed the gap between SSL and Supervised Learning at a fraction of the number of labels. However, recent performance improvements have often come \textit{at the cost of significantly increased training computation}. To address this, we propose Curriculum Batch Size (CBS), \textit{an unlabeled batch size curriculum which exploits the natural training dynamics of deep neural networks. } A small unlabeled batch size is used in the beginning of training and is gradually increased to the end of training. A fixed curriculum is used regardless of dataset, model or number of epochs, and reduced training computations is demonstrated on all settings. We apply CBS, strong labeled augmentation, Curriculum Pseudo Labeling (CPL) \citep{FlexMatch} to FixMatch \citep{FixMatch} and term the new SSL algorithm Fast FixMatch. We perform an ablation study to show that strong labeled augmentation and/or CPL do not significantly reduce training computations, but, in synergy with CBS, they achieve optimal performance. Fast FixMatch also achieves substantially higher data utilization compared to previous state-of-the-art. Fast FixMatch achieves between $2.1\times$ - $3.4\times$ reduced training computations on CIFAR-10 with all but 40, 250 and 4000 labels removed, compared to vanilla FixMatch, while attaining the same cited state-of-the-art error rate \citep{FixMatch}. Similar results are achieved for CIFAR-100, SVHN and STL-10. Finally, Fast MixMatch achieves between $2.6\times$ - $3.3\times$ reduced training computations in federated SSL tasks and online/streaming learning SSL tasks, which further demonstrate the generializbility of Fast MixMatch to different scenarios and tasks.
CoNeS: Conditional neural fields with shift modulation for multi-sequence MRI translation

Multi-sequence magnetic resonance imaging (MRI) has found wide applications in both modern clinical studies and deep learning research. However, in clinical practice, it frequently occurs that one or more of the MRI sequences are missing due to different image acquisition protocols or contrast agent contraindications of patients, limiting the utilization of deep learning models trained on multi-sequence data. One promising approach is to leverage generative models to synthesize the missing sequences, which can serve as a surrogate acquisition. State-of-the-art methods tackling this problem are based on convolutional neural networks (CNN) which usually suffer from spectral biases, resulting in poor reconstruction of high-frequency fine details. In this paper, we propose Conditional Neural fields with Shift modulation (CoNeS), a model that takes voxel coordinates as input and learns a representation of the target images for multi-sequence MRI translation. The proposed model uses a multi-layer perceptron (MLP) instead of a CNN as the decoder for pixel-to-pixel mapping. Hence, each target image is represented as a neural field that is conditioned on the source image via shift modulation with a learned latent code. Experiments on BraTS 2018 and an in-house clinical dataset of vestibular schwannoma patients showed that the proposed method outperformed state-of-the-art methods for multi-sequence MRI translation both visually and quantitatively. Moreover, we conducted spectral analysis, showing that CoNeS was able to overcome the spectral bias issue common in conventional CNN models. To further evaluate the usage of synthesized images in clinical downstream tasks, we tested a segmentation network using the synthesized images at inference. The results showed that CoNeS improved the segmentation performance when some MRI sequences were missing and outperformed other synthesis models. We concluded that neural fields are a promising technique for multi-sequence MRI translation.