Miracle-dz/UniFAR-FTUs huggingface dataset:
Co-modeling the Sequential and Graphical Routes for Peptide Representation Learning

Peptides are formed by the dehydration condensation of multiple amino acids. The primary structure of a peptide can be represented either as an amino acid sequence or as a molecular graph consisting of atoms and chemical bonds. Previous studies have indicated that deep learning routes specific to sequential and graphical peptide forms exhibit comparable performance on downstream tasks. Despite the fact that these models learn representations of the same modality of peptides, we find that they explain their predictions differently. Considering sequential and graphical models as two experts making inferences from different perspectives, we work on fusing expert knowledge to enrich the learned representations for improving the discriminative performance. To achieve this, we propose a peptide co-modeling method, RepCon, which employs a contrastive learning-based framework to enhance the mutual information of representations from decoupled sequential and graphical end-to-end models. It considers representations from the sequential encoder and the graphical encoder for the same peptide sample as a positive pair and learns to enhance the consistency of representations between positive sample pairs and to repel representations between negative pairs. Empirical studies of RepCon and other co-modeling methods are conducted on open-source discriminative datasets, including aggregation propensity, retention time, antimicrobial peptide prediction, and family classification from Peptide Database. Our results demonstrate the superiority of the co-modeling approach over independent modeling, as well as the superiority of RepCon over other methods under the co-modeling framework. In addition, the attribution on RepCon further corroborates the validity of the approach at the level of model explanation.
Hoeffding's Inequality for Markov Chains under Generalized Concentrability Condition

This paper studies Hoeffding's inequality for Markov chains under the generalized concentrability condition defined via integral probability metric (IPM). The generalized concentrability condition establishes a framework that interpolates and extends the existing hypotheses of Markov chain Hoeffding-type inequalities. The flexibility of our framework allows Hoeffding's inequality to be applied beyond the ergodic Markov chains in the traditional sense. We demonstrate the utility by applying our framework to several non-asymptotic analyses arising from the field of machine learning, including (i) a generalization bound for empirical risk minimization with Markovian samples, (ii) a finite sample guarantee for Ployak-Ruppert averaging of SGD, and (iii) a new regret bound for rested Markovian bandits with general state space.
Generalized Stochastic Dynamic Aggregative Game for Demand-Side Management in Microgrids With Shared Battery

In this letter, we focus on modeling and analysis of demand-side management in a microgrid where agents utilize grid energy and a shared battery charged by renewable energy sources. We model the problem as a generalized stochastic dynamic aggregative game with chance constraints that capture the effects of uncertainties in the renewable generation and agents’ demands. Computing the solution of the game is a complex task due to probabilistic and coupling constraints among the agents through the state of charge of the shared battery. We investigate the Nash equilibrium of this game under uncertainty considering both the uniqueness of the solution and the effect of uncertainty on the solution. Simulation results demonstrate that the presented stochastic method is superior to deterministic methods.
Signature of topology via heat transfer analysis in the Su–Schrieffer–Heeger (SSH) model

In this work, we explore how thermodynamics can be a potential tool for identifying the topological phase transition. Specifically, we focus on a one-dimensional Su–Schrieffer–Heeger (SSH) chain sandwiched between two fermionic baths. To investigate distinctive thermodynamic signatures associated with the topological phase, we employ heat flow analysis. Our results, derived using a global master equation, unveil a significant suppression of heat flow as we transition from the trivial to the topological phase. This decline in heat flow can be attributed to the reduction in transmission coefficients of non-zero energy modes within the topological phase. It may serve as an indicator of a phase transition. Furthermore, we investigate the heat flow asymmetry to search for phase transition indicators. Interestingly, no asymmetry is observed when employing fermionic baths. However, upon substituting fermionic baths with bosonic ones, we report a non-zero heat flow asymmetry. For SSH model with few fermionic sites, this asymmetry is more pronounced in the topological phase compared to the trivial phase. Therefore, the observed behavior of the heat diode provides an additional means of distinguishing between the topological and trivial phases. Finally, we delve into the contributions from both bulk and edge effects in heat flow and rectification to explore the impact of small system sizes on our findings.
Effects of feedback-free starburst galaxies on the 21-cm signal and reionization history

Different star-formation models at Cosmic Dawn produce detectable signatures in the observables of upcoming 21-cm experiments. In this work, we consider the physical scenario of feedback-free starbursts (FFB), according to which the star-formation efficiency (SFE) is enhanced in sufficiently massive halos at early enough times, thus explaining the indication from the James Webb Space Telescope for an excess of bright galaxies at $z \geq 10$. We model the contribution of FFBs to popII SFE and compute the impact these have on the 21-cm global signal and power spectrum. We show that FFBs affect the evolution of the brightness temperature and the 21-cm power spectrum, but they only have a limited effect on the neutral hydrogen fraction. We investigate how the observables are affected by changes in the underlying star formation model and by contribution from popIII stars. Finally, we forecast the capability of next-generation Hydrogen Epoch of Reionization Array (HERA) to detect the existence of FFB galaxies via power spectrum measurements. Our results show the possibility of a significant detection, provided that popII stars are the main drivers of lowering the spin temperature. Efficient popIII star formation will make the detection more challenging.
Patterns of Shear and Turbulence across the Florida Current

Abstract Measurements by the Multi-Scale Profiler (MSP) at seven stations spanning the Straits of Florida characterize levels and patterns of internal wave activity and mixing in this vertically sheared environment. Contrasting properties suggest five mixing regimes. The largest and most vast is the interior regime, where the background flow has an inverse Richardson number (Ri−1) ranging up to 0.55, shear is dominated by fluctuations that are 1–4 times stronger than in the open ocean, and turbulent diffusivities are similarly moderate at (1–4) × 10−5 m2 s−1. The high-velocity core of the current, near the surface at midchannel, is associated with weak mixing. To its west is a zone of high mean shear, where strong stratification results in background Ri−1 of only 0.4, fluctuations are weak, and diffusivity is moderate. Intermittent shear features beneath the core have mean Ri−1 > 1 and strong turbulence. Two regimes are related to channel topography. Adjacent to the steep eastern slope, finescale shear is.
Variance Reduced Halpern Iteration for Finite-Sum Monotone Inclusions

Machine learning approaches relying on such criteria as adversarial robustness or multi-agent settings have raised the need for solving game-theoretic equilibrium problems. Of particular relevance to these applications are methods targeting finite-sum structure, which generically arises in empirical variants of learning problems in these contexts. Further, methods with computable approximation errors are highly desirable, as they provide verifiable exit criteria. Motivated by these applications, we study finite-sum monotone inclusion problems, which model broad classes of equilibrium problems. Our main contributions are variants of the classical Halpern iteration that employ variance reduction to obtain improved complexity guarantees in which $n$ component operators in the finite sum are ``on average'' either cocoercive or Lipschitz continuous and monotone, with parameter $L$. The resulting oracle complexity of our methods, which provide guarantees for the last iterate and for a (computable) operator norm residual, is $\widetilde{\mathcal{O}}( n + \sqrt{n}L\varepsilon^{-1})$, which improves upon existing methods by a factor up to $\sqrt{n}$. This constitutes the first variance reduction-type result for general finite-sum monotone inclusions and for more specific problems such as convex-concave optimization when operator norm residual is the optimality measure. We further argue that, up to poly-logarithmic factors, this complexity is unimprovable in the monotone Lipschitz setting; i.e., the provided result is near-optimal.
USB-NeRF: Unrolling Shutter Bundle Adjusted Neural Radiance Fields

Neural Radiance Fields (NeRF) has received much attention recently due to its impressive capability to represent 3D scene and synthesize novel view images. Existing works usually assume that the input images are captured by a global shutter camera. Thus, rolling shutter (RS) images cannot be trivially applied to an off-the-shelf NeRF algorithm for novel view synthesis. Rolling shutter effect would also affect the accuracy of the camera pose estimation (e.g. via COLMAP), which further prevents the success of NeRF algorithm with RS images. In this paper, we propose Unrolling Shutter Bundle Adjusted Neural Radiance Fields (USB-NeRF). USB-NeRF is able to correct rolling shutter distortions and recover accurate camera motion trajectory simultaneously under the framework of NeRF, by modeling the physical image formation process of a RS camera. Experimental results demonstrate that USB-NeRF achieves better performance compared to prior works, in terms of RS effect removal, novel view image synthesis as well as camera motion estimation. Furthermore, our algorithm can also be used to recover high-fidelity high frame-rate global shutter video from a sequence of RS images.
Tackling Hybrid Heterogeneity on Federated Optimization via Gradient Diversity Maximization

Federated learning refers to a distributed machine learning paradigm in which data samples are decentralized and distributed among multiple clients. These samples may exhibit statistical heterogeneity, which refers to data distributions are not independent and identical across clients. Additionally, system heterogeneity, or variations in the computational power of the clients, introduces biases into federated learning. The combined effects of statistical and system heterogeneity can significantly reduce the efficiency of federated optimization. However, the impact of hybrid heterogeneity is not rigorously discussed. This paper explores how hybrid heterogeneity affects federated optimization by investigating server-side optimization. The theoretical results indicate that adaptively maximizing gradient diversity in server update direction can help mitigate the potential negative consequences of hybrid heterogeneity. To this end, we introduce a novel server-side gradient-based optimizer \textsc{FedAWARE} with theoretical guarantees provided. Intensive experiments in heterogeneous federated settings demonstrate that our proposed optimizer can significantly enhance the performance of federated learning across varying degrees of hybrid heterogeneity.
Efficient similar waveform search using short binary codes obtained through a deep hashing technique

A similar waveform search plays a crucial role in seismology for detecting seismic events, such as small earthquakes and low-frequency events. However, the high computational costs associated with waveform cross-correlation calculations represent bottlenecks during the analysis of long, continuous records obtained from numerous stations. In this study, we developed a deep-learning network to obtain 64-bit hash codes containing information on seismic waveforms. Using this network, we performed a similar waveform search for ~35 million moving windows developed for the 30 min waveforms recorded continuously at 10 MHz sampling rates using 16 acoustic emission transducers during a laboratory hydraulic fracturing experiment. The sampling points of each channel corresponded to those of the 5.8-year records obtained from typical seismic observations at 100 Hz sampling rates. Of the 35 million windows, we searched for windows with small average Hamming distances among the hash codes of 16 channel waveforms against template hash codes of 6057 events that were catalogued using conventional autoprocessing techniques. The calculation of average Hamming distances is 1000 times faster than that of the corresponding network correlation. This hashing-based template matching enabled the detection of 23,462 additional events. We also demonstrated the feasibility of the hashing-based autocorrelation analysis, where similar event pairs were extracted without templates, by calculating the average Hamming distances for all possible pairs of the ~35 million windows. This calculation required only 15.5 h under 120 thread parallelisation. This deep hashing approach significantly reduced the required memory compared with locality-sensitive hashing approaches based on random permutations, enabling similar waveform searching on a large-scale dataset.
Reward Model Ensembles Help Mitigate Overoptimization

Reinforcement learning from human feedback (RLHF) is a standard approach for fine-tuning large language models to follow instructions. As part of this process, learned reward models are used to approximately model human preferences. However, as imperfect representations of the"true"reward, these learned reward models are susceptible to overoptimization. Gao et al. (2023) studied this phenomenon in a synthetic human feedback setup with a significantly larger"gold"reward model acting as the true reward (instead of humans) and showed that overoptimization remains a persistent problem regardless of the size of the proxy reward model and training data used. Using a similar setup, we conduct a systematic study to evaluate the efficacy of using ensemble-based conservative optimization objectives, specifically worst-case optimization (WCO) and uncertainty-weighted optimization (UWO), for mitigating reward model overoptimization when using two optimization methods: (a) best-of-n sampling (BoN) (b) proximal policy optimization (PPO). We additionally extend the setup of Gao et al. (2023) to include 25% label noise to better mirror real-world conditions. Both with and without label noise, we find that conservative optimization practically eliminates overoptimization and improves performance by up to 70% for BoN sampling. For PPO, ensemble-based conservative optimization always reduces overoptimization and outperforms single reward model optimization. Moreover, combining it with a small KL penalty successfully prevents overoptimization at no performance cost. Overall, our results demonstrate that ensemble-based conservative optimization can effectively counter overoptimization.
Identifying predictors of glioma evolution from longitudinal sequencing

Clonal evolution drives cancer progression and therapeutic resistance. Recent studies have revealed divergent longitudinal trajectories in gliomas, but early molecular features steering posttreatment cancer evolution remain unclear. Here, we collected sequencing and clinical data of initial-recurrent tumor pairs from 544 adult diffuse gliomas and performed multivariate analysis to identify early molecular predictors of tumor evolution in three diffuse glioma subtypes. We found that CDKN2A deletion at initial diagnosis preceded tumor necrosis and microvascular proliferation that occur at later stages of IDH-mutant glioma. Ki67 expression at diagnosis was positively correlated with acquiring hypermutation at recurrence in the IDH–wild-type glioma. In all glioma subtypes, MYC gain or MYC-target activation at diagnosis was associated with treatment-induced hypermutation at recurrence. To predict glioma evolution, we constructed CELLO2 (Cancer EvoLution for LOngitudinal data version 2), a machine learning model integrating features at diagnosis to forecast hypermutation and progression after treatment. CELLO2 successfully stratified patients into subgroups with distinct prognoses and identified a high-risk patient group featured by MYC gain with worse post-progression survival, from the low-grade IDH-mutant-noncodel subtype. We then performed chronic temozolomide-induction experiments in glioma cell lines and isogenic patient-derived gliomaspheres and demonstrated that MYC drives temozolomide resistance by promoting hypermutation. Mechanistically, we demonstrated that, by binding to open chromatin and transcriptionally active genomic regions, c-MYC increases the vulnerability of key mismatch repair genes to treatment-induced mutagenesis, thus triggering hypermutation. This study reveals early predictors of cancer evolution under therapy and provides a resource for precision oncology targeting cancer dynamics in diffuse gliomas. Description Machine learning of 544 initial-recurrent glioma pairs identified c-MYC as an early predictor of hypermutation and histological grade progression. Editor’s summary Diffuse gliomas are the most common primary brain tumors in adults, and the underlying molecular mechanism of its evolution is unknown. To shed light on this, Mu et al. collected 544 matching initial and recurrent samples from patients with diffuse glioma and performed whole-genome sequencing to evaluate correlations to progression. They identified that MYC gain or activation at diagnosis correlated to the worst patient prognosis and was able to drive temozolomide resistance in vitro. These data provide a resource that can lead to potential targets for therapy to overcome these therapeutic limitations in diffuse gliomas. —Dorothy Hallberg
COOLer: Class-Incremental Learning for Appearance-Based Multiple Object Tracking

Continual learning allows a model to learn multiple tasks sequentially while retaining the old knowledge without the training data of the preceding tasks. This paper extends the scope of continual learning research to class-incremental learning for multiple object tracking (MOT), which is desirable to accommodate the continuously evolving needs of autonomous systems. Previous solutions for continual learning of object detectors do not address the data association stage of appearance-based trackers, leading to catastrophic forgetting of previous classes' re-identification features. We introduce COOLer, a COntrastive- and cOntinual-Learning-based tracker, which incrementally learns to track new categories while preserving past knowledge by training on a combination of currently available ground truth labels and pseudo-labels generated by the past tracker. To further exacerbate the disentanglement of instance representations, we introduce a novel contrastive class-incremental instance representation learning technique. Finally, we propose a practical evaluation protocol for continual learning for MOT and conduct experiments on the BDD100K and SHIFT datasets. Experimental results demonstrate that COOLer continually learns while effectively addressing catastrophic forgetting of both tracking and detection. The code is available at https://github.com/BoSmallEar/COOLer.
Hybrid Inception Architecture with Residual Connection: Fine-tuned Inception-ResNet Deep Learning Model for Lung Inflammation Diagnosis from Chest Radiographs

Diagnosing lung inflammation, particularly pneumonia, is of paramount importance for effectively treating and managing the disease. Pneumonia is a common respiratory infection caused by bacteria, viruses, or fungi and can indiscriminately affect people of all ages. As highlighted by the World Health Organization (WHO), this prevalent disease tragically accounts for a substantial 15% of global mortality in children under five years of age. This article presents a comparative study of the Inception-ResNet deep learning model's performance in diagnosing pneumonia from chest radiographs. The study leverages Mendeleys chest X-ray images dataset, which contains 5856 2D images, including both Viral and Bacterial Pneumonia X-ray images. The Inception-ResNet model is compared with seven other state-of-the-art convolutional neural networks (CNNs), and the experimental results demonstrate the Inception-ResNet model's superiority in extracting essential features and saving computation runtime. Furthermore, we examine the impact of transfer learning with fine-tuning in improving the performance of deep convolutional models. This study provides valuable insights into using deep learning models for pneumonia diagnosis and highlights the potential of the Inception-ResNet model in this field. In classification accuracy, Inception-ResNet-V2 showed superior performance compared to other models, including ResNet152V2, MobileNet-V3 (Large and Small), EfficientNetV2 (Large and Small), InceptionV3, and NASNet-Mobile, with substantial margins. It outperformed them by 2.6%, 6.5%, 7.1%, 13%, 16.1%, 3.9%, and 1.6%, respectively, demonstrating its significant advantage in accurate classification.
Quantum algorithms: A survey of applications and end-to-end complexities

The anticipated applications of quantum computers span across science and industry, ranging from quantum chemistry and many-body physics to optimization, finance, and machine learning. Proposed quantum solutions in these areas typically combine multiple quantum algorithmic primitives into an overall quantum algorithm, which must then incorporate the methods of quantum error correction and fault tolerance to be implemented correctly on quantum hardware. As such, it can be difficult to assess how much a particular application benefits from quantum computing, as the various approaches are often sensitive to intricate technical details about the underlying primitives and their complexities. Here we present a survey of several potential application areas of quantum algorithms and their underlying algorithmic primitives, carefully considering technical caveats and subtleties. We outline the challenges and opportunities in each area in an"end-to-end"fashion by clearly defining the problem being solved alongside the input-output model, instantiating all"oracles,"and spelling out all hidden costs. We also compare quantum solutions against state-of-the-art classical methods and complexity-theoretic limitations to evaluate possible quantum speedups. The survey is written in a modular, wiki-like fashion to facilitate navigation of the content. Each primitive and application area is discussed in a standalone section, with its own bibliography of references and embedded hyperlinks that direct to other relevant sections. This structure mirrors that of complex quantum algorithms that involve several layers of abstraction, and it enables rapid evaluation of how end-to-end complexities are impacted when subroutines are altered.