Miracle-dz/UniFAR-FTUs huggingface dataset:
Straggler-Resilient Decentralized Learning via Adaptive Asynchronous Updates

With the increasing demand for large-scale training of machine learning models, fully decentralized optimization methods have recently been advocated as alternatives to the popular parameter server framework. In this paradigm, each worker maintains a local estimate of the optimal parameter vector, and iteratively updates it by waiting and averaging all estimates obtained from its neighbors, and then corrects it on the basis of its local dataset. However, the synchronization phase is sensitive to stragglers. An efficient way to mitigate this effect is to consider asynchronous updates, where each worker computes stochastic gradients and communicates with other workers at its own pace. Unfortunately, fully asynchronous updates suffer from staleness of the stragglers' parameters. To address these limitations, we propose a fully decentralized algorithm DSGD-AAU with adaptive asynchronous updates via adaptively determining the number of neighbor workers for each worker to communicate with. We show that DSGD-AAU achieves a linear speedup for convergence (i.e., convergence performance increases linearly with respect to the number of workers). Experimental results on a suite of datasets and deep neural network models are provided to verify our theoretical results.
Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Compute

Self-supervised learning (SSL) has led to great strides in speech processing. However, the resources needed to train these models has become prohibitively large as they continue to scale. Currently, only a few groups with substantial resources are capable of creating SSL models, which harms reproducibility. In this work, we optimize HuBERT SSL to fit in academic constraints. We reproduce HuBERT independently from the original implementation, with no performance loss. Our code and training optimizations make SSL feasible with only 8 GPUs, instead of the 32 used in the original work. We also explore a semi-supervised route, using an ASR model to skip the first pre-training iteration. Within one iteration of pre-training, our models improve over HuBERT on several tasks. Furthermore, our HuBERT Large variant requires only 8 GPUs, achieving similar performance to the original trained on 128. As our contribution to the community, all models, configurations, and code are made open-source in ESPnet.
Triple crossing positivity bounds, mass dependence and cosmological scalars: Horndeski theory and DHOST

Scalars are widely used in cosmology to model novel phenomena such as the late-time cosmic acceleration. These are effective field theories with highly nonlinear interactions, including Horndeski theory/generalized galileon and beyond. We use the latest fully crossing symmetric positivity bounds to constrain these cosmological EFTs. These positivity bounds, based on fundamental principles of quantum field theory such as causality and unitarity, are able to constrain the EFT coefficients both from above and below. We first map the mass dependence of the fully crossing symmetric bounds, and find that a nonzero mass generically enlarges the positivity regions. We show that fine-tunings in the EFT construction can significantly reduce the viable regions and sometimes can be precarious. Then, we apply the positivity bounds to several models in the Horndeski class and beyond, explicitly listing the ready-to-use bounds with the model parameters, and discuss the implications for these models. The new positivity bounds are found to severely constrain some of these models, in which positivity requires the mass to be parametrically close to the cutoff of the EFT, effectively ruling them out. The examples include massive galileon, the original beyond Horndeski model, and DHOST theory with unity speed of gravity and nearly constant Newton's coupling.
Silent cerebral lesions following catheter ablation for atrial fibrillation: a state-of-the-art review

Abstract Atrial fibrillation is associated with neurocognitive comorbidities such as stroke and dementia. Evidence suggests that rhythm control—especially if implemented early—may reduce the risk of cognitive decline. Catheter ablation is highly efficacious for restoring sinus rhythm in the setting of atrial fibrillation; however, ablation within the left atrium has been shown to result in MRI-detected silent cerebral lesions. In this state-of-the-art review article, we discuss the balance of risk between left atrial ablation and rhythm control. We highlight suggestions to lower the risk, as well as the evidence behind newer forms of ablation such as very high power short duration radiofrequency ablation and pulsed field ablation.
ADE SUBALGEBRAS OF THE TRIPLET VERTEX ALGEBRA $\mathcal{W}(p)$: A-SERIES

Motivated by [On the triplet vertex algebra , Adv. Math.217 (2008) 2664–2699], for every finite subgroup Γ ⊂ PSL(2, ℂ) we investigate the fixed point subalgebra of the triplet vertex , of central charge , p ≥ 2. This part deals with the A-series in the ADE classification of finite subgroups of PSL(2, ℂ). First, we prove the C2-cofiniteness of the Am-fixed subalgebra . Then we construct a family of -modules, which are expected to form a complete set of irreducible representations. As a strong support to our conjecture, we prove modular invariance of (generalized) characters of the relevant (logarithmic) modules. Further evidence is provided by calculations in Zhu's algebra for m = 2. We also present a rigorous proof of the fact that the full automorphism group of is PSL(2, ℂ).
The ligation of CD4 molecules, expressed on monocytes by an anti-CD4 monoclonal antibody, inhibits T cell activation and monocyte mobility.

BACKGROUND CD4, a leukocyte surface glycoprotein, is mainly expressed on CD4+ T cells, but is also expressed on monocytes. The difference in the expression level and structure of CD4 on T cells and monocytes predicts the different functions of this molecule in both cell types. Although the function of CD4 on T cells is well characterized, little is known about that expressed on primary monocytes. OBJECTIVE In this study, we investigated the immunoregulation function of CD4 on peripheral blood monocytes. METHODS Methods: CD4 molecule on monocyte was ligated by anti-CD4 monoclonal antibody (mAb), MT4/3. The effect of mAb MT4/3 on T cell proliferation, cytokine production, the expression of monocyte costimulatory molecules, monocyte migration, and macrophage differentiation were investigated. Moreover, the molecular weight of CD4 on peripheral blood monocyte was carried out by Western immunoblotting. RESULTS We demonstrated that mAb MT4/3 inhibited anti-CD3 induced T cell proliferation, cytokine production, and the expression of monocyte costimulatory molecules. The ligation of only CD4 on monocytes was sufficient to inhibit T cell activation. Moreover, mAb MT4/3 could inhibit monocyte migration in a transwell migration assay, but not affect the differentiation of monocytes to macrophages. Using purified primary monocytes, the molecular weight of CD4 expressed on monocytes was identified as 55 kDa. CONCLUSIONS The CD4 molecule expressed on monocytes might play an important role in the regulation of immune responses in both innate and adaptive immunity. Understanding the novel role of CD4 on monocytes in immunoregulation is valuable in the development of new therapeutic approaches.
The Haar theorem for lattice-ordered abelian groups with order-unit

Given a finite union $P$ of rational simplexes, we assign to $P$ numerical invariants $\lambda_{0}, \lambda_{1},\ldots,\lambda_{\dim P};$ each $\lambda_{i}$ is the suitably normalized volume of the $i$-dimensional part of $P$. We then prove that every finitely generated projective lattice-ordered abelian group $G$ with order-unit $u$ has a faithful invariant positive linear functional $s: G\to \mathbb R$. For each $g\in G$, $s(g)$ is the integral of $g$ over the maximal spectrum of $G$, the latter being canonically identified with a rational polyhedron $P$. Volume elements are measured by the $\lambda_{i}$'s. The proof uses the polyhedral versions of the Wlodarczyk-Morelli theorem on decompositions of birational toric maps in blow-ups and blow-downs, and of the De Concini-Procesi theorem on elimination of points of indeterminacy.
A proof that convection in a porous vertical slab may be unstable

The stability of the natural convection parallel flow in a vertical porous slab is reconsidered, by reformulating the problem originally solved in Gill’s classical paper of 1969 (J. Fluid Mech., vol. 35, pp. 545–547). A comparison is made between the set of boundary conditions where the slab boundaries are assumed to be isothermal and impermeable (Model A), and the set of boundary conditions where the boundaries are modelled as isothermal and permeable (Model B). It is shown that Gill’s proof of linear stability for Model A cannot be extended to Model B. The question about the stability/instability of the basic flow is examined by carrying out a numerical solution of the stability eigenvalue problem. It is shown that, with Model B, the natural convection parallel flow in the basic state becomes unstable when the Darcy–Rayleigh number is larger than 197.081. The normal modes selected at onset of instability are transverse rolls. Direct numerical simulations of the nonlinear regime of instability are carried out.
DVFO: Learning-Based DVFS for Energy-Efficient Edge-Cloud Collaborative Inference

Due to limited resources on edge and different characteristics of deep neural network (DNN) models, it is a big challenge to optimize DNN inference performance in terms of energy consumption and end-to-end latency on edge devices. In addition to the dynamic voltage frequency scaling (DVFS) technique, the edge-cloud architecture provides a collaborative approach for efficient DNN inference. However, current edge-cloud collaborative inference methods have not optimized various compute resources on edge devices. Thus, we propose DVFO, a novel DVFS-enabled edge-cloud collaborative inference framework, which co-optimizes DVFS and offloading parameters via deep reinforcement learning (DRL). Specifically, DVFO automatically co-optimizes 1) the CPU, GPU and memory frequencies of edge devices, and 2) the feature maps to be offloaded to cloud servers. In addition, it leverages a thinking-while-moving concurrent mechanism to accelerate the DRL learning process, and a spatial-channel attention mechanism to extract DNN feature maps of secondary importance for workload offloading. This approach improves inference performance for different DNN models under various edge-cloud network conditions. Extensive evaluations using two datasets and six widely-deployed DNN models on three heterogeneous edge devices show that DVFO significantly reduces the energy consumption by 33% on average, compared to state-of-the-art schemes. Moreover, DVFO achieves up to 28.6%-59.1% end-to-end latency reduction, while maintaining accuracy within 1% loss on average.
Operator Learning with Neural Fields: Tackling PDEs on General Geometries

Machine learning approaches for solving partial differential equations require learning mappings between function spaces. While convolutional or graph neural networks are constrained to discretized functions, neural operators present a promising milestone toward mapping functions directly. Despite impressive results they still face challenges with respect to the domain geometry and typically rely on some form of discretization. In order to alleviate such limitations, we present CORAL, a new method that leverages coordinate-based networks for solving PDEs on general geometries. CORAL is designed to remove constraints on the input mesh, making it applicable to any spatial sampling and geometry. Its ability extends to diverse problem domains, including PDE solving, spatio-temporal forecasting, and inverse problems like geometric design. CORAL demonstrates robust performance across multiple resolutions and performs well in both convex and non-convex domains, surpassing or performing on par with state-of-the-art models.
EFdA efficiently suppresses HIV replication in the male genital tract and prevents penile HIV acquisition

ABSTRACT Sexually transmitted HIV infections in heterosexual men are acquired through the penis. Low adherence to condom usage and the fact that 40% of circumcised men are not protected indicate the need for additional prevention strategies. Here, we describe a new approach to evaluate the prevention of penile HIV transmission. We demonstrated that the entire male genital tract (MGT) of bone marrow/liver/thymus (BLT) humanized mice is repopulated with human T and myeloid cells. The majority of the human T cells in the MGT express CD4 and CCR5. Direct penile exposure to HIV leads to systemic infection including all tissues of the MGT. HIV replication throughout the MGT was reduced 100-1,000-fold by treatment with 4′-ethynyl-2-fluoro-2′-deoxyadenosine (EFdA), resulting in the restoration of CD4+ T cell levels. Importantly, systemic preexposure prophylaxis with EFdA effectively protects from penile HIV acquisition. IMPORTANCE Over 84.2 million people have been infected by the human immunodeficiency virus type 1 (HIV-1) during the past 40 years, most through sexual transmission. Men comprise approximately half of the HIV-infected population worldwide. Sexually transmitted HIV infections in exclusively heterosexual men are acquired through the penis. However, direct evaluation of HIV infection throughout the human male genital tract (MGT) is not possible. Here, we developed a new in vivo model that permits, for the first time, the detail analysis of HIV infection. Using BLT humanized mice, we showed that productive HIV infection occurs throughout the entire MGT and induces a dramatic reduction in human CD4 T cells compromising immune responses in this organ. Antiretroviral treatment with novel drug EFdA suppresses HIV replication in all tissues of the MGT, restores normal levels of CD4 T cells and is highly efficient at preventing penile transmission. Over 84.2 million people have been infected by the human immunodeficiency virus type 1 (HIV-1) during the past 40 years, most through sexual transmission. Men comprise approximately half of the HIV-infected population worldwide. Sexually transmitted HIV infections in exclusively heterosexual men are acquired through the penis. However, direct evaluation of HIV infection throughout the human male genital tract (MGT) is not possible. Here, we developed a new in vivo model that permits, for the first time, the detail analysis of HIV infection. Using BLT humanized mice, we showed that productive HIV infection occurs throughout the entire MGT and induces a dramatic reduction in human CD4 T cells compromising immune responses in this organ. Antiretroviral treatment with novel drug EFdA suppresses HIV replication in all tissues of the MGT, restores normal levels of CD4 T cells and is highly efficient at preventing penile transmission.
PUF60‐related developmental disorder: A case series and phenotypic analysis of 10 additional patients with monoallelic PUF60 variants

PUF60‐related developmental disorder (also referred to as Verheij syndrome), resulting from haploinsufficiency of PUF60, is associated with multiple congenital anomalies affecting a wide range of body systems. These anomalies include ophthalmic coloboma, and congenital anomalies of the heart, kidney, and musculoskeletal system. Behavioral and intellectual difficulties are also observed. While less common than other features associated with PUF60‐related developmental disorder, for instance hearing impairment and short stature, identification of specific anomalies such as ophthalmic coloboma can aid with diagnostic identification given the limited spectrum of genes linked with this feature. We describe 10 patients with PUF60 gene variants, bringing the total number reported in the literature, to varying levels of details, to 56 patients. Patients were recruited both via locally based exome sequencing from international sites and from the DDD study in the United Kingdom. Eight of the variants reported were novel PUF60 variants. The addition of a further patient with a reported c449‐457del variant to the existing literature highlights this as a recurrent variant. One variant was inherited from an affected parent. This is the first example in the literature of an inherited variant resulting in PUF60‐related developmental disorder. Two patients (20%) were reported to have a renal anomaly consistent with 22% of cases in previously reported literature. Two patients received specialist endocrine treatment. More commonly observed were clinical features such as: cardiac anomalies (40%), ocular abnormalities (70%), intellectual disability (60%), and skeletal abnormalities (80%). Facial features did not demonstrate a recognizable gestalt. Of note, but remaining of unclear causality, we describe a single pediatric patient with pineoblastoma. We recommend that stature and pubertal progress should be monitored in PUF60‐related developmental disorder with a low threshold for endocrine investigations as hormone therapy may be indicated. Our study reports an inherited case with PUF60‐related developmental disorder which has important genetic counseling implications for families.
PoET: A generative model of protein families as sequences-of-sequences

Generative protein language models are a natural way to design new proteins with desired functions. However, current models are either difficult to direct to produce a protein from a specific family of interest, or must be trained on a large multiple sequence alignment (MSA) from the specific family of interest, making them unable to benefit from transfer learning across families. To address this, we propose $\textbf{P}$r$\textbf{o}$tein $\textbf{E}$volutionary $\textbf{T}$ransformer (PoET), an autoregressive generative model of whole protein families that learns to generate sets of related proteins as sequences-of-sequences across tens of millions of natural protein sequence clusters. PoET can be used as a retrieval-augmented language model to generate and score arbitrary modifications conditioned on any protein family of interest, and can extrapolate from short context lengths to generalize well even for small families. This is enabled by a unique Transformer layer; we model tokens sequentially within sequences while attending between sequences order invariantly, allowing PoET to scale to context lengths beyond those used during training. In extensive experiments on deep mutational scanning datasets, we show that PoET outperforms existing protein language models and evolutionary sequence models for variant function prediction across proteins of all MSA depths. We also demonstrate PoET's ability to controllably generate new protein sequences.
Modeling the impact of thermal stresses induced by wellbore cooldown on the breakdown pressure and geometry of a hydraulic fracture

Wellbore cooldown is often employed before well stimulation and/or hydraulic fracture stress testing in EGS and high temperature petroleum reservoirs. The thermo-elastic stress resulting from heat conduction during the cooling activity can have important influence on the behavior of the hydraulic fractures. A coupled numerical model has been developed to study the thermo-mechanical effect associated with pre-injection wellbore cooldown on the wellbore pressure and geometry of the hydraulic fracture (either longitudinal or transverse to the wellbore axis). The main novelty of this numerical study is the consideration of significant near-wellbore thermal stresses in the coupled non-linear problem of hydraulic fracturing initiation and propagation, which enables investigation of the thermo-mechanical effect under different fracture propagation regimes. Simulation results show earlier fracture initiation and lower breakdown pressure caused by cooling circulation. Extensive wellbore cooling also significantly alters the evolution of wellbore pressure, as evidenced by the differences observed under various cooling conditions. Most importantly, cooling promotes the transverse initiation of hydraulic fractures in situations where the initiation would have been longitudinal in the absence of cooling. The cases most susceptible to the complete change of fracture initiation geometry are those horizontal wells drilled parallel to the minimum horizontal stress but also applicable to vertical wells in cases where the vertical stress is the lower in magnitude than either horizontal principal stress. These results combine to indicate a profound potential for cooling to impact hydraulic fracture initiation and early growth, and therefore needs to be considered in the planning and interpretation of stress testing and reservoir stimulation when cooling operations are necessary.
Causal Inference via Style Transfer for Out-of-distribution Generalisation

Out-of-distribution (OOD) generalisation aims to build a model that can generalise well on an unseen target domain using knowledge from multiple source domains. To this end, the model should seek the causal dependence between inputs and labels, which may be determined by the semantics of inputs and remain invariant across domains. However, statistical or non-causal methods often cannot capture this dependence and perform poorly due to not considering spurious correlations learnt from model training via unobserved confounders. A well-known existing causal inference method like back-door adjustment cannot be applied to remove spurious correlations as it requires the observation of confounders. In this paper, we propose a novel method that effectively deals with hidden confounders by successfully implementing front-door adjustment (FA). FA requires the choice of a mediator, which we regard as the semantic information of images that helps access the causal mechanism without the need for observing confounders. Further, we propose to estimate the combination of the mediator with other observed images in the front-door formula via style transfer algorithms. Our use of style transfer to estimate FA is novel and sensible for OOD generalisation, which we justify by extensive experimental results on widely used benchmark datasets.
FLSL: Feature-level Self-supervised Learning

Current self-supervised learning (SSL) methods (e.g., SimCLR, DINO, VICReg,MOCOv3) target primarily on representations at instance level and do not generalize well to dense prediction tasks, such as object detection and segmentation.Towards aligning SSL with dense predictions, this paper demonstrates for the first time the underlying mean-shift clustering process of Vision Transformers (ViT), which aligns well with natural image semantics (e.g., a world of objects and stuffs). By employing transformer for joint embedding and clustering, we propose a two-level feature clustering SSL method, coined Feature-Level Self-supervised Learning (FLSL). We present the formal definition of the FLSL problem and construct the objectives from the mean-shift and k-means perspectives. We show that FLSL promotes remarkable semantic cluster representations and learns an embedding scheme amenable to intra-view and inter-view feature clustering. Experiments show that FLSL yields significant improvements in dense prediction tasks, achieving 44.9 (+2.8)% AP and 46.5% AP in object detection, as well as 40.8 (+2.3)% AP and 42.1% AP in instance segmentation on MS-COCO, using Mask R-CNN with ViT-S/16 and ViT-S/8 as backbone, respectively. FLSL consistently outperforms existing SSL methods across additional benchmarks, including UAV17 object detection on UAVDT, and video instance segmentation on DAVIS 2017.We conclude by presenting visualization and various ablation studies to better understand the success of FLSL. The source code is available at https://github.com/ISL-CV/FLSL.