Miracle-dz/UniFAR-FTUs huggingface dataset:
Convergence Rates of Non-Convex Stochastic Gradient Descent Under a Generic Lojasiewicz Condition and Local Smoothness

Training over-parameterized neural networks involves the empirical minimization of highly non-convex objective functions. Recently, a large body of works provided theoretical evidence that, despite this non-convexity, properly initialized over-parameterized networks can converge to a zero training loss through the introduction of the Polyak-Łojasiewicz condition. However, these analyses are restricted to quadratic losses such as mean square error , and tend to indicate fast exponential convergence rates that are seldom observed in practice. In this work, we propose to extend these results by analyzing stochastic gradient descent under more generic Łojasiewicz conditions that are applicable to any convex loss function, thus extending the current theory to a larger panel of losses commonly used in practice such as cross-entropy. Moreover, our analysis provides high-probability bounds on the approximation error under sub-Gaussian gradient noise and only requires the local smoothness of the objective function, thus making it applicable to deep neural networks in realistic settings.
Distributionally-Aware Kernelized Bandit Problems for Risk Aversion

The kernelized bandit problem is a theoretically justiﬁed framework and has solid applications to various ﬁelds. Recently, there is a growing interest in generalizing the problem to the optimization of risk-averse metrics such as Conditional Value-at-Risk (CVaR) or Mean-Variance (MV). However, due to the model assumption, most existing meth-ods need explicit design of environment random variables and can incur large regret because of possible high dimensionality of them. To address the issues, in this paper, we model the environment using a family of the output distributions (or more precisely, probability kernel) and Kernel Mean Embeddings (KME), and provide novel UCB-type algorithms for CVaR and MV. More-over, we provide algorithm-independent lower bounds for CVaR in the case of Mat ´ ern kernels, and propose a nearly optimal algorithm. Furthermore, we empirically verify our theoretical result in synthetic environments, and demonstrate that our proposed method signiﬁcantly outperforms a baseline in many cases.
ButterflyFlow: Building Invertible Layers with Butterfly Matrices

Normalizing flows model complex probability distributions using maps obtained by composing invertible layers. Special linear layers such as masked and 1x1 convolutions play a key role in existing architectures because they increase expressive power while having tractable Jacobians and inverses. We propose a new family of invertible linear layers based on butterfly layers, which are known to theoretically capture complex linear structures including permutations and periodicity, yet can be inverted efficiently. This representational power is a key advantage of our approach, as such structures are common in many real-world datasets. Based on our invertible butterfly layers, we construct a new class of normalizing flow models called ButterflyFlow. Empirically, we demonstrate that ButterflyFlows not only achieve strong density estimation results on natural images such as MNIST, CIFAR-10, and ImageNet 32x32, but also obtain significantly better log-likelihoods on structured datasets such as galaxy images and MIMIC-III patient cohorts -- all while being more efficient in terms of memory and computation than relevant baselines.
Efficient Approximate Inference for Stationary Kernel on Frequency Domain

Based on the Fourier duality between a stationary kernel and its spectral density, modeling the spectral density using a Gaussian mixture density enables one to construct a flexible kernel, known as a Spectral Mixture kernel, that can model any stationary kernel. However, despite its expressive power, training this kernel is typically difficult because scalability and overfitting issues often arise due to a large number of training parameters. To resolve these issues, we propose an approximate inference method for estimating the Spectral mixture kernel hyperparameters. Specifically, we approximate this kernel by using the finite random spectral points based on Random Fourier Feature and optimize the parameters for the distribution of spectral points by sampling-based variational inference. To improve this inference procedure, we analyze the training loss and propose two special methods: a sampling method of spectral points to reduce the error of the approximate kernel in training, and an approximate natural gradient to accelerate the convergence of parameter inference.
Unsupervised Detection of Contextualized Embedding Bias with Application to Ideology

We propose a fully unsupervised method to detect bias in contextualized embeddings. The method leverages the assortative information latently encoded by social networks and combines orthogonality regularization, structured sparsity learning, and graph neural networks to find the embedding subspace capturing this information. As a concrete example, we focus on the phenomenon of ideological bias: we introduce the concept of an ideological subspace, show how it can be found by applying our method to online discussion forums, and present techniques to probe it. Our experiments suggest that the ideological subspace encodes abstract evaluative semantics and reflects changes in the political left-right spectrum during the presidency of Donald Trump.
Counterfactual Transportability: A Formal Approach

Generalizing causal knowledge across environments is a common challenge shared across many of the data-driven disciplines, including AI and ML. Experiments are usually performed in one environment (e.g., in a lab, on Earth, in a train-ing ground), almost invariably, with the intent of being used elsewhere (e.g., outside the lab, on Mars, in the real world), in an environment that is related but somewhat different than the original one, where certain conditions and mechanisms are likely to change. This generalization task has been studied in the causal inference literature under the rubric of transportability (Pearl and Bareinboim, 2011). While most transportability works focused on generalizing associational and interventional distributions, the generalization of counterfactual distributions has not been formally studied. In this paper, we investigate the transportability of counterfactuals from an arbitrary combination of observational and experimental distributions coming from disparate domains. Speciﬁcally, we introduce a sufﬁcient and necessary graphical condition and develop an efﬁcient, sound, and complete algorithm for transporting counterfactual quantities across domains in nonparametric settings. Failure of the algorithm implies the impossibility of generalizing the target counterfactual from the available data without further assumptions.
A modular platform for bioluminescent RNA tracking

A complete understanding of RNA biology requires methods for tracking transcripts in vivo. Common strategies rely on fluorogenic probes that are limited in sensitivity, dynamic range, and depth of interrogation, owing to their need for excitation light and tissue autofluorescence. To overcome these challenges, we developed a bioluminescent platform for serial imaging of RNAs. Small RNA tags were engineered to recruit light-emitting luciferase fragments (termed RNA lanterns) upon transcription. Robust photon production was observed for RNA targets both in cells and in live animals. Importantly, only a single copy of the tag was necessary for sensitive detection, in sharp contrast to fluorescent platforms requiring multiple repeats. Overall, this work provides a foundational platform for visualizing RNA dynamics from the micro to the macro scale.
Specific deletion of Axin1 leads to activation of β-catenin/BMP signaling resulting in fibular hemimelia phenotype in mice

Axin1 and Axin2 are key regulators of canonical Wnt signaling pathway. The roles of Axin1 and Axin2 in skeletal development and in disease development have not been fully defined. Here, we reported that Axin1 and Axin2 are essential for lower limb development. Specific deletion of Axin1 in limb mesenchymal cells leads to fibular hemimelia (FH)-like phenotype, associated with tarsal coalition. Further studies demonstrated that FH disease is associated with additional defects resembling to the proximal femoral focal deficiency (PFFD) in Axin1/Axin2 double knockout (KO) mice. We then provided in vivo evidence showing that Axin1 controls limb development through both canonical β-catenin and BMP signaling pathways. We demonstrated that inhibition of β-catenin or BMP signaling could significantly reverse the FH phenotype in mice. Together, our findings revealed that integration of Wnt and BMP signaling by Axin1 is required for lower limb development. Defects in Axin1 and Axin2 signaling could lead to the development of FH disease.
GIP receptor reduces osteoclast activity and improves osteoblast survival by activating multiple signaling pathways

Bone is a dynamic tissue that is remodeled throughout life by bone resorbing osteoclasts and bone forming osteoblasts, to adapt to physiological or mechanical demands. These processes are impaired in osteoporosis, and understanding how bone remodeling is regulated could improve anti-osteoporotic treatments. Clinical investigations show that short-term treatment with glucose-dependent insulinotropic polypeptide (GIP) acutely decreases serum markers of bone resorption and may increase bone formation. However, evidence for direct effects of GIP intracellular signaling and functions in mature human osteoclasts and osteoblasts have not been investigated. We report that the GIP receptor (GIPR) is robustly expressed in mature human osteoclasts. Exposure of osteoclasts to GIP inhibits osteoclastogenesis, delays bone resorption, and increases osteoclast apoptosis by acting upon multiple signaling pathways (cAMP, Src, Akt, calcium, p38) to impair nuclear translocation of nuclear factor of activated T cells 1 (NFATc1) and nuclear factor-κB (NFκB). Human osteoblasts also express GIPR, and GIP improves osteoblast survival via cAMP and Akt-mediated pathways. GIP treatment of co-cultures of osteoclasts and osteoblasts also decreased bone resorption. Antagonizing GIPR with GIP(3-30)NH2 abolished the effects of GIP on osteoclasts and osteoblasts. This study demonstrates that GIP inhibits bone resorption and improves survival of human osteoblasts, which could increase bone mass and strength, supporting clinical investigations of the effect of GIP on bone. Moreover, this study demonstrates that GIPR agonism could be beneficial in the treatment of disorders of bone remodeling, such as osteoporosis. One-sentence Summary GIP acts directly on bone cells to regulate bone remodeling
A Multivariate Comparison of EEG and fMRI to ECoG Using Visual Object Representations in Humans

Today most neurocognitive studies in humans employ the non-invasive neuroimaging techniques fMRI and EEG. However, how the data provided by fMRI and EEG relate exactly to the underlying neural activity remains incompletely understood. Here, we pursued this open question by relating EEG and fMRI data at the level of neural population codes using multivariate pattern analysis. For this, we recorded EEG and fMRI data while participants viewed everyday objects in different viewing conditions, and then related the data to ECoG data recorded for the same stimulus set. The comparison of EEG and ECoG data showed that object category signals emerge swiftly in the visual system and can be detected by both EEG and ECoG at similar temporal delays after stimulus onset. The correlation between EEG and ECoG reduces and onset latencies increase when object representations tolerant to changes in scale and orientation are considered. The comparison of fMRI and ECoG overall revealed a tighter relationship in occipital than in temporal regions, related to differences in fMRI signal-to-noise ratio. Together, our results reveal a complex relationship between fMRI, EEG and ECoG signals at the level of population codes that critically depends on the time point after stimulus onset, the region investigated, and the visual contents used.
Induction of Bacteriophage PinR Facilitates the Evolution of Antibiotic Resistance

A recent work reports that the single treatment of β-lactams can cause a SOS-independent superfast evolution of multi-drug resistance in the DNA repair deficiency Escherichia coli (E. coli), but the mechanism is not yet clear. Here, we find that the induction of PinR, a lambdoid prophage Rac, is involved in this process and facilitates the evolution of antibiotic resistance in DNA repair deficiency bacteria through the repression on the transcription of antioxidative genes and the thereafter ROS burst in cells. It is highlighted that the bacteriophage PinR can orchestrate the mutagenesis induced by the overaccumulation of ROS in cells. More importantly, we for the first time demonstrate that the deletion of pinR can avoid the rapid evolution of antibiotic resistance induced by either the single or long-term exposure to antibiotics, while strategies to target RecA, e.g., the inactivation on RecA, can be safely implemented to disarm the bacterial resistance to other antibiotics. Therefore, from a drug development perspective, our work suggests future studies on the “evolutionary potentiators” towards a safe and more effective strategy to be developed for infectious disease treatment.
Bayesian and evolutionary temperature dependent

10 The metabolism of all living organisms is dependent on temperature, and therefore, having a good method to predict temperature effects at a systems level is of importance. Here, we investigate the properties and robustness of a recently developed computational framework for enzyme and temperature constrained genome-scale models (etcGEM). The approach predicts the temperature dependence of an organism’s metabolic network from thermodynamic properties of the metabolic enzymes, markedly expanding the scope and applicability of constraint-based metabolic modelling. We show that the existing Bayesian algorithm for ﬁtting an etcGEM converges under a range of different priors and random seeds, yet the solutions differ both in parameter space and on a phenotypic level. We quantiﬁed the phenotypic consequences by studying the impact of different solutions on six metabolic network signature reactions. While two of these reactions showed little phenotype variation between the solutions, the remainder displayed huge variation in ﬂux-carrying capacity. Furthermore, we made software improvements to reduce the running time of the compute-intensive Bayesian algorithm by a factor of 8.5. In addition, we develop an evolutionary algorithm as an alternative to the Bayesian parameter ﬁtting algorithm. Our analysis shows that it also gives decent convergence, but it yields solutions with lower variability.
SRRM2 organizes splicing condensates to regulate alternative splicing

SRRM2 is a nuclear-speckle marker containing multiple disordered domains, whose dysfunction is associated with several human diseases. Using mainly EGFP-SRRM2 knock-in HEK293T cells, we show that SRRM2 forms biomolecular condensates satisfying most hallmarks of liquid-liquid phase separation, including spherical shape, dynamic rearrangement, coalescence, and concentration dependence supported by in vitro experiments. Live-cell imaging shows that SRRM2 organizes nuclear speckles along the cell cycle. As bona-fide splicing factor present in spliceosome structures, SRRM2 deficiency induces skipping of cassette exons with short introns and weak splice sites, tending to change large protein domains. In THP-1 myeloid-like cells, SRRM2 depletion compromises cell viability, upregulates differentiation markers, and sensitizes cells to anti-leukemia drugs. SRRM2 induces a FES splice isoform that attenuates innate inflammatory responses, and MUC1 isoforms that undergo shedding with oncogenic properties. We conclude that SRRM2 acts as a scaffold to organize nuclear speckles, regulating alternative splicing in innate immunity and cell homeostasis.
Comparative genomics of orobanchaceous species with different parasitic lifestyles reveals the origin and stepwise evolution of plant parasitism

Orobanchaceae is the largest family of parasitic plants, containing autotrophic and parasitic plants with all degrees of parasitism. This makes it by far the best family for studying the origin and evolution of plant parasitism. Here we provide three high-quality genomes of orobanchaceous plants, the autotrophic Lindenbergia luchunensis and the holoparasitic plants Phelipanche aegyptiaca and Orobanche cumana. Phylogenomic analysis of these three genomes together with those previously published and the transcriptomes of other orobanchaceous species, created a robust phylogenetic framework for Orobanchaceae. We found that an ancient whole-genome duplication (WGD; about 73.48 Mya), which occurred earlier than the origin of Orobanchaceae, might have contributed to the emergence of parasitism. However, no WGD events occurred in any lineage of orobanchaceous parasites except for Striga after divergence from their autotrophic common ancestor, suggesting that, in contrast to previous speculations, WGD is not associated with the emergence of holoparasitism. We detected evident convergent gene loss in all parasites within Orobanchaceae and between Orobanchaceae and dodder Cuscuta australis. The gene families in the orobanchaceous parasites showed a clear pattern of recent gains and expansions. The expanded gene families are enriched in functions related to the development of the haustorium, suggesting that recent gene family expansions may have facilitated the adaptation of orobanchaceous parasites to different hosts. This study illustrates a stepwise pattern in the evolution of parasitism in the orobanchaceous parasites, and will facilitate future studies on parasitism and the control of parasitic plants in agriculture.
Multidepot UAV Routing Problem with Weapon Configuration and Time Window

In recent wars, there is an increasing trend that unmanned aerial vehicles (UAVs) are utilized to conduct military attacking missions. In this paper, we investigate a novel multidepot UAV routing problem with consideration of weapon configuration in the UAV and the attacking time window of the target. A mixed-integer linear programming model is developed to jointly optimize three kinds of decisions: the weapon configuration strategy in the UAV, the routing strategy of target, and the allocation strategy of weapons to targets. An adaptive large neighborhood search (ALNS) algorithm is proposed for solving the problem, which is tested by randomly generated instances covering the small, medium, and large sizes. Experimental results confirm the effectiveness and robustness of the proposed ALNS algorithm.
Deep Learning Segmentation of the Nucleus Basalis of Meynert on 3T MRI

BACKGROUND AND PURPOSE: The nucleus basalis of Meynert is a key subcortical structure that is important in arousal and cognition and has been explored as a deep brain stimulation target but is difficult to study due to its small size, variability among patients, and lack of contrast on 3T MR imaging. Thus, our goal was to establish and evaluate a deep learning network for automatic, accurate, and patient-specific segmentations with 3T MR imaging. MATERIALS AND METHODS: Patient-specific segmentations can be produced manually; however, the nucleus basalis of Meynert is difficult to accurately segment on 3T MR imaging, with 7T being preferred. Thus, paired 3T and 7T MR imaging data sets of 21 healthy subjects were obtained. A test data set of 6 subjects was completely withheld. The nucleus was expertly segmented on 7T, providing accurate labels for the paired 3T MR imaging. An external data set of 14 patients with temporal lobe epilepsy was used to test the model on brains with neurologic disorders. A 3D-Unet convolutional neural network was constructed, and a 5-fold cross-validation was performed. RESULTS: The novel segmentation model demonstrated significantly improved Dice coefficients over the standard probabilistic atlas for both healthy subjects (mean, 0.68 [SD, 0.10] versus 0.45 [SD, 0.11], P = .002, t test) and patients (0.64 [SD, 0.10] versus 0.37 [SD, 0.22], P < .001). Additionally, the model demonstrated significantly decreased centroid distance in patients (1.18 [SD, 0.43] mm, 3.09 [SD, 2.56] mm, P = .007). CONCLUSIONS: We developed the first model, to our knowledge, for automatic and accurate patient-specific segmentation of the nucleus basalis of Meynert. This model may enable further study into the nucleus, impacting new treatments such as deep brain stimulation.
A novel quantification method for RPE phagocytosis using a VLC-PUFA-based strategy

The vertebrate retinal pigment epithelium (RPE) lies adjacent to the photoreceptors and is responsible for the engulfment and degradation of shed photoreceptor outer segment fragments (POS) through receptor-mediated phagocytosis. Phagocytosis of POS is critical for maintaining photoreceptor function and is a key indicator of RPE functionality. Popular established methods to assess RPE phagocytosis rely mainly on quantifying POS proteins, especially their most abundant protein rhodopsin, or on fluorescent dye conjugation of bulk, unspecified POS components. While these approaches are practical and quantitative, they fail to assess the fate of POS lipids, which make up about 50% of POS by weight and whose processing is essential for life-long functionality of RPE and retina. Here, we have developed a novel VLC-PUFA-based approach for evaluating RPE phagocytic activity by primary bovine and rat RPE and the human ARPE-19 cell line and validated its results using traditional methods. This new approach can be used to detect the dynamic process of phagocytosis at varying POS concentrations and incubation times and offers a robust, unbiased, and reproducible assay that will have utility in studies of POS lipid processing.
Synapses, predictions, and prediction errors: A neocortical computational study of MDD using the temporal memory algorithm of HTM

Background Synapses and spines are central in major depressive disorder (MDD) pathophysiology, recently highlighted by ketamine’s and psilocybin’s rapid antidepressant effects. The Bayesian brain and interoception perspectives formalize MDD as being “stuck” in affective states constantly predicting negative energy balance. We examined how synaptic atrophy relates to the predictive function of the neocortex and thus to symptoms, using temporal memory (TM), an unsupervised machine-learning algorithm. TM represents a single neocortical layer, learns in real-time using local Hebbian-learning rules, and extracts and predicts temporal sequences. Methods We trained a TM model on random sequences of upper-case alphabetical letters, representing sequences of affective states. To model depression, we progressively destroyed synapses in the TM model and examined how that affected the predictive capacity of the network. Results Destroying 50% of the synapses slightly reduced the number of predictions, followed by a marked drop with further destruction. However, reducing the synapses by 25% dropped the confidence in the predictions distinctly. So even though the network was making accurate predictions, the network was no longer confident about these predictions. Conclusions These findings explain how interoceptive cortices could be stuck in limited affective states with high prediction error. Growth of new synapses, e.g., with ketamine and psilocybin, would allow representing more futuristic predictions with higher confidence. To our knowledge, this is the first study to use the TM model to connect changes happening at synaptic levels to the Bayesian formulation of psychiatric symptomatology, making it possible to understand treatment mechanisms and possibly, develop new treatments. Graphical abstract