Miracle-dz/UniFAR-FTUs huggingface dataset:
Transforming Transformers for Resilient Lifelong Learning

Lifelong learning without catastrophic forgetting (i.e., resiliency) remains an open problem for deep neural networks. The prior art mostly focuses on convolutional neural networks. With the increasing dominance of Transformers in deep learning, it is a pressing need to study lifelong learning with Transformers. Due to the complexity of training Transformers in practice, for lifelong learning, a question naturally arises: Can Transformers be learned to grow in a task aware way, that is to be dynamically transformed by introducing lightweight learnable plastic components to the architecture, while retaining the parameter-heavy, but stable components at streaming tasks? To that end, motivated by the lifelong learning capability maintained by the functionality of Hippocampi in human brain, we explore what would be, and how to implement, Artificial Hippocampi (ArtiHippo) in Transformers. We present a method to identify, and learn to grow, ArtiHippo in Vision Transformers (ViTs) for resilient lifelong learning in four aspects: (i) Where to place ArtiHippo to enable plasticity while preserving the core function of ViTs at streaming tasks? (ii) How to represent and realize ArtiHippo to ensure expressivity and adaptivity for tackling tasks of different nature in lifelong learning? (iii) How to learn to grow ArtiHippo to exploit task synergies (i.e., the learned knowledge) and overcome catastrophic forgetting? (iv) How to harness the best of our proposed ArtiHippo and prompting-based approaches? In experiments, we test the proposed method on the challenging Visual Domain Decathlon (VDD) benchmark and the 5-Dataset benchmark under the task-incremental lifelong learning setting. It obtains consistently better performance than the prior art with sensible ArtiHippo learned continually. To our knowledge, it is the first attempt of lifelong learning with ViTs on the challenging VDD benchmark.
Photometric and spectroscopic monitoring of YSOs in nearby star-forming regions – I. Eruptive YSOs

Mid-infrared (mid-IR) variability in young stellar objects (YSOs) is driven by several physical mechanisms, which produce a variety of amplitudes and light-curve shapes. One of these mechanisms, variable disc accretion, is predicted by models of episodic accretion to drive secular variability, including in the mid-IR. Because the largest accretion bursts are rare, adding new objects to the YSO eruptive variable class aids our understanding of the episodic accretion phenomenon and its possible impact on stellar and planetary formation. A previous analysis of 6.5 yr of NeoWISE light curves (3–5 μm) of ∼7000 nearby YSOs found an increase in the fraction of variability and variability amplitude for objects at younger stages of evolution. To help interpret these light curves, we have obtained low- and high-resolution near-IR spectra of 78 objects from this sample of YSOs. In this work, we present the analysis of nine nearby YSOs (d<1 kpc) that show the characteristics of known classes of eruptive variable YSOs. We find one FU Orionis (FUor)-like source, one EX Lupi type object, and six YSOs with mixed characteristics or V1647 Ori like objects. The varied characteristics observed in our sample are consistent with recent discoveries of eruptive YSOs. We discuss how a wide range in YSO outburst parameters (central mass, maximum accretion rate during outburst, evolutionary stage, and/or instability leading to the outburst) may play a significant role in the observed spectrophotometric properties of YSO outbursts.
Halide Perovskite Artificial Solids as a New Platform to Simulate Collective Phenomena in Doped Mott Insulators

The development of quantum simulators, artificial platforms where the predictions of many-body theories of correlated quantum materials can be tested in a controllable and tunable way, is one of the main challenges of condensed matter physics. Here we introduce artificial lattices made of lead halide perovskite nanocubes as a new platform to simulate and investigate the physics of correlated quantum materials. We demonstrate that optical injection of quantum confined excitons in this system realizes the two main features that ubiquitously pervade the phase diagram of many quantum materials: collective phenomena, in which long-range orders emerge from incoherent fluctuations, and the excitonic Mott transition, which has one-to-one correspondence with the insulator-to-metal transition described by the repulsive Hubbard model in a magnetic field. Our results demonstrate that time-resolved experiments provide a quantum simulator that is able to span a parameter range relevant for a broad class of phenomena, such as superconductivity and charge-density waves.
Cascaded Zoom-in Detector for High Resolution Aerial Images

Detecting objects in aerial images is challenging because they are typically composed of crowded small objects distributed non-uniformly over high-resolution (in terms of pixel size) images. Density cropping is a widely used method to improve this small object detection where the crowded small object regions are extracted and processed in high image-resolution. However, this is typically accomplished by adding other learnable components, thus complicating the training and inference over a standard detection process. In this paper, we propose an efficient Cascaded Zoom-in (CZ) detector that re-purposes the detector itself for density-guided training and inference. During training, density crops are located, labeled as a new class, and employed to augment the training dataset. During inference, the density crops are first detected along with the base class objects, and then input for a second stage of inference. This approach is easily integrated into any detector, and creates no significant change in the standard detection process, like the uniform cropping approach popular in aerial image detection. Experimental results on the aerial images of the challenging VisDrone and DOTA datasets verify the benefits of the proposed approach. The proposed CZ detector also provides state-of-the-art results over uniform cropping and other density cropping methods on the VisDrone dataset, increasing the detection mAP of small objects by more than 3 percentage points. 1
NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptions

The Natural Language for Optimization (NL4Opt) Competition was created to investigate methods of extracting the meaning and formulation of an optimization problem based on its text description. Specifically, the goal of the competition is to increase the accessibility and usability of optimization solvers by allowing non-experts to interface with them using natural language. We separate this challenging goal into two sub-tasks: (1) recognize and label the semantic entities that correspond to the components of the optimization problem; (2) generate a meaning representation (i.e., a logical form) of the problem from its detected problem entities. The first task aims to reduce ambiguity by detecting and tagging the entities of the optimization problems. The second task creates an intermediate representation of the linear programming (LP) problem that is converted into a format that can be used by commercial solvers. In this report, we present the LP word problem dataset and shared tasks for the NeurIPS 2022 competition. Furthermore, we investigate and compare the performance of the ChatGPT large language model against the winning solutions. Through this competition, we hope to bring interest towards the development of novel machine learning applications and datasets for optimization modeling.
Thermally driven phase transitions in freestanding low-buckled silicene, germanene, and stanene

Low-buckled silicene, germanene, and stanene are group$-IV$ graphene allotropes. They form a honeycomb lattice out of two interpenetrating ($A$ and $B$) triangular sublattices that are vertically separated by a small distance $\Delta_z$. The atomic numbers $Z$ of silicon, germanium, and tin are larger to carbon's ($Z_C=6$), making them the first experimentally viable two-dimensional topological insulators. Those materials have a twice-energy-degenerate atomistic structure characterized by the buckling direction of the $B$ sublattice with respect to the $A$ sublattice [whereby the $B-$atom either protrudes {\em above} ($\Delta_z>0$) or {\em below} ($\Delta_z<0$) the $A-$atoms], and the consequences of that energy degeneracy on their elastic and electronic properties have not been reported thus far. Here, we uncover {\em ferroelastic, bistable} behavior on silicene, which turns into an {\em average} planar structure at about 600 K. Further, the creation of electron and hole puddles obfuscates the zero-temperature SOC induced band gaps at temperatures as low as 200 K, which may discard silicene as a viable two-dimensional topological insulator for room temperature applications. Germanene, on the other hand, never undergoes a low-buckled to planar 2D transformation, becoming amorphous at around 675 K instead, and preserving its SOC-induced bandgap despite of band broadening. Stanene undergoes a transition onto a crystalline 3D structure at about 300 K, preserving its SOC-induced electronic band gap up to that temperature. Unlike what is observed in silicene and germanene, stanene readily develops a higher-coordinated structure with a high degree of structural order. The structural phenomena is shown to have deep-reaching consequences for the electronic and vibrational properties of those two dimensional topological insulators.
The Devil's Advocate: Shattering the Illusion of Unexploitable Data using Diffusion Models

Protecting personal data against exploitation of machine learning models is crucial. Recently, availability attacks have shown great promise to provide an extra layer of protection against the unauthorized use of data to train neural networks. These methods aim to add imperceptible noise to clean data so that the neural networks cannot extract meaningful patterns from the protected data, claiming that they can make personal data"unexploitable. "This paper provides a strong countermeasure against such approaches, showing that unexploitable data might only be an illusion. In particular, we leverage the power of diffusion models and show that a carefully designed denoising process can counteract the effectiveness of the data-protecting perturbations. We rigorously analyze our algorithm, and theoretically prove that the amount of required denoising is directly related to the magnitude of the data-protecting perturbations. Our approach, called AVATAR, delivers state-of-the-art performance against a suite of recent availability attacks in various scenarios, outperforming adversarial training even under distribution mismatch between the diffusion model and the protected data. Our findings call for more research into making personal data unexploitable, showing that this goal is far from over. Our implementation is available at this repository: https://github.com/hmdolatabadi/AVATAR.
Translating predictive distributions into informative priors

When complex Bayesian models exhibit implausible behaviour, one solution is to assemble available information into an informative prior. Challenges arise as prior information is often only available for the observable quantity, or some model-derived marginal quantity, rather than directly pertaining to the natural parameters in our model. We propose a method for translating available prior information, in the form of an elicited distribution for the observable or model-derived marginal quantity, into an informative joint prior. Our approach proceeds given a parametric class of prior distributions with as yet undetermined hyperparameters, and minimises the difference between the supplied elicited distribution and corresponding prior predictive distribution. We employ a global, multi-stage Bayesian optimisation procedure to locate optimal values for the hyperparameters. Three examples illustrate our approach: a cure-fraction survival model, where censoring implies that the observable quantity is a priori a mixed discrete/continuous quantity; a setting in which prior information pertains to $R^{2}$ -- a model-derived quantity; and a nonlinear regression model.
The posterior probability distribution of traffic flow: a new scheme for the assignment of stochastic traffic flow

This study proposes a new scheme for assigning traffic flows that aims to capture the stochastic nature of route traffic flows. We consider the route traffic flows to be random variables. The distribution of these random variables is formulated as a conditional probability distribution for a given assumption: the traffic network is in stochastic user equilibrium. From a Bayesian perspective, we treat the conditional distribution as a posterior distribution of route traffic flows, which is obtained using Bayes’ theorem. We develop a basic Metropolis–Hastings (M–H) sampling scheme, as well as a M–H within Gibbs sampling scheme, to draw samples from the posterior distribution. We estimate characteristics such as the means and variances of route traffic flows from simulated samples. The proposed model can directly output the route traffic flows, and has a highly flexible computation process.
Statistical Learning Methods Applied to Process Monitoring: An Overview and Perspective

The increasing availability of high-volume, high-velocity data sets, often containing variables of different data types, brings an increasing need for monitoring tools that are designed to handle these big data sets. While the research on multivariate statistical process monitoring tools is vast, the application of these tools for big data sets has received less attention. In this expository paper, we give an overview of the current state of data-driven multivariate statistical process monitoring methodology. We highlight some of the main directions involving statistical learning and dimension reduction techniques applied to control charts in research from supply chain, engineering, computer science, and statistics. The goal of this paper is to bring into better focus some of the monitoring and surveillance methodology informed by data mining techniques that show promise for monitoring large and diverse data sets. We introduce an example using Wikipedia search information and illustrate a few of the complexities of applying the available methods to a high-dimensional monitoring scenario. Throughout, we offer advice to practitioners and some suggestions for future research in this emerging area of research.
Rate of COVID‐19 vaccination among patients with cancer who tested positive for severe acute respiratory syndrome‐coronavirus 2: Analysis of the American Society of Clinical Oncology Registry

The availability of safe and effective COVID‐19 vaccines has enabled protections against serious COVID‐19 outcomes, which are particularly important for patients with cancer. The American Society of Clinical Oncology Registry enabled the study of COVID‐19 vaccine uptake in patients with cancer who were positive for severe acute respiratory syndrome‐coronavirus 2.
Utility of Person–Environment–Occupation model in exploring sex-specific causes of work-related traumatic brain injury: a retrospective chart review

ABSTRACT Background Work-related traumatic brain injury (wr-TBI) is on the rise. The pre-injury period, a significant consideration for preventive initiatives, is largely unexplored. Objectives To identify Person–Environment–Occupation (PEO) variables associated with wr-TBI to inform sex-specific primary prevention. Methods Retrospective chart review data were analyses. Two-tailed t-test and chi-squared tests were used to study sex differences. Multivariate logistic regression models of wr-TBI were fit with a priori defined PEO variables. Results The sample comprised 330 consecutive workers with wr-TBI (40.8 ± 11.1 years old, 71% male). Sex differences were observed across PEO variables. In multivariable logistic regression analyses the odds of sustaining a wr-TBI from a fall increased with the presence of a mood disorder and participation in non-labourer occupations (odds ratio (OR) 2.89 (95% CI 1.06–7.89) and OR 2.89 (95% CI 1.06–7.89), respectively) and decreased being a male (OR 0.31 (95% CI 0.17–0.54)). The odds of sustaining a wr-TBI from being striken by an object was greater in workers with prior head injury (OR 2.8 (95% CI 1.24–6.45)). None of the variables studied were associated with wr-TBI sustained from being striken against an object. Conclusions Workers’ health status pre-injury is associated with external causes of wr-TBI. Sex differences across PEO categories warrant further study.
NR3C2 mediates oxidised low-density lipoprotein-induced human coronary endothelial cells dysfunction via modulation of NLRP3 inflammasome activation

Abstract Nuclear receptor subfamily 3 group C member 2 (NR3C2) has been revealed to affect the progression of multiple inflammatory diseases, while NR3C2’s efficacy in coronary artery disease (CAD) remains largely unsolved. The study intended to elucidate the possible mechanisms of NR3C2 in oxidised low density lipoprotein (ox-LDL)-induced inflammation in human coronary endothelial cells (HCAECs) via regulating NACHT, LRR, and PYD domains-containing protein 3 (NLRP3). Patients who underwent CT angiography or coronary angiography for suspected CAD in our hospital were collected. The patients were divided into the CAD and the non-CAD (NCAD) groups. The expression of NR3C2 and NLRP3 in the peripheral blood of patients in both groups was examined by RT-qPCR. HCAECs were treated with ox-LDL to establish the model. The expression of NR3C2 and NLRP3 in ox-LDL-induced HCAECs was tested by RT-qPCR. The proliferation of HCAECs was measured using CCK-8 assay, the apoptosis of HCAECs was assessed by flow cytometry, and the levels of inflammation-related factors IL-1β and IL-18 in the cell supernatant were evaluated by ELISA. The molecular mechanisms of these factors in the proliferation and apoptosis of HCAECs and in the inflammatory response were further determined by knockdown and overexpression systems. The relationship between NR3C2 and NLRP3 was determined by ChIP and luciferase activity assays and bioinformatics analysis. NR3C2 and NLRP3 levels were elevated in the serum of CAD patients. The ox-LDL treatment elevated NR3C2 levels, evoked apoptosis and inflammation, and impeded cell viability in HCAECs, whereas downregulation of NR3C2 increased cell viability and reduced apoptosis and inflammatory response in ox-LDL-induced inflammation in HCAECs. NR3C2 levels were positively correlated with NLRP3, and NR3C2 elevated NLRP3 expression through transcription. Overexpression of NLRP3 counteracted the impacts of silencing NR3C2 on cell viability, cell apoptosis, and inflammatory response in ox-LDL-induced HCAECs. Our research stresses that NR3C2 transcription promotes NLRP3 to induce inflammatory responses in ox-LDL-induced HCAECs.
Dynamical Mechanisms for Stratospheric Influences on the Troposphere.

The dynamical mechanisms through which stratospheric forcing can influence tropospheric annular modes are explored. A torque is applied to the stratosphere of an idealized general circulation model, and, under some circumstances, a robust tropospheric response is observed. These tropospheric responses, while initiated by stratospheric forcing, are maintained locally by interactions with transient eddies, and they closely resemble the intrinsic annular modes of the model. Manipulations of the model are consistent in showing that planetary waves, and not only the zonally symmetric secondary circulations induced by stratospheric forcing, are important for transmitting dynamical signals to the troposphere. Specifically, it is found that the tropospheric response is significantly reduced when planetary waves are suppressed in the stratosphere by additional damping or when the strength of the stratospheric jet is increased. Wave diagnoses indicate that the confinement of these waves within the troposphere, when stratospheric winds are enhanced, leads to increased planetary wave deceleration of the zonal winds in the high-latitude upper troposphere.
Effect of Home-Based Self-Management Intervention for Community-Dwelling Patients with Early Parkinson’s Disease: A Feasibility Study

ABSTRACT Purpose This study aimed to evaluate the effect of a home-based self-management intervention in community-dwelling patients with early Parkinson’s diseases (PD). Design A randomized-controlled design. Methods Thirty-two patients participated (15=intervention, 17=control), and the intervention group received 16 weeks of the intervention. Findings Physical activity and non-motor symptoms improved more in the intervention group than in the control group. Conclusion Home-based self-management intervention was effective in improving physical activity and non-motor symptoms for them. Clinical evidence Home-based intervention – comprising education, telephone counseling, smartphone-based message and information, and smart wearable devices – was feasible for patients with early PD.
Prevalence and descriptive epidemiology of Turner syndrome in the United States, 2000–2017: A report from the National Birth Defects Prevention Network

The lack of United States population‐based data on Turner syndrome limits assessments of prevalence and associated characteristics for this sex chromosome abnormality. Therefore, we collated 2000–2017 data from seven birth defects surveillance programs within the National Birth Defects Prevention Network. We estimated the prevalence of karyotype‐confirmed Turner syndrome diagnosed within the first year of life. We also calculated the proportion of cases with commonly ascertained birth defects, assessed associations with maternal and infant characteristics using prevalence ratios (PR) with 95% confidence intervals (CI), and estimated survival probability. The prevalence of Turner syndrome of any pregnancy outcome was 3.2 per 10,000 female live births (95% CI = 3.0–3.3, program range: 1.0–10.4), and 1.9 for live birth and stillbirth (≥20 weeks gestation) cases (95% CI = 1.8–2.1, program range: 0.2–3.9). Prevalence was lowest among cases born to non‐Hispanic Black women compared to non‐Hispanic White women (PR = 0.5, 95% CI = 0.4–0.6). Coarctation of the aorta was the most common defect (11.6% of cases), and across the cohort, individuals without hypoplastic left heart had a five‐year survival probability of 94.6%. The findings from this population‐based study may inform surveillance practices, prenatal counseling, and diagnosis. We also identified racial and ethnic disparities in prevalence, an observation that warrants further investigation.