Miracle-dz/UniFAR-FTUs huggingface dataset:
Learning from Multiple Cities: A Meta-Learning Approach for Spatial-Temporal Prediction

Spatial-temporal prediction is a fundamental problem for constructing smart city, which is useful for tasks such as traffic control, taxi dispatching, and environment policy making. Due to data collection mechanism, it is common to see data collection with unbalanced spatial distributions. For example, some cities may release taxi data for multiple years while others only release a few days of data; some regions may have constant water quality data monitored by sensors whereas some regions only have a small collection of water samples. In this paper, we tackle the problem of spatial-temporal prediction for the cities with only a short period of data collection. We aim to utilize the long-period data from other cities via transfer learning. Different from previous studies that transfer knowledge from one single source city to a target city, we are the first to leverage information from multiple cities to increase the stability of transfer. Specifically, our proposed model is designed as a spatial-temporal network with a meta-learning paradigm. The meta-learning paradigm learns a well-generalized initialization of the spatial-temporal network, which can be effectively adapted to target cities. In addition, a pattern-based spatial-temporal memory is designed to distill long-term temporal information (i.e., periodicity). We conduct extensive experiments on two tasks: traffic (taxi and bike) prediction and water quality prediction. The experiments demonstrate the effectiveness of our proposed model over several competitive baseline models.
Overcomplete Independent Component Analysis via SDP

We present a novel algorithm for overcomplete independent components analysis (ICA), where the number of latent sources k exceeds the dimension p of observed variables. Previous algorithms either suffer from high computational complexity or make strong assumptions about the form of the mixing matrix. Our algorithm does not make any sparsity assumption yet enjoys favorable computational and theoretical properties. Our algorithm consists of two main steps: (a) estimation of the Hessians of the cumulant generating function (as opposed to the fourth and higher order cumulants used by most algorithms) and (b) a novel semi-definite programming (SDP) relaxation for recovering a mixing component. We show that this relaxation can be efficiently solved with a projected accelerated gradient descent method, which makes the whole algorithm computationally practical. Moreover, we conjecture that the proposed program recovers a mixing component at the rate k < p^2/4 and prove that a mixing component can be recovered with high probability when k < (2 - epsilon) p log p when the original components are sampled uniformly at random on the hyper sphere. Experiments are provided on synthetic data and the CIFAR-10 dataset of real images.
The Multi-Agent Reinforcement Learning in MalmÖ (MARLÖ) Competition

Learning in multi-agent scenarios is a fruitful research direction, but current approaches still show scalability problems in multiple games with general reward settings and different opponent types. The Multi-Agent Reinforcement Learning in Malm\"O (MARL\"O) competition is a new challenge that proposes research in this domain using multiple 3D games. The goal of this contest is to foster research in general agents that can learn across different games and opponent types, proposing a challenge as a milestone in the direction of Artificial General Intelligence.
PAC Identification of Many Good Arms in Stochastic Multi-Armed Bandits

We consider the problem of identifying any $k$ out of the best $m$ arms in an $n$-armed stochastic multi-armed bandit. Framed in the PAC setting, this particular problem generalises both the problem of `best subset selection' and that of selecting `one out of the best m' arms [arcsk 2017]. In applications such as crowd-sourcing and drug-designing, identifying a single good solution is often not sufficient. Moreover, finding the best subset might be hard due to the presence of many indistinguishably close solutions. Our generalisation of identifying exactly $k$ arms out of the best $m$, where $1 \leq k \leq m$, serves as a more effective alternative. We present a lower bound on the worst-case sample complexity for general $k$, and a fully sequential PAC algorithm, \GLUCB, which is more sample-efficient on easy instances. Also, extending our analysis to infinite-armed bandits, we present a PAC algorithm that is independent of $n$, which identifies an arm from the best $\rho$ fraction of arms using at most an additive poly-log number of samples than compared to the lower bound, thereby improving over [arcsk 2017] and [Aziz+AKA:2018]. The problem of identifying $k > 1$ distinct arms from the best $\rho$ fraction is not always well-defined; for a special class of this problem, we present lower and upper bounds. Finally, through a reduction, we establish a relation between upper bounds for the `one out of the best $\rho$' problem for infinite instances and the `one out of the best $m$' problem for finite instances. We conjecture that it is more efficient to solve `small' finite instances using the latter formulation, rather than going through the former.
Development and validation of the Japanese Moral Foundations Dictionary

The Moral Foundations Dictionary (MFD) is a useful tool for applying the conceptual framework developed in Moral Foundations Theory and quantifying the moral meanings implicated in the linguistic information people convey. However, the applicability of the MFD is limited because it is available only in English. Translated versions of the MFD are therefore needed to study morality across various cultures, including non-Western cultures. The contribution of this paper is two-fold. We developed the first Japanese version of the MFD (referred to as the J-MFD) using a semi-automated method—this serves as a reference when translating the MFD into other languages. We next tested the validity of the J-MFD by analyzing open-ended written texts about the situations that Japanese participants thought followed and violated the five moral foundations. We found that the J-MFD correctly categorized the Japanese participants’ descriptions into the corresponding moral foundations, and that the Moral Foundations Questionnaire (MFQ) scores correlated with the frequency of situations, of total words, and of J-MFD words in the participants’ descriptions for the Harm and Fairness foundations. The J-MFD can be used to study morality unique to the Japanese and also multicultural comparisons in moral behavior.
Heavy-Tailed Universality Predicts Trends in Test Accuracies for Very Large Pre-Trained Deep Neural Networks

Given two or more Deep Neural Networks (DNNs) with the same or similar architectures, and trained on the same dataset, but trained with different solvers, parameters, hyper-parameters, regularization, etc., can we predict which DNN will have the best test accuracy, and can we do so without peeking at the test data? In this paper, we show how to use a new Theory of Heavy-Tailed Self-Regularization (HT-SR) to answer this. HT-SR suggests, among other things, that modern DNNs exhibit what we call Heavy-Tailed Mechanistic Universality (HT-MU), meaning that the correlations in the layer weight matrices can be fit to a power law (PL) with exponents that lie in common Universality classes from Heavy-Tailed Random Matrix Theory (HT-RMT). From this, we develop a Universal capacity control metric that is a weighted average of PL exponents. Rather than considering small toy NNs, we examine over 50 different, large-scale pre-trained DNNs, ranging over 15 different architectures, trained on ImagetNet, each of which has been reported to have different test accuracies. We show that this new capacity metric correlates very well with the reported test accuracies of these DNNs, looking across each architecture (VGG16/./VGG19, ResNet10/./ResNet152, etc.). We also show how to approximate the metric by the more familiar Product Norm capacity measure, as the average of the log Frobenius norm of the layer weight matrices. Our approach requires no changes to the underlying DNN or its loss function, it does not require us to train a model (although it could be used to monitor training), and it does not even require access to the ImageNet data.
Semi-Unsupervised Learning with Deep Generative Models: Clustering and Classifying using Ultra-Sparse Labels

We introduce $\textit{semi-unsupervised learning}$, an extreme case of semi-supervised learning with ultra-sparse categorisation where some classes have no labels in the training set. That is, in the training data some classes are sparsely labelled and other classes appear only as unlabelled data. Many real-world datasets are conceivably of this type. We demonstrate that effective learning in this regime is only possible when a model is capable of capturing both semi-supervised and unsupervised learning. We develop two deep generative models for classification in this regime that extend previous deep generative models designed for semi-supervised learning. By changing their probabilistic structure to contain a mixture of Gaussians in their continuous latent space, these new models can learn in both unsupervised and semi-unsupervised paradigms. We demonstrate their performance both for semi-unsupervised and unsupervised learning on various standard datasets. We show that our models can learn in an semi-unsupervised manner on Fashion-MNIST. Here we artificially mask out all labels for half of the classes of data and keep $2\%$ of labels for the remaining classes. Our model is able to learn effectively, obtaining a trained classifier with $(77.2\pm1.3)\%$ test set accuracy. We also can train on Fashion-MNIST unsupervised, obtaining $(75.2\pm1.5)\%$ test set accuracy. Additionally, doing the same for MNIST unsupervised we get $(96.3\pm0.9)\%$ test set accuracy, which is state-of-the art for fully probabilistic deep generative models.
Learning and Its Neural Correlates in a Virtual Environment for Honeybees

The search for neural correlates of operant and observational learning requires a combination of two (experimental) conditions that are very difficult to combine: stable recording from high order neurons and free movement of the animal in a rather natural environment. We developed a virtual environment (VE) that simulates a simplified 3D world for honeybees walking stationary on an air-supported spherical treadmill. We show that honeybees perceive the stimuli in the VE as meaningful by transferring learned information from free flight to the virtual world. In search for neural correlates of learning in the VE, mushroom body extrinsic neurons were recorded over days during learning. We found changes in the neural activity specific to the rewarded and unrewarded visual stimuli. Our results suggest an involvement of the mushroom body extrinsic neurons in operant learning in the honeybee (Apis mellifera).
Elucidating Conserved Transcriptional Networks Underlying Pesticide Exposure and Parkinson's Disease: A Focus on Chemicals of Epidemiological Relevance

While a number of genetic mutations are associated with Parkinson's disease (PD), it is also widely acknowledged that the environment plays a significant role in the etiology of neurodegenerative diseases. Epidemiological evidence suggests that occupational exposure to pesticides (e.g., dieldrin, paraquat, rotenone, maneb, and ziram) is associated with a higher risk of developing PD in susceptible populations. Within dopaminergic neurons, environmental chemicals can have an array of adverse effects resulting in cell death, such as aberrant redox cycling and oxidative damage, mitochondrial dysfunction, unfolded protein response, ubiquitin-proteome system dysfunction, neuroinflammation, and metabolic disruption. More recently, our understanding of how pesticides affect cells of the central nervous system has been strengthened by computational biology. New insight has been gained about transcriptional and proteomic networks, and the metabolic pathways perturbed by pesticides. These networks and cell signaling pathways constitute potential therapeutic targets for intervention to slow or mitigate neurodegenerative diseases. Here we review the epidemiological evidence that supports a role for specific pesticides in the etiology of PD and identify molecular profiles amongst these pesticides that may contribute to the disease. Using the Comparative Toxicogenomics Database, these transcripts were compared to those regulated by the PD-associated neurotoxicant MPTP (1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine). While many transcripts are already established as those related to PD (alpha-synuclein, caspases, leucine rich repeat kinase 2, and parkin2), lesser studied targets have emerged as “pesticide/PD-associated transcripts” [e.g., phosphatidylinositol glycan anchor biosynthesis class C (Pigc), allograft inflammatory factor 1 (Aif1), TIMP metallopeptidase inhibitor 3, and DNA damage inducible transcript 4]. We also compared pesticide-regulated genes to a recent meta-analysis of genome-wide association studies in PD which revealed new genetic mutant alleles; the pesticides under review regulated the expression of many of these genes (e.g., ELOVL fatty acid elongase 7, ATPase H+ transporting V0 subunit a1, and bridging integrator 3). The significance is that these proteins may contribute to pesticide-related increases in PD risk. This review collates information on transcriptome responses to PD-associated pesticides to develop a mechanistic framework for quantifying PD risk with exposures.
“It Can’t Be Like Last Time” – Choices Made in Early Pregnancy by Women Who Have Previously Experienced a Traumatic Birth

Background: A significant number of women experience childbirth as traumatic. These experiences are often characterized by a loss of control coupled with a perceived lack of support and inadequate communication with health care providers. Little is known about the choices women make in subsequent pregnancy(s) and birth(s), or why they make these choices. This study aimed to understand these choices and explore the reasons behind them. Methods: A longitudinal grounded theory methods study involving nine women was conducted. Over half of the participants had a formal diagnosis of post-traumatic stress disorder (PTSD) and/or PND related to the previous birth. Interviews were carried out at three timepoints perinatally. These findings are from the first interviews at 12–20 weeks. Results: From the first days of pregnancy, this cohort of women were focused on concerns that this birth would be a repeated traumatic experience. The women were deliberately searching out and analyzing information about their choices in this pregnancy and birth, and making plans which had two aims; firstly to avoid a repeat of their previous birth experience and secondly to avoid a loss of control to other people during the birth. The women considered a range of birth choices, from elective cesareans to freebirth. Some women felt well supported by those around them, including care providers, partners, friends, and family. Others did not feel supported and were anticipating conflict in trying to assert their birth choices. Many early relationships with healthcare professionals were characterized by fear and mistrust. Discussion: If women who have previously experienced a traumatic birth become pregnant again, they have a strong desire to avoid a repeat experience and to feel in control of their birth choices. Access to robust information appears to help reduce uncertainty and arm women in their discussions with professionals. Similarly making plans and seeking to have them agreed with care providers at an early stage is used a way to reduce the risk of having a further traumatic experience. Implications for practice include supporting women in formulating and confirming pregnancy and birth plans at an early stage to reduce uncertainty and foster a sense of control.
Psychosocial Stress Before a Nap Increases Sleep Latency and Decreases Early Slow-Wave Activity

Sleep disturbances are an important risk factor for stress-related diseases such as burnout or depression. In particular, slow-wave activity (SWA) during sleep might be eminently relevant for optimal maintenance of mental health and cognitive functioning. In spite of the clinical importance and the pertinence of stress-related processes in everyday life, the physiological mechanisms of the association between stress, sleep, and cognition are not well-understood. In the present study, we carefully mapped the time course of the influence of a psychosocial stressor on sleep architecture and sleep-related oscillations during a midday nap. We induced stress using a psychosocial laboratory stressor, the Montreal Imaging Stress Task, vs. a neutral control task. Afterward, participants were allowed to take a 90-min nap (n = 20) or stayed awake (n = 19) and cortisol was measured via saliva samples. We hypothesized that stress would decrease sleep efficiency and SWA in a time-dependent manner, with impairing effects on cognitive functioning. Psychosocial stress resulted in increased cortisol levels, which were elevated throughout the study interval. In the nap group, psychosocial stress increased sleep latency, but had only minor effects on sleep architecture. Still, SWA in the first 30 min of sleep was significantly reduced, whereas alpha activity was enhanced. These effects vanished after approximately 30 min. No impairing effect on cognitive functioning occurred. Our results show that psychosocial stress before sleep has an impact on sleep latency and early SWA during sleep. In contrast to our hypothesis, the effects were rather small and short-lasting. Importantly, cognitive functioning was maintained. We conclude that the effects of psychosocial stress before a nap are possibly better compensated than previously believed.
Social Memory and Social Patterns Alterations in the Absence of STriatal-Enriched Protein Tyrosine Phosphatase

STriatal-Enriched protein tyrosine Phosphatase (STEP) is a neural-specific protein that opposes the development of synaptic strengthening and whose levels are altered in several neurodegenerative and psychiatric disorders. Since STEP is expressed in brain regions implicated in social behavior, namely the striatum, the CA2 region of the hippocampus, cortex and amygdala, here we investigated whether social memory and social patterns were altered in STEP knockout (KO) mice. Our data robustly demonstrated that STEP KO mice presented specific social memory impairment as indicated by the three-chamber sociability test, the social discrimination test, the 11-trial habituation/dishabituation social recognition test, and the novel object recognition test (NORT). This affectation was not related to deficiencies in the detection of social olfactory cues, altered sociability or anxiety levels. However, STEP KO mice showed lower exploratory activity, reduced interaction time with an intruder, less dominant behavior and higher immobility time in the tail suspension test than controls, suggesting alterations in motivation. Moreover, the extracellular levels of dopamine (DA), but not serotonin (5-HT), were increased in the dorsal striatum of STEP KO mice. Overall, our results indicate that STEP deficiency disrupts social memory and other social behaviors as well as DA homeostasis in the dorsal striatum.
Assessing Beach and Dune Erosion and Vulnerability Under Sea Level Rise: A Case Study in the Mediterranean Sea

In this study, we estimate the shoreline retreat, the vulnerability and the erosion rates of an open beach-dune system under projected sea level rise and the action of wind-waves (separately and in combination). The methodology is based on the combination of two state-of-the-art numerical models (XBeach and Q2D-morfo) applied in a probabilistic framework and it is implemented in an open sandy beach in Menorca Island (Western Mediterranean). We compute the shoreline response to sea level rise during the 21st century and we assess the changing impacts of storm waves on the aerial beach-dune system. Results demonstrate the relevant role that the beach backshore features, such as the berm, play as coastal defence, reducing the shoreline retreat and dune vulnerability rates in the near-term (a few decades ahead) and highlighting the importance of simulating the beach morphodynamic processes in coastal impacts assessments. Our findings point at sea level rise as the major driver of the projected impacts over the beach-dune system, leading to an increase of ~25% of the volume eroded due to storm waves by the end of the century with respect to present-day conditions.
Health Outcomes Associated With Internalizing Problems in Early Childhood and Adolescence

Physical and mental health problems are becoming more common among the general population. Studies examining mental and physical health often indicate that mental illness early in life is associated with more detrimental health outcomes later. However, additional work is needed to better identity which psychological problems may contribute to poorer health outcomes. Given recent increases in childhood anxiety and depression specifically, it is beneficial to further investigate the relationship between internalizing problems, both early and later in life, and related health problems. Furthermore, little work has focused on studying internalizing problems in children as young as preschool-aged. Therefore, the current project used a longitudinal design to assess the effects of preschool and adolescent internalizing problems on health-related problems in adolescence. We analyzed data from 70 youth (47% male) who had been tested in our lab when they were 5 years old and then were administered questionnaires over a telephone interview when they were adolescents, between the ages of 12 and 20 years old. We used multi-informant measures, including parent-report at age 5 and youth-report at follow-up, 7 to 15 years later. Parents reported on children’s internalizing behavior problems and negative emotionality (NE). Youth reported on their own internalizing behavior problems as well as health problems, physical activity, and overeating behaviors. Path modeling was used to examine predictions of internalizing and health behaviors. At age 5, parent-reported NE and internalizing problems were related, in addition to 5-year-old internalizing predicting health problems and overeating at follow-up. At follow-up, youth-reported internalizing was positively related to health problems and negatively related to physical activity, suggesting some similarities and differences between parent and youth responses. Additionally, girls reported significantly higher rates of internalizing and health problems at follow-up. These results indicate a significant relationship between preschool-aged and adolescent internalizing problems and related health outcomes experienced in adolescence.