VictorJuiz/thesis huggingface dataset:
More sample efficient and robust reinforcement learning with domain knowledge

Reinforcement Learning has achieved great success on environments with good simulators (for example, Atari, Starcraft, Go, and various robotic tasks). In these settings, agents were able to achieve performance on par with or exceeding human performance. However, the application of reinforcement learning to real world human-facing applications has been limited due to various issues such as large sample complexity. This dissertation proposes methods that work towards addressing these issues by utilizing domain knowledge and structure. Domain knowledge was the main component of the first class of successful AI systems, expert-rule based systems. However, due to many challenges, including the large amount of expensive expert time required, the research community has shifted towards data-driven methods that learn automatically. This dissertation presents methods that aim to combine the benefits of expert-based systems with the strengths of reinforcement learning. These methods can have better performance, achieving better sample efficiency or more robust performance, with only minimal burden on the experts. This dissertation proposes multiple novel methods for leveraging different types of domain knowledge in multiple different reinforcement learning settings. It will introduce methods for incorporating expert domain knowledge and heuristics to speed online reinforcement learning; for incorporating repeated structure in procedure/imitation learning; for incorporating anticipated domain distribution shift for batch contextual bandit settings; and for incorporating a curriculum graph to create better personalized adaptive progressions in a real world educational webgame. We empirically evaluate our methods in simulators designed with real-world data, such as recommendation systems and educational activities sequencing. We additionally test one of our methods in a real world Korean language learning webgame. For all our methods, we demonstrate that we can achieve faster or more robust performance. This shows promise for reinforcement learning methods to be helpful in human-facing applications

Keywords: Electrical Engineering.
Advisor: Brunskill Emma
Sadigh Dorsa
Van Roy Benjamin
Morphology, chemical composition, and defect evolution during synthesis and annealing of core/shell germanium/germanium-tin nanowires

Nanowire structures provide a platform for fundamental research. Due to the inherent nano-scale diameter and large aspect ratio, nanowires demonstrate structure-property relationships that often differ from those of bulk materials, including dimensional confinement of electronic carriers and light, a large surface area to volume ratio, and size- and shape-dependent partitioning of mechanical strain and barriers to inelastic deformation. Herein Ge and core/shell Ge/GeSn nanowires were investigated with a range of techniques to investigate their synthesis and annealing properties. First, a technique using in-situ laser reflectometry was investigated for measuring the axial growth rate in chemical vapor deposition of assemblies of well-aligned vertical germanium nanowires grown epitaxially on single crystal substrates. Finite difference frequency domain optical simulations were performed to facilitate quantitative analysis and interpretation of the measured reflectivity data. The results showed an insensitivity of reflected intensity oscillation period to nanowire diameter and density within the range of experimental conditions investigated. Furthermore, we showed that the measured reflectivity decay can be used to determine the germanium nanowire nucleation time with good precision. The link between the optical emission and atomic structure of individual vertical Ge/GeSn core/shell nanowires was investigated using transmission electron microscopy (TEM) and cathodoluminescence (CL). Optical measurements of individually as-grown nanowires were performed using scanning electron microscopy (SEM) CL. Electron beam deposited fiducial markers were utilized to identify and track individual nanowires between CL measurements and focused ion beam preparation for TEM analysis. High angle annular dark field scanning TEM (HAADF STEM) was then performed as a probe into the origin of the non-homogeneous CL emission between the identified nanowires based on observable line defects. X-ray photoelectron spectroscopy was used to analyze results from in-situ and ex-situ annealing of core/shell Ge/GeSn nanowire arrays. The annealing characteristics of the germanium-tin surface and native oxide were investigated for tin contents in the range of 2 at% to 12 at% in both air-exposed and oxide-free nanowires. For air-exposed samples, we observed the presence of a tin-rich oxide that exhibits a composition dependent temperature for thermal decomposition via in-situ annealing. Utilizing nominally air-free transfer of freshly synthesized and rapid thermally annealed core/shell Ge/GeSn nanowires, we showed the protective effects of the surface oxide when comparing samples annealed under a rough vacuum environment compared to hydrogen annealing

Keywords: Materials Science and Engineering.
Advisor: McIntyre Paul Cameron
Brongersma Mark L.
Dionne Jennifer Anne
Mothers' Experiences of Institutional Betrayal during Childbirth and Postpartum Mental Health Outcomes

Purpose: Emerging research suggests that betrayal by healthcare institutions could explain how negative birth experiences may lead to postpartum mental health disorders. The current study is the first to examine the role of experiencing institutional betrayal during childbirth (failure of a health system to meet a mother’s expectations in a way that harms the mother) in the prediction of and causal pathways to poor postpartum mental health outcomes for mothers in the US.
Participants & Methods: Mothers who had given birth in the last 3 years in the US as of June 2021 were recruited online. Participants (n = 612; 72% White; 100% women; median age between 27 and 36 years) responded to Qualtrics survey questions about their birth experiences, including adverse medical events, negative birth experiences, and experiences of institutional betrayal.
Data Analysis: Multiple logistic regression analysis was used to examine if experiencing instances of institutional betrayal (i.e., having procedure that was more painful than expected, not being notified of test results, etc.) significantly predicted postpartum mental health condition. Hierarchical regressions were used to examine whether institutional betrayal is a moderating variable, and R’s mediate package was used to determine if institutional betrayal mediates the causal pathway between negative birth experiences and poor postpartum outcomes.
Results: 48% of respondents experienced at least once instance of Institutional Betrayal, and their average number of IB experiences was 1.9 (SD = 1.10, max=5). Mothers who experienced institutional betrayal compared to those who didn’t were more likely to have been diagnosed with any postpartum mental health condition (p < 0.001). Experiencing more institutional betrayal instances increased a mother’s odds of a postpartum mental condition by 1.54 times (p < 0.005). Institutional betrayal was not a moderating variable, but it was a mediator between negative birth experiences and poor postpartum mental health outcomes.
Discussion: These findings can inform interventions to prevent poor postpartum mental health outcomes at the level of the individual patient as well as at the level of the healthcare system or institution. Further research should focus on postpartum mental health and birth equity by developing an additional adaptation of the IB questionnaire for childbirth for mothers of color, recruit a sample of mothers of color, and conduct analyses into causal pathways to postpartum mental health disorders for that population.

Keywords: Postpartum mental health
Institutional Betrayal
Birth experience
Text
Article
Thesis
Advisor: Fielding-Singh Priya
Lee Henry
Delecourt Solène
Motor neuron heterogeneity and vulnerability

The spinal cord is a fascinating structure that is responsible for coordinating movement in vertebrates. Spinal motor neurons control muscle activity by transmitting signals from the spinal cord to diverse peripheral targets in healthy adults. This biological process is disrupted in numerous neurodegenerative disorders, most notably amyotrophic lateral sclerosis (ALS). In ALS patients, motor neurons selectively detach from the muscles that they control and subsequently degenerate, causing sequential paralysis that culminates in death within 3-5 years. The key feature of ALS, as well as most other neurodegenerative diseases, is that it can be caused by mutations in numerous genes that are ubiquitously expressed throughout the central nervous system, yet the functional consequences of these mutations converge to induce similar cellular pathology and patient phenotypes. This phenomenon, known as selective vulnerability of neuronal subtypes, holds the key to understanding the molecular etiology of neurodegenerative disease. In the following work, we endeavored to uncover the core transcriptional differences that distinguish vulnerable and resistant neurons using a combination of single-cell transcriptomics, epigenetics, in situ hybridization, and mouse genetics. In the process, we identified unprecedented levels of transcriptional divergence throughout the healthy adult mouse motor system, including the discovery of functionally important subtypes of motor neurons, and described key gene expression differences that underlie their highly diversified function. Our characterization of the molecular machinery that enables neuronal control of the visceral and skeletal motor systems will provide a roadmap for future exploration of adult motor neuron function

Advisor: Gitler Aaron D.
Bassik Michael
Greenleaf William James
Reimer Richard J.
Multi-fidelity simulation framework with large-eddy simulations for predicting natural ventilation

Natural ventilation can play an important role in preventing the spread of airborne diseases in indoor environments. For example, a previous study in Dhaka, Bangladesh identified an association between the ventilation status of slum homes and the occurrence of pneumonia in children under five, which is the leading cause of death in this age group. However, quantifying natural ventilation flow rates is a challenging task due to significant variability in the boundary conditions that drive the flow. Thus, the objective of this research is to establish a modeling framework to precisely estimate ventilation flow patterns and ventilation rates in terms of air change per hour (ACH). The test case considered in this study is a typical low-income house in Dhaka, but the modeling strategy can be readily applied to any naturally ventilated building. The framework encompasses two computational models with different levels of fidelity: a computationally efficient building thermal model (BTM) and a high-fidelity computational fluid dynamics (CFD) model. The BTM is combined with methods for uncertainty quantification to predict mean and 95% confidence intervals for two primary quantities of interest, i.e., the average indoor air temperature and the ACH, at very low computational cost. For the CFD model, computationally extensive large-eddy simulations (LES) are performed to provide a detailed solution for the flow pattern an accurate ACH prediction at a much higher computational cost. LES is leveraged to propose a similarity relation that efficiently represents the effect of variable wind and temperature conditions on the ACH with a single non-dimensional number, the ventilation Richardson number. This similarity relationship is incorporated in the BTM to improve the accuracy of the ACH predictions. The model results are validated against field measurements and interpreted to identify efficient and robust ventilation strategies that will work under a variety of housing and weather conditions

Advisor: Gorle Catherine
Fringer Oliver B. (Oliver Bartlett)
Luby Stephen
Multifocus Convolutional Neural Networks for Short-term Solar PV Power Output Prediction

Reliable forecasts for the power output from variable renewable energy generators like solar photovoltaic systems are important for balancing the load on real-time electricity markets and ensuring the reliability of the grid. However, solar PV power output is highly uncertain, with significant variations occurring over longer (daily or seasonally) and shorter (minutely) timescales due to weather conditions, especially cloud cover. Therefore, accurate models that can provide these predictions are necessary.
This thesis builds on existing work, the SUNSET model, that uses a convolutional neural network applied to the computer vision tasks of nowcasting and forecasting solar PV power output. Multifocus SUNSET Nowcast and Forecast model architectures are presented and compared to the baseline SUNSET models. The multifocus models utilize a two-stream approach: one branch ingests a 64 x 64 pixel downsampled full sky image and the other branch ingests a 64x64 pixel sun patch cropped from a 256 x 256 pixel resolution full sky image. Various locations of fusion within the model were performed in order to optimize for accuracy as well as for model size and parameter count.
It was found that the best Multifocus SUNEST Nowcast model outperformed the baseline model by 9.8% on the test set when considering images without border masking and by 14.8% when considering images with border masking. This large improvement comes from better performance on cloudy test days. Further, the best Multifocus SUNSET Nowcast model fused the two branches after the fully connected layers but before the regression. However, the multifocus model that fuses after the 1st convolution layer had a similar performance on the validation set with far fewer parameters. Thus, it may be worth utilizing the multifocus model that fuses after the 1st convolution layer instead if computational resources are limited. When considering the best Multifocus SUNSET Forecast model, with respect to the baseline model, only minor improvements in performance were noticed on the order of 0.01 kW measured as RMSE, relative to baseline RMSE of 2.62 kW. Thus, the Multifocus SUNSET Forecast model may not be the best choice for forecasting because it incurs additional computational costs compared to the baseline model without improving performance.

Keywords: Solar forecasting
Convolutional Neural Networks
Machine learning
Deep learning
Text
Thesis
Advisor: Brandt Adam
Multi-level resilience of small-scale fisheries in Baja California, Mexico

Social-ecological systems (SES) are those that couple the human and natural world through resource exploitation and utilization. They are intricate multi-layered systems that are defined by a complex network of interactions between the resource units, resource systems, governance systems and users within the SES, which can result in a variety of different outcomes. Global climate change is increasingly putting communities in these systems, and the resources they rely on at risk. As such, economically and ecologically important resources within SES are facing stress on two fronts, human extraction, and human induced climate change. Understanding these stressors individually and deciphering the interplay between the two is crucial for informed decision-making to promote resilience of a system and may be the key to the longevity of these SESs. In this dissertation, I explore how current fishing practices and future climate conditions interact across three components of the social-ecological system framework within the context of cooperative fisheries systems of Baja California, Mexico. First, I assess population structure and catch demographics of an economically important fishery resource unit, highlighting the occurrence of multiple recaptures of sub-legal individuals and potential need for bycatch reduction in the face of future climate scenarios. Expanding out, I then adapt a data-poor vulnerability assessment methodology to assess risk of fisheries species within a resource system to multiple climate change stressors. I provide considerations of drivers of risk to climate change between groups of species and highlight the need to account for compounding pressure of multiple climate stressors, multidimensional risk from fishing and climate, and spatial scale of application of the methodology when assessing results for management decisions. Finally, I survey resource users within a Baja fishing cooperative on their knowledge and access to information about climate change and find broad understanding of climate change, spotlight the importance of informal methods of knowledge acquisition, and provide recommendations on the utility of engaging stakeholders in information sharing through social media platforms. In studying multiple components of an SES, I argue co-consideration of fishing and climate impact understanding at every level is crucial for implementation of management schemes for future resilience

Keywords: Biology.
Advisor: Micheli Fiorenza
Crowder Larry B.
De Leo Giulio A.
Multimodal, Self-Supervised Deep Learning-Based Estimation of Symptoms and Severity of Depression and Anxiety

Depression and anxiety are common mental health disorders, affecting 264 million and 284 million people worldwide, respectively. Although effective treatments for depression exist, healthcare providers in America diagnose only 47% of patients. Consequently, barriers to effective care include the lack of accurate, feasible screening and diagnosis tools. Currently, screening and diagnosis methods employ self-reported patient surveys, which are highly subjective, along with in-person clinical interviews to assess symptom severity, which are resource-intensive due to shortage of clinicians. In this work, we collect data from 110 patients before their regularly scheduled visit, either in-person or virtual, to the Stanford Family Medicine clinic. We collect ground truth PHQ-9 and GAD-7 scores, which are clinically validated instruments to screen for depression and anxiety, and record video, audio, and text transcriptions of patient responses to 11 questions. We train unimodal deep learning models that use audio, video, or text transcriptions of the conversations to detect the presence (binary classification) and severity (four-class classification) of depression and anxiety. We also train multimodal models that use all three modalities. Multimodal deep learning models detect the presence and severity of depression and anxiety with high balanced accuracy: 93.8% and 60.4% for depression, and 100% and 62.5% for anxiety. Additionally, we utilize a self-supervised, multi-modal pre-training scheme that jointly pre-trains audio, video, and text autoencoders to converge representations across the three modalities, followed by decoders that classify the patient into score buckets. Finally, we investigate model interpretability using saliency mapping to uncover specific behavioral markers, such as facial areas and emotionally valent words, that our models utilize in making estimations of depression or anxiety. Saliency mapping also illuminates specific questions out of the 11 total questions that provide the maximum signal to our text-based models. These results have both high-level technical and clinical significance. Technically, we suggest ways of conducting optimal representation learning in small data regimes by leveraging semantic alignment in data across several modalities. Clinically, we show the potential of an objective, automated tool for depression and anxiety screening that takes an average of three minutes per person to administer during casual conversations in clinical settings. Such a tool could be easily incorporated into current clinical workflows and facilitate low-cost, universal access to mental health care.

Keywords: Computer Science
Machine learning
Deep learning (Machine learning)
Depression, Mental > Diagnosis
Anxiety disorders
Text
Thesis
Advisor: Adeli Ehsan
Li Fei-Fei
Multi-omic characterization of segmental graft dysfunction in liver transplant patients

Liver transplantation was pioneered in canine models in the 1950s and extended to humans in the 1960s. Since then, improvements in recipient-donor matching, organ allocation policies, and immunosuppression have made transplantation a viable intervention for patients with end-stage liver disease. Nowadays, the 5-year survival rate is 75%, with the majority of transplant failures occurring within 90 days after surgery. The success of liver transplantation is overshadowed by the significant organ shortage which motivated the inclusion of living donors. Within the first week after receiving a partial liver graft, patients could experience segmental graft dysfunction (SGD), a form of early allograft dysfunction (EAD). Although the majority of patients recover, an SGD diagnosis is still predictive of graft failure. This thesis focuses on determining the molecular signatures underpinning the development of SGD and developing predictive models of graft failure to identify high-risk patients

Keywords: Chemistry.
Advisor: Snyder Michael
Bertozzi Carolyn R.
Khosla Chaitan
Multiomics reveals critical metabolites for colonization of salmonella in superspreaders hosts

The molecular understanding of host-pathogen interactions in the GI tract of superspreader hosts is incomplete. In a mouse model of chronic, asymptomatic Salmonella enterica serovar Typhimurium (S. Tm) infection, we performed untargeted metabolomics on the feces of mice and found superspreader hosts possess distinct metabolic signatures compared to non-superspreaders, including differential levels of L-arabinose. RNA-seq on S. Tm in superspreader fecal samples showed increased expression of the L-arabinose catabolism pathway in vivo. By combining bacterial genetics and diet manipulation, we demonstrate that diet-derived L-arabinose provides S. Tm a competitive advantage in the GI tract. We show that the expansion of S. Tm in the GI tract requires a previously uncharacterized alpha-N-arabinofuranosidase that can liberate L-arabinose from dietary polysaccharides. Ultimately, we demonstrate that pathogen-liberated L-arabinose from the diet provides a competitive advantage to S. Tm in vivo. These findings propose L-arabinose as a critical driver of S. Tm expansion in the GI tracts of superspreader hosts

Keywords: Microbiology and Immunology.
Advisor: Monack Denise M.
Bhatt Ami (Ami Bharat)
Bollyky Paul
Sonnenburg Justin