VictorJuiz/thesis huggingface dataset:
Sample Complexity Bounds for the Linear Quadratic Regulator

Reinforcement learning (RL) has demonstrated impressive performance in various domains such as video games, Go, robotic locomotion, and manipulation tasks. As we turn towards RL to power autonomous systems in the physical world, a natural question to ask is, how do we ensure that the behavior observed in the laboratory reflects the behavior that occurs when systems are deployed in the real world? How much data do we need to collect in order to learn how to control a system with a high degree of confidence?/nThis thesis takes a step towards answering these questions by establishing the Linear Quadratic Regulator (LQR) as a baseline for comparison of RL algorithms. LQR is a fundamental problem in optimal control theory for which the exact solution is efficiently computable with perfect knowledge of the underlying dynamics. This makes LQR well suited as a baseline for studying the sample complexity of RL algorithms which learn how to control from observing repeated interactions with the system./nThe first part of this thesis focuses on model-based algorithms which estimate a model of the underlying system, and then build a controller based on the estimated dynamics. We show that the classic certainty equivalence controller, which discards confidence intervals surrounding the estimated dynamics, is efficient in regimes of low uncertainty. For regimes of moderate uncertainty, we propose a new model-based algorithm based on robust optimization, and show that it is also sample efficient./nThe second part studies model-free algorithms which learn intermediate representations instead, or directly search for the parameters of the optimal controller. We first look at the classical least-squares policy iteration algorithm, and establish an upper bound on its sample complexity. We then use tools from asymptotic statistics to characterize the asymptotic behavior of both the certainty equivalence controller and the popular policy gradient method on a particular family of LQR instances, which allows us to directly compare the bounds. This comparison reveals that the model-free policy gradient method has polynomial in state/input dimension and horizon length worse sample complexity than the model-based certainty equivalence controller. Our experiments corroborate this finding and show that model-based algorithms are more sample efficient than model-free algorithms for LQR.

Advisor: Recht, Benjamin
Feeling Alive: Unofficial Jewish Practices in the USSR in the 1970s and 1980s

This dissertation explores unofficial “Jewish practices” in the Soviet Union in the 1970s and 1980s. The interest and participation in such practices in the late Soviet period are usually analyzed within either “totalitarian” or Zionist frameworks, which represent them as manifestations of “anti-Soviet” sentiments and goals, and as part of a political movement struggling for the chance to leave the Soviet Union, particularly for Israel within the Zionist framework. However, I argue, this interpretation reflects much less the real Soviet context than the set of assumptions about the Soviet system that have been shaped in Western liberal discourses during and since the Cold War. In this dissertation I show that Jewish practices and pursuits in the late Soviet period could not be grouped under the concept of a “political movement” or reduced to an anti-Soviet agenda and Zionist activity. Instead, they should be understood as a particular example of “searching for the extraordinary” – a much broader cultural phenomenon that developed among the Soviet intelligentsia during that period. My informants often mentioned a unique feeling of “liveliness” that they felt when engaging with Jewish practices and pursuits. Sometimes they linked this feeling with “freedom” – a concept that cannot be easily equated with the liberal conception of freedom understood in terms of individual choice. In fact, many of them also reported that after having emigrated from the Soviet Union to market democracies (Israel, USA, Canada etc.) they, surprisingly, experienced a “loss” of freedom. Analyzing what the experience of “freedom” might mean in the Soviet context, and what unique forms it could take in the context of the “Jewish practices,” I argue that a number of ideological, economic and cultural realities of state socialism made this experience of freedom part and parcel of the Soviet system itself. From this perspective, gaining the actual right to emigrate was just as important as, if not more important than, performing the actual act of emigration. The real “Exodus” happened within the borders of the Soviet Union rather than outside of them. This dissertation further reflects on the role of the global political context in shaping the unofficial Jewish practices and pursuits in the Soviet Union and in giving them a unique form that made them different from many other unofficial practices that developed around that time.

Advisor: Yurchak, Alexei
Passionate Failures: The Diva Onscreen

I place the origins of the diva in the 19th century artistic movement l’art pour l’art (“art for art’s sake”) in order to show that the diva was born of mechanization and the reaction against it. Therefore, to consider the diva “onscreen” is to acknowledge the very dialectic that birthed this paradoxical prima donna: uniqueness vs. reproduction, private vs. public, ritual vs. politics./nI understand the notion of the female voice as a way to anchor the meaning of “diva,” which has come to be used too promiscuously, sometimes denoting just about anyone whose manner of self-presentation puts them on some imagined scale between self-confidence and vanity. I do not aim to offer a strict definition of the term, but I do aim to draw from certain tropes of the presentation of the diva onscreen the contours of particular relations of desire that hem the specificity of what “diva” might mean. The common ancestor of the operatic prima donna, I argue, accounts for many other iterations of the diva type, which goes well beyond singers. The virtuosic female voice has historically been thought of as “expressing the inexpressible,” a melodramatic striving for an unattainable meaning that is also manifest in the grand theatrical postures of 19th century actors, in the violent and tortured D’Annunzian body language of the Italian silent film divas, as well as in the languid and swooning gestures of Hollywood silent film stars, and in popular American singers of the 20th century, such as Judy Garland, Barbra Streisand, Liza Minnelli, and Whitney Houston./nIn this dissertation, I define “diva” as a cultural type that has broadened beyond opera singers to include nearly any woman with a theatrical manner of self-presentation who is narrated as a “public success,” and a “private failure.” Still, I posit that it is crucial to understand the diva’s origins in discourse around female opera singers because the diva’s overdetermined relationship to the notion of presence is determined by a history of ideas regarding the female voice. I divide the dissertation into three tropes surrounding the diva: The Voice, The Narcissist, and The Comeback. Using studies in musicology, and citing a history of ideas about the female voice, including psychoanalytic notions and queer theory, I explain the importance of the fact that “diva” came into currency particularly in association with a series of two-soprano duets in the late 1820s. It is this redoubled and self-referential femininity, coupled with ideas of the female voice as “beyond language” which trouble standard regimes of representation, such as those codified by mainstream cinema. Finally, I use theories of melodrama and cinematic time to explain the meaning of “the comeback” and the diva’s relation to nostalgia and pathos.

Advisor: Williams, Linda
Urbanization, its Exposure to Environmental Hazards, and Equity in Adaptation Programs

Urban areas are increasingly exposed to weather-related environmental hazards under climate change, and equity concerns exist in adaptation and mitigation programs for these exposures and their consequences. To address these issues above, we used data-driven approaches to investigate three unique cases. We first developed a novel, cost-effective, and spatially-explicit classification of urbanization trends over time, using time-series nighttime light remote sensing images and unsupervised classification, in mainland China between 1992 and 2013. We identified five temporal typologies of urbanization, namely stable urban activity, high-level steady growth, acceleration, low-level steady growth, and fluctuation. Our classification characterizes distinct urbanization patterns over time and can be applied to environmentally sensitive and hazard-prone areas where monitoring of urbanization is critical. Next, we perform a multitemporal and multi-scenario projection of exposure to flooding, caused by sea level rise and storm surge, in the highly urbanized San Francisco Bay Area. We found increased uncertainty in exposure over time and in scenarios with higher greenhouse gas concentrations. Such elevated uncertainty suggests that stakeholders should employ adaptation strategies that are no-regret, reversible, and flexible, and that regulators may explicitly require a long-term planning horizon for adaptation programs and new developments. Finally, we investigated the allocation of a widely-adopted adaptation and mitigation program, clean vehicle rebates, from two major policy programs in California. We evaluated rebate allocation rates with respect to community characteristics including socioeconomic and environmental disadvantages, household income, racial and ethnical composition, and ambient air pollution. We found that when rebate assignment and amount were only based on vehicle technology and did not consider the varied socioeconomic backgrounds of potential applicants, rebate allocation rates were higher in advantaged, wealthier communities, and communities with intermediate levels of nitrogen dioxide concentration, but the rates were lower in communities with higher percentages of Hispanics and Non-Hispanic Blacks. After introducing an income cap, expanded vehicle eligibility, and income- and geography-tiered rebate amounts, rebate allocation rates increased in communities with lower-household income, higher percentages of Hispanics, and slightly higher nitrogen dioxide concentration. The findings of this study implied the need for and effectiveness of equity-related policy designs to spread the benefits of adaptation and mitigation programs to more diverse populations. In all, these studies seek to engage a broader conversation about environmental and societal challenges in urban areas under climate change, as well as how data-driven approaches can reveal the underlying processes of these challenges and inform better policy and decision-making.

Advisor: Radke, John D
Production of Non-Multiple-of-Five Terpenes Using the Lepidoptera Mevalonate Pathway and ATP Positive Pathway for Five-Carbon Alcohol Production

Metabolism represents at least three and a half billion years of research and development in chemical synthesis and degradation. Human have been taking advantage of this resource for thousands of years, first with the advent of agriculture and the invention of cheese, beer and other fermentation products. More recently selective breeding, metabolic engineering and synthetic biology have given humanity more control over organisms’ metabolism, enabling food, chemical and fuel production on massive scales. Given the extensive previous work optimize existing metabolism to make products, the two most exciting frontiers for metabolic engineering are 1) making new chemicals and 2) making old chemicals with new and improved pathways. To explore the area of new chemical production, we have heterologously expressed the lepidopteran mevalonate pathway in Escherichia coli to make sixteen carbon homoterpenes. Because the pathway is modular, we demonstrate that making novel homoterpenes requires only altering one gene. As terpenes are an economically valuable class of molecules, we hope that this pathway will enable the exploration of new chemical space difficult to search with traditional organic synthesis techniques. To explore the area of new pathways we designed a pathway to make the five-carbon alcohols prenol and isopentanol. While there are already pathways to five-carbon alcohols, the novel pathway does not require ATP. In fact, if glucose is the carbon source, the pathway is ATP positive when redox balanced with non-oxidative glycolysis. Because of this it is theoretically possible to ferment these five carbon alcohols the same way microbes ferment ethanol, potentially easing many engineering problems around titer improvements and scaling to industrial production levels. Finally, the challenges of CO2 emissions will be considered. Many technologies will need to be developed to sustain a prosperous global society. While some of these technologies will be based in traditional engineering fields, biology and metabolism offer unique opportunities and will be critical to enabling a CO2 neutral civilization in the future.

Advisor: Keasling, Jay D
Norme: a chamber opera

This dissertation is an opera for three singers and eleven instrumentalists, divided into six groups: (1) piano; (2) soprano, flute, clarinet, soprano saxophone; (3) harp; (4) soprano, percussion (one player); (5) string quartet; (6) soprano, percussion (one player). The document includes a program note, performance notes, libretto (authored by the composer), notation guide, and score (in C).

Advisor: Campion, Edmund
Understanding Grafted Cations in Controlled Environments for Heterogeneous Catalysis

Catalytic processes occurring on the surfaces of heterogeneous catalysts are controlled by the molecular structures of active sites where these reactions occur. These active sites can be broadly thought to consist of an active center, where bond making and breaking events occur, surrounding by the surface of the support. These are the inner-sphere (i.e. active center) and outer-sphere (i.e. surface) environments of the active site. Catalyst design typically focuses on the choice of the optimal inner-sphere environment, while surfaces are often regarded as inert oxide supports onto which active sites are dispersed to facilitate catalyst recovery. In this thesis, I demonstrate that the outer-sphere surface environment is, in fact, an essential element for controlling the structure and reactivity of active sites supported on silicates./nThe theoretical concepts, silicate supports, and synthetic approaches that are used in this thesis are introduced in Chapter 1. Given the importance of silanol groups as grafting sites in synthetic approaches used in this thesis, I begin by providing a detailed study of silanol speciation across zeotypes and amorphous supports, in Chapter 2. Subsequently, I provide an example of how silanol environments control grafting processes and how crystalline silicates provide stable support environments for FeIII cations, in Chapter 3. I then introduce an approach to synthesizing well-defined active sites by controlling the structure of a grafted cation using an organic ligand, applied to calix[4]arene-TiIV complexes grafted on amorphous SiO2 as epoxidation catalysts, in Chapter 4./nHaving established the structure of silicates and approaches to synthesize well-defined active sites on their surface, I present three studies where this enables the study of structure and catalytic properties. In Chapter 5, I demonstrate how this approach enables the unambiguous deconvolution of the effect of support outer-sphere on epoxidation catalysis. In Chapter 6, I investigate how the support outer-sphere can also control the conformation and structure of grafted complexes, while providing insight into adsorption processes occurring on surfaces. Finally, in Chapter 7, I provide a detailed mechanistic study of how partially confining outer-sphere environments impact catalytic reactivity for olefin epoxidation./nTaken together, this work provides fresh insights into the structure of silicate supports and their ability to control catalysis, providing an additional and important avenue to the design of heterogeneous catalysts.

Advisor: Katz, Alexander
A Variational Principle for Modeling Electronic Excitations in Gas and Condensed Phase

Accurate modeling of electronic excited states is one of the most important and challenging problems in electronic structure theory. This thesis focuses on a recently developed excited state variational principle and its applications in gas and condensed phase. In contrast to the widely used excited states method such as linear response (LR) and many-body perturbation theory (MBPT), which find excited states by perturbing around the ground state wave function or a zeroth order particle-hole excitation picture, the new excited state variational principle directly targets excited states with the full flexibility of an approximate wave function ansatz. Due to its non-perturbative nature, this method offers balanced and systematically improvable descriptions to excited states. We will also discuss the efficient implementation of the new excited state variational principle through variational Monte Carlo and the Linear Method optimization algorithm.The new excited state variational principle is applied to predict both the excitation energies of low lying excited states in small molecules and optical gaps in solids. In molecules, the new method yields order-of-magnitude of improvements over the state-of-art excited state methods based on LR theory in double excitations. In solids, not only is the new method demonstrated to be more accurate than the commonly used MBPT method, but it could also be used to analyze and provide insights into MBPT. In order to further extend the method’s applicability, we introduce a modified optimization method that addresses a fatal memory bottleneck in the original algorithm. With only minor lose in accuracy, the modified algorithm reduces the required memory per parallel process from tens of gigabytes to hundreds of megabytes. With the aid of the new optimization method, we show that the new excited state variational principle could systematically converge the excitation energy in a strongly correlated, Mott-insulating hydrogen ring with respect to increasing flexibility in the wave function ansatzes.

Advisor: Neuscamman, Eric W
Agreement, case, and switch-reference in Amahuaca

This dissertation probes the nature of the syntactic operation of Agree through the lens of the morphosyntax of Amahuaca, an endangered Panoan language of the Peruvian Amazon. I take as my empirical focus two interrelated case studies in Amahuaca syntax: 1) the split ergative case system, and 2) the extensive switch-reference system. In the domain of case, I argue that overt ergative case morphology in Amahuaca expones agreement of the transitive subject with multiple functional heads. This leads to a distinction between the features needed for abstract ergative case (agreement only with v), and the features needed to trigger overt ergative case (agreement with both v and T). This distinction between abstract and morphological case factors into the analysis of the switch-reference system of Amahuaca, which I argue is sensitive to abstract case. In addition to case-sensitivity, Amahuaca’s switch-reference system shows the typologically unusual property of tracking the reference of all arguments of the verb, not only subjects. I propose that this system arises through adjunct complementizer agreement that probes both the adjunct and matrix arguments directly through cyclic expansion of the probe. Through these two investigations, I conclude that Amahuaca provides support for a narrowly cyclic model of Agree in which each instance of Merge defines a new cycle of Agree (Rezac 2003, 2004; Béjar and Rezac 2009). Further, the empirical facts can be most straightforwardly accounted for if we assume that some probes are insatiable, agreeing with all possible goals in their search space (Deal 2015b). Finally, despite the fact that some agreement in Amahuaca appears to be long distance, I argue that the data can be captured under the fairly conservative assumption that Agree is always under c-command and is always phase-bound.

Advisor: Deal, Amy Rose
Playfighting: Encountering Aviddhakarṇa and Bhāvivikta in Śāntarakṣita's Tattvasaṃgraha and Kamalaśīla's Pañjikā

The present study collects, translates, and analyzes the surviving fragments of two lost Naiyāyika authors, Aviddhakarṇa and Bhāvivikta, principally as they have been preserved in the works of the eighth-century Buddhist philosophers Śāntarakṣita and Kamalaśīla. (The present study argues, without coming to a definite conclusion as yet, that there is strong evidence Aviddhakarṇa and Bhāvivikta are not two distinct authors but different names for the same man.) The fragments themselves often contain fascinating and idiosyncratic arguments but are also often difficult to interpret. Unpacking them requires close consultation of major Nyāya-Vaiśeṣika works, primarily the Nyāyasūtra, Vātsyāyana’s Nyāyabhāṣya, which is the direct source material for most of the fragments, Uddyotakara’s Nyāyavārttika, which often parallels the arguments in the fragments, and Praśastapāda’s Padārthadharmasaṃgraha, which clarifies much of the technical terminology tersely packed into the most difficult of the fragments. The majority of the fragments are preserved in Kamalaśīla’s Pañjikā, his commentary on his teacher Śāntarakṣita’s Tattvasaṃgraha. Śāntarakṣita invokes, and Kamalaśīla cites, Aviddhakarṇa and Bhāvivikta in chapters concerning cosmology (2), the self (7), momentariness (8), the Vaiśeṣika categories of substance (10), quality (11), and universals (13), and the epistemological issues of perception (17), inference (18), and the existence of other means of knowledge (19). The fragments, accordingly, cover an extremely broad range of issues and, so, serve as an occasion to consider a number of questions about the intellectual commitments of the early Nyāya-Vaiśeṣika community and various disputes between Buddhists and Naiyāyikas. Several of the fragments also derive from Aviddhakarṇa’s Cārvāka commentary, allowing for a discussion of the relation between Brahmanical philosophical traditions and the materialist Cārvāka philosophy. Finally, because the fragments are preserved in the Pañjikā, they make possible a thorough analysis of the structure and style of the Tattvasaṃgraha as a whole, as well as the way the Buddhists represent the many rival thinkers they cite. The present study mirrors the structure of the Tattvasaṃgraha, using the fragments as anchor points in a reading of the text’s overall engagement with Nyāya-Vaiśeṣika, in order to argue that Śāntarakṣita organizes his work in a dialogical manner.

Advisor: von Rospatt, Alexander