VictorJuiz/thesis huggingface dataset:
Model-Based Design for Legged Robots: Predictive Control and Reinforcement Learning

Legged robots have the potential to provide extreme mobility in highly rugged terrain. Developing such locomotion capabilities within these robots is challenging for several reasons, including their inherent hybrid and nonlinear dynamics. Moreover, uncertainties in the robot's model arising due to external payloads, joint friction, wear and tear, and uncertainties introduced by environmental factors such as changing contact conditions or force perturbations can significantly hinder the performance and reliability of these robots. Developing safe, reliable, and robust feedback controllers is crucial as we begin to deploy such robots into environments with humans. Moreover, the underlying feedback controllers must be able to quickly adapt to rapidly changing environmental factors that can significantly affect their performance./nRecent developments in Reinforcement Learning have shown tremendous success and impressive results on several legged robot platforms to navigate challenging terrain. While these methods can generate very complex behaviors, they are highly sample inefficient as they do not take into account any knowledge of the dynamical structure of the robot. Model-based methods such as Control Lyapunov Functions (CLFs), Control Barrier Functions (CBFs), and Model Predictive Control (MPC), on the other hand, provide us with a set of tools to achieve desired control objectives while remaining within specified constraints for the closed-loop system. The performance of both RL-based control and model-based methods can significantly degrade if an accurate model of the underlying system is not known. In such cases, achieving the best performance will require learning from real-world data. This thesis develops planning algorithms and feedback controls through the lens of model-based techniques to achieve safe, stable, and robust legged locomotion on challenging terrain. In particular, we present a trajectory generation method to achieve aperiodic running with precise foot placement for a 2D bipedal robot model. We then develop a novel coordinate-free geometric MPC for the Cassie biped and validate our approach through several hardware experiments. We apply the developed trajectory generation method and geometric MPC on a quadruped robot to navigate challenging terrain with visual feedback. Using tools from model-based methods such as CLFs and CBFs, we then develop novel reward function shaping methods to achieve safe and robust locomotion policies for bipedal and quadrupedal robots. We show that our method can significantly reduce the sample complexity to learn a stabilizing controller, which allows us to finetune policies directly on hardware using only a few seconds to a few minutes of data.

Advisor: Sreenath, Koushil
Towards Adaptive, Continual Embodied Agents

In recent years, artificial learning systems have demonstrated tremendous advances on a number of challenging domains such as computer vision, natural language processing and speech recognition. A striking characteristic of these recent advances has been the seemingly simple formula of combining flexible deep function approximators with large datasets collected for specific problems. These systems struggle however to leverage their learned capabilities when generalizing to new inputs for acquiring new capabilities, often requiring re-training from scratch on a similarly large dataset from scratch. This is in stark contrast to humans, who have a remarkable ability to build upon their prior experiences and learn new concepts from only a few examples. In the first part of this thesis, we will study the question of how to construct systems that mimic this ability to adapt rapidly to new tasks. One of the core principles that will underlie this part of the thesis will be to leverage structure in a large number of prior experiences/tasks to enable fast adaptation and uncertainty. We will start first by studying the setting of reward specification, a common challenge in reinforcement learning, and next study how a probabilistic framing of the meta-learning setting can enable reasoning under uncertainty./nIn the second part of this thesis, given the established potential that a prior datasets of tasks can play in accelerating learning, we will ask the natural question of how to enable agents to collect data completely autonomously. This would remove the need of a human to "curate" the dataset of tasks for the artificial agent and enable fully scalable never ending embodied learning. The central theme of the approach we take will be to consider the online real world nature of "tasks" that an agent must solve, and through it revisit the basic assumptions of episodic RL. Finally, we will conclude with a demonstration of these ideas in the domain of real world dexterous manipulation and provide some hints for future work in this "autonomous" reinforcement setting.

Advisor: Levine, Sergey
Large-Scale Real-World Robotic Manipulation Using Diverse Data

Recent breakthroughs in computer vision and natural language processing have been largely propelled by scaling up both dataset diversity as well as model capacity, leading to robust generalization. In this thesis I am addressing the question of 1) whether for learning-based robotic manipulation we can similarly scale up dataset diversity and model capacity in order to achieve generalization and adaptation to new scenes and environments, new objects, new tasks and even different types of robots, and 2) the question of how re-collecting data from scratch for every new task and environment can be avoided, since this often leads to poor generalization and performance. To answer these questions we propose two different methodologies, a model-based reinforcement learning approach based on video-prediction, and a model-free and imitation-learning-based approach. We collect several of the biggest robotic interaction datasets to date, and show that by leveraging and effectively reusing diverse prior datasets, we can allow an agent to generalize to never-before-seen objects, learn new tasks based on only a handful of demonstrations, and even adapt to new robot types.

Advisor: Levine, Sergey
Finn, Chelsea
Quantum sensing at high pressures using nitrogen-vacancy centers in diamond

Pressure alters all properties of matter. The development of the diamond anvil cell enables the exploration of high pressure phenomena in tabletop experiments. This thesis introduces an optical sensing platform based on nitrogen-vacancy color centers embedded into the culet (tip) of a diamond anvil. The micron-scale spatial resolution and high sensitivity of this platform to stress, magnetic and electric fields opens the door to a hitherto unexplored range of experiments. We demonstrate the versatility of this platform through several applications. In the context of stress, we demonstrate imaging of all normal and shear components of the stress tensor, which enables us to probe the accumulation and dissipation of shear stresses of chrysotile serpentine (Mg_3(Si_2 O_5)(OH)_4) undergoing brittle failure. In the context of magnetism, we demonstrate imaging of vector magnetic fields under gigapascal pressure, enabling measurement of the \alpha-->\epsilon transition in elemental iron as well as the complex pressure-temperature phase diagram of elemental gadolinium. We further extend these magnetic imaging capabilities to probe the pressure-induced demagnetization of 4C monoclinic pyrrhotite (Fe_7 S_8), an iron sulfide mineral found in the Earth's crust as well as in Martian and chondritic meteorites. In the context of electric fields, we show that a spectral feature commonly observed in NV centers originates from the local charge environment of the diamond lattice, and we utilize this understanding to image individual electronic charges with nanometer precision. We extend this understanding to the NV center orbital excited state, whose strong coupling to electric fields enables a protocol that enhances measurement sensitivity by several orders of magnitude. Finally, motivated by the sensitivity of NV centers to electric noise, we theoretically consider polarization fluctuations from polar and dielectric materials and show that this electric noise encodes valuable information about dielectric properties over a range of frequencies and length scales.

Advisor: Yao, Norman
Lower bounds on the complexity of quantum proofs

The quantum PCP conjecture is one of the central open questions in quantum complexity theory.It asserts that calculating even a rough approximation to the ground energy of a local Hamiltonian is intractable even for quantum devices. The widely believed separation between the complexity classes NP and QMA necessitates that polynomial length classical proofs do not exist for calculating the ground energy. This further implies that low-energy states of local Hamiltonians cannot be described by constant depth quantum circuits. The No low-energy trivial states (NLTS) conjecture by Freedman and Hastings posited the existence of such Hamiltonians./nThis thesis describes a line of research culminating in a proof of the NLTS conjecture, first presentedby Anshu, Breuckmann, and Nirkhe. The construction is based on quantum error correction and the thesis elaborates on how error correction, local Hamiltonians, and low-depth quantum circuits are related.

Advisor: Vazirani, Umesh
Looking Up from the Page: Scenes of Reading in Medieval and Modern German Literature

This dissertation examines scenes of reading – a literary motif where one or more figures are portrayed in the act of reading – in German literature from the High Middle Ages to today. Against the pressure to interpret such scenes as depictions of historic or exemplary reading practices, I analyze them as imaginative theorizations of reading, as nexuses of imagery, form, and content that reflect on, explicate, and influence the role of reading in a textual culture. As an integral part of a text’s poetical structure, these scenes render the reading process visible, exposing it to critique and reflection. In my analysis of several texts, including a medieval Arthurian romance, a late medieval devotional text, a nineteenth-century novella, and a contemporary narrative of migration, I demonstrate how the matrix of people, practices, and technologies that constitute reading scenes have affected and continue to affect popular as well as academic discourses on this fundamental cultural practice. By elucidating the theoretical and meaning-making capacities of scenes of reading, my project offers a unique approach to the study of reading: it incorporates the diversity of historic reading practices with the symbolic potency of reading’s representation to explicate reading’s poetological role in literature.

Advisor: Largier, Niklaus
Fabrication and Testing of Zinc Oxide Superhydrophilic Nanostructured Surfaces on Metal Substrates and Wavy Fin Geometries for Use in Spray Cooling

The vaporization process in the spray cooling of power plant condensers can greatly be enhanced by applying a ZnO nanostructure to the heat exchanger. Experiments have shown that the superhydrophilicity of the ZnO nanostructure on copper, results in ultra-low apparent contact angles that increase the wetting of liquid water and facilitate evaporation. While previous studies focused on ZnO nanostructures on copper substrates, this research demonstrates the successful fabrication of ZnO nanostructures on three non-copper metallic substrates. In the research reported here, single droplet adiabatic and evaporation experiments on the ZnO nanostructure coated surfaces of aluminum, stainless steel, and titanium substrates, result in increased wettability and heat transfer performance compared with the bare substrate surfaces. This enhancement offers insight about applying ZnO nanoporous coatings to complex heat exchangers./nFurthermore, this research tested the direct evaporative cooling of plain and ZnO nanostructured coated copper wavy fins. Three copper wavy fin matrices were fabricated and their ability to precool air was tested in a custom-built test loop consisting of an airflow and water flow system. Fabrication of a ZnO nanostructure coating on the three wavy fin matrices exemplified and demonstrated the feasibility of scaling the ZnO nanostructure to larger geometries. The presence of the ZnO nanostructure on the wavy fins enhances the matrix fin cooling effectiveness, which is a parametric measure of the matrix’s ability to precool air. Machine learning tools were used to extract fundamental heat transfer and wetting performance parameters from the evaporative cooling results. A genetic algorithm was used to predict the heat transfer coefficient and wetted interfacial area of the fin matrices at various flow conditions. In addition to providing greater insight about the performance of the heat exchanger, a genetic algorithm was used establish a basis for optimizing the design of a wavy fin matrix coated with a ZnO nanostructure. Overall, the findings presented in this investigation demonstrated the potential advantages of using an evaporative cooler/eliminator coated with a ZnO nanostructure to precool air in an air cooled condenser in Rankine power plants and building HVAC systems.

Advisor: Carey, Van
Symmetry Dictated Properties in Two-dimensional Systems

The physical properties of crystalized materials are closely related to the symmetry elements of the point group of the crystal. To the contrary, engineered symmetry breaking would allow the emergence of new physical phenomena which are originally forbidden. With the successful exfoliation of two-dimensional Van der Waals materials, the restricted stacking sequence between layers is relaxed and thus, the symmetry of artificially stacked heterostructure can be readily controlled./nIn this dissertation, we present two approaches to engineer the symmetry of solid-state and photonic two-dimensional systems respectively. The resulting optical and topological property changes are explored and related back to the underlying symmetry breaking. Chapter 1 gives an overview of the various symmetry breaking mechanisms and their distinct consequences. The emphasis is given to the inversion-symmetry breaking which can be easily implemented in two-dimensional systems via structural or electrical means. In Chapter 2, we show that introducing a twisting between two layers of graphene, the symmetry of bilayer structure is decisively changed, and new nonlinear optical effects are allowable. We further verify that the observed nonlinear signals are controlled by the composited bilayer instead of the linear combination of two individual layers experimentally and confirmed by DFT calculations. The methodology proposed is generic and can be applied to other composite systems. In Chapter 3, we break the symmetry of two-dimensional photonic lattice by ‘tilting’ the constitutive parameter tensors and demonstrate spin degeneracy lifting via electromagnetic simulations. The approach allows us to enlarge the scale of spin-orbit coupling effect in a photonic platform and at the same time, the controllability of spin-polarized energy bands associated with symmetry is revealed./nTo sum up, we propose two methods to engineer the symmetry of the 2D system in a controllable way and show their potential for new physical properties and applications.

Advisor: Yao, Jie
Efficient and trusted operation of quantum computers and quantum simulators

Quantum computers have the potential to revolutionize the computational power accessible to humanity. But because the scale and quality of current devices is limited, maximizing their usefulness requires that researchers make efficient use of the resources that they provide./nMotivated by this, we introduce in this work several techniques for efficient verification and operation of near-term quantum devices. We first develop and demonstrate a stochastic quantum compilation technique (STOQ) which produces approximate compilations of unitaries in terms of any arbitrary native instruction set. We then construct a novel verification protocol for quantum devices called randomized analog verification (RAV), which uses STOQ to generate verification sequences that ideally leave the system near a measurement basis state, which allows for efficient estimation of the success rate./nWe first apply RAV to the context of quantum computers with continuously-parameterized native gate sets. We demonstrate both numerically and experimentally that RAV provides an efficiency advantage over cross-entropy benchmarking (XEB) when estimating an error rate experimentally. We then adapt the RAV protocol for verification of analog quantum simulators, and we demonstrate its sensitivity to various types of error both numerically and experimentally./nIn addition, we discuss practical techniques for efficient operation of quantum computers. We develop and implement a surrogate-based optimization (SBO) technique for variational quantum algorithms, and we demonstrate that the technique has significant performance advantages over the most commonly-used optimization methods. Then, applying a more experimental lens, we describe work toward realistic simulation of various aspects of a trapped-ion experiment, such as experimental control sequences and calibration runs./nTaken together, the techniques introduced in this work are a collection of incremental steps toward the broader goal of increasing efficiency in near-term quantum computers and quantum simulators.

Advisor: Häffner, Hartmut
Diagnosing oxidation in the free troposphere and in cities over North America using space-based observations

The oxidative capacity of the troposphere – primarily characterized by the burden of the most abundant and reactive of oxidants, the hydroxyl radical (OH) – determines the lifetime of many trace gases of importance to climate and human health, including air pollutants and the greenhouse gas methane. In the upper troposphere, there is large uncertainty in the oxidative capacity associated with the lightning NOx production, for which reported results range from 16-700 mol NO flash-1. I implemented a lightning parameterization which significantly improves the representation of lightning in one chemical transport and and showed that this parameterization should be more effective in any model. I combined the model simulations configured with this new lightning parameterization with the satellite observations of NO2 column to yield a better estimate of lightning NOx production over the continental US. In cities, urban OH controls the removal rate of primary pollutants and triggers the production of ozone. Interannual trends of OH in urban areas are not well documented or understood due to the short lifetime and high spatial heterogeneity of OH and of OH precursors. Here I synthesized a machine learning technique, satellite observations and simulations from a state-of-art chemical transport model to estimate OH trends between 2005 and 2014 in 49 North American cities. I described trends in the summertime OH with wide variation among different cities. The variation of OH is explained by the shift in chemical regime from one where additional NOx slows chemistry to one where additional NOx speeds chemistry over the years. The identification of chemical regime, in turn, sheds light on the effective policy for controlling ozone.

Advisor: Cohen, Ronald
Learning to Generalize in Dynamic Environments

A model must be able to adapt itself to generalize to new environments. Deep networks achieved great success in the past decade, especially when training and testing data come from the same distribution. Unfortunately, the performance suﬀers when the training (source) diﬀers from the testing (target) data, a condition known as domain shift. Models should update themselves to deal with these unexpected natural and adversarial perturbations, such as weather change, sensor degradation, adversarial attack, and so on. If we have some labeled target data, several transfer learning methods, such as ﬁne-tuning and few-shot learning, could be utilized to optimize the model in a supervised way. However, the requirement for target labels is not practical for most real-world scenarios. Therefore, we instead focus on the unsupervised learning approach to generalize the model to the target domain./nIn this dissertation, we study the setting of fully test-time adaptation, updating the model to the uncontrollable target data distribution, without access to target labels and source data. In other words, the model only has its parameters and unlabeled target data in this setting. The core idea is to leverage the test-time optimization objective, entropy minimization, as a feedback mechanism to the learnable model to close the loop during the test time. We optimize the model for conﬁdence as measured by output entropy in either an online or oﬄine manner. Such a simple yet eﬀective method could reduce the generalization error for image classiﬁcation on naturally corrupted and adversarial perturbed images. Also, the adaptive nature of the semantic segmentation model could be exploited to cope with the dynamic scale inference for scene understanding. With the help of contrastive learning and diﬀusion models, we could learn target domain features and generate source-style images to further boost the recognition performance in dynamic environments.

Advisor: Darrell, Trevor
Principle and applications of quantum metrology in diamond anvil cells using the Nitrogen Vacancy color center

Tuning pressure using diamond anvil cells (DACs) provides an interesting playground for exploring novel condensed phases and understanding important geophysical processes. However, the extension of conventional probes for materials characterization to extreme pressure environments is often constrained by large stress gradients and limited signal strengths. Here, I outline a novel platform for high pressure metrology in DACs based on the Nitrogen Vacancy (NV) color center in diamond. By integrating the NV center, a robust and versatile quantum sensor, directly into the diamond culet, we demonstrate in situ vector magnetometry and tensorial stress imaging with high sensitivity and diffraction-limited sub-micron spatial resolution. Proof of principle measurements demonstrate the platform’s versatility by imaging the dipole strength of Iron (Fe) across the α ↔ ε structural transition, mapping the full stress tensor across the culet surface, and exploring the pressure-temperature phase diagram of gadolinium (Gd). Applying this technique in synergy with piezoelectric displacement sensing and X-ray diffraction (XRD) to study the amorphization of chrysotile Serpentine, we observe phase transformational nanoseismicity driven by shear stress - a possible mechanism underpinning deep earthquakes. Furthermore, we explore surprising qualitative changes in the behavior of the NV center up to megabar pressures, motivating further studies to understand the spin-orbit, and spin-spin coupling in the excited electronic state and the inter-system crossing mechanism. We demonstrate the viability of using the NV center as a robust pressure calibrant capable of easily measuring spatial gradients in multiple stress components. By playing with different crystal cuts of diamond anvils, we extend continuous wave and pulsed NV sensing to pressures nearing ∼ 150 GPa achieving excellent signal contrast and sensitivity. Finally, we use simultaneous transport measurements and NV magnetometry to probe the dual hallmarks superconductivity in a superhydride material at megabar pressures. Our work constitutes the first spatially resolved measurement of the Meissner effect and provides clear evidence of flux trapping in these novel compounds.

Advisor: Yao, Norman Y