VictorJuiz/thesis huggingface dataset:
Efficient and scalable transfer learning for natural language processing

Neural networks work best when trained on large amounts of data, but most labeled datasets in natural language processing (NLP) are small. As a result, neural NLP models often overfit to idiosyncrasies and artifacts in their training data rather than learning generalizable patterns. Transfer learning offers a solution: instead of learning a single task from scratch and in isolation, the model can benefit from the wealth of text on the web or other tasks with rich annotations. This additional data enables the training of bigger, more expressive networks. However, it also dramatically increases the computational cost of training, with recent models taking up to hundreds of GPU years to train. To alleviate this cost, I develop transfer learning methods that learn much more efficiently than previous approaches while remaining highly scalable. First, I present a multi-task learning algorithm based on knowledge distillation that consistently improves over single-task training even when learning many diverse tasks. I next develop Cross-View Training, which revitalizes semi-supervised learning methods from the statistical era of NLP (self-training and co-training) while taking advantage of neural methods. The resulting models outperform pre-trained LSTM language models such as ELMo while training 10x faster. Lastly, I present ELECTRA, a self-supervised pre-training method for transformer networks based on energy-based models. ELECTRA learns 4x--10x faster than previous approaches such as BERT, resulting in excellent performance on natural language understanding tasks both when trained at large scale or even when it is trained on a single GPU

Advisor: Manning Christopher D.
Jurafsky Dan
Le Quoc V.
Efficient predictive coding in neural networks in the presence of disorder and delays

Biological neural networks face a formidable task: performing reliable computations in the face of intrinsic stochasticity in individual neurons, imprecisely specified synaptic connectivity, and nonnegligible delays in synaptic transmission. A common approach to combatting such biological heterogeneity involves averaging over large redundant networks of N neurons resulting in coding errors that decrease classically as 1/sqrt(N). Recent work demonstrated a novel mechanism whereby recurrent spiking networks could efficiently encode dynamic stimuli, achieving a superclassical scaling in which coding errors decrease as 1/N. This specific mechanism involved two key ideas: predictive coding, and a tight balance, or cancellation between strong feedforward inputs and strong recurrent feedback. However, the theoretical principles governing the efficacy of balanced predictive coding and its robustness to noise, synaptic weight heterogeneity and communication delays remain poorly understood. To discover such principles, we introduce an analytically tractable rate-neuron model of balanced predictive coding, in which the degree of balance and the degree of weight disorder and noise can be dissociated unlike in previous balanced network models, and we develop a mean field theory of coding accuracy. Furthermore, we extend our analysis and elucidate the dependence of coding accuracy on delays and noise in a spiking neural model. Overall, our work provides and solves a general theoretical framework for dissecting the differential contributions neural noise, synaptic disorder, chaos, synaptic delays, and balance to the fidelity of predictive neural codes, reveals the fundamental role that balance plays in achieving superclassical scaling, and unifies previously disparate models in theoretical neuroscience

Keywords: Physics.
Advisor: Boahen Kwabena (Kwabena Adu)
Ganguli Surya
Doniach S.
Efficient simulation for complex systems

This dissertation consists of two parts, with the central topic on efficient simulation. The first part focuses on the problem of optimal stratified sampling with finitely many and infinitely many strata. We present a more general framework for analysis and design of adaptive sampling algorithms achieving optimal convergence rate. We show that asymptotic optimal variance can be achieved. The second part focuses on hospital-level COVID demand forecast, where we provide a method to generate consistent forecast intervals. We show that no stationarity assumption of the underlying point process is required for the proposed method. Furthermore, the model is computationally lightweight to estimate, as compared to epidemiology and machine learning based models

Keywords: Management Science and Engineering.
Advisor: Blanchet Jose H.
Glynn Peter W.
Bambos Nicholas
Scheinker David
Electric field driven microfluidics for acceleration of CRISPR enzyme kinetics and its application in molecular diagnostics

Rapid and early-stage screening is crucial, especially during pandemics, for early identification of infected persons and control of disease spread. CRISPR biology offers new and effective methods for rapid and accurate pathogen detection. Despite their versatility and specificity, existing CRISPR diagnostic methods suffer from the requirements of up-front nucleic acid extraction, large reagent volumes, and several manual steps — factors which prolong the process and impede use in low-resource settings. Moreover, the sensitivity and assay time for CRISPR-based approaches are fundamentally limited by the enzyme kinetic rates. In this experimental and theoretical work, we combine microfluidics, on-chip electric field control, and CRISPR to directly address limitations of current CRISPR diagnostic methods. We find that electric field gradients can be used to control and accelerate CRISPR reactions by co-focusing Cas--gRNA, reporters, and target nucleic acids within a microfluidic chip. We achieve an appropriate electric field gradient using a selective ionic focusing technique known as isotachophoresis (ITP) implemented on a microfluidic chip. In this thesis, we first develop and validate a model based on Michaelis-Menten enzyme kinetics theory and tailor it to regimes common in conventional CRISPR diagnostics. We map out an estimate of the limits of detection and achievable assay speeds of existing CRISPR-based diagnostics given current known and consistent (i.e., physically possible) kinetics rate data. We then develop a kinetics model which couples ITP physics with CRISPR reactions. We experimentally validate the model and identify key regimes for product formation and reaction acceleration in ITP. We then apply our ITP-CRISPR method to the rapid (30 min) detection of the RNA of SARS-CoV-2, the virus that causes COVID-19, starting from raw nasopharyngeal swab samples. Unlike previous CRISPR diagnostic assays, we also use ITP for automated purification of target RNA from raw samples. This electric field control enables a new modality for a suite of microfluidic CRISPR-based diagnostic assays

Keywords: Aeronautics and Astronautics.
Advisor: Santiago Juan G.
Fordyce Polly
Lele Sanjiva K. (Sanjiva Keshava)
Electrically reconfigurable phase-change antennas and metasurfaces

The success of semiconductor electronics is built on the creation of compact, low-power switching elements that offer routing, logic, and memory functions. The availability of nanoscale optical switches could have a similarly transformative impact on the development of dynamic and programmable metasurfaces, optical neural networks, and quantum information processing. Phase-change materials are uniquely suited to enable the creation of dynamically tunable optical elements and metasurfaces, as they offer high-speed electrical switching between amorphous and crystalline states with notably different optical properties. Their non-volatility also enables metasurface pixels to afford convenient programming of desired functions and a reduced power consumption. The high refractive index of phase change materials has already been harnessed to fashion them into compact optical antennas. Here, we take the next important step, by showing electrically switchable phase-change antennas and metasurfaces that offer strong, reversible, non-volatile, multi-phase switching and spectral tuning of light scattering in the visible and near-infrared spectral ranges. Their successful implementation relies on a careful joint thermal and optical optimization of each optical device that comprises a metal strip that simultaneously serves as a plasmonic resonator and a miniature heating stage. In the dissertation, three functional optical devices are demonstrated with electrical tunability. An optical antenna featuring 30% modulation of the scattered light intensity is first presented and helps us understand our approach of switching phase-change materials. Based on this knowledge, a metasurface is developed, which can afford electrical modulation of the reflectance by more than 4-fold. Lastly, optical antennas are carefully arranged together to build a metasurface that can direct light into different directions and shows promise in individual control of each antenna. The works presented in this dissertation open the opportunity to create a wide range of dynamic random access metasurfaces capable of programmable and active wavefront manipulation

Keywords: Electrical Engineering.
Advisor: Brongersma Mark L.
Fan Shanhui
Vuckovic Jelena
The electrochemical phase transformation in LiXFePO4

Phase separation induces heterogeneity and are common in many industries, such as energy storage, metallurgy, polymers and life sciences. Such heterogeneity, often in the form of spatially distributed phases of different compositions, directly impacts the performance and degradation of a material. For example, in electrochemical phase-separating materials such as lithium ion batteries, the phase separation boundaries has been shown to impact the transport and stress accumulation, thus controls the rate performance and lifetime. Understanding the spatiotemporal electrochemical transformation dynamics is therefore important; but the thermodynamically heterogeneous nature, as well as the non-linear dynamics underlying the non-equilibrium electrochemical reaction kinetics makes this goal challenging. More importantly, in a realistic device operation cycle, fluctuating current is unavoidable, but most of the transient behavior beyond steady current conditions are under-explored. In this dissertation, I demonstrate an approach that integrates advanced characterization, phase field theory and numerical computation to understand electrochemical phase transformation, extending beyond typical constant current conditions. Using LiXFePO4 as a model system, I will break the challenge into two parts based on phase-field theory. First, I will focus on understanding the stress/strain development during phase separation. This allows for a proper construction of the materials behavior at equilibrium. To be specific, the chemo-mechanical constitutive relation that governs the material deformation upon lithiation is extracted through a partial differential equation constrained optimization from correlative images composed of strain and composition. Additional chemo-mechanical insights such as generations of dislocations can be drawn, which may be key to understanding battery lifetime. Having established the equilibrium picture of the material, I will next discuss the transient transformation dynamics beyond constant current conditions. Mechanisms of electrode activation that minimizes resistance through current pulses, as well as the power law relaxation of the metastable solid solution will be discussed. These results provide fundamental insights on optimal battery operation strategies. These findings are generalizable to many other phase-separating systems and demonstrate the potential of merging advanced characterization, phase-field theory, and numerical computation for scientific discovery

Keywords: Materials Science and Engineering.
Advisor: Chueh William
Cai Wei
Lindenberg Aaron Michael
Elucidating molecular and cellular transport mechanisms of the innate immunotransmitter cGAMP for paracrine STING signaling

The innate immune system represents a crucial first-line of defense against infection and disease. Moreover, the quality and quantity of the innate immune response shapes the subsequent adaptive immune response in dramatic ways. As a result, uncovering the mechanisms by which innate immunity is activated has broad implications not only for understanding cellular and organismal biology, but also for the development of powerful immunotherapeutic drugs. One innate immune pathway, the cGAS-STING pathway, has been implicated as a major factor across diverse contexts in human health: protection against viral infection, response to cancer and cancer therapies, and also pathogenic autoinflammatory and autoimmune states. A unique aspect of the cGAS-STING pathway is that activation of the sensor and receptor are linked by the production of a distinct small molecule second messenger, 2'3'-cyclic GMP-AMP (cGAMP). While cGAMP plays important roles within a single cell, striking biological effects are observed when cGAMP is able to "spread the message" that innate immunity should be activated across cells throughout in the local environment. This dissertation focuses on paracrine cGAS-STING signaling mechanisms and therapeutic applications in detail. In Chapter 1, I will review the molecular mechanisms of the cGAS-cGAMP-STING pathway, its signal transduction, and implications in human health and disease. In Chapter 2, I will describe the identification of LRRC8A:C/E heteromeric channels as widely-expressed, bidirectional transporters of cGAMP that enable cGAMP's paracrine signaling function. In Chapter 3, I will report ongoing efforts to develop cGAMP as a vaccine adjuvant through the use of hydrogels to control cGAMP release and delivery in a highly localized manner

Advisor: Li Lingyin
Kim Peter
Rohatgi Rajat
Elucidating the role of Axin in beta-catenin destruction and stabilization

Wnts are a family of secreted ligands that regulate embryonic development and stem cell renewal. One way they do so is by increasing cellular concentrations of the transcriptional coactivator beta-catenin by blocking its degradation. As the key scaffold of the beta-catenin destruction complex, Axin promotes beta-catenin degradation. However, it is also essential for the recruitment of the destruction complex to the Wnt-bound receptors to halt beta-catenin destruction in response to a Wnt signal. While most of the essential proteins and many of the key events in Wnt/beta-catenin signal transduction are known, the precise mechanistic logic of both how the beta-catenin destruction completely efficiently destroys beta-catenin in the absence of Wnt and how Wnt binding to its receptors halts this destruction remains unclear. To probe the function of Axin in the Wnt ON and OFF states, structural, biochemical, and enzymatic analyses of the interactions among Axin, GSK-3, beta-catenin, and Dishevelled were carried out. These results inform our understanding of this fundamental signaling pathway

Keywords: Structural Biology.
Advisor: Weis William I.
Ferrell James Ellsworth
Nusse Roel
Emotional acknowledgment in organizations : the benefits and risks of verbalizing the affect of others

Emotions are omnipresent in organizations, but emotional sensemaking is not. Although emotions have the potential to serve as rapid conduits of social information, organizational practices often impede effective communication using emotions. In this dissertation, I investigate how using verbal language can facilitate the co-construction, calibration, and clarification of emotional communication. In everyday conversations, people acknowledge the emotions of others using language (e.g., saying "You seem excited" or "You look distraught"). I examine the role of this simple, but potentially meaningful act of emotional acknowledgment—verbal communication by which a perceiver signals the recognition of an expresser's emotions—in facilitating emotional sensemaking (Weick, 1995). Because emotional communication usually relies heavily on drawing inferences from non-verbal cues (van Kleef, 2009), I argue that explicitly verbalizing these inferences through emotional acknowledgment can reap important social and informational benefits while also coming with considerable risk. Using an organizational lens, I explore the consequences of emotional acknowledgment in three dissertation chapters. In the first chapter, I examine how emotional acknowledgment provides information about the quality of our social relationships, thereby facilitating the development of interpersonal trust. In the second chapter, I show how emotional acknowledgment can facilitate team performance by surfacing hidden team information. In the third and final chapter, I investigate how expressers use emotional acknowledgment as feedback to change their own self-presentation

Keywords: Business.
Advisor: Berg Justin 
Flynn Francis J. 
Sorensen Jesper B.
Emotional disturbance and its constructive role in the learning process of collaborative design

Design work is filled with rich emotions and feelings resulting from resolving team conflicts to breaking impasses to discovering creative insight. In this dissertation, I have furthered the theoretical understandings about the constructive role of emotion and emotional disturbance in group design innovation activities and proposed a method of studying it. With a field study of novice designers and a mixed-methods study of experienced designers, I have presented ample evidence about the relationship between intense emotional experiences of disturbance and knowledge construction. In the mixed-methods study, I integrated speech acoustics (e.g., vocal pitch) and other bodily signals into video interaction analysis and examined how emotions develop as self-organizing social processes, how they are experienced as ongoing constructed interactions, and how they interact with other key processes of learning and discovery. Together, the current dissertation (1) advances theorizing on the emotional process underlying designers' learning, growth and performance, and (2) proposes a situated and integrated methodology for accessing, characterizing and understanding designers' moment-by-moment emotional dynamics in naturalistic settings. My work on the inherently emotional nature of design fills a gap in research that most often leaves out emotion in the study of designer cognition and behavior. For engineering and design practitioners, my work challenges the stereotypical view that emotion is separate from the intellectual process of engineering and design. My work affords a new way of seeing emotion as functional to creative engineering and design work, and opens up an important path for improving its practice and education

Keywords: Mechanical Engineering.
Advisor: Leifer Larry J.
Sheppard S. (Sheri)
Markus Hazel Rose
Pea Roy D.
Empire in Action: US Operations and the Making of Narratives Around the 1953 Iranian Coup d'État

How did US diplomatic and intelligence officers construct narratives of the 1953 Iranian coup d'état, one of the most famous and foundational instances of Anglo-American foreign intervention of the Cold War? The US role in the 1953 coup has been examined through myriad lenses, including post-World War II historical analysis, early Cold War foreign policy, consequences of covert intervention, and intelligence successes and failures. Historians, social scientists, and other scholars of political, intelligence, and military history have also long debated the significance of the 1953 coup. However, the coup has yet to be studied through an anthropological lens and examined as a site for the perpetuation and expansion of US empire as well as the imagination of narratives characterized by a new form of imperial mythos. As such, this thesis takes an ethnographically and historiographically inspired approach by investigating volumes of foreign policy writings, intelligence histories, personal memoirs produced by American planners, administrators, and governmental historians of the 1953 coup. This thesis argues that US operatives from the Central Intelligence Agency and the US Department of State embodied and reproduced empire via a set of practices that constituted a collective US imperial habitus. These practices involved exerting textual control over Tehran and Iranian individuals, construing themselves as committed to a sublime struggle against terror, and constantly repositioning covert operations as a form of valiant resistance in their written accounts. This thesis also contends that the narratives initially created by US operatives in archival and personal writings have imbued the history of the coup with a form of their ideological character and become the accepted US history of events.

Keywords:  Program in Science
STS
science technology and society
ethnography
Iran
coup
coup d'état
archives
empire
imperialism
United States
covert operations
Cold War
intelligence
diplomacy
narratives
sublime
prestige
Tehran
memoirs
foreign policy
history
habitus
Thesis
Advisor: Kohrman Matthew
Empire in the backlands : mobility and the interiorization of Portuguese colonization in Brazil, seventeenth and eighteenth centuries

This dissertation examines the process of interiorization of the Portuguese colonization in Brazil during the seventeenth and eighteenth centuries. Portuguese empire-building in the early modern era was a quintessentially maritime enterprise based on settlements established along the coast of Africa, Asia, and South America. In parallel with seaborne expansionism, the Portuguese adopted the vague spatial concept of the backlands—sertão—to refer to regions in the interior that were beyond imperial control. However, in the last decade of the seventeenth century, the discovery of gold deposits in central Brazil prompted a massive spatial transformation, leading to the formation of permanent pockets of settlement in the interior. This dissertation asks how the Portuguese came to integrate vast portions of the backlands into the empire. Specifically, it examines the mechanisms that enabled movement and exchanges over long distances in this fragmented geographic setting. I argue that the mobility of imperial subjects at the margin of the empire enabled the formation of Luso-Brazilian settlements away from the coast as well as formed and maintained inter-regional linkages that made them part of the empire. These spatial practices were not extraordinary acts of pioneering nor centrally directed by the crown, but part of routinized acts carried out by colonists and often rooted in family ties to incorporate resources from the backlands, exploit indigenous labor, and unlock new economic opportunities created by the settlement of the interior. While historians tend to emphasize the transplantation of institutions and the formation of territorial jurisdictions as the primary mode of the imperial territorial extension, this dissertation reveals the practices of mobility that enabled and sustained this process of occupation

Keywords: History.
Advisor: Frank Zephyr L.
Findlen Paula
Wolfe Mikael