VictorJuiz/thesis huggingface dataset:
Neural reading comprehension and beyond

Teaching machines to understand human language documents is one of the most elusive and long-standing challenges in Artificial Intelligence. This thesis tackles the problem of reading comprehension: how to build computer systems to read a passage of text and answer comprehension questions. On the one hand, we think that reading comprehension is an important task for evaluating how well computer systems understand human language. On the other hand, if we can build high-performing reading comprehension systems, they would be a crucial technology for applications such as question answering and dialogue systems. In this thesis, we focus on neural reading comprehension: a class of reading comprehension models built on top of deep neural networks. Compared to traditional sparse, hand-designed feature-based models, these end-to-end neural models have proven to be more effective in learning rich linguistic phenomena and improved performance on all the modern reading comprehension benchmarks by a large margin. This thesis consists of two parts. In the first part, we aim to cover the essence of neural reading comprehension and present our efforts at building effective neural reading comprehension models, and more importantly, understanding what neural reading comprehension models have actually learned, and what depth of language understanding is needed to solve current tasks. We also summarize recent advances and discuss future directions and open questions in this field. In the second part of this thesis, we investigate how we can build practical applications based on the recent success of neural reading comprehension. In particular, we pioneered two new research directions: 1) how we can combine information retrieval techniques with neural reading comprehension to tackle large-scale open-domain question answering; and 2) how we can build conversational question answering systems from current single-turn, span-based reading comprehension models. We implemented these ideas in the DrQA and CoQA projects and we demonstrate the effectiveness of these approaches. We believe that they hold great promise for future language technologies.

Advisor: Manning Christopher D. 
Jurafsky Dan
Liang Percy 
Zettlemoyer Luke S.
Neuronal Expression of the MHCI Qa-1 and Its Role in Activity-Dependent Plasticity

During development, brain connectivity is modified by changes in experience and neural activity. These plastic modifications in the structure and function of synapses, the connections between neurons, are known as synaptic plasticity. Synaptic plasticity can alter the way neural circuits compute and ultimately lead to changes in behavior. In the visual system, vision is required for the refinement of connections in primary visual cortex (V1), especially during a developmental critical period. However, the molecular mechanisms by which changes in experience are translated into synapse modifications are unclear. This study describes a potential role for Qa-1, a mouse major histocompatibility complex protein, in visual cortical plasticity during the postnatal critical period. The role of Qa-1 in the immune system has been investigated, but we show that it may have a separate function in the brain that depends on neural activity. Multiplex in situ hybridization experiments showed that Qa-1 mRNA is specifically expressed in deep layer cortical excitatory neurons. This expression begins during the visual cortical critical period, correlating with a period of increased plasticity. Furthermore, Qa-1 mRNA in V1 is regulated by changes in visual experience. Exposure to darkness during the critical period resulted in lower expression of Qa-1 in mouse V1, and re-exposure to light restored Qa-1 expression to control levels. These results support a role for Qa-1 in transducing changes in neural activity into molecular signals for synaptic plasticity. Our results indicate that Qa-1 may be involved in activity-dependent plasticity and help clarify the molecular mechanisms by which synapses change in response to experience during development. Uncovering the molecular basis of synaptic plasticity in development will lead to a better understanding of neural developmental disorders and neurodegenerative disease.

Keywords:  Department of Biology
synaptic plasticity
critical period
visual cortex
MHCI
Qa-1
HLA-E
neurobiology
biology
Thesis
Advisor: Shatz Carla J.
Shen Kang
New platforms for the generation and extraction of single photons

At the core of next generation computing and communication paradigms lies the burgeoning field of quantum information science, which involves the encoding, communication, manipulation, and measurement of information using quantum-mechanical objects. One key example is quantum cryptography, which relies on quantum key distribution protocols to ensure secure communication channels. A vital enabling technology for this application is the single photon source. This thesis explores platforms for on-chip generation and extraction of single photons. First, we demonstrate a hybrid metal-dielectric nanocavity coupled to a single quantum dot that emits single photons directly into free space. In introducing this nanocavity, we conduct a detailed exploration of the design and subsequent nanofabrication of the device. Then, we introduce the design of a platform for high-purity single photon generation based on a waveguide with a partially transmitting element coupled to a photonic crystal cavity. This system generates single photons in an integrated fashion and without loss, allowing for further processing on a single chip. Finally, we test a method for angled fabrication of devices using a focused ion beam (FIB) system. We probe the feasibility of this technique in two ways: (a) undercutting of a nanobeam in 4H-SiC, and (b) angled milling of high-efficiency grating couplers using inverse design to convert from on-chip to free-space and vice versa. Taken together, we hope these individual advances provide a framework upon which to build in order to move closer to scalable, industry-ready single photon sources.

Keywords: Electrical Engineering.
Advisor: Vuckovic Jelena 
Miller D. A. B. 
Safavi-Naeini Amir H.
No escape: light waves in AdS

The AdS/CFT correspondence represents a major advance in our understanding of string theory and quantum gravity. It provides a powerful toolkit for studying strongly coupled quantum field theories and understanding quantum gravity. To apply this framework effectively, it is important to understand AdS in the context of string theory. Calculations involving string theory in AdS rely heavily on the assumption that light starting from the center of Anti-de Sitter (AdS) cannot reach its boundary. However, it can be shown that light, if it follows null geodesics, will reach the boundary and then come back to the origin in finite time. How can we resolve this tension? This work proposes a simple resolution that involves modeling light as a massless scalar field instead of as rays. Using this new representation and applying some scattering theory, we show that light in AdS cannot be modelled as rays due to diffraction caused by the curvature of the spacetime. In other words, as the light wave travels up the gravitational potential of AdS, it is redshifted and its wavelength increases, which renders the geometric optics approximation inadequate after some finite distance. We then show how this implies that light does not reach the boundary of AdS.

Keywords:  Department of Physics
Stanford Institute of Theoretical Physics
SITP
AdS
General Relativity
Klein-Gordon Equation
light waves
scalar fields
gravity
Thesis
Advisor: Silverstein Eva
Brown Adam
Scherlis Adam
Noising and denoising natural language

Written communication is a crucial human activity, and one that has increasingly been altered by technology, including Artificial Intelligence. In this thesis, we describe an application of AI to build computational writing assistants that suggest edits to text, with the aim of correcting errors, improving fluency, and modifying style. Existing AI writing assistants are highly limited in their ability to provide useful suggestions due to the challenge of data sparsity. When state-of-the-art neural sequence transduction models are fed inputs that do not match the training distribution, low-quality and noninterpretable outputs often result. Thus such models must be trained on large parallel corpora—often on the order of millions of sentence pairs—to attain good performance, and even then, they continue to improve with more data. With the end goal of developing more effective AI writing assistants, this thesis addresses the challenge of data sparsity by investigating the effects and applications of noise in the sequence modeling setting. We begin with a theoretical analysis of the effects of sequence-level noise that illustrates the insufficiency of existing approaches for understanding and modeling such noise. With this understanding in place, we develop a method for synthesizing more diverse and realistic noise in natural language, thus remedying the need for parallel data for the task of "denoising" or suggesting edits to writing. To demonstrate the broader applicability of this method, we describe an extension to generate stylistic edits without parallel data between different styles. We close by describing an AI writing assistant that we deployed to validate the methods proposed in this thesis, along with findings to improve such AI assistants in production.

Advisor: Jurafsky Dan
Ng Andrew Y.
Savarese Silvio
Non-canonical roles for the E2F family in the survival and proliferation of stem and cancer cells

The E2F family of transcription factors has a well-known, critical role in cell cycle regulation. At the G1/S checkpoint, the canonical "repressor" E2Fs inhibit the expression of cell cycle genes in conjunction with the RB family proteins, while the canonical "activator" E2Fs activate the expression of cell cycle genes in response to mitogenic cues. However, accumulating evidence points toward diverse functions for the E2F family outside of G1/S checkpoint regulation. For instance, loss-of-function studies suggest that specific E2F family members are required for cellular differentiation, self-renewal, and apoptosis in a context-dependent manner. In addition, the "repressor" and "activator" designations are based on in vitro studies in a few cell types, and the E2Fs may also have different molecular functions in physiological conditions, including in the cytoplasm. These non-canonical roles remain largely unexplored. Our work shows that E2F4, a canonical "repressor" E2F, can switch to an activator of cell cycle genes to promote the survival and proliferation of embryonic stem cells (ES cells). We demonstrate that E2F4 loss leads to cell cycle defects at the G1 and S phases, decreased cell viability, and global downregulation of cell cycle genes, and that rescue of E2F4 function relies on both its DNA binding and transactivation domains. We further demonstrate that the function of E2F4 in ES cells is RB family-independent, as E2F4 can translocate into the nucleus and bind to gene promoters in cells that completely lack RB family activity. Finally, we characterized the interactomes of E2F4 in ES cells and somatic cells, and found ES cell-specific interactors (e.g. components of the MLL methyltransferase complex) that may be critical for the activator function of E2F4. Our data reveal a non-canonical function for a well-known repressor E2F, and may lead to insights into the biology of other rapidly cycling cell types (e.g. cancer cells and progenitor cells) that lack RB family activity. Future work will be needed to examine the mechanisms driving gene activation by E2F4, the different protein complexes to which E2F4 contributes, and the plethora of cellular functions controlled by the E2F family members.

Keywords: Genetics.
Advisor: Sage Julien 
Attardi Laura 
Kundaje Anshul
Lipsick Joseph Steven
Nonmagnetic optical isolation and circulation with meta-surfaces and integrated photonics

Nonreciprocal optical components, such as optical isolators and circulators, allow light to transmit in one direction but block its transmission in the opposite direction. Such devices play an indispensable role in photonic systems, as they provide lasers with feedback protection and mitigate multi-path interference in communication channels. Constructing nonreciprocal devices require the breaking of Lorentz reciprocity, which has traditionally been achieved with magneto-optical materials. This method, however, is incompatible with integrated photonics platforms, most notably silicon photonics, because most standard optoelectronic materials do not exhibit magneto-optical effects. As such, there has been a significant demand for achieving optical isolation without magneto-optic materials. In this Thesis, we present our recent advancements in using nonmagnetic approaches for constructing nonreciprocal optical devices. First, we discuss the use of nonlinear optics for optical isolation and highlight some of their fundamental limitations. Next, we focus on the use of dynamic modulation of refractive index to achieve nonreciprocal manipulation of light. In particular, we discuss our recent developments in the simulation of dynamically modulated optical devices, as well as the design and demonstration of optical isolators and circulators on both meta-surface and integrated photonics platforms. These elements provide the basis for robust photonic systems, improve the flexibility in optical designs, and expand the degrees of control over light propagation.

Keywords: Electrical Engineering.
Advisor: Fan Shanhui
Solgaard Olav 
Vuckovic Jelena
Non-negative matrix factorization and topic models

Matrix factorization algorithms provide a powerful tool for data analysis and statistical inference. As an important class of these algorithms, nonnegative matrix factorization (NMF) suggests expressing a collection of data points as convex combinations of a small set of 'archetypes' with nonnegative entries. As a result, NMF enables us to extract the latent structure and underlying 'pure components' in a broad range of applications from chemometrics to image processing and topic modeling. In this thesis, we study the NMF problem from two different perspectives: 1. We consider the NMF problem under a deterministic model and propose a new estimator that robustly recovers the factors under a more general settings than the state-of-the-art estimators, with theoretical guarantees. 2. We analyze the NMF problem in a Bayesian setting under which the data is generated using an LDA model. We show negative results about variational inference in this problem, showing that it is not 'stable'. In particular, we show that in a certain region for signal-to-noise ratio, in which the underlying signal cannot be recovered, the variational method is randomly biased and outputs a non-trivial solution. This output, of course, is uncorrelated with the underlying truth. We also suggest a possible correction to remedy this issue.

Keywords: Electrical Engineering.
Advisor: Montanari Andrea 
Boyd Stephen P. 
Candès Emmanuel J. (Emmanuel Jean) 
Weissman Tsachy
Nonreciprocity and topology in dynamically modulated photonic systems [electronic resource]

The propagation of light in linear, time-independent and non-magnetic media is governed by Lorentz reciprocity. As a result, nonreciprocal optical devices such as isolators and one-way waveguides traditionally rely on magneto-optical materials, which are hard to incorporate in an integrated photonics platform. Dynamic modulation of the refractive index breaks Lorentz reciprocity, and provides a CMOS-compatible and reconfigurable approach to optical nonreciprocity. Moreover, dynamically modulated photonic systems also provide a versatile platform for studying photonic topological states that support robust unidirectional photonic transport. Dynamic modulation also enables one to explore the synthetic dimension associated with the frequency axis in certain photonic resonator systems. With a synthetic dimension, one can access higher dimensional physics in a lower dimensional photonic structure. In this dissertation, we theoretically and numerically explore a rich set of nonreciprocal and topological photonic effects associated with dynamically modulated optical waveguides and resonators. Potential experimental realizations of our proposals are also discussed and evaluated. In the first part of this dissertation, I show that the phase of dynamic modulation creates an effective gauge field for photons, which is analogous to the vector potential for electrons in the sense that they both impart a direction-dependent propagation phase on the particles. Such a gauge field enables the creation of nonreciprocal photonic devices such as dynamic optical isolators and gauge-field one-way waveguides, as well as the realization of photonic quantum Hall effect in a nonreciprocal waveguide network. These results illustrate the power of the effective gauge field in manipulating the propagation of electromagnetic wave and enabling new designs for nonreciprocal photonic elements. In the second half of this dissertation, I show that the set of optical resonances supported in a ring resonator forms a synthetic dimension along the frequency axis, with dynamic modulation controlling the coupling between modes along the synthetic dimension. This synthetic dimension enables the realization of three-dimensional physics in a planar photonic structure, bypassing the complex three-dimensional geometries required by previous studies. I will demonstrate the creation of three-dimensional topological insulators and Weyl points in a two-dimensional array of ring resonators, which paves the way for experimental demonstration of three-dimensional topological effects that are difficult to study in other systems.

Keywords: Applied Physics.
Advisor: Fan Shanhui
Digonnet Michel J. F. advisor. Thesis advisor
Schleier-Smith Monika
Non-uniform Fourier Analysis on Active Seismic Data recorded by 3-component Geophones and the Stanford Fiber Seismic Observatory

A distributed acoustic sensing (DAS) system allows seismologists to record seismic data through optical fibers and provides denser spatial sampling than geophones. Optical fiber cables can be deployed in areas that are difficult to access with geophones, such as under the ocean and in urban areas. Pre-existing telecommunication cables can also be utilized, which can reduce the cost of a project. We conducted active-source seismic experiment on Stanford campus to test the performance of DAS with optical fiber cable installed along an existing conduit used for telecommunication cables. The data we collected are heavily contaminated by surface wave noises. To attenuate these noises, we analyzed filters that utilize two different techniques, regular and Non-equispaced Discrete Fourier Transform. Our study showed that overall the Non-equispaced Discrete Fourier Transform improves signal coherence better than regular Fourier Transform; however, the Non-equispaced Discrete Fourier Transform on DAS data can become unstable when offsets of the data cause instability in matrix inversion.

Keywords:  Department of Geophysics
School of Earth Energy &amp; Environmental Sciences
Geophysics
Seismic Processing
Thesis
Advisor: Biondi Biondo L.
A novel enhancer near the Pitx1 gene influences vertebrate hindlimb development [electronic resource]

The evolution and development of the vertebrate limb has long fascinated biologists. Vertebrates show a huge degree of diversity in limb form and function, ranging from the heavily modified forelimb of a bat which enables powered flight, to the partial loss of limbs in whales, or the complete loss of limbs in snakes. The mechanisms that produce such differences have been largely unknown. Studying limb modifications that have evolved recurrently can help reveal ecological and molecular factors that shape evolutionary outcomes. One classic example of repeated limb evolution is the loss of pelvic and hindlimb structures, a trait that has independently evolved in many clades, including fish, amphibians, reptiles, and mammals. To date, the molecular basis of hindlimb reduction is still poorly understood in most species. However, in threespine stickleback fish, a combination of genetic and genomic studies has previously identified the Pitx1 gene as a major locus controlling much of the variation in pelvic size in natural species. Recurrent deletion of a pelvic enhancer region upstream of the Pitx1 gene (PelA) is correlated with pelvic reduction in many independently evolved stickleback populations. Despite the widespread use of this enhancer in stickleback populations, the PelA enhancer is not well conserved outside of fish. In my Dissertation research, I have identified and characterized a second pelvic enhancer in the Pitx1 gene, PelB, which is functionally conserved from fish to mammals. The PelB enhancer drives robust reporter activity in the hindlimb autopod of a developing mouse embryo, and the orthologous stickleback sequence is able to drive robust activity in the pelvic spine of a developing fish. In mice, deletion of the PelB enhancer sequence leads to reductions in length of multiple bones in the hind feet. In addition, loss of the PelB enhancer on a genetic background with reduced Pitx1 expression results in complete loss of the patella (kneecap). These experiments confirm that PelB is required for normal hindlimb development, and suggest that mutations in this region might also contribute to evolutionary changes in limb size. To further test whether PelB is altered in species that show natural hindlimb reduction, I compared PelB sequences in pelvic complete and reduced species. While no obviously disabling lesions were observed in whales or manatees, partial loss of PelB was seen in several snake genomes. In addition, I identified one population of pelvic-reduced stickleback fish that has a complex insertion-deletion mutation in PelB. Using functional assays in transgenic sticklebacks, I showed that this naturally occurring mutation causes loss of PelB activity in the developing pelvis, while other expression domains remain intact. These experiments suggest that PelB is an important driver of evolutionary pelvic reduction in some species.

Keywords: Genetics.
Advisor: Kingsley David M. (David Mark) primary advisor. Thesis advisor
Fuller Margaret
Sherlock Gavin advisor. Thesis advisor
Vollrath Doug advisor. Thesis advisor
A novel high-throughput screening system for directed evolution of robust optical reporters of electrical activity in the brain

Dynamic patterns of neuronal activity within intact neuronal circuits underly the astounding complexity of the brain. To decipher this code, we need tools that can monitor neuronal electrical activity with sufficient resolution in space and time, in awake behaving animals. Genetically encoded voltage indicators (GEVIs) offer great potential in achieving this goal but require significant improvements before they can be routinely deployed in vivo. A major limitation to developing better voltage sensors has been a lack of high-throughput screening methods that can reproducibly induce a fast change in transmembrane potential of a cell. In my thesis work, I present the development of a new electroporation-based screening approach, and mutant library construction and expression methods that enabled functional testing of GEVI variants at unprecedented throughput. I further describe using the screening system and a mechanism-guided approach to improve ASAP-family voltage indicators by directed evolution. One of engineered variants, ASAP3, exhibits up to 51% fluorescence change in the physiological range, millisecond response kinetics, efficient membrane localization and full activity under two-photon illumination. Finally, I show imaging ASAP3 signals in the awake mammalian brain using volumetric kHz-rate random-access multiphoton microscopy. The improved responsivity of ASAP3 to voltage allowed imaging of oscillations and spikes in single cells in single trials, revealing correlations between neuronal activity and animal behavior.

Keywords: Bioengineering.
Advisor: Lin Michael
Baer Thomas
Bryant Zev David