VictorJuiz/thesis huggingface dataset:
Abstractions for scaling stateful cloud applications

As the scale of both computing and data grows, developers are increasingly building distributed stateful systems in the cloud. However, these systems are challenging to build at scale because they must provide fault tolerance and consistency for stateful computations while managing both compute and data resources. Thus, we need new high-level abstractions that hide the complexity of distributed state management from developers. This dissertation proposes three such abstractions at multiple levels of the stack of a stateful cloud application. The first part of this dissertation targets cloud application developers, proposing Apiary, a database-oriented transactional function-as-a-service (FaaS) platform for stateful cloud applications. FaaS is an increasingly popular programming model because it abstracts away resource management concerns and reduces the complexity of cloud deployment, but existing FaaS platforms struggle to efficiently or reliably serve stateful applications. Apiary solves this problem by tightly integrating function execution with data management, improving FaaS performance on stateful applications by 2-68x while providing fault tolerance and strong transactional guarantees. The second part of this dissertation targets developers of the data management systems on which stateful cloud apps depend, proposing data-parallel actors (DPA), a framework for scaling data management systems. DPA targets an increasingly important class of data management systems called query serving systems, which are characterized by data-parallel, low-latency computations and frequent bulk data updates. DPA allows developers to construct query serving systems from purely single-node components while automatically providing critical properties such as data replication, fault tolerance, and update consistency. We use DPA to build a new query serving system, a simplified data warehouse based on MonetDB, and port existing ones, such as Druid, Solr, and MongoDB, enhancing them with new features such as a novel parallelism-optimizing data placement policy that improves query tail latency by 7-64%. The third part of this dissertation targets application developers utilizing multiple data management systems, proposing Epoxy, a protocol for providing ACID transactions across diverse data stores. Such applications are increasingly common because developers often use multiple data stores to manage heterogeneous data, for example doing transaction processing in Postgres and text search in Elasticsearch while storing image data in a cloud object store like AWS S3. To provide transactional guarantees for these applications, Epoxy adapts multi-version concurrency control to a cross-data store setting. We implement Epoxy for five data stores: Postgres, Elasticsearch, MongoDB, Google Cloud Storage, and MySQL, finding it outperforms existing distributed transaction protocols like XA while providing stronger guarantees and supporting more systems

Keywords: Engineering.
Advisor: Bailis Peter
Zaharia Matei
Ousterhout John K.
Accountability or Appeasement? An Exploration of the World Bank's Inspection Panel

The World Bank’s Inspection Panel was the first accountability mechanism of its kind at an international finance institution. The Panel was created in response to civil society protests regarding the human rights violations that were increasing in frequency because of Bank-funded projects. Since its inception in 1993, the Inspection Panel has received over 160 complaints citing concerns about project implementation and design. The central question of this thesis surfaced when it became apparent that, despite the Panel’s promises to address the concerns of communities adversely affected by Bank-funded projects, the Panel has taken decisive action in a concerningly low number of cases. There is a clear gap between the Panel’s mandate and the actions that it took. But why? As such, the question emerged: What determines how cases move through the World Bank’s Accountability Mechanism complaint process? Through a combination of case studies and Panel data, supplemented by interviews with Accountability Counsel staff members and a former Inspection Panel member, I argue that both time and civil society organization (CSO) involvement were significant factors that influenced a complaint’s mobility through the Panel’s process. The thesis found that, while these factors could play a role, ultimately, it boils down to the whims of the World Bank’s Management, who are heavily involved in the Inspection Panel process despite the Panel’s design as an independent body. Management’s overt involvement in the Panel’s process severely undermines the ability of the Panel to practice its mandate and blurs the necessary lines of separation between the Panel and the Bank. This major finding raises concerns on the practice of accountability at these institutions and highlights the need for the Panel, and the Bank, to amend the ways in which communities can seek redress within its walls. Evidently, the Inspection Panel still has a long way to go in the way of meeting its original mandate.

Keywords: World Bank
World Bank. Inspection Panel
Accountability
Responsibility
Financial institutions, International
Text
Thesis
Advisor: Stedman Stephen
Achieving order with two-photon lithography : colloidal self-assembly and direct laser writing

Structural or spatial order at the nanometer/micron regime is an avenue to improve material properties. The field of photonics and metamaterials have shown that size-effects at these regimes, in combination with purposefully designed architected structures, can enhance mechanical and optical performance. A common approach to achieve these types of ordered structures is through colloidal self-assembly or direct laser writing of 3D structures. In this work, I propose using direct laser writing to fabricate colloidal particles and to fabricate complex 3D structures that have enhanced mechanical properties. In the first part of my work, I focus on colloidal self-assembly as a method to achieve order. Due to the limited chemistries and shapes of colloids available to self-assemble, a large majority of self-assembled structures remain elusive. I propose using two-photon lithography to fabricate micron-sized particles and experimentally study the effect of shape (both concave and convex) on the final self-assembled structure. This method allows for highly monodisperse fabrication of colloidal particles which can then be imaged using optical techniques due to their micron size. I fabricate colloidal conical shapes that self-assemble under entropic conditions (depletants) and tune the degree of assembly by changing the effective driving force through size. I then use a custom machine learning framework to identify these assembled structures (columnar grains) and recover self-assembly trends in which larger particles show a higher degree of self-assembly. Building upon this work, convex particles, specifically the Archimedean truncated tetrahedron, are also fabricated using the same framework and studied under another entropic condition (hard-particle interaction). These particles assemble in a six-fold symmetry upon interaction with an interface and transition to a three-fold symmetry upon application of a potential field. Analytical and computational results show that the six-fold symmetry state is a quasi-stable state and upon additional energy input, a transition occurs to achieve the lower energy state. In the second part of my work, I use two-photon lithography in conjunction with nanoclusters to enhance the direct laser writing process and improve the mechanical properties. I fabricate lattices with micron sized features and test them mechanically. The resulting nanocomposite lattices shows high stiffness and best-of-class energy absorbance by suppressing layer by layer collapse that is commonly seen with these types of structures

Keywords: Engineering.
Mechanical Engineering.
Advisor: Gu Wendy (Professor of mechanical engineering)
Cai Wei
Tang Sindy (Sindy K.Y.)
Adapting SnapDx Platform For Low-Cost, Electricity-Free Molecular Amplification Based Detection of Schistosomiasis in LMIC Settings

Schistosomiasis, caused by parasitic worms, is a neglected tropical disease affecting nearly 240 million people globally. The World Health Organization released a Target Product Profile in 2021 calling for the development of new diagnostics for monitoring and evaluation of Schistosomiasis, to move towards elimination. Traditional diagnostics using microscopy are reliant on laboratory infrastructure and technician skill. They also lack sufficient sensitivity and specificity. This thesis develops a loop-mediated isothermal amplification (LAMP) direct from urine Schistosomiasis assay for translation onto SnapDx platform, an electricity-free, point-of-care nucleic-acid diagnostics platform developed in the Prakash Lab for respiratory infections like Sars-CoV-2 detection. Here we demonstrate that the current version of the assay with genomic DNA from Schistosomiasis spiked in human urine in controlled lab settings displays a sensitivity of 77.50% at 1 ng/uL and 62.50% at 100 pg/uL and a specificity of 97.5% - all read via a simple colorimetric readout, in Eppendorf tubes. We further test the impact of various sample preparation methods on the assay. The lowest limit of detection reached on simulated urine samples is 1 pg/uL using samples purified with MidiPrep prior to the introduction of genomic DNA. We further find 11% of all the reactions run showing differing results between fluorescence and colorimetric readout. The mismatch is found to be greater at lower sample concentrations. Next we report on testing the functionality of the assay with wet bath inactivation and run, as required in the electricity-free SnapDx usage in field context. In future work, we intend to investigate different primer combinations, urine-based reaction inhibitors, and compact, low-cost sample preparation methods (such as urine egg concentration) are required to further optimize the assay. Next, reliability, sensitivity, specificity and limit of detection will need to be evaluated with real patient samples. Our work lays the foundation for the future implementation of this assay in field conditions without electricity, opening the door to molecular based surveillance approaches , which satisfies the usability and infrastructure requirements of the World Health Organization’s Target Product Profile. It also contributes to a body of work aiming to develop SnapDx applications for a wide range of infectious diseases, compatible with urine, saliva, blood, and other biological fluids.

Keywords: Global Health
Bioengineering > Research
Molecular Amplification
Schistosomiasis > Diagnosis
Point-of-care testing
Text
Thesis
Advisor: Prakash Manu
Adapting Whisker Sensors to Deep Sea Environment For Ocean One Robot

Stanford’s Ocean One robot has been paving the way in human robot collaboration and underwater archaeology. The humanoid form factor of this scuba diving robot makes it so humans can easily control it remotely to perform fine manipulation with its 3 fingered hands. Currently, Ocean One’s wrists measure forces and torque and convey it back to the user using haptic feedback. It may seem strange to send a robot in place of a human, but explorations, such as Ocean One’s exploration of King Louis XIV’s sunken ship (La Lune), require diving 100 meters below. While humans can technically dive to 100 meters, these are dangerous depths for humans due to factors like the pressure, so 40 meters is the recreational diving limit. Therefore, Ocean One makes going deeper much safer, while still allowing human-like interaction with the environment.
Similarly to human scuba divers, the Oceans One robot has an Achilles heel. While exploring uncharted underwater territories, it could get hooked on a piece of ship wreckage. Even for a human, it can be difficult to unhook oneself, but it is even harder for the Oceans One robot since it does not yet have integrated sensors signaling how it is caught. Without sensors, it is difficult to convey what is stuck, further complicating the job for remote teammates helping it break free.
Adding sensors could not only make it easier for the robot to stay clear and get out of tangles, but it could also provide helpful information on currents and surroundings. Sensors on the arms would provide helpful feedback on obstacles for Ocean One to avoid before getting caught at its shoulders. My first quarter of research involved defining sensor requirements, looking at various sensor possibilities, and narrowing the scope to whisker sensors. My second quarter involved developing code to filter ocean noise. My final quarter involved adapting the existing whisker sensors to high pressure underwater conditions. After testing against sensor requirements, I concluded that sensitivity, robustness, and resolution of the whisker sensor, make it a promising option to improve the Ocean One’s safety and sensing as it navigates cluttered underwater environments.
My project highlights the potential of underwater whisker sensors to address the issue of sensing objects before getting caught and ultimately extend the lifespan of the Ocean One Robot.

Keywords: Mechanical Engineering
Whiskers
Underwater exploration
Academic theses
Text
Thesis
Advisor: Cutkosky Mark
Lin Michael
Brouwer Dane
Adaptive algorithms for data science and computational genomics

Recent years have seen a sustained exponential growth in the volume of data generated within the domains of data science and biology. This dramatic pace of data generation has outstripped the growth in computational power predicted by Moore's Law, creating a computational bottleneck. Existing algorithmic techniques are computationally insufficient, and cannot scale to meet this demand. My research has focused on developing new algorithmic techniques to handle this massive scale of data, using probabilistic modeling to design algorithms that adapt to the input dataset's structure. In this thesis, I present my work on data-adaptive algorithms, highlighting their application in computational genomics. The first part of this thesis focuses on how we can utilize methods from multi-armed bandits to construct computationally adaptive methods for data-scientific problems. We show that adaptivity can be used to design algorithms that are more efficient than their brute-force counterparts, while still providing theoretical guarantees discarded by common heuristic approaches. These methods leverage the fact that practical problems have structured data that is generated by nature, so classical worst-case analyses are overly pessimistic. To this end, we develop a general framework for the use of adaptivity in a class of computational tasks arising in data-science, including in clustering and randomized numerical linear algebra. Tackling each separate problem highlights the modularity of this framework, which we show can be further improved to exploit the structure and constraints of the specific task, which sometimes yield new multi-armed bandit settings requiring the development of new bandit algorithms. The second part of this thesis focuses on the application of these ideas to computational genomics. Nowhere is the computational bottleneck of exploding data more apparent than in genomics, where the rapidly decreasing cost of sequencing has fallen orders of magnitude faster than Moore's Law. The first problem I discuss is that of estimating pairwise sequence similarity, a common first step in genome assembly, for which we develop a novel statistical measure of similarity which adapts to the overall sequence similarity in the dataset, and a computationally efficient adaptive SVD to estimate it. The second problem I highlight is that of reference-free inference, analyzing sequencing data without the use of a reference genome. For this task we develop a novel method for analyzing sequencing data without a reference genome, and a novel finite-sample valid test on contingency tables. In both of these problems, we highlight the importance of improved probabilistic modeling and show how it can be used to develop new algorithms that adapt to structure in the data and are both statistically and computationally efficient.

Keywords: Engineering.
Electrical Engineering.
Advisor: Tse David
Pilanci Mert
Salzman Julia
Addressing discontinuous root-finding for subsequent differentiability in machine learning, inverse problems, and control

There are many physical processes that have inherent discontinuities in their mathematical formulations. This thesis is motivated by the specific case of collisions between two rigid or deformable bodies and the intrinsic nature of that discontinuity. The impulse response to a collision is discontinuous with the lack of any response when no collision occurs, which causes difficulties for numerical approaches that require differentiability which are typical in machine learning, inverse problems, and control. We theoretically and numerically demonstrate that the derivative of the collision time with respect to the parameters becomes infinite as one approaches the barrier separating colliding from not colliding, and use lifting to complexify the solution space so that solutions on the other side of the barrier are directly attainable as precise values. Subsequently, we mollify the barrier posed by the unbounded derivatives, so that one can tunnel back and forth in a smooth and reliable fashion facilitating the use of standard numerical approaches. Moreover, we illustrate that standard approaches fail in numerous ways mostly due to a lack of understanding of the mathematical nature of the problem (e.g. typical backpropagation utilizes many rules of differentiation, but ignores L'Hopital's rule)

Keywords: Engineering.
Advisor: Fedkiw Ronald P.
Farhat Charbel
Liu Cheng-Yun Karen
Advancements in wireless communication : AI-driven error correcting code design and multiuser system optimization

The rapid expansion of wireless technology in recent years has ushered in an era where an increasing number of devices are interconnected. These devices, from the Internet of Things sensors to industrial automation systems, place stringent demands on wireless networks in terms of data rates and latency. As the wireless landscape evolves towards 5G and next-generation networks, the need for efficient and reliable communication systems becomes paramount. This dissertation presents a comprehensive exploration of advancements in wireless communication design, focusing on two pivotal areas: AI-based polar code design and multi-user system optimization. The first part of the dissertation ventures into the realm of AI-based polar code design. Polar codes, known for their error correction capabilities, play a pivotal role in ensuring reliable data transmission. This part introduces two novel techniques that harness the power of reinforcement learning, transforming polar code design into maze-traversing game and graph evolution process, respectively. Both learning-based approaches successfully find polar codes that outperform the existing ones, and the graph-based method offers scalability and adaptability that breaks through bottlenecks seen in other learning-based methods for code design. The dissertation's second part turns its focus to multi-user system optimization within next-generation networks, particularly in cases where the number of users exceeds the wireless environment's capacity. This part conducts an in-depth analysis of optimization algorithms meticulously crafted to maximize system capacity. Additionally, it highlights the practical implementation of newly-developed, readily-available software solutions. In an age characterized by the intricate interplay of connected devices and users, these optimization techniques assume a central role in shaping the seamless, interconnected experiences that next-generation networks are expected to deliver. This dissertation's findings underscore the transformative potential of interdisciplinary research in wireless communication. By addressing the complex challenges posed by evolving wireless technologies, it paves the way for innovative solutions that can push network capacities to their limits.

Keywords: Engineering.
Electrical Engineering.
Advisor: Cioffi John
Wootters Mary
MondelliMarco
Advancing ambient intelligence in healthcare : granularity, efficiency, and privacy

Artificial Intelligence (AI) is revolutionizing the healthcare industry by providing cutting-edge solutions that can significantly enhance patient care and medical research. However, this wave of technological transformation has largely overlooked the physical environment of clinical care delivery. This dissertation illuminates the significant impact of integrating video activity recognition, powered by advanced machine learning and intelligent sensors, into ICU and residential care, demonstrating its potential to alleviate healthcare burdens and improve patient outcomes. We first detail how healthcare spaces can be equipped with ambient intelligence using smart sensors and machine learning algorithms. We then introduce three crucial technical foundations that enable this application. Firstly, we instantiate a novel task, benchmark, and model for discerning complex human activities, offering a hierarchical comprehension of actor roles, object attributes, and their relationships. Secondly, we unveil a suite of data and label-efficient algorithms tailored to address overcome the scarcity of annotated data, a challenge amplified by privacy constraints, proprietary considerations, and the substantial costs associated with domain-specific annotation. Finally, we embark on a comprehensive dialogue on ensuring trustworthy machine learning from ethical and privacy perspectives. We introduce a scalable, differentially private algorithm, specifically designed for large-scale video activity recognition tasks. To demonstrate the real-world impact of our ambient intelligence system, we conclude the dissertation by showcasing two clinical deployments - one within the hospital environment and one in daily living spaces.

Keywords: Engineering.
Advisor: Li Fei-Fei
AdeliEhsan
Wu Jiajun
Yeung Serena
Advancing generative models for real-world applications

While generative models hold thrilling potential, their limited usability presents substantial challenges for their widespread adoption in real-world applications. Specifically, existing methods tend to amplify harmful societal biases ingrained in their training data and often fail to accurately reflect subjective user specifications such as style in the generated outputs. Furthermore, a notable performance gap exists when handling data distributions with unique structures, such as periodicity, restricting their applicability beyond image and text data. This dissertation considers all such facets to help construct safe, reliable generative AI systems for practical integration and deployment. First, we present a methodological framework for tackling the challenges of bias mitigation and controllability. Building on the classical approach of density ratio estimation (DRE), we develop techniques to correct a learned model distribution such that it exhibits characteristics that are more closely aligned with an alternative target distribution of interest. Together, these contributions not only yield a new theoretical framework for DRE, but also improve performance on a diverse set of downstream tasks such as domain adaptation, data augmentation, and mutual information estimation. Next, we present two real-world applications of these methods for societal applications. We demonstrate that: (a) our reweighted generative modeling framework successfully mitigates dataset bias, and (b) more controllable models can better customize AI-generated music to individual preferences and assist the creative process. Finally, we conclude by developing new learning algorithms that incorporate domain-specific inductive biases into generative models for wireless communications, as well as for discrete data distributions.

Keywords: Engineering.
Advisor: Ermon Stefano
Koyejo Sanmi
Re Chris