VictorJuiz/thesis huggingface dataset:
Learning Visual Groupings and Representations with Minimal Human Labels

Making a computer system understand complex image scenes is challenging. Complex imagescenes often have multiple objects, which are not isolated but related to each other in different aspects. Identifying certain object categories may not be enough to understand complex scenes. Categories have multiple granularities. We need such knowledge to capture semantic correlation thoroughly. In addition, objects have numerous interactions/relationships. We need to localize these objects, recognize scene environments, and figure out their interaction- s/relationships. In computer vision, recognizing what the categories are, where the objects are, and how objects interact to each other is often formulated as the classification, segmentation, and relationship recognition problem./nExisting approaches often tackle all these formulations in supervised settings. Despite theirtremendous progress, we identify three major limitations. 1) Human annotation is too time- and labor-consuming to scale up to real-world scenarios. 2) The sets of human labels are pre-selected arbitrarily, providing limited/biased perspectives to understand images. 3) Such supervised methods conduct inference in terms of discrete labeling. They isolate labels from each other, ignoring the similarity/dissimilarity among each other. Also, they can only put images to the known labels seen during training and fail to recognize novel images sampled from unknown labels during testing./nIn this dissertation, we address the issues of current supervised approaches by replacingdiscrete labeling with grouping and using minimal human labels. Specifically, we tackle the recognition problem from four perspectives. 1) We address weakly-supervised semantic segmentation, where partial semantic pixel labels are used. 2) We address unsupervised semantic segmentation, where only low-level edge detections are used. 3) We address unsupervised concurrent image classification and segmentation in a single framework, where our model does not use any human labels. 4) We address unsupervised human-object recognition, where semantic and instance pixels labels, no relationship labels, are used. This dissertation explores more general and robust approaches to understanding the highly-complex and fast-changing real-world scene.

Advisor: Yu, Stella X.
Whitney, David
An Embarrassment of Queer Possibility: The Black Dandy in the Harlem Renaissance Fictions of Langston Hughes, Richard Bruce Nugent, and Nella Larsen

Dandyism is not only a praxis of representational conflict waged through sartorial aesthetics, fashion, and clothing; it is also a practice of stylistic refusal. In this dissertation, I examine the Harlem Renaissance as a site of conflict and resistance where a diverse array of collective actions produced meanings and narratives of a syncretic cultural style in direct conflict with the distortions and myths of Western racial production. In each author’s readings I locate how the Black dandy indexes the notion that Black modern life was structured in and around the circulation of affective embarrassment and the violence that disciplines transgressive subjects. Consequently, I read Black dandyism as a practice of cultural syncretism that transforms into a practice that makes space for stylistic refusal and an expression of freedom./nAside from the conflict between Harlem’s subculture and a regime of anti-Black racist production, Harlem’s artists were keenly aware of the tension amongst Black leaders and artists as they battled over the politics of Black representation. Harlem’s Black dandies embodies both conflicts. Their extravagant style, defiant demeanor, and radical gender identities and sexual practices queer Blackness, race, gender, sexuality, and modern notions of citizenship and national belonging./nIn the first chapter, I read Langston Hughes’s short story “Home” as an adaptation of the tragic ending familiar to nineteenth century tragic mulatto/a narratives. In the second chapter I explore the overt expression of queer Blackness in Richard Bruce Nugent’s posthumous novel, Gentleman Jigger. Nugent, as one of the few openly queer artists of the period, straddles a Black cultural and European decadent tradition in a thoroughly modern novel that includes a critique of the limiting forms of Blackness within the Renaissance and a critique of the violence that structures early twentieth century “white” American culture. In the third chapter’s reading of Nella Larsen’s Passing, I argue that the novel contains a rich merging of the Black dandy with the tragic mulatta, mixed race subject, and sex worker, four figures that deeply implicated and distorted conceptions of Black womanhood.

Advisor: Wagner, Bryan
Learning when Objectives are Hard to Specify

Deploying learning systems in the real-world requires aligning their objectives with those of the humans they interact with. Existing algorithmic approaches for this alignment try to infer these objectives through human feedback. The correctness of these algorithms crucially depends on several simplifying assumptions on 1) how humans represent these objectives, 2) how humans respond to queries given these objectives, and 3) how well the hypothesis space represents these objectives. In this thesis, we question the robustness of existing approaches to misspecifications in these assumptions and develop principled approaches to overcome such misspecifications./nWe begin by studying misspecifications in the hypothesis class assumed by the learner and propose an agnostic learning setup where we demonstrate that all existing approaches based on learning from comparisons would incur constant regret. We further show that it is necessary for humans to provide more detailed feedback in the form of higher-order comparisons and obtain sharp bounds on the regret as a function of the order of comparisons. Next, we focus on misspecifications in human behavioral models and establish, through both theoretical and empirical analyses, that inverse RL methods can be extremely brittle in worst case. However, under reasonable assumptions, we exhibit that these methods do exhibit robustness and are able to recover underlying reward functions up to a small error term. We then proceed to study misspecifications in assumptions on how humans represent objective functions. We begin by showing that taking a uni-criterion approach to modeling human preferences fails to capture real-world human objectives and propose a new multi-criteria comparison based framework which overcome these limitations. In the next part, we shift our focus to hand-specified reward functions in reinforcement learning, an alternative to learning rewards from humans. We empirically study the effects of such misspecifications showing that over-optimizing such proxy rewards can hurt performance in the long run.

Advisor: Bartlett, Peter
Dragan, Anca
The Provider’s Letter: Cisgender Anxieties, Specters of Regret, and Trans Agency as Liberation

Transgender individuals worldwide are bound to the necessity of the provider’s letter—a letter from healthcare professional certifying the validity of their identity—for medical care and legal recognition. The use of the provider’s letter continues, despite decades of community pushback, a lack of any evidence of efficacy or positive effect of the requirement, and multiple literatures supporting its unethical and damaging impacts on the trans community. This project uses three approaches, in the form of three separate but thematically connected articles, to understand the letter and its current usage in the trans medical and legal world. The first article defines the providers letter and performs an integrated historical review of the letter and the bodies of literature that have engaged with it; bioethics, critical literatures of medicalization/pathologization, and legal scholarship. This review concludes that the letter, rather than being a supportive or helpful practice, is used to control trans bodies and performances of gendered subjectivity. Further, that the justification of the letter focuses on the specter of regret, which has not been supported by evidence to be a legitimate concern. The second article digs into the history of the letter as an object, revealing its origins as a part of the application for a transvestite certificate in early 20th century Germany. This article focuses on sexologist Dr. Magnus Hirschfeld as the creator of the modern understanding of the letter, and its use in the application for the transvestite certificate, a pragmatic solution to the problem of presumed transvestites disturbing the peace. Hirschfeld’s pupil, Dr. Harry Benjamin, brought the letter to the United States and served as the architect for early trans medicine in the mid 20th century, so its Germanic origins are deeply relevant to the modern practice. The final article focuses on an alternative approach to transgender medicine, first laying out the issues with the current medical model, then laying out an alternative model, the gender wellness model. This model rests on the idea that all people have an optimum level of gender wellness, and for some, reaching that optimum level may require intervention. While the letter is a complex object that resists simple categorization, understanding what it was created to do and how it functions practically in the legal and medical realm opens a variety of productive avenues for exploration.

Advisor: Morrill, Calvin
Symplectic geometric methods in microlocal sheaf theory

The main goal of this dissertation is to import symplectic geometric methods into microlocal sheaf theory based on the foundational sheaf quantization construction by Guillermou, Kashiwara, and Schapira. This construction provides the notion of isotopies of sheaves and a sheaf-theoretic analogue of the notion of continuation maps in Lagrangian Floer theory./nBased on previous work by Ganatra, Pardon, and Shende, we make further ex- amination on the category of unbounded sheaves microsupported in some singular isotropic Λ in the cosphere bundle. We show that various categorical constructions concerning this category can be described in symplectic geometric terms by using isotopies of sheaves./nThe main construction is a sheaf-theoretic analogue of the wrapped Fukaya category, by localizing a category of sheaves microsupported away from some given Λ along continuation maps. When Λ is a subanalytic singular isotropic, we also construct a comparison map to the category of compact objects in the category mentioned above, and show that it is an equivalence. The last statement can be seen as a sheaf-theoretical incarnation of the sheaf-Fukaya comparison theorem of Ganatra-Pardon-Shende.

Advisor: Shende, Vivek
The Geometry of Hilbert Schemes on Projective Space

In this thesis we study singularities of Hilbert schemes and show that there are many (components) of Hilbert schemes that are smooth or mildly singular and use them to explore phenomena in birational geometry and commutative algebra. Specifically, we study the Hilbert scheme compactification of a pair of linear spaces, describe all the sub- schemes parameterized by this component and show that it is a smooth Mori dream space. We study Hilbert schemes with two Borel-fixed points and prove that they are reduced, and that their irreducible components have normal and Cohen-Macaulay singularities. We study the Hilbert scheme of points on a threefold and extend results on the Hilbert scheme of points of a surface to this case; we also provide bounds on the dimension of this Hilbert scheme. Finally, we generalize the Hilbert and Quot schemes to construct the fiber-full scheme, which is a fine moduli space that controls all the cohomological data of a variety instead of just the Hilbert polynomial.

Advisor: Eisenbud, David
Understanding and Exploring Serverless Cloud Computing

The past few years have seen a wave of enthusiasm for serverless computing, and we begin this work by analyzing the marketplace trends and underlying technical factors that have shaped the movement. We find that serverless computing addresses programming challenges in the same class as those that high-level programming languages address, suggesting that serverless computing may be viewed as high-level programming for distributed systems./nWe next turn our attention to one of the key shortcomings of serverless: the lack of integration between compute and state. We develop FaaSFS, a distributed file system that is compatible with POSIX applications but uses a novel consistency model with relaxed real-time ordering constraints. We call this model externally consistent sequential consistency (ECSC) and use it to scale a pre-existing single-server application to 10,000 serverless processes. We also show that under reasonable assumptions ECSC is indistinguishable from linearizability, a widely accepted strong form of consistency./nLastly, we explore whether serverless computing might lead to the demise of server hardware. By applying Amdahl's law and scaling rules for interconnect costs, we show that applications that rely on coordination protocols are particularly dependent on large servers for scalability. In contrast, those implemented with coordination-free protocols can run well on collections of small, low-cost servers or on disaggregated hardware. These approaches will likely continue to coexist, suggesting that a need for underlying server hardware will remain even as serverless abstractions thrive.

Advisor: Hellerstein, Joseph M
High Order Implicit Shock Tracking using an Optimization-Based Discontinuous Galerkin Method

We present a framework for resolving discontinuous solutions of conservation laws using implicit tracking and a high-order discontinuous Galerkin (DG) discretization. Central tothe framework is an optimization problem and associated sequential quadratic programming solver which simultaneously solves for a discontinuity-aligned mesh and the corresponding high-order approximation to the flow that does not require explicit meshing of the a priori unknown discontinuity surface. We utilize an error-based objective function that penalizes violation of the DG residual in an enriched test space, which endows the method with r-adaptive behavior: mesh nodes move to track discontinuities with element faces and improve the conservation law approximation in smooth regions of the flow. This method is shown to deliver highly accurate solutions on coarse, high-order discretizations without nonlinear stabilization and recover optimal convergence rates O(hp+1) for problems with discontinuous solutions. We demonstrate this framework on a series of inviscid steady and unsteady conservation laws, the latter of which using both a space-time and method of lines discretization. We also develop local mesh operations for curved meshes that are required to maintain mesh and solution quality as our high-order meshes deform to track the shock.

Advisor: Zahr, Matthew J
Persson, Per-Olof S
Neuroprosthetic Skill Learning and Stimulation-Based Neuromodulation

Brain-machine interface (BMI) systems hold great promise for improving quality of life in patients with a number of motor and cognitive disabilities, but significant advancements must be made before these systems are clinically viable. Developing neuroprosthetic devices and adaptive deep brain stimulators that are neurobiologically-informed is essential for increasing the usability and scalability of these systems. To do so, we must first develop an understanding of how the brain interacts with these systems. Previous work has shown that learning to control a BMI is associated with neural adaptations that depend on large-scale networks across multiple brain regions. In this work, we aim to develop a better understanding of this large-scale learning process by analyzing how motor cortical population dynamics differ in groups of neurons used to directly control a BMI and the remaining recorded population. Then, we investigate the role of corticostriatal circuits in BMI control by simultaneously recording from dorsolateral prefrontal cortex, the caudate nucleus of the striatum, and motor cortex as a nonhuman primate controls a motor cortical BMI. Finally, we assess how high-frequency microstimulation administered in the striatum changes functional decision-making behavior and underlying neural representations of value in the caudate and anterior cingulate cortex with the goal of developing a better understanding of how stimulation-based therapies can be used for neuropsychiatric disorders. Altogether, this work aims to improve our understanding of the neurobiology of BMIs with the goal of developing neurobiologically informed neuroprosthetic devices.

Advisor: Carmena, Jose M
Wallis, Joni
Mobile Robot Motion Prediction and Control Under Uncertainties

Over the past years, wheeled mobile robots have attracted considerable attention in society. They are often used in civilian or military fields, such as agronomy, processing, environmental monitoring, and national defense. However, many uncertainties exist regarding the motion prediction and control of mobile robots. They often exhibit highly non-linear behaviors, which are difficult, or even impossible, to analytically model with sufficient accuracy. Fur- thermore, many existing prediction and control strategies suffer from parameter uncertainties and external disturbances./nThis dissertation research mainly focuses on two specific tasks for mobile robots, namely pushing and trajectory tracking under slippery conditions. These tasks are highly uncertain in nature: First, in terms of pushing dynamics, it is difficult to determine all of the physical parameters (e.g., friction) and motion prediction and control are coupled with the pushed object; second, when tracking mobile robots’ trajectory on slippery ground, their behavior cannot be modeled accurately and models are highly dependent on the ground condition./nThis dissertation is divided into two parts. Part I focuses on improving the performance of mobile robot pushing. First, we explore the use of a combined prediction model and an online learning framework for planar push prediction. By adjusting low-dimensional analytical parameters in online situations, our robot is able to quickly adapt to new environments. Second, a different approach is discussed, which discards the analytical model all together and uses model-free reinforcement learning to perform robot pushing./nPart II focuses on mobile robot tracking under slippery conditions. The central methodology involves using reinforcement learning at a high level to adjust low-level model parameters, thus achieving robust performance. Specifically, we explore the adaptation of key parameters in a model predictive control optimization formulation, enabling the control scheme to be adjusted to be more aggressive or conservative. Then, online gain tuning in lateral and longitudinal controllers is explored.

Advisor: Tomizuka, Masayoshi
Identifying new risk factors for antimicrobial resistance in urinary tract infections through microbial, genotypic, and geospatial analyses

Antimicrobial resistance and the subsequent loss of effective antimicrobial treatments is one of the greatest threats facing public health in the 21st century. The increasing prevalence of antimicrobial resistant (AMR) bacteria threatens the effective treatment of common and previously easily treated bacterial infections, such as urinary tract infections (UTIs), and if unaddressed could effectively turn back the clock to a pre-antibiotic era. The complex dynamics of horizontal gene transfer, selective pressure from antimicrobial agent usage and the dissemination of resistant bacterial strains, present a multifaceted problem that will challenge researchers and policy makers, requiring interdisciplinary problem-solving and political will. To slow the spread of resistant strains and limit the impact of antimicrobial resistance selection, global and local public health agencies will require detailed surveillance and improved diagnostic efforts to guide clinical practice. This work attempted to address some of these concerns by examining potential risk factors for AMR UTI through genotypic and geospatial analysis. Our goal was to improve understanding of AMR UTI by leveraging whole genome, patient level and geospatial data. We examined co-resistance of bacteria causing UTI and identified genotypic candidates for resistance markers that may aid in both diagnostic development, as well as surveillance. We also investigated geospatial clustering of UTI by examining common Escherichia coli strains associated with AMR UTI within the community and examined patient characteristics associated with common strains. These approaches to characterize the epidemiology of AMR UTIs have challenged the traditional view that the community prevalence of these infections is largely due to antibiotic selective pressure on E. coli pathogens. Our observations demonstrate that a large proportion of community-onset AMR UTIs are caused by strains of E. coli that are already resistant at the time of infection. This body of work provides the foundation for future investigations of AMR UTI risk factors that will help provide a basis for public health surveillance and intervention.

Advisor: Riley, Lee W
Interactive Cross-Dimensional Media for Collaboration and Guidance in Mixed Reality Environments

Collaboration and multi-user interactions are key aspects of many software tasks. In traditional desktop interfaces, such elements are well supported through built-in collaboration functions or general-purpose techniques such as screen and video sharing. Collaboration and guidance may also be required in Mixed Reality environments, where users carry out spatial actions in a three-dimensional space. However, not all users may have access to the same Mixed Reality interface. All of them may not have access to the same information, the same visual representation, or the same interaction affordances. Such asymmetries make communication and collaboration between users harder. To address these issues, we introduce Interactive Cross-Dimensional Media. In these media, the visual representation of information streams can be changed between 2D and 3D. Different representations can be chosen automatically based on context or through associated interaction techniques that give users control over exploring spatial, temporal, and dimensional levels of detail. This ensures that users understand and interpret any information or interaction across different dimensions, interfaces, and spaces. We have deployed these techniques in four different contexts: (1) Mixed Reality telepresence for remote instruction of physical tasks, (2) asynchronous video-based instruction of virtual tasks, (3) live asymmetric guidance of virtual tasks, and (4) live interactive spectating of virtual tasks. Through user studies of these systems, we show that Mixed Reality environments that provide users with interactive cross-dimensional media interfaces improve performance and user experience in multi-user and collaboration settings.

Advisor: Hartmann, Bjoern