VictorJuiz/thesis huggingface dataset:
Sequential Decision-Making in Networking Algorithms using Deep Reinforcement Learning

Networking algorithms perform sequential decision-making on the Internet, where they take decisions, e.g., on when to transmit a packet. Traditional networking algorithms use fixed mappings between network-level events and control responses, which may result in inefficient usage of network resources in some network environments. The main issue with existing networking algorithms is the lack of generalization to diverse network environments. This results in poor performance in a wide variety of network environments. This dissertation proposes new techniques in reinforcement learning (RL) to solve the lack of generalization issue. We propose new RL techniques and study them in congestion control, network adaptive coding, and adaptive bitrate selection. First, we present Pareto, a congestion control algorithm fueled by deep reinforcement learning (DRL). Different from existing RL-based congestion control algorithms, Pareto uses expert demonstrations, a new staged training process, multi-agent RL fairness training framework, and online adaptation to new environments. All of these techniques enable Pareto to perform well in a wide variety of environments in terms of high throughput, low latency, low loss rate and fairness to competing flows. In network adaptive coding, we propose Ivory, a new DRL-based network adaptive coding algorithm that utilizes an existing low-latency forward error correction (FEC) scheme. Ivory chooses better coding parameters to ensure a low loss rate and latency while having lower coding overhead compared to the state-of-the-art. Lower coding overhead makes Ivory a better fit for limited bandwidth networks. Since interactions with the environment are costly in online RL, we propose Lethe, a new online RL approach to quickly acquire new knowledge by biasing against old interfering knowledge. In Lethe, we solve issues resulting from interference between knowledge acquired in different time slots. On a different note, in federated learning (FL), interference occurs when the server simultaneously accumulates models from clients with interfering objectives. To avoid inter-client interference, we propose Cascade, a curriculum-federated RL framework. Cascade is tested by training an adaptive bitrate selection algorithm, and it achieves a better asymptotic behavior with fairer knowledge accumulation as compared to other FL algorithms. This results in a better performance over diverse network environments.

Keywords: Adaptive Bitrate Selection
Congestion Control
Machine Learning
Network Adaptive Coding
Reinforcement Learning
Advisor: Li, Baochun
Where is Communication Breaking Down? Narrative Tensions in Obesity-in-Pregnancy Clinical Encounters

Clinical encounters in the context of obesity in pregnancy can be complex to navigate. The biomedical literature on obesity in pregnancy focuses on numerous adverse outcomes that these patients are at an increased risk of, and the associated procedural difficulties. In contrast, the literature on patients’ lived experiences characterizes obesity in pregnancy as challenging due to stigma, and feel weight is overemphasized. To date, studies lack an exploration of stories of both patients and healthcare providers (HCPs) in tandem, as they relate to these clinical encounters. The objective of the study was to understand the communication challenges within obesity-in-pregnancy clinical encounters. I conducted in-depth, semi-structured narrative interviews with 16 patients and 19 HCPs. Stigma theory informed this work, along with Goffman’s dramaturgical approach, which allowed for an exploration of how participants portrayed their identities when describing these encounters and communication. In my analysis of their accounts, I identified five narrative tensions that contributed to communication challenges. The first three tensions related to obesity: 1) obesity is only seen as a detriment to health versus acceptance of obesity; 2) obesity is a result of personal choice versus a result of uncontrollable circumstances; and 3) a regular pregnancy versus a high-risk diagnosis. The fourth and fifth tensions related to characterizations of the obesity-in-pregnancy clinical encounter and the communication therein: 4) a typical and problem-free clinical encounter versus a tremendously difficult clinical encounter; and 5) talking openly about BMI and related co-morbidities versus sidestepping the topic. How participants positioned themselves relative to prevailing societal discourse regarding obesity and being a ‘good’ HCP/patient influenced these tensions. These narrative tensions revealed specific areas where communication is vulnerable to breaking down. They also illuminated the experiences of both HCPs and patients (and the meaning they ascribe), as well as the complexities in this interactive space. This research has important implications for improving clinical practice and education. Beyond the implications in the realm of obesity in pregnancy, this approach and these findings may be applicable to other obesity contexts, to improve clinical encounters for people of larger body sizes.

Keywords: Clinical Encounters
Narrative
Obesity
Obstetrics
Pregnancy
Qualitative
Advisor: Parsons, Janet
D'Souza, Rohan D
Navigating Conflicts of Interest and Psychological Need Satisfaction in Romantic Relationships

In four experiments (Studies 1-4; combined N=2,088) and a twice-weekly diary study (Study 5; N=252) with partnered individuals, I investigated how resolving conflicts of interest with a romantic partner—situations in which partners’ interests are misaligned—affects psychological needs for autonomy, competence, and relatedness. In Studies 1-4, participants imagined or recalled resolving a conflict of interest with a partner through making or receiving a sacrifice (Studies 1-4), compromising (Studies 2-4), or going separate ways and pursuing interests independently (Studies 3-4). In Study 5, participants completed twice-weekly surveys for one month about their daily conflicts of interest and needs. In Studies 4 and 5, I also explored goal-, power-, compatibility-, and self-clarity-related mechanisms through which conflict resolutions impact needs, as well as contextual and sociodemographic moderators to determine when or for whom resolutions are more or less costly. Three core findings emerged. First, resolving conflicts of interest consistently predicted need satisfaction: self-sacrifice undermined autonomy, self- and partner sacrifice undermined competence, and self-sacrifice and separate ways undermined relatedness. Second, goal obstruction and feeling a shared reality with a partner explained links among nearly all resolutions and needs. Some mechanisms more robustly predicted specific needs (e.g., relationship inequity and autonomy; personal-relational balance and competence; matching values, relationship closeness, and relatedness) and uniquely explained links among self-sacrifice and needs (i.e., self-concept clarity, relationship power, preference communication clarity). Third, severe conflicts of interest made self-sacrifice more detrimental—but partner sacrifice more beneficial—for autonomy and relatedness (Studies 4-5), as well as made compromising more beneficial for relatedness (Study 4 only). In Study 4, when people had a strong preference during a conflict, self-sacrifice was more detrimental for all needs, but preference strength attenuated the negative impact of partner sacrifice on competence. In Study 5, self-sacrifice undermined relatedness more strongly for women and undermined autonomy more strongly for women and people from lower socioeconomic status (SES) backgrounds. Overall, self-sacrifice seems to be destructive for people’s psychological needs through multiple mechanisms—especially during high-stakes conflicts and for women and lower SES individuals—and alternative resolution strategies (e.g., compromising, going separate ways) may better protect couple’s needs.

Advisor: Impett, Emily A
A Quantitative Study of Occupational Stress in Animal Shelter Workers

Animal shelter workers are a unique cohort of care workers who have been shown to have high levels of burnout, compassion fatigue, and occupational stress. The full extent of the burden of this type of work is still being discovered. We deployed an online survey (n = 113) which included a novel occupational stress questionnaire and established mental health measures to evaluate levels of compassion fatigue, potential for moral injury, anxiety, and depression in animal shelter workers. Specific variables of interest were type of worker (full/part- time/volunteer), reason for euthanasia (medical vs. non-medical), and level of training. Results reveal only a small percentage of shelters did not practice any form of euthanasia (9%). Overall, respondents exceeded normative levels of burnout, compassion fatigue/secondary trauma, depression, and anxiety. Shelter work was shown to place respondents at a higher-than-average risk of moral injury, which was found to be significantly correlated with depression, anxiety, and compassion fatigue, and inversely related to optimism. Moral injury was higher for shelter workers whose work included euthanasia for non-medical reasons. Euthanasia for non-medical reasons and having formal animal care training, however, were not associated with significant increases in depression, compassion fatigue, or optimism. Full-time workers were found to have significantly higher levels of anxiety, depression, and risk of moral injury than part-time workers and volunteers. A factor analysis of the 43 workplace stressors included in the novel occupational stressor questionnaire revealed four distinct dimensions: 1) involvement in policy and decision-making, 2) public perceptions and stigma, 3) staff and the environment, and 4) job rigidity and social support. Of these scales, job rigidity and social support were found to have the greatest impact on burnout, depression, anxiety, optimism, and compassion fatigue/secondary trauma. Involvement in policy and decision-making had the greatest impact on moral injury. It is clear from our findings that the mental health burden of caring for shelter animals is great. Our findings add four distinct areas of focus that policy and decision-makers may consider to improve the wellbeing of animal shelter workers.

Keywords: Animal
Care Worker
Occupational
Shelter
Stress
Advisor: Schneider, Marg
Evolution and Ecological Consequences of Temperature-Dependent Sex Determination in Long-Lived Reptiles

Temperature exerts strong and pervasive effects on reptile development, resulting in lasting phenotypic impacts and consequences for individual fitness. An interesting example of pervasive effects of early-life temperature is in temperature-dependent sex determination (TSD), whereby incubation temperature irreversibly determines an individual’s sex. TSD has been described in 137 reptile species, but we have no taxonomically widespread explanation for its adaptive significance. Nor do we understand how TSD may have persisted through climatic upheaval in deep time. My thesis addresses both key questions.The adaptive significance of TSD is likely rooted in the Charnov-Bull model, which states that TSD is adaptive if the ratio of male to female fitness varies with temperature. This model predicts a sex-by-temperature interaction on offspring fitness, but given the diversity of reptile natural histories, it is not clear how this interaction could manifest in a manner that is taxonomically widespread. I test three hypotheses for the driver underlying this interaction: 1) immune system strength; 2) an extension of the Trivers-Willard hypothesis (males should be produced in high-quality environments as their fitness is strongly dependent on condition); 3) the Survival to Maturity hypothesis (sex differences in age at maturity compound survival differences resulting from incubation temperatures). I find that the Survival to Maturity hypothesis is consistent with the results of my thesis. Given that sex is determined by temperature, widespread feminization of populations is predicted under climate change. However, our knowledge of how climate change will affect species with TSD is limited to species with MF TSD (males produced at cool temperatures and females at warm temperatures). To address this gap, I investigate long-term trends in primary sex ratios of snapping turtles (Chelydra serpentina), an FMF TSD species. Despite documented warming, the sex ratio remained unchanged over 21 years in Algonquin Park, Canada. I attribute this resilience to remarkable intra-annual variation in sex ratio which is, in turn, related to individual-level attributes of nests that affect development temperature. My thesis provides further direction for tests into the adaptive significance of TSD and suggests that aspects of TSD and species biology contribute to resilience to climate change.

Advisor: Rollinson, Njal
A Geometric Approach for Generating Feasible Configurations of Robotic Manipulators

Most robotic manipulators, and especially those designed with autonomous operation in mind, consist of a series of joints that rotate about a single axis, also known as revolute joints. These mechanisms give robotic manipulators the degrees of freedom and versatility similar to that of the human arm, which they are designed to outperform. However, this results in a geometry of motion or kinematics that makes all aspects of robotic manipulation challenging from a computational perspective. A major part of this challenge lies in the fact that computing joint configurations adhering to a specific set of constraints (i.e., gripper pose) is a non-trivial problem. The procedure of finding feasible joint configurations and the mathematical problem associated with it are known as inverse kinematics - a core part of motion planning, trajectory optimization, calibration and other important challenges in successfully performing robotic manipulation. In recent years, the overall decrease of computation time required to perceive and process environmental and proprioceptive information has helped realize the potential of robotic manipulation in dynamic environments. Concurrently, a new standard in manipulator design has emerged, where additional degrees of freedom are added in order to increase their overall dexterity and capacity for motion. These two developments have vastly increased the requirements for inverse kinematics algorithms, which are now expected to deal with infinite solution spaces and difficult, nonlinear constraints. On the other hand, the addition of degrees of freedom in recent robot designs has enabled algorithms to search for locally optimal configurations with respect to some performance criteria in an infinitely large solution space. This property has motivated approaches that leverage non-Euclidean geometries to replace conventional constraints and optimization criteria, thereby overcoming computational bottlenecks and common failure modes. The contributions presented in this thesis propose three such approaches, that aim to develop new ways of looking at the problems associated with inverse kinematics through the use of geometric representations that are not widely utilized in robotic manipulation.

Keywords: Distance geometry
Kinematics
Manipulability
Manipulation
Riemannian geometry
Advisor: Kelly, Jonathan
Petrovic, Ivan
Improving Reliability and Performance of Storage Stacks on next generation Solid State Drives

Solid State Drives (SSDs) are increasingly replacing HDDs as the primary source of persistent storage medium in both data centers and personal storage devices. SSDs offer higher performance and consume less power. They also bring two challenges in the storage space: First, SSDs have different reliability characteristics than HDDs. However, the impact of various SSD-based errors on the overlying software stack is not well known. Furthermore, as the demand for high-capacity SSDs increases, more bits are stored within the same voltage range of the underlying SSD medium. Second, with the end of Moore’s law, there is increased emphasis on disaggregated computing where data-intensive computation takes place on a processor within the device. Applications can offload computation from the host to the device, save host CPU cycles, lower host-to-device latency, and reduce bandwidth utilization. However, such devices are expensive, hard to program, and may not be suitable for all application computation offloads. We address these reliability and performance challenges for SSDs as follows: First, we evaluate file system reliability in the presence of SSD errors. We notice that real-world file systems often make performance assumptions for SSDs that impact their reliability. We find that real-world file systems incur severe failures, such as unmountable drives in 16% of our extensive test suite emulating 7000 SSD-based errors. Second, we invent a novel voltage-based Write Once Memory (WOM-v) coding scheme that reduces the number of erase operations that take place on every overwrite to the SSD. We show that for QLC SSD, the number of erase operations can be reduced from 4.4x to 11.1x for real-world workloads. Third, we build Nescafe, a computational storage framework to evaluate performance gains of offloading computational tasks to an emulated near storage device. We show that real-world, compute-intensive applications like zlib compression, jbd2 checksum computation, and LSM Tree compaction are easy to offload to a computational device using our framework. We show that Nescafe is generic and simple to extend to different types of block, stream, and application-based workloads.

Keywords: Computational Storage Drives
Performance
Reliability
Solid State Drives
Write Once Memory Codes
Advisor: Schroeder, Bianca
Integrative Genomics of Human Cerebellar Development and Group 3/4 Medulloblastoma

Medulloblastoma (MB) comprises a group of heterogeneous pediatric neuronal neoplasms of the hindbrain with strong links to early development of the hindbrain. While Sonic Hedgehog MB arises in the upper rhombic lip (RL) granule cell lineage due to mutations that activate Shh signaling, and WNT MB arises in the lower RL due to mutations activating WNT signaling, little is known about the more common Group 4 (G4) MB, thought to arise in the unipolar brush cell (UBC) lineage. We demonstrate that somatic mutations in G4 MB converge on the Core Binding Factor Alpha (CBFA) complex, observing mutually exclusive alterations affecting CBFA2T2, CBFA2T3, PRDM6, KDM6A, and OTX2. CBFA2T2 is expressed early in the progenitor cells of the Homo sapiens cerebellar RL subventricular zone (RLSVZ), and G4 MB transcriptionally resemble these RLSVZ progenitors but are stalled in developmental time. Knockdown of OTX2 in model systems relieves this differentiation blockade, allowing MB cells to spontaneously proceed along normal developmental differentiation trajectories. The unique nature of the split human RL, destined to generate the majority of the neurons in the human brain, with its high level of susceptible EOMES+ KI67+ UBC progenitor cells, likely predisposes our species to the development of G4 MB.

Keywords: CBFA2T2
CBFA2T3
Group 3
Group 4
Medulloblastoma
Pediatric
Advisor: Taylor, Michael D
Consequences of Multiple Stability and Connectivity on Coral Reef Ecosystem Dynamics

Coral larval dispersal networks are changing due to multiple anthropogenic stressors that are destroying coral habitat and shifting reefs from coral-dominated to macroalgal-dominated states that support smaller fish communities. However, it is unclear whether these connectivity changes could themselves propagate or contain shifts to macroalgal-dominated states. My research investigates how the combined effects of connectivity and local multiple stability affect coral persistence with the goal of informing conservation decisions. In Chapter 2, I explore how plausible scenarios of climate-change-induced loss could change coral larval dispersal networks and their reseeding potential. I find that these scenarios of loss (yielding the same number of reefs) lead to vastly different global reef networks, from which full reseeding is impossible. In Chapter 3, I develop a two-reef mathematical model and determine that two connected reefs may exhibit a novel third, mixed coral-macroalgal stable state at low dispersal. Furthermore, I find that high- and low-grazing reefs have stable coral-dominated and macroalgal-dominated states if connected by high dispersal to a low or high-grazing reef (respectively). In Chapter 4, I extend this two-reef model to a model of 75 Fijian reefs to determine the interplay among connectivity, local multiple stability, and management strategies in larger networks across three grazing scenarios spanning the stability regimes elucidated in Chapter 3. I find that increasing the grazing rate of a few reefs leads to higher overall coral cover in the entire network, even in reefs with low grazing rates. I also find evidence that a combined strategy of increasing the water quality and increasing a few reefs’ grazing rates is more effective than either strategy in isolation, across all grazing scenarios. Overall, my research increases our general understanding of how ecosystems with multiple stable states respond to connectivity changes, stresses the importance of collecting connectivity and grazing rate data for coral reefs and pioneers novel methods for informing reef management design. Specifically, my thesis stresses the importance of connectivity and local stability dynamics to coral reef management by showing that natural larval dispersal can tip connected reefs to higher coral states but is not sufficient to restore coral habitat worldwide.

Keywords: anthropocene
climate change
coral reefs
dispersal
mathematical models
theoretical ecology
Advisor: Krkošek, Martin
Fortin, Marie-Josée