VictorJuiz/thesis huggingface dataset:
Discount Factor Estimation in Inverse Reinforcement Learning

In reinforcement learning, the reward that guides the learning process of a decision-maker acting in an environment to achieve a goal can be non-trivial to specify. As such, inverse reinforcement learning (IRL) seeks to observe an expert and explain behaviour in terms of a reward. However, behaviour of humans is linked to the choice of discount factor and arbitrary choices of discount factor in IRL can lead to different rewards and optimal behaviours. Thus, discount factor estimation as part of the IRL framework is crucial to mitigate such bias in the learning process. This thesis evolves a utility-based objective within the maximum entropy IRL algorithmic framework to simultaneously estimate the discount factor and reward in a gradient-based manner via explicit computations of derivatives. The case of a single expert is first investigated prior to extension to multiple experts, in which Expectation-Maximization procedure is employed to learn each expert's cluster of trajectories and obtain maximum likelihood estimates of interest. Experimental and numerical studies on MDP environments, e.g., Grid-World are carried out with evaluative performance measures. Furthermore, an application to real-world traffic data is presented. Driving behaviour of humans may react sub-optimally to congestion leading to loss in traffic capacity. Therefore, a comparison of the observed traffic flow with theoretical maximum flow gives a quantified loss. Recommendations are also made on how a human driver should be nudged to minimize the loss. This is possible by solving three optimization problems: IRL given traffic data to learn actual reward and discount factor, forward reinforcement learning using a flow-based reward to generate ideal driving behaviour leading to theoretical maximum traffic flow, and a least-square fitting procedure to compute ideal rewards. Empirical evaluations on highway 401 in Greater Toronto Area are carried out to validate the approach.

Keywords: discount factor
inverse reinforcement learning
maximum entropy
Advisor: Lee, Chi-Guhn
Fractures: A History and Philosophy of Patient Suffering in 20th-century American Medicine

My dissertation explores the history and philosophy of patient suffering in 20th-century American medicine. Chapter One argues that historians of medicine colloquially synonymize suffering with related phenomena, such as pain, which risks treating suffering as a transhistorical object. That is a problem, first because suffering appears to be historically distinct, and second because neglecting it has undesirable consequences in the history of medicine and beyond. In response, Chapters Two and Three modestly enlarge the historical scholarship by presenting an intellectual and cultural history of American physician Eric Cassell’s (1928-2021) influential theory of suffering. This narrative argues that legal influences in Cassell’s early intellectual development and the medico-legal milieu in which he wrote provided the impetus, concepts, and language for his seminal theory. Chapter Four brings my historical findings to bear on current philosophical debates over Cassell’s view. Some critics argue that his account is too narrowly focused on damage, an objection I contextualize historically using the legal descriptions of suffering that influenced him by way of an explosion in medical malpractice lawsuits. My historical research thus lends credence to existing philosophical critiques. To further reinforce these critiques, I also introduce a case of suffering excluded by Cassell’s narrow account, which I call ‘paradoxical purposes.’ On the basis of this exclusion, I conclude that his view does not exhaust suffering as he intended. To rectify this shortcoming, Chapter Five amends his theory in two different ways. Both locate personal integrity, which Cassell says suffering affects, on a spectrum that ranges by ‘existential degrees.’ I refer to the lower end of this spectrum as ‘local suffering,’ which includes paradoxical purposes, whereas Cassell’s focus is on the higher end, ‘global suffering.’ Chapter Six explores two ways scholars can theorize about suffering along this spectrum. One exhausts suffering in general accounts, which I refer to as ‘monistic theories.’ The other involves a multiplicity of narrower models aimed at types of suffering, which I call ‘pluralistic theories.’ Next, I associate these theories with the conceptual questions to which they are most relevant in a bid to facilitate greater collaboration among theorists.

Keywords: bioethics
Eric Cassell
history of medicine
pain
suffering
Advisor: Upshur, Ross
Jones-Imhotep, Edward
Measuring, Predicting, and Applying the Octanol-air Partition Ratio (KOA) and its Temperature Dependence

The partition ratio of chemical concentrations in octanol and air at equilibrium (KOA) is often used in environmental chemistry to describe numerous partitioning processes involving the gas phase. A database comprising 2 500 experimental and 10 000 estimated KOA values was assembled and curated to consolidate decades of previous measurements and estimations. Analysis of the database showed the advantages, gaps and applicability range of different measurement and prediction techniques. Relying on the experimental KOA values from the database, five different methods for estimating KOA were compared in terms of their suitability for producing reliable predictions with low prediction intervals and wide applicability. Poly-parameter linear free energy relationships were identified as the preferred prediction technique. A new, curated dataset of high-quality internal energies of octanol-air phase transfer dUOA, which is key to predicting the temperature dependence of the KOA, was assembled and used to develop a new equation that predicts dUOA using only the log KOA at 25 °C. Direct measurements of the log KOA in the range from 4.5 to 7 are rare, but could be made with a modified generator column technique. Quantification of the octanol concentration during the experiments was identified as a key requirement for obtaining reliable results for substances in this volatility range. This thesis expanded current measurement techniques and identified and created reliable methods for predicting the KOA and its temperature dependence.

Keywords: environmental chemistry
internal energy
octanol-air
partition coefficient
partition ratios
physical-chemical properties
Advisor: Wania, Frank
Design and Characterization of Touch Sensing, EMI Shielding and Wireless Power Transfer Structures for Wireless Devices

With the market for wireless devices as competitive as ever, the stricter performance specifications imposed on new devices test the limits of current design tools. This motivates further research on design optimization approaches that can meet these stricter specifications under fabrication constraints. The thesis demonstrates noveldesign optimization approaches with experimental validation, and material level investigations that can solve three significant problems associated with the design of modern wireless devices. First, an electromagnetically-aware design methodology is proposed for touch sensor panels, to facilitate the integration of antennas. Building on the observation that touch sensor panels are finite periodic structures with transmission pass-bands that depend on the shape of their electrodes, these shapes are optimized to facilitate the integration of antennas at frequency bands of interest. To that end, a hybrid simulation approach is leveraged for touch sensor panels, based on the combination of a quasi-static simulation of a sub-section of the panel for the extraction of an equivalent circuit of the entire panel that is simulated in SPICE, reducing the simulation time by an order of magnitude. The feasibility of this methodology is experimentally demonstrated, by integrating an antenna with a touch sensor panel with minimal impact on its radiation pattern. As an increasing number of wireless services co-exist in various indoor and outdoor environments, the importance of electromagnetic interference (EMI) shielding is as high as ever. Shielding structures for such applications, such as frequency selective surfaces and absorbers, often need to be optically transparent for aesthetic and practical reasons. This thesis introduces optically transparent EMI shield designs based on copper mesh and aluminum-doped zinc oxide, with a unique combination of absorbance and optical transparency that advance the state of the art. Finally, a semi-definite relaxation (SDR) design optimization framework is applied for wireless power transfer systems with a single transmitter and multiple parasitically excited loops on a surface, for wireless power transfer in free space and in biological tissues. The potential of this approach is demonstrated to achieve significantly improved wireless power transfer efficiency to implanted devices, designing a proof-of-concept system including a small receiver at 50 MHz.

Keywords: Electromagnetic Interference
Electronic Skin
EM Shielding
Optically Transparent
Touch Sensors
Wireless Power Transfer
Advisor: Sarris, Costas D
The Impact of Essential Amino Acid Supplementation on Free-living Protein Metabolism and Recovery following Resistance Exercise in Healthy Adults

The consumption of essential amino acids (EAAs) plays an important role in enhancing adaptation and recovery after exercise. The two studies of this thesis sought to determine the effect of crystalline EAA supplementation following resistance exercise (RE) on protein metabolism and muscle recovery in free-living settings. The first study examined the effect of daily EAA supplementation on myofibrillar protein synthesis following RE and how this may relate to the recovery of muscle damage. We found that EAA supplementation did not enhance myofibrillar protein synthesis integrated over 96 hours post-RE in free-living young men consuming 1.2g/kg/day of dietary protein. We also found that EAA supplementation moderately improved muscle damage recovery over the same timeframe, but this was unrelated to changes in myofibrillar protein synthesis. The second study examined the retention of exogenous leucine (an index of whole-body protein synthesis) and the urinary 3-methylhistidine:creatinine ratio (3MH:Cr; an index of myofibrillar protein breakdown) with different EAA supplements following bodyweight RE in a home-based setting. We found that exogenous leucine retention and urinary 3MH:Cr improved with the consumption of an EAA formulation or branched-chain amino acids (BCAAs) following RE in young adults. However, these outcomes generally improved to a greater extent with EAA supplementation as compared to BCAAs. The overall findings of this thesis indicate that the consumption of all EAAs optimizes the anabolic response to RE, but is less impactful in the context of adequate dietary protein. Additionally, the ability of EAAs to ameliorate muscle damage recovery is not contingent on increases in myofibrillar protein synthesis, suggesting they primarily enhance recovery via a different mechanism. Finally, we demonstrate that a home-based study design to assess postprandial protein metabolism, which had not previously been conducted, is feasible and provides meaningful results. This approach should be further applied in various settings outside the laboratory to understand how the beneficial effects of exercise are best supported by nutrition in different populations and/or circumstances.

Keywords: amino acids
muscle recovery
performance
protein metabolism
resistance exercise
young adults
Advisor: Moore, Daniel R
Undergraduate Students’ Experiences of Academic Probation and Suspension

This study involved case studies of the lived experiences of six undergraduate students who received notice of academic probation/suspension at a large university. The students participated in a voluntary academic retention program, and all returned to good academic standing. Two interviews with each student explored their backgrounds, high school experiences and challenges in first year university that resulted in notice of academic probation/suspension. The interviews also covered the students’ participation in an academic retention program and the actions they took to regain good academic standing.The interviews revealed a link between students’ assumptions from high school and lack of information at university that hindered their acquisition of needed knowledge and skills to succeed in first year. The mostly first-generation students encountered challenges in a system that expects students to have a baseline of knowledge, interactional skills to self-advocate and academic skills to manage their time and commitments effectively. The concepts of cultural capital, habitus and field were valuable for understanding how students handled academic probation and suspension, gained knowledge of the institution and obtained advice from trusted knowledgeable individuals. The students demonstrated determination and resilience which helped them return to good academic standing. Individual student profiles and cross-case analysis revealed areas of knowledge and understanding that were vital for students’ timely progress through university and provided insight into the kinds of advice and guidance that were most beneficial to them. The students’ completion of an academic retention program boosted their confidence, enhanced their academic skills for university and enabled them to interact more effectively with staff, mentors, academic advisors, faculty and peers. In addition, with guidance from academic advisors, they made strategic course choices that helped them improve their marks and return to good academic standing. Their university experience was transformative as they developed their interests, gained academic self-efficacy and even took on leadership roles. This study adds to our understanding of students’ experiences of academic probation and suspension, raises awareness of the challenges involved that merit the attention of administrators and hopefully encourages further work to understand and support the talents and needs of underrepresented students.

Keywords: academic probation
cultural knowledge
first-generation
habitus
suspension
underrepresented
Advisor: Childs, Ruth A.
Statistical Learning Changes Across Development

Statistical learning is widely credited with empowering learners of all ages to extract the environmental regularities necessary to piece together the structure of their worlds. Its availability to learners of all ages, however, belies important differences in how statistical learning likely changes across development. Acknowledging this developmental change has broad implications for understanding the cognitive architecture of statistical learning, but very little past work has addressed possible developmental differences across childhood. In this thesis, I present three experiments highlighting the ways in which ongoing cognitive and neural development shape the operation of statistical learning, demonstrating that statistical learning changes in quality with age and experience. In Chapter 2, I ask whether the memory representations formed as a result of statistical learning vary with age. I report that while adults and older children (8-9-year-olds) form general and specific memories for statistical structures, young children (5-7-year-olds) remember only specific information. In Chapter 3, I directly investigate the neural underpinnings of statistical learning in 9-10-year-old children and young adults, and show that children rely more on parietal and temporal cortices and posterior hippocampus to support statistical learning than adults, who rely on ventral prefrontal cortex and anterior hippocampus. Accordingly, children represent general memories in the posterior hippocampus and IFG, while adults represent them in the vmPFC. In Chapter 4, I characterize the interaction between prior knowledge and attention during statistical learning in adulthood, laying the groundwork for developmental investigations of how children’s attention and minimal experience shape statistical learning. Specifically, I ask how experience in an environment shifts the focus of attention during learning, and show support for the idea that as a learner gains experience in an environment, they attend to successively more complex aspects of that environment. In Chapter 5, I review these interconnected findings, and reiterate that our understanding of statistical learning will be incomplete without considering how the input and output change with cognitive and neural development. Together, the studies in this dissertation offer clear insight into how developmental changes in statistical learning fundamentally alter the ways in which children interact with, learn about, and remember their experiences.

Keywords: Cognitive Development
Developmental Cognitive Neuroscience
Statistical Learning
Advisor: Finn, Amy S
Emplaced, Relational and Therapeutic? An Exploration of Parkrun’s Role in Mental Health Recovery

Community-based physical activity (PA) is beneficial for individuals with mental health conditions. In an explicit link between parkrun and mental health, the Bethlem Royal Hospital in South London was established in May 2019 as the first-ever site of a parkrun on the grounds of a National Health Service (NHS) trust. With the increasing use of PA as a multimodal treatment of mental illness, parkrun participation may have clinical implications. However, engagement in parkrun by individuals with mental illness is unknown. Therefore, the current program of research sought to understand the parkrun experiences of those experiencing mental health conditions. Study 1 explored discussions of parkrun in electronic mental health records. The findings were organized into 4 themes: i) More than Just a Run; ii) Clinicians as Gatekeepers; iii) The Role of parkrun in Recovery; iv) When parkrun Isn’t Always for Everyone. Study 2 examined the health, social and wellbeing impacts of parkrun engagement among runners and volunteers living with mental health conditions. It was found that those who run and volunteer, compared to those who only run, interact with significantly more individuals at parkrun, and feel more connected to their communities. Further, the benefits of parkrun participation are greater for those who run and volunteer, compared to those who only run. Using ethnographic methods, Study 3 aimed to capture the experiences of those involved in the Bethlem parkrun. The findings illustrated the emplaced and relational experiences of parkrun engagement. Given the context of the global pandemic, the Bethlem parkrun allowed for experiences of collective healing for many, while others were excluded or unable to engage in these practices. Collectively, this work may provide further understanding of the individual and collective experiences of parkrun. Such understandings may ultimately be of interest to both the parkrun organization and the Bethlem Royal Hospital. Furthermore, the findings regarding stigmatizing and discriminatory experiences may provide evidence for parkrun to explore ways that they may better welcome and accommodate individuals experiencing various mental health conditions and recoveries. These findings can be used to make parkrun a more welcoming and inclusive place for those experiencing mental illness.

Advisor: Sabiston, Catherine M
Measuring and Incorporating Family Spillover Cost and Health Consequences in Economic Evaluation of Child Health Interventions

Background: A child's health conditions affect family members’ health, well-being, and economic well-being. However, these spillover effects are ignored in conventional economic evaluations, resulting in an incomplete understanding of the cost and consequences of child health interventions. Frameworks for including family spillover effects have been proposed but are not specific to pediatric economic evaluations.Objective: The primary aim was to develop a theoretical framework for incorporating family spillover effects in pediatric economic evaluation. Specific objectives were to (1) summarize methods used in pediatric cost-utility analyses (CUAs) to include family health spillover effects and maternal-perinatal CUAs to integrate health outcomes of pregnant women and children, (2) integrate evidence from theories, theoretical frameworks and models into a theoretical framework, and (3) measure health-related quality of life (HRQoL), care-related quality of life (QoL), mental health service use and time losses from paid labour and/or usual activities for parents of children with a neuroinflammatory disorder(s) (ND). Methods: A systematic review of the literature was conducted to determine methods used to incorporate health outcomes of family members in pediatric and maternal-perinatal CUAs. A critical interpretive synthesis was performed to develop a theoretical framework for incorporating family spillover effects in pediatric economic evaluation. Empiric data were collected prospectively from the Hospital for Sick Children in a cross-sectional study. Descriptive statistics were used to describe time lost from paid labour and/or usual activities, HRQoL, carer-related QoL and mental health services. Results: Considerable heterogeneity was observed in methods applied to incorporate health outcomes of family members in pediatric and maternal-perinatal CUAs. In the proposed theoretical framework ‘conducting economic evaluation from a pediatric family perspective,’ the family is a unit of analysis, where family costs and consequences related to a child's illness or disabilities are derived from all family members and incorporated into the analysis. Findings from the cross-sectional study indicated reduced HRQoL and carer-related QoL, and time lost from paid labour and/or usual activities of parents of children with ND. Conclusion: The thesis presents a theoretical framework ‘conducting economic evaluation from a family perspective’ for incorporating family spillover effects in pediatric CUAs. This can be used to develop empirical methods to include family spillover effects in pediatric CUAs.

Keywords: child
cost utility analyses
economic evaluation
family spillover effects
parents
theoretical framework
Advisor: Ungar, Wendy J