VictorJuiz/thesis huggingface dataset:
Temporal abstraction and generalisation in reinforcement learning

The ability of agents to generalise---to perform well when presented with previously unseen situations and data---is deeply important to the reliability, autonomy, and functionality of artificial intelligence systems. The generalisation test examines an agent's ability to reason over the world in an \emph{abstract} manner. In reinforcement learning problem settings, where an agent interacts continually with the environment, multiple notions of abstraction are possible. State-based abstraction allows for generalised behaviour across different \mccorrect{observations in the environment} that share similar properties. On the other hand, temporal abstraction is concerned with generalisation over an agent's own behaviour. This form of abstraction allows an agent to reason in a unified manner over different sequences of actions that may lead to similar outcomes. Data abstraction refers to the fact that agents may need to make use of information gleaned using data from one sampling distribution, while being evaluated on a different sampling distribution. This thesis develops algorithmic, theoretical, and empirical results on the questions of state abstraction, temporal abstraction, and finite-data generalisation performance for reinforcement learning algorithms. To focus on data abstraction, we explore an imitation learning setting. We provide a novel algorithm for completely offline imitation learning, as well as an empirical evaluation pipeline for offline reinforcement learning algorithms, encouraging honest and principled data complexity results and discouraging overfitting of algorithm hyperparameters to the environment on which test scores are reported. In order to more deeply explore state abstraction, we provide finite-sample analysis of target network performance---a key architectural element of deep reinforcement learning. By conducting our analysis in the fully nonlinear setting, we are able to help explain the strong performance of nonlinear neural-network based function approximation. Finally, we consider the question of temporal abstraction, providing an algorithm for semi-supervised (partially reward-free) learning of skills. This algorithm improves on the variational option discovery framework---solving a key under-specification problem in the domain---by defining skills which are specified in terms of a learned, reward-dependent state abstraction.

Keywords: imitation learning
generalisation
reinforcement learning
target networks
temporal abstraction
Responsive lanthanide-doped nanoparticles for bioimaging

With recent advances in nanotechnology, the tailored synthesis of functional nanoparticles has emerged as a new frontier for biomedical diagnostics, especially in the field of (pre-)clinical bioimaging. In this thesis, biocompatible superparamagnetic iron oxide nanoparticles (SPIONs) and Gd3+-doped mesoporous silica nanoparticles (Gd-MSNs) have been investigated as versatile, stimuli-responsive magnetic resonance imaging (MRI) contrast agents (CAs). One of the principal aims was to optimise T1 relaxivities (r1) for these paramagnetic nanoparticles, with a distinct emphasis on exploiting the often overlooked outer-sphere (OS) relaxation pathway. Another key objective was to design T1-active CAs whose OS relaxivities could be tuned in response to biologically-relevant stimuli (e.g., environmental pH), enabling disease-specific reporting capability by MRI. Following an introduction to the fundamental theory/project aims in Chapter 1, and a discussion of detailed synthetic/characterisation procedures (Chapter 2), the research results are presented in Chapters 3, 4 and 5. In Chapter 3, strategies for enhancing T1 contrast using Gd-MSNs are discussed. This enhancement is achieved by incorporating strong hydrogen-bonding acceptors (HBAs) into the channel sidewalls, influencing the mobility of the internalised water and hence amplifying OS relaxivities. This chapter also introduces a novel, one-pot Ugi-coupling reaction for the synthesis of bifunctional MSNs with dual-modal MRI and fluorescence imaging capabilities. In Chapter 4 it is shown that externally grafting a pH-responsive, hydrophilic polymer onto Gd-MSNs results in a powerful T1 contrast agent which possesses the highest reported r1 switches at clinically-relevant magnetic fields (1.4 T, 1.5 T and 3 T). These ultrahigh switches result from modulating the mobility of nanoconfined water, and consequently the OS relaxivities, through a conformational change of the polymer. Building on this polymer-mediated OS mechanism, Chapter 5 describes polymer-modified SPIONs possessing significant, reversible r1 switching, along with antifouling characteristics. Chapter 6 summarises the conclusions from Chapters 3 - 5. Chapters 7 and 8 provide relevant supplementary information and a list of references.

Keywords: lanthanides
f-block chemistry
magnetic resonance imaging
bioimaging
surface-initiated polymerisation
inorganic nanoparticles
luminescence
nanoparticle synthesis
Advisor: Davis, J
Faulkner, S
Political economy of state capacity

How does society react politically when the state tries to increase its capacity? This thesis presents three experimental studies that shed light on this question by examining the micro-behavioral consequences of such increases in state capacity. The first article explores how delegating tax collection to local elites such as city chiefs in the Democratic Republic of the Congo affects citizen demand for accountability, both from the state and the chiefs. It examines data collected from a randomized controlled trial in the city of Kananga where neighborhoods were assigned to either state or chief tax collection, and citizens had a collective action opportunity to hold both accountable through the request of citizen meetings. The results show that delegating tax collection to local elites does not impede citizens’ ability or willingness to hold both chiefs and the state accountable for policy outcomes. The second article asks whether there are electoral costs associated when candidates for political office promise to fight corruption instead of improving economic growth. The study draws on evidence from two conjoint experiments conducted in India and the United Kingdom where respondents could choose between different hypothetical candidates who varied along multiple dimensions. The results suggest that promising to fight corruption as a central policy platform may be electorally punished by voters compared to promising to improve the economy. The third article investigates the bottom-up accountability pressures in response to earmarking tax revenue for particular public expenditures. It does so by leveraging variation in the extent of citizens’ beliefs about the prevalence of earmarking in an online survey experiment conducted in Ghana. Results show that earmarking revenue does not have positive accountability effects by virtue of increasing citizens’ incentives to engage in politics. Overall, this thesis contributes original data, novel evidence and new theoretical frameworks to the study of state-society relations and the political economy of state capacity.

Keywords: state capacity
political economy
Advisor: Rueda, D
Kosmidis, S
The relationship between education and political doctrine : the Isocratic heritage and a Socratic alternative

This thesis is a philosophical and historical examination of the question of the nature of education. There are two ideas of the nature of education. The Isocratic idea is defined by the claim that the normative intentions of education are deductively derived from, and therefore relative to, an axiomatic political doctrine. The Socratic idea is defined by the claim that such intentions are constructed on the basis of permanent and universal philosophical questions, and not relative to political belief and aspiration. The thesis is primarily intended to clarify the question of the nature of education by explicating these two alternative ideas in critical contrast with one another.
The argument of the thesis proceeds as follows. Part One (Chapters One and Two) argues that the heritage of Isocrates has influenced virtually every aspect of the historical development of Western civilization, particularly education, and yet does not appear in contemporary educationist literature. Part Two (Chapters Four to Seven) explicates the logical structure of the Isocratic idea of education, as well as the Isocratic tradition of educational thought and practice generally, and shows that both the idea and the tradition provides the unity of Western education from classical antiquity until the present day. Part Three (Chapters Eight to Ten) argues that the Isocratic idea of education may no longer be tenable in contemporary educational thought or practice, and that we ought therefore to reconsider the alternative Socratic idea. The Socratic idea of the nature of education, as well as some aspects of the practical educational programme based upon it, are examined in some detail.
The thesis draws two conclusions. First, for practical, theoretical, and philosophical reasons, we must return to the question of the nature of education in full awareness of the two alternative ideas available to us. Second, and by implication, the future status and credibility of educational philosophy as an academic discipline cannot be improved without a return to careful study of the historical foundations of the discipline.

Keywords: Socrates
political doctrine
Isocratic
education
Socratic
Isocrates
A study of a christian conception of vocation in networked life: in dialogue with Oliver O’Donovan and Robert Adams

In networked life, individuals are encouraged to live with a sense of limitlessness, and yet are provided with less, agency diminished, if not corrupted. Action within networked life seeks control or dominance in the material realm, such short-termism antithetical to freedom. These challenges are clearly presented, albeit from contrasting angles, in the work of the psychoanalyst Sherry Turkle and the sociologist Ronald Burt (contextualised by Marshall McLuhan, Jacques Ellul and George Grant), each converging on control as the aim of networked life.
We discover that networked life is, in its corrupted state, a life in the flesh. Networked life, however, can be redeemed. In this dissertation, I argue that a certain Christian conception of vocation is the way through networked life. Through discussion with Robert Adams and Oliver O’Donovan, vocation is defined as ‘An offering from God, involving an ensemble of worldly relations and created goods, through which we are given, in particular, to pursue excellence, realise our agency and serve God.’ I stress Adams’ focus on pursuit of excellence, but in a manner that is reflective of O’Donovan’s restraint.
Both interlocutors provide compelling conceptions of vocation, with ample opportunity for critical dialogue (this contextualised by Grisez, Bonhoeffer, Ellul, Stackhouse, Jr., Williams and others), particularly in areas such as faith, love and responsibility. Moreover, Adams proposes a theory of value grounding excellence in resemblance to the Transcendent Good, whereas O’Donovan embeds value in moral order, which seizes individuals. Each sees the measure of value as residing outside of individuals, discussion on value important in a world that otherwise prioritises the securing of short-term material advantage. This dissertation favours an evangelical Christian ethics, engaging constructively with networked life, and making use of its clear benefits, using such benefits for mutual service in the world, and in the ultimate service of God.

Keywords: networks
excellence
calling
service
agency
infinite
vocation
Advisor: Biggar, N
The crossmodal correspondences between basic tastes and visual features: examining visual synergies and the underlying psychological mechanisms

This thesis presents several explorations on vision-taste crossmodal correspondences, which delve into the patterns, underlying mechanisms, and implications of the systematic associations that have been documented between visual features and basic taste qualities. The research findings presented in this thesis demonstrate the significant role of colour properties, curvilinearity, typeface, geometric configurations, and texture in establishing consensual associations with basic taste qualities. In the previous efforts to identify the possible origins of these correspondences, some promising theories were highlighted by other researchers to explain the causes guiding people to make certain associations between visual and taste attributes. Building on these popular explanations, a theoretical model is proposed that conceptualises the dynamics of crossmodal correspondences, introducing a putative system of hierarchy in decision-making for these associations. The relevant studies featured in this thesis have placed an emphasis on the role of emotion in mediating some of the correspondences observed, and adopted the semantic differential technique to gauge how various feelings associated with sensory attributes might have contributed to the crossmodal correspondences. Experiments in this thesis also reveal that specific pairings of visual features, such as colour and curvilinearity, could work in synergistic ways to influence taste expectations. Finally, the applications of these correspondences in various contexts, from influencing consumer expectations to enhancing dining experiences, are explored with respect to the mapping patterns and potential of visual synergy. The thesis concludes with a reflection on the research achievements, their impact on the field of crossmodal correspondences, the future directions of research in this field, and an acknowledgement of limitations, along with proposals for follow-up studies to address these limitations.

Keywords: taste
colour
visual features
semantic differential technique
shape
vision
crossmodal correspondences
curvilinearity
Advisor: Spence, C
Smithson, H
Cognitive mechanisms, neural circuitry, and individual differences in human foraging under threat

In nature, survival often requires foraging while staying vigilant for potential predators. The ability to flexibly switch between foraging and vigilance is ubiquitous across species and thought to have shaped the evolution of neural circuitry in animals. Balancing attention between rewards and threats is also relevant to contemporary human life, and changes in responding to rewarding versus threatening stimuli have been linked with symptoms of psychiatric disorders such as apathy and anxiety. However, it is not fully understood how people make these behavioural transitions. Characterising the cognitive mechanisms, neural substrate, and role of individual differences in this process would improve our knowledge of ecological decision-making and could provide insights of clinical relevance. This thesis aims to address these knowledge gaps by presenting three experiments. First, Chapter 1 introduces relevant background information including theories, methods, frameworks, and questions that inspired the design and interpretation of the following experiments. Next, Chapter 2 presents a unique ecological task for studying natural foraging and vigilance behaviours. I use a computational model to show how people solve this task by tracking information about reward and threat levels, demonstrate model validity with simulation, and show potential clinical relevance. Third, Chapter 3 describes a large online study inspired by computational psychiatry and transdiagnostic approaches. Results show that dimensions of psychiatric symptoms are characterised by distinct patterns in switching between foraging and vigilance, and suggest that emotion plays an adaptive, homeostatic role in decision-making. Chapter 4 presents a study that uses ultra-high field fMRI to map brain areas involved in behavioural transition. It shows that two distinct networks are involved in tracking reward and threat and initiating behavioural change. Finally, a general discussion considers how this work contributes to our knowledge of fundamental cognitive and neural processes and supports emerging methodologies for studying mental health and disease.

Keywords: decision-making
transdiagnostic
cognitive neuroscience
computational modelling
Advisor: Rushworth, M
Scholl, J
An investigation into the mechanisms underpinning the pH-sensitivity of GLP-1 release from the enteroendocrine cells of the gut

GLP-1 is a gut hormone primarily secreted from the enteroendocrine L-cell in response to a meal. GLP-1 acts to potentiate insulin secretion from pancreatic beta-cells and suppress appetite through actions on POMC and NPY neurons in the PVN. Reports of a lower average gut pH in obese and diabetic patients, prompted a preliminary investigation into the pH dependence of GLP-1 release. Work from our laboratory has shown that GLP-1 secretion is pH dependent when using a non-physiological HEPES buffering system. To further dissect the underlying mechanisms of pH-dependent GLP-1 secretion, the pH sensitive ratiometric dye, cSNARF1, was used to visualize the extra- to intracellular pH relationship in GLUTag cells, a model L-cell line, I show that, comparable to other cell lines, pHi of GLUTag cells is slightly more acidic and directly responded to changes in pHo. Decreasing extracellular [Cl-] or [HCO3-] resulted in significant intracellular alkalinisation, demonstrating a significant role of the proton-loading anion exchanger (AE) for pH regulation in GLUTag cells. Using a FRET-based and ELISA assay, the pH-dependence of GLP-1 secretion was confirmed in both CO2/HCO3− buffered and HCO3−-free (HEPES) media despite inconsistent pHi regulation. Furthermore, I show that, specifically, the rate of GLP-1 secretion is markedly lower in acidic conditions. Considering the multifactorial nature of multiple mechanisms for both intracellular pH regulation and GLP-1 secretion, more sophisticated methods will be required to robustly investigate in real time the underlying pH-dependent intracellular mechanisms which impact GLP-1 secretion from gut enteroendocrine cells.

Keywords: GLP-1
GLUTag
gut hormone
pH
Advisor: de Wet, H
Swietach, P
Why do whales exist? cancer resistance in cetaceans

Cancer is the inevitable fate for all multicellular organisms should they live long enough. However, if all mammalian cells have approximately equivalent probabilities of cancer-associated gene mutation then, all else being equal, the number of cells susceptible to transformation should scale with body size and longevity, implying larger, longer-lived mammals such as whales should be disproportionately more prone to cancer than smaller mammals, such as mice, simply due to cell number alone. Empirical evidence to date does not support this prediction, and the discrepancy between theory and observation is known as Peto’s paradox.
The mere existence of whales and other large mammals demands the evolution of a suite of cancer suppressive mechanisms in order to resolve the paradox; the establishment and understanding of which being the focus of this research.
First, cancer incidence rates across the animal kingdom are collected and assessed, quantitatively revealing the veracity of Peto’s paradox and the relationship between incidence risk and scaling basal metabolic rate. Next, computational models are developed to test the hypothesis that intrinsic metabolic scaling may itself be sufficient to explain Peto’s paradox. Results reveal that though incorporating scaling life-history traits into the narrative helps, they alone aren’t sufficient to explain cancer suppression in large mammals. Instead, genetic control is established as playing a crucial role. Finally, comparative analysis of the available cetacean genomes and transcriptomes uncovers several gene duplication events in key cancer-associated pathways, alongside lineage-specific and size-dependent selection in gene families associated with cancer resistance. This work demonstrates though there exists no singular explanation for Peto’s paradox, the disparate mechanisms that underlie the trajectory towards cancer resistance in independent lineages are all variations on a theme - the understanding of which may elucidate insight towards the development of future targets for human cancer therapies.

Keywords: Whales
Cetaceans
Genomics
Zoology
Cancer
Computational Biology
Peto's Paradox
Advisor: Shimeld, S
Exploring the role of the microbiome in inflammatory arthritis disease pathogenesis

Spondyloarthritis (SpA) is characterised by chronic, destructive joint inflammation and in many patients, intestinal inflammation. Microbiome profiling in SpA has identified increased abundances of inflammation-associated bacteria, and patients with reactive arthritis (ReA); a subset of SpA, develop joint inflammation following gastrointestinal (GI) infection. Despite the presence of these microbiome associations, the host-microbe interactions supporting disease remain largely unknown and may provide an avenue for microbiota-based therapies. The SKG mouse model of SpA develops microbiota-dependent joint and intestinal inflammation following dectin-1 activation with curdlan. Thus, in this thesis we aimed to study two forms of SpA; ReA and axial spondyloarthritis (AxSpA) in the SKG mouse model to delineate the host-microbe interactions driving joint disease. Firstly, we developed a microbiota-dependent model for ReA utilising GI infection with a murine enteropathogen in place of curdlan and characterised the microbiome and the T cell responses in this model. Secondly, we leveraged gnotobiotic SKG mice to delineate host-microbe interactions driving disease in the SKG-curdlan model. We found that although germ-free SKG mice did not develop disease, monocolonisation followed by depletion of the bacterium Bifidobacterium animalis did not prevent onset of disease. Furthermore, administration of heat-killed B. animalis to germ-free SKG mice enabled the induction of joint disease, suggesting that a component of B. animalis programs the host immune system to respond to curdlan and trigger disease. Finally, to determine the clinical relevance of studying the intestinal microbiome in SpA we completed immune phenotyping on AxSpA blood and stool, as well as stool metabolomic analysis. We identified a Th17 signature in the stool and blood of patients with AxSpA and identified several stool amino acids negatively associated with stool IL-23, suggesting an association between the host immune system and amino acid availability to microbes. Collectively, this work identifies several host-microbe associations in murine models of SpA as well as in patients with SpA and opens new avenues for the exploration of the host-microbe interactions supporting disease in SpA.

Keywords: Microbiome
AxSpA
Advisor: Powrie, F
Foster, K