VictorJuiz/thesis huggingface dataset:
Reinforcement Learning for Stochastic Control and Games in Algorithmic Trading

Algorithmic trading in electronic markets is a well-studied field in finance with a plethora of different possible approaches. This thesis explores how agents participating in electronic markets should optimally trade when accounting for latent factors and the behaviour of other participating agents. It investigates the problem by developing a modified Deep Q-Learning method for a single agent trader, generalizing the approach for multi-agent games using Nash equilibria, and constructing a new method for generating arbitrage-free implied volatility surfaces used for derivative pricing. The thesis contains three main parts. In the first part we take a model free approach to optimal trade execution and develop a variation of Deep Q-Learning to estimate the optimal actions of a trader. The model is a fully connected Neural Network trained using Experience Replay and Double DQN with input features given by the current state of the limit order book, other trading signals, and available execution actions, while the output is the Q-value function estimating the future rewards under an arbitrary action. We apply our model to nine different stocks and find that it significantly outperforms a baseline approach. In the second part, we generalize this approach into the multi-agent setting by developing a new data efficient Deep Q-learning methodology for model-free learning of Nash equilibria for general-sum stochastic games. The algorithm uses a local linear-quadratic approximation of the stochastic game, which leads to analytically solvable optimal actions. The approximation is parameterized by deep neural networks to provide sufficient flexibility to learn the environment without the need to experience all state-action pairs. We study symmetry properties of the algorithm stemming from label-invariant stochastic games and as a proof of concept, apply our algorithm to learning optimal trading strategies in various simulated competitive electronic markets. In the third part, we take the first steps in extending our approach into the derivatives market by proposing a new hybrid method for generating arbitrage-free implied volatility (IV) surfaces, which is a key feature used in derivative pricing. Our approach combines model-free Variational Autoencoders (VAEs) with continuous time stochastic differential equation (SDE) driven models. We focus on two classes of SDE models: regime switching models and L\'evy additive processes. By projecting historical surfaces onto the space of SDE model parameters, we obtain a distribution on the parameter subspace faithful to the data on which we then train a VAE. Arbitrage-free IV surfaces are then generated by sampling from the posterior distribution on the latent space, decoding to obtain SDE model parameters, and finally mapping those parameters to IV surfaces. We further refine the VAE model by including conditional features and demonstrate its superior generative out-of-sample performance.

Keywords: Algorithmic Trading
Implied Volatility
Machine Learning
Nash Equilibrium
Reinforcement Learning
Advisor: Jaimungal, Sebastian
TMS-EEG of FrontSensory Co-activation and Brain State in Suprathreshold Frontal Cortex

Transcranial magnetic stimulation (TMS) combined with electroencephalography (EEG) can provide an opportunity to study the intact brain by perturbing it and recording its response. It is crucial to have a clear understanding of the level of inherent sensory co-activation in the TMS-EEG signal, and to devise methods to minimize these effects. Additionally, TMS-EEG recordings are performed without considering the state of the brain at the time of stimulation. Investigating the possible modulatory effect of phase and power of EEG oscillation before the TMS pulse on the outcome of TMS-EEG recordings is of interest. We explored the differentiation of the auditory co-activation from TMS response in healthy participants (Study-1). We further studied the topic of sensory co-activation and state-of-the-art methods for masking it (Study-2). Additionally, we investigated the effect of phase and power of EEG oscillations before stimulation on single trial TMS-EEG response (Study-3). We observed a distinctively higher level of cortical reactivity to active TMS compared to the auditory sham and the sham coil. The early response (<80 ms) in the active conditions (state-of-the-art- masking vs conventional) did not show significant differences over the stimulation site and across all of the electrodes (global activity). The late response (<275 ms) over the stimulation site was not significantly different between the active conditions, but the difference was significant in the global activity. Only active paired-pulse protocols in both studies resulted in a significant level of cortical inhibition measured by LICI. No effect of phase was observed on the single trial cortical reactivity while a significant effect for pre-stimulus power was evident in theta and beta bands with higher power resulting in larger single trial cortical reactivity to TMS compared with lower power. While sensory co-activation in suprathreshold TMS-EEG of dorsolateral prefrontal cortex is a concerning factor, it can be masked effectively with appropriate masking procedures, and reliable data regarding cortical reactivity to TMS can be achieved. Though we found no modulation of cortical reactivity to TMS by the phase of EEG oscillations, we did observe that higher power in theta and beta bands prior to stimulation can lead to larger cortical reactivity.

Keywords: DLPFC
EEG
TMS-EEG
Transcranial Magnetic Stimulation
Advisor: Blumberger, Daniel M
The Evolution and Consequences of Sexual Reproduction

Sex is predicted to be a costly mode of reproduction associated with drawbacks such as a genetic transmission disadvantage, the destruction of favourable genetic associations, and the cost of finding mates. Despite this, most eukaryotic species reproduce sexually at least occasionally. In my thesis I use genomics and field work to explore the possible benefits and consequences of sexual reproduction. In Chapter 2, I analysed data from a mutation accumulation experiment using two highly asexual duckweed species. My analyses revealed some of the lowest known multicellular eukaryotic mutation rates, possibly a consequence of asexuality. In Chapter 3, I analysed patterns of linkage disequilibria (LD) in two population genomics datasets to explore the potential role of negative LD and synergistic epistasis among deleterious mutations in favouring sex. I found that patterns of genome-wide LD among deleterious mutations differed from those of neutral mutations, but that this was likely a by-product of complex demographic processes. Further analyses focusing specifically on deleterious mutations affecting genes in established genetic networks did suggest a signal of within-network negative LD, hinting at a possible role of negative epistasis in favoring the evolution of sex. In Chapter 4, I used exploratory population genomics analyses to examine signatures of selection and recombination in the facultatively sexual liverwort Marchantia polymorpha. My findings suggested that despite the likely limited dispersal of bryophyte sperm, M. polymorpha populations appear to engage in sexual reproduction at least occasionally. However, other analyses suggested that the M. polymorpha genome harbors a large fraction of deleterious genetic diversity, a possible hint of inefficient selection associated with predominantly asexual reproduction. Finally in Chapter 5, I attempted to experimentally compare the fitness of sexual and asexual offspring in M. polymorpha in a field experiment to broadly ask which mechanisms may favour the evolution of sex. However, my analyses suggested that almost all variation in fitness in my experiment was due to environmental heterogeneity, precluding any biologically meaningful comparisons of sexual and asexual offspring. Overall, my thesis contributes to our understanding of the consequences of sexual and asexual reproduction, particularly in species where both reproductive modes are employed.

Advisor: Agrawal, Aneil AFA
Wright, Stephen SIW
Michael Wyschogrod and Jewish Existentialism

This dissertation analyzes the theological writings of Michael Wyschogrod through the lens of existentialism. While acknowledging the importance of Wyschogrod as a disciple of Karl Barth and an exponent of Jewish-Christian dialogue, the dissertation largely brackets these aspects of Wyschogrod’s intellectual background in order to focus on the ways in which his thought deploys themes and tropes drawn especially from the work of Søren Kierkegaard, but also from that of Martin Heidegger and Jean-Paul Sartre. The study examines four major topics in Wyschogrod’s philosophy of Judaism: his indictment of the inauthenticity Modern Orthodox community in the United States; his critique of halakhah (Jewish law) as a spiritual deadening agent which tends to rob the individual of her responsibility before God; his attempt to make Judaism harder by emphasizing the extreme challenges which Jewish revelation poses not only to theoretical reason, but also in light of the historical suffering of the Jewish people, to the sense of decency; and his attack on the role of universalizing ethics in modern Jewish thought. While the dominant source for Wyschogrod’s existentialist philosophy of Judaism is the work of Kierkegaard, the dissertation also makes continual reference to Wyschogrod’s most important influences in the world of modern Jewish thought itself — namely Joseph Soloveitchik and Martin Buber. Wyschogrod can be understood as torn between the conflicting influences of his former teacher Soloveitchik, whose thinking on halakhah Wyschogrod took to be a particularly baleful influence on American Orthodoxy, and the Biblicist, antinomian tendencies of Buber which were deeply attractive to him despite his unwillingness to abandon traditional Jewish practice or the Orthodox community.

Keywords: Buber
Existentialism
Kierkegaard
Modern Jewish Thought
Soloveitchik
Wyschogrod
Advisor: Novak, David
Nightcrawler Commodities: How capital subsumes nature and labour in Ontario’s underground bait worm industry

This dissertation uncovers the development and contemporary constitution of the ‘underground’ nightcrawler (Lumbricus terrestris) industry that has consolidated in southwestern Ontario. My theoretical starting point is the ‘subsumption of nature’ framework that identifies the strategies by which ‘nature-based industries’ harness biophysical processes to produce surplus value. While Boyd, Prudham and Schurman (2001), initially treated the subsumption of nature as “analogous” to the subsumption of labour in how they reflect the dynamics of capitalist design and control over the production process, I argue the subsumption of labour and nature are not merely parallel processes but are, in fact, conjoined; the formal and real subsumption of labour is dialectically related to the formal and real subsumption nature. The extent to which capitalists can control the ecological conditions of production will shape how labour is arranged and subsumed in the capitalist production processes. The characteristics of the peculiar nightcrawler industry result from the capitalist’s uneven ability to subsume the nightcrawler’s biological characteristics and ecological conditions necessary for commercial production. The research relies primarily on semi-structured interviews, as well as newspaper archives, and secondary natural science research. Through these methods, I describe the history of recreational angling and bait worms, why southwestern Ontario became the ‘worm capital of the world,’ why dairy farmers rent their land, how capital investments are deployed, as well as the social composition of the labour force and their working conditions. The conjoined subsumption framework I propose in this dissertation provides a way to go “beyond the antinomies of posthumanist and political economy inquiry” (Barua, 2019) and engage in the open-ended associations of socio-ecological relations while following how capital surplus value is produced. No one set out to develop a profitable bait worm industry in Ontario; it emerged as an industry and took shape through the contingent socio-ecological conditions of production that constituted who worked (under what conditions) and who profited (with what investments). Despite the non-deterministic trajectories of commodifying living organisms, value relations continue to take form and discipline producers; in other words, the pursuit of surplus value is the glue that holds these disparate people, soils, and worms together.

Keywords: Capitalism
Earthworms
Marxism
Political Ecology
Subsumption of Nature
Advisor: Prudham, Scott
Mobile Health (mHealth) in Older Adults with Airways Disease

The use of mobile health (mHealth) in asthma and COPD is growing in tandem with shifting demographic trends towards an older population. While mHealth innovations may hold great promise in helping older adults manage asthma and COPD, the two most common chronic airways conditions, many apps and mobile digital devices are not designed with older adults in mind, and older age is known to be a barrier to health technology adoption in general. Furthermore, little is currently known about how older adults with asthma and COPD specifically experience mHealth use. The purpose of this thesis is to illuminate these knowledge gaps and achieve a greater understanding of mHealth research and use in older adults with chronic airways disease. In the first study, I quantitatively analyzed uptake of and engagement with an mHealth asthma questionnaire among older adults, demonstrating that older age (>/=65 years) was associated with decreased uptake, but not engagement. This finding suggests that adverse perceptions of mHealth among older adults are an important barrier to adoption, even among tools optimized for older adult usability. In the second study, I systematically explored the asthma and COPD mHealth literature using a scoping review approach. In studies across the mHealth research cycle, I found that needs and perspectives of older adults have seldom been specifically considered, including in tool design. Even when included in the study populations, study designs and analyses seldom considered older adults separately. In the third study I used a phenomenological approach to qualitatively assess the perspectives and experiences of older adults with asthma and COPD in their use of mHealth. Through this analysis, I demonstrated a central tension among older adults between the use of mHealth to maintain independence and apprehension to a changing healthcare technology landscape. Overall, this thesis resulted in a deeper understanding of mHealth use among older adults with asthma and COPD, including quantitative and qualitative analyses of use and the state of current research, leading to identification of important evidence gaps that may inform future research, clinical practice, and health policy decisions with the goal of promoting effective and equitable use of mHealth in asthma and COPD across demographics.

Keywords: asthma
COPD
digital health
mHealth
Advisor: Straus, Sharon E
Development of a Protocol for Measuring the Tensile Properties of the Swallowing Musculature: A Potential Methodology for Quantifying Radiofibrosis

Radiotherapy for head and neck cancer (HNC) can lead to fibrosis (RF) and dysphagia. However, the relationship between these conditions remains unclear due to a lack of valid/objective RF measurement tools. Three studies were designed to address this knowledge gap. First, a systematic review was conducted to identify and evaluate existing tools for measuring RF in HNC patients. Eight electronic databases and five HNC journals were searched. Two blinded raters reviewed each reference. Included studies were critically appraised using Quality Assessment of Diagnostic Accuracy Studies-2. A narrative review was also conducted to identify tools for measuring fibrosis elsewhere in the body. One low cost tool with good clinical availability was shear wave elastography (SWE). SWE is sensitive to muscle fiber orientation; however, little is known about the musculoaponeurotic architecture of the swallowing musculature. Therefore, the second study evaluated the musculoaponeurotic architecture of the suprahyoid muscles (AD, anterior digastric/GH, geniohyoid/PD, posterior digastric/MH, mylohyoid/SH, stylohyoid). Ten cadaveric specimens were volumetrically dissected, digitized, and modeled, as in situ. Architectural parameters for each muscle were quantified and functionally distinct regions identified. The final study assessed the potential of using SWE to measure the stiffness of GH and genioglossus (GG), in vivo, in ten healthy adult subjects. Ten stiffness measurements were obtained bilaterally at rest and during swallowing exercises. The systematic review found nine tools that had published data regarding their reliability/validity. No valid and reliable tools had been applied to the swallowing musculature. All studies were at a high risk of bias. The anatomical study found that GH, SH, and PD had a single belly; MH had distinct anterior and posterior regions; AD varied by participant, with 1-3 distinct bellies. GH had the most consistent fiber bundle orientation and was therefore selected for initial SWE investigations. The third study showed that using SWE to measure the stiffness of GH and GG at rest and during certain exercises is possible and well-tolerated. Measuring the tensile properties of the swallowing musculature could offer insight into the underlying causes of dysphagia post-radiotherapy for HNC, and could thereby guide the development of novel, individualized, and targeted interventions.

Keywords: Dysphagia
Elastography
Head and Neck Cancer
Radiofibrosis
Swallowing
Ultrasound
Advisor: Martino, Rosemary
The Field of Higher Education in Pakistan as Experienced by Faculty: A Bourdieusian Analysis

Faculty play a key role in the development of a nation. Their ideologies, work, knowledge, and attitudes are all significant in not only determining the students’ success but also the overall quality of the education system. This study explores the experiences and struggles faced by the faculty in a public research university in Pakistan and their approaches to navigating their everyday work lives. It employs Bourdieu’s theoretical triad of habitus, capital, and field to highlight the struggles faced by the faculty in their everyday academic lives. The study is designed to understand the shaping and reshaping of academic habitus and the resultant formation of their sense of place. The interviewed participants included PhD holders who earned their degrees internationally as well as those who did so locally, belonging to the departments of Chemistry and Management Sciences. The study highlights ideological differences between the two disciplines, their varying backgrounds and dispositions. The study showed that not all faculty members working in the Pakistani academic environment hail from the same environment or have had similar access to resources and experiences. Their varying backgrounds and dispositions act as paradigms to help them understand and navigate their academic careers, starting with their choice of pursuing a Ph.D. The interviewed faculty experienced a clash between their expectations and reality when it came to new habitus formation, resulting in a very docile and compliant faculty. Overall, the study highlights a key common concern from participants across the board in relation to higher education reforms and systemic issues, which leads to multi-level issues of varying gravity. A lack of importance given to Social Sciences and overt importance given to Sciences were identified as creating rivalry and jealousy amongst the faculties. However, there were some commonalities between the faculties, including the use of the word ‘we’, non-uniform interpretation and implementation of policies, lack of job security, communication, and appreciation. Despite the significant differences amongst international and local PhDs, there did not appear to be consistent differences amongst any two subgroups, irrespective of how one divides the participant pool.

Advisor: Bascia, Nina Dr
Understanding Pulsar Scintillation with the Power of Straight Lines

Pulsars provide an exciting laboratory for the study of physics over a wide range of scales and regimes. Their extreme densities give us a window into the properties of matter in exotic conditions realized almost nowhere else in the universe. Their stable spins and regular emission allow them to be used as clocks to test general relativity and its alternatives, both in their local environment, and on a larger scale through gravitational waves. Their small size makes them effectively point sources whose coherent emission leads to an interference pattern when scattered by structures in the Ionized Interstellar Medium (ISM). However, this interference not only provides information about the ISM, but can be a nuisance parameter for other pulsar observations including timing. Fortunately, there is increasing evidence that not only is the scattering of any given pulsar dominated by a thin region along the line of sight, but also that in many cases it is restricted to a collection of scattering points forming a nearly straight line on the sky. In this thesis, I will present a new approach to understanding the scattering structure of the ISM in the direction of a pulsar. By taking advantage of the linearity of the scattering region, we map pulsar observations into a new space where the interference of pairs of images is more apparent and the properties of each image can be extracted. The power of this new technique is first demonstrated on simulated data where the assumption of a straight line of images is exact. Using archival data of pulsar B0834+06, we show how this new method is able to function on data where this assumption breaks down using both single dish and Very Long Baseline Interferrometry (VLBI) observations. These techniques will aid in future observations to determine physical properties of the pulsars, such as their size and distance, as well as reducing the effects of scattering from timing measurements that may limit the precision of gravitational wave measurements.

Keywords: Pulsar
scintillometry
VLBI
Advisor: Pen, Ue-Li