VictorJuiz/thesis huggingface dataset:
Integrating Linguistic Theory and Neural Language Models

Transformer-based language models have recently achieved remarkable results in many natural language tasks. However, performance on leaderboards is generally achieved by leveraging massive amounts of training data and computation, and rarely by encoding explicit linguistic knowledge into neural models. This has led many to question the relevance of linguistics for modern natural language processing. In this dissertation, I present several case studies to illustrate how theoretical linguistics and deep neural language models are still relevant to each other. First, language models are useful to linguists by providing an automatic and objective tool to measure semantic distance, which is difficult to do using traditional methods. On the other hand, linguistic theory contributes to language modelling research by providing frameworks and sources of data to probe our language models for specific aspects of language understanding. This thesis contributes three studies that explore different aspects of the syntax-semantics interface in language models. In the first part of my thesis, I apply language models to the problem of word class flexibility, a long-debated issue in theoretical linguistics. Using mBERT as a source of semantic distance measurements across many languages, I present evidence in favour of analyzing word class flexibility as a directional process. In the second part of my thesis, I propose a method to measure surprisal at intermediate layers of language models, using Gaussian models for density estimation. My experiments show that sentences containing morphosyntactic anomalies trigger surprisals earlier in language models than semantic and commonsense anomalies. Finally, in the third part of my thesis, I adapt several psycholinguistic studies to show that language models contain knowledge of argument structure constructions (a proposed analysis of verb valency from construction grammar theory). In summary, my thesis develops new connections between natural language processing, linguistic theory, and psycholinguistics to provide fresh perspectives for the interpretation of language models.

Keywords: computational linguistics
deep learning
language models
natural language processing
neural networks
Advisor: Rudzicz, Frank
Towards a Molecular Taxonomy for Intracranial Meningiomas

Meningiomas are the most common primary intracranial tumour and their incidences continues to rise in the face of the aging population. This dissertation investigated the hypothesis that outcomes of patients with intracranial meningiomas as determined by routine clinical classifications are heterogenous and can more accurately predicted using molecular features of tissue and blood. First, consensus expert definitions of a set of minimum clinical data elements required for molecular studies reporting on meningioma were established and vetted using discovery/development, internal validation, external validation and distribution phases. Second, the definitions established for common data elements were used to inform on the development and validation of a nomogram that combines genome-wide methylation data with known prognostic clinical features to more accurately predict tumour recurrence than standard clinical factors alone. Third, a Cluster of Cluster Assignments algorithm was used to perform comprehensive integrative clustering analysis of meningiomas combining DNA methylation data with mRNA data and gene-level copy number alterations. This algorithm revealed four novel molecular groups of meningioma with highly distinct clinical outcomes. Classification by molecular groups superseded all existing classifications and each molecular group showed distinctive and prototypical biology that informed on novel therapeutic options. Lastly, methylation profiling of circulating tumour DNA derived from plasma revealed the potential for non-invasive diagnosis and subtyping according to molecular alterations in meningiomas

Keywords: biology
genetics
meningioma
molecular
Advisor: Zadeh, Gelareh
Dynamic ER-PM Contact Sites and their Impact on Lipid Homeostasis of Phosphotyrosine Signaling

Membrane contact sites (MCS) are specialized cellular structures where two opposed membranes of intracellular organelles form a close gap (~20nm). These regions serve as communication hubs for various cellular processes such as inter-organellar exchange of lipids, calcium homeostasis, and phosphatase activity. The endoplasmic reticulum (ER) extends to the cell periphery and contacts the plasma membrane (PM) to form metastable ER-PM contact sites maintained by membrane tethering factors linking the two membranes. Cellular functions of ER-PM MCS depend on membrane tether proteins and distinct enzymes in the vicinity, but unsolved questions remain as to how ER-PM MCS is dynamically regulated. In this dissertation, I investigate the mechanism of ER-PM MCS formation and the functional role in physiological contexts. Specifically, the study aims to i) dissect the intrinsic properties of one of ER-PM contact/tether proteins, oxysterol-binding protein – related protein 8 (ORP8), to determine the deciding factors for its membrane targeting and ii) describe the role of cortical actin in regulating ER-PM contacts during phagocytosis and determine the physiological function of rapidly formed ER-PM contacts. In the first study, I found that both the N-terminal polybasic region and the PH domain of ORP8 are required for proper targeting of the short splice variant ORP8S to the PM-ER contact site. ORP8 localization is responsive to PM phosphatidylinositol phosphate (PIP) and phosphatidylserine (PtdSer) levels and its capability to bind PtdSer. Together, I describe that electrostatic interaction, PH domain binding of PIPs, and PtdSer ligand all play a role in defining ORP8 localization. In the second study, I characterize that depletion of cortical actin at the phagocytic cup leads to rapid ER-PM/early phagosome contact site formation. PTP1B accumulates in this region and acts in trans to dephosphorylate Syk. By modulating Syk signaling at ER-phagosome contacts, PTP1B regulates phagosomal superoxide production. In addition, my thesis includes a separate chapter investigating the role of lipid post-modification in SARS-CoV2 infectivity. I characterize that CoV-2 Spike protein is palmitoylated on multiple juxtamembrane cysteine residues and is required for efficient syncytium formation. Next, TVB-3166, a FASN inhibitor, significantly reduces Spike palmitoylation that reduces viral infectivity and grants protection against CoV infection.

Keywords: ER-PM membrane contact site
ORP8
Phagocytosis
PTP1B
SARS-CoV-2
Advisor: Fairn, Gregory
Effects of miR-137 on Pyramidal Neuron Morphology and GABA Interneuron Neurogenesis

A recent large genome-wide association study identified MIR137 as a leading candidate gene for schizophrenia. The promoter SNP rs2660304 may contribute to schizophrenia susceptibility of schizophrenia in the Ashkenazi Jewish population, and the major allele A of rs2660304 is associated with lower activity of the MIR137 promoter in vitro. However, the role of miR-137 in the pathophysiology of schizophrenia remains largely unknown, and in vivo genetic animal models could be useful in this regard. Therefore, we generated a humanized transgenic mouse carrying the major allele A of rs2660304 and examined the effects of this mutation on aspects of brain development and histology relevant to schizophrenia. Female mutant mice had selective loss of parvalbumin (PV) and calretinin (CR)-positive cells in the frontal cortex and/or hippocampus relative to wild-type controls, and those with severe loss of PV interneurons demonstrated the greatest impairment in working memory as measured in the Y-maze. The reduction in PV+ interneurons could have resulted from decreased proliferation and mitosis and increased apoptosis of medial ganglionic eminence-derived progenitors in the embryonic brains of mutants, but not altered tangential migration. RNA sequencing of miR-137 mutant mouse brain tissue revealed increased transcripts from gene sets involved in mitotic cell cycle and proliferation. In addition, mutant mice had increased dendritic spine density and branching complexity of pyramidal neurons, elongated axonal length and reduced axonal branches of primary hippocampal neurons, which could be due to indirect down-regulation of Notch1 by miR-137. Moreover, the hippocampal membrane NR2A protein level was significantly reduced in mutants relative to wild-type controls. Knocking-down miR-137 of zebrafish using morpholinos produced a major decrease in GAD67+, PV+, CR+ and mitotic cells. RNA-seq of these zebrafish morphants produced results similar to those in transgenic miR-137 mutant mice. Our findings are consistent with a large body of literature implicating GABA interneurons in the pathophysiology of schizophrenia and provides novel insights connecting a prominent candidate gene for schizophrenia with abnormal aspects of cortical development and histology. Our findings contribute new knowledge to understanding the genetic and developmental origins of schizophrenia that can inform future research aimed at creating better treatments targeting specific disease mechanisms.

Keywords: miR137
neurodevelopment
Parvalbumin interneuron
rs2660304
schizophrenia
zebrafish
Advisor: Wong, Albert HC
Careers After Breast Cancer: Agency, Meaning and Context at Work

This qualitative study explored the career experiences of breast cancer survivors after their diagnosis. Previous literature addressed the post-cancer return to work rates and specific work accommodations, yet little was known about how breast cancer affects women’s meaning of work and how they dynamically utilise and adjust their personal and contextual resources to overcome this obstacle. This study aimed to understand the career experiences of breast cancer survivors, including; the role of personal agency, the shifting career meaning and the contextual influences that impact their return to work. This is also the first study to utilise the Career Human Agency Theory (CHAT) as a framework for the work experiences of cancer survivors and to gather empirical evidence for this emerging meta-theory. Twelve women, ranging in age from 34 to 61, shared their career stories since being diagnosed with breast cancer within the past five years. Open-ended, semi-structured interviews were completed with each participant and then analysed using interpretive phenomenological analysis. Shared themes include changes in their sense of self-efficacy, shifted career priorities and goals, and common factors that supported or hindered their process of returning to work. A unique theme that emerged was the importance of having a sense of belonging at work; as an indicator of meaningful work, a motivator to return, and an integral contextual factor. The theoretical implications include a proposed extension of the CHAT model specific to a population of cancer survivors (CHAT-CS). This model details the relevant contextual barriers and supports which must be considered in the working-lives of cancer survivors. The model provides useful considerations for clinicians and practitioners working with survivors on their career planning and goals. Lastly, the limitations and directions for future research are discussed.

Keywords: Agency
Cancer
Career
Meaning
Psychosocial Oncology
Advisor: Chen, Charles P.
Irregular Welded Rectangular Hollow Section Truss-type Connections

This thesis presents research on four welded rectangular hollow section truss-type connections. Each type of connection was mainly investigated using numerical methods, with finite element models validated against experimental tests that were performed by either the author or other researchers. Design recommendations are proposed based on the numerical results. In addition, a numerically based failure mode is verified using the results from each study. For longitudinally offset X-connections, 264 numerical models were analysed with the end distance varied, along with other geometric parameters. By comparing the strength of longitudinally offset models and corresponding control models, a required minimum chord end distance is proposed, along with a strength reduction factor in case the minimum end distance is not satisfied. For laterally offset X-connections, a set of 504 numerical analyses was used to evaluate the validity of two analytical models. By performing a series of reliability analyses, one of the analytical models – a combined yield-line mechanism – is recommended to be used for the design of laterally offset RHS X-connections. For overlapped K-connections, the effect of hidden toe welds was investigated both experimentally and numerically. Five full-scale welded specimens with high overlap ratios were designed and fabricated for experimental testing in the laboratory, and for validation of numerical models. 628 numerical analyses were considered for a series of parametric studies and design recommendations on the inclusion of hidden toe welds are proposed for various contemporary codes and guides. Meanwhile, revised design equations for overlapped K-connections are also proposed. For zero-gap K-connections, a new chord face shear analytical model is proposed, based on a yield-line mechanism. Two full-scale connections were tested in the laboratory and were also used to validate corresponding numerical models. Then, 171 numerical models were analysed and were used to verify the proposed analytical model for different gap sizes.

Keywords: Connections
Design recommendations
Experiments
Finite element models
Hollow structural sections
Structural Engineering
Advisor: Packer, Jeffrey
Divergent Ecological Adaptation in the Speciation Cycle

In this thesis I attempt to better understand the general role played by adaptive ecological divergence in the evolution of new species. For the past two decades, evolutionary ecologists have largely viewed diversification through the lens of ecological theory of adaptive radiation (ETAR), an intuitive and influential synthesis in which divergent natural selection is the ultimate engine of biodiversity. Evidence for the theory’s main tenets has been reported in several detailed studies, leading many authors to suggest that adaptive radiation is a generally important mechanism in diversification. But much of this evidence is derived from the study of naturally occurring model systems that reside in conditions that may be especially favourable to rapid ecological divergence. Newly-discovered patterns of morphological and genetic diversity in non-model taxa, meanwhile, appear to contradict some expectations of this framework, calling into question whether the dynamics observed in model systems can in fact scale up to explain non-model diversity. It is thus unclear if our leading model of diversification applies in the complex ecological theatres in which diversity most typically arises. To address this problem, I design and deploy new statistical tools for studying divergence between speciating and recently speciated lineages (i.e. sister taxa). With these tools, I assess broader support for alternative hypotheses regarding two key processes in the diversification cycle: speciation and the evolution of coexistence between close relatives.

Keywords: Character Displacement
Ecomorphology
Speciation
Trait Divergence
Trait Evolution
Advisor: Weir, Jason T
Assessing the Unintended Consequences of Antipsychotic Reduction in Nursing Homes: Medication Substitution and Changes to Diagnosis Coding

Antipsychotic medication misuse and overuse have been persistent quality of care issues in nursing homes for decades. Although potentially inappropriate antipsychotic use has declined following various quality improvement efforts, increases in other psychotropic medications, such as trazodone, and increases in the indications used to define criteria for appropriate antipsychotic use (e.g., schizophrenia) have generated concerns about medication substitution and diagnosis upcoding – two potential unintended consequences of antipsychotic reduction efforts. To understand the extent of these potential consequences, I conducted three observational studies of nearly all nursing home residents in Ontario, Canada between 2010 and 2019 using linked health administrative data. In a population-level repeated cross-sectional study, I found that sedating antidepressants and anticonvulsants continued to increase over time. Overall antipsychotic use stabilized following a modest decline in prevalence. The documentation of delusions increased substantially over time, with a clear inflection coinciding with efforts to reduce potentially inappropriate antipsychotics. Increases in sedating antidepressants and documented delusions were greatest among residents with dementia and severe aggressive behaviors.In a facility-level repeated cross-sectional study, I found modest variation in the time trends for potentially inappropriate antipsychotic and antidepressant use across facilities. More substantial variation in the documentation of delusions and associated conditions was observed and shown to be strongly correlated with a homes’ reduction in potentially inappropriate antipsychotic use over time. In a resident-level retrospective cohort study of new admissions to nursing homes, the incidence of trazodone modestly increased while the incidence of antipsychotics significantly decreased over time. Among residents receiving a potentially inappropriate antipsychotic at admission, those who were off an antipsychotic at subsequent assessments were significantly more likely to receive an incident trazodone dispensation compared to those who were on an antipsychotic. These studies provide complementary evidence consistent with antipsychotic medication substitution and changes to diagnosis coding. While these findings likely reflect an increasingly complex nursing home population, they suggest a role for improved quality of care monitoring, such as the expansion of public reporting programs to consider a broader range of quality indicators, investigations into the validity of diagnosis coding, and assessment of the safety and effectiveness of medication substitution.

Keywords: Antipsychotics
Diagnosis coding
Medication substitution
Nursing homes
Ontario
Pharmacoepidemiology
Advisor: Bronskill, Susan E
Neural Networks as Add-on Modules for Improved Performance of Robot Control Systems

Robots are envisioned to become reliable companions in domains ranging from advanced industrial applications to our daily lives. In the literature, well-established control techniques provide the foundation for designing high-performance robot systems with desired theoretical guarantees. However, these control techniques often rely on a dynamics model of the robot, and any inaccuracies in the model can result in suboptimal performance or even unsafe actions. These limitations motivate the incorporation of machine learning into the traditional robot decision-making stack to identify and thereby compensate for model uncertainties. In this dissertation, learning-based control approaches are developed to enhance the performance of black-box robot systems. While different machine learning models may be similarly applied to enhance the performance of a robot control system, in this thesis, we focus on neural-network-based approaches and analyze the properties that the neural network modules must satisfy for safe and data-efficient deployment. This dissertation contains five contributions. In the first contribution, we introduce a neural network inverse dynamics learning approach for improving the performance of black-box robot control systems in arbitrary trajectory tracking tasks (i.e., impromptu trajectory tracking). In the second contribution, we extend the inverse dynamics approach to non-minimum phase robot systems (i.e., robot systems with unstable inverse dynamics). In the third contribution, we propose an active training trajectory generation framework for systematically training a neural network inverse dynamics model. In the fourth contribution, we outline a data-efficient online learning scheme that allows us to transfer the inverse dynamics model trained on one robot to enhance the tracking performance of another robot. In the last contribution, we propose a learning-based model reference adaptive controller where a Lipschitz network is updated online to bridge the model-reality gap in uncertain robot systems and the system's stability is guaranteed by exploiting the architectural design of the network. Through theoretical analysis and physical experiments using quadrotors, we demonstrate that, by combining control theory and the expressiveness of neural networks, we can design safe and efficient robot learning algorithms to achieve high performance despite the uncertainties present in the environment.

Keywords: Learning-based control
Machine learning
Neural networks
Quadrotors
Robotics
System control
Advisor: Schoellig, Angela
Quadrature-based Moment Closures for Predicting Soot Formation in Laminar and Turbulent Non-premixed Flames

Fractional-order moment formulations of the quadrature method of moments (QMOM) and conditional quadrature method of moments (CQMOM) with a fixed quadrature point at the inception size, resulting respectively in so-called fractional-order QMOM-Radau and CQMOM-Radau closures, are proposed for the prediction of soot formation in laminar and turbulent non-premixed flames. The univariate QMOM-Radau formulation assumes spherical particles while the bivariate CQMOM-Radau formulation allows for the treatment of more complex soot particle geometries by accounting for both the particle size and number of primary particles in the aggregates. The predictions of the proposed QMOM-Radau and CQMOM-Radau approaches, as well as those of other univariate and bivariate moment methods, are compared in detail to reference Monte-Carlo (MC) solutions, as well as to sectional-type methods, for several space-homogeneous problems. A seven-moment CQMOM-Radau closure based on fractional-order moments of soot particle volume is then used to explore the influence of the choice of polycyclic aromatic hydrocarbon (PAH) precursors on the prediction of soot formation, evolution, and oxidation in ethylene-air laminar diffusion flames at atmospheric pressures. The seven-moment closure allows detailed modelling of the gas-phase chemistry along with treatments for soot particle nucleation, condensation, surface growth, oxidation, coagulation, sintering, obliteration, and fragmentation. The relative performances of the various PAH-based precursors for predicting the observed soot concentrations, particles sizes, and structure are discussed. Finally, both five-moment QMOM-Radau and seven-moment CQMOM-Radau moment closures are applied to the prediction of soot formation in ethylene turbulent non-premixed jet flames using a Reynolds/Favre-Averaged Navier-Stokes (RANS/FANS) simulation framework coupled with tabulated chemistry. For the tabulated chemistry, the relative performances of several tabulation approaches, namely the Flame-Prolongation of ILDM (FPI), Steady Laminar Flamelet Method (SLFM), and Flamelet/Progress Variable (FPV) approach, are used in conjunction with the quadrature-based closures. The capabilities of the fractional-order moment closures for predicting soot formation in this class of turbulent flame are discussed.

Advisor: Groth, Clinton P. T.