VictorJuiz/thesis huggingface dataset:
Benchmarking, Profiling and White-box Performance Modeling for DNN Training

Training a modern deep learning model is extremely time-consuming. The software/hardware deployments that machine learning (ML) programmers use in practice are widely diverse.Differences in algorithms, hardware features, or even software versions could all shift the performance bottlenecks of practical deployments. Therefore, it usually requires extensive amount of domain knowledge and engineering effort for ML programmers to profile and pinpoint ad-hoc performance bottlenecks. In this thesis, we explore the approach of performance modeling in the context of ML workloads to enable efficient optimizations.Our key idea is to decompose a training workload into atomic traces of heterogeneous hardware components, and properly structure these traces to build a complete profile. Using this profile, we allow ML practitioners to effectively identify the performance bottlenecks and efficient optimizations for any given software/hardware deployments. In summary, this thesis makes the following major contributions. First, we propose a new benchmark suite for DNN training workloads, called TBD, that contains nine mainstream DNN models, covering six major DNN applications. We also propose a set of performance metrics that can directly indicate performance bottlenecks on CPU/GPU, and build a tool chain that enables end-to-end profiling for DNN training workloads. Second, we propose a new performance predictor (Daydream), that allows users to accurately and effectively estimate the efficacy of various optimization techniques for DNN training workloads. Daydream utilizes a dependency graph approach to model the hardware execution, which tracks the task dependencies at kernel level and addresses several unique challenges in the ML context. Third, we propose two tensor compilers, Sokoban and Roller, that can generate high-performance tensor programs in seconds.They view the underlying architecture as a data processing pipeline, and the computation of an operator as independent tiles. Sokoban is a search-based compiler that utilizes an end-to-end cost model based on the pipeline model, while Roller is a construction-based compiler that exploits heuristics from the pipeline model and directly constructs efficient schedules. Both compilers manage to generate tensor programs that are comparable against manually-written libraries and state-of-the-art compilers, as well as reduce the compilation time to seconds.

Keywords: Benchmarking
Compiling
Deep Learning
Performance Analysis
System
Advisor: Pekhimenko, Gennady
Exploration of New Nucleic Acid Factors to Improve the Development of Synthetic Biosensing Switches

In biosensor development, synthetic nucleic acid or protein responsive switches are critical for improving disease diagnostic performance. My thesis investigates several topics related to the exploration of new nucleic acid factors to improve the performance of current strand displacement switches and to discover RNA/ssDNA-responsive prokaryotic promoters. Incorporating a secondary looped structural element (flex-loop) into a switching domain allows the acceleration of toehold-mediated strand displacement in various constructs, including toehold-exchange, associative toehold, remote toehold, and allosteric toehold systems. The flex-loop switch has the ability to boost the i) efficiency of DNA-based computing, ii) analytical performance of an allosteric toehold technique for DNA sensing, and iii) performance of artificial gene circuits that regulate gene expression. Additionally, the flex-loop regulatory element enabled protein responsive proximity-strand displacement switch was investigated. The proximity flex-loop switch-functionalized antibodies allowed on-cell multi-receptor detection. This strategy, combined with the droplet microfluidic technique, enables a fluorescent-droplet cytometry assay that allows the characterization and profiling of live cancer cells with the single-cell resolution based on surface expression phenotype. Using this approach, we achieve live-cell phenotype profiling of multi-surface markers acquired with small (< 40 cells) collections of cells. We also develop an in vitro, non-traditional promoter structure selection strategy, in which a cell-free lacZ gene expression screening method carries on a new type of promoter structure acting as transcription factors. These T7 promoter structures performed as cis-acting transcription factors under the control of prokaryotic RNA polymerase. A rationally designed sequence responsive T7 promoter, termed Omega promoter, was developed to control gene expression via a toehold-mediated promoter rearrangement principal. Transcription-only Omega promoter-based sensors demonstrate engineered reporting activity through the use of RNA aptamers or control of Cas12a cleavage. We also discover that Omega promoter sensors could be operated from a circular DNA template for sequence-specific detection of ssDNA and RNA. The Omega promoter design can be directly adapted to the control of SP6 RNAP using prescribed sequences of ssDNA/RNA complexes. A single-stranded Omega promoter enables DNA-dependent RNA polymerase to accomplish the simplest in vitro RNA replication. Together, these studies explore nucleic acid factors to improve synthetic biosensing switches development in disease diagnostic applications.

Keywords: Biosensing Switches
DNA Nanotechnology
Omega Promoter
RNA Amplification
Single-Cell Analysis
Synthetic Biology
Advisor: Kelley, Shana O
An Examination of the Genetics of Sleep Disturbances in Depression

It was not until large consortia were established that genome-wide association studies of major depressive disorder (MDD) were able to identify significant variants. However, given the nature of MDD, these studies often have considerable heterogeneity, partly due to various symptom clusters that can be labelled as MDD. Research suggests that focusing on one symptom cluster of MDD may reduce this heterogeneity and allow for smaller sample sizes for genetic studies. Sleep disturbances, mainly insomnia and hypersomnia affect 25-88% of depressed individuals and lead to poorer patient outcomes, including increased suicide risk. Both animal and human studies have shown that sleep is largely controlled by circadian rhythms, which, at the cellular level are regulated by a set of circadian genes. More recently, mitochondria have been suggested to be involved in sleep due to their well-known role in energy metabolism. However, prior to the work in this thesis, there were only a handful of candidate gene studies in sleep and depression, all of which focused on circadian genes. Therefore, the overall goal of this thesis was to extend upon previous work by using state-of-the-art analytical approaches to examine the role of common genetic variants in sleep disturbances among depressed individuals. This thesis is divided into four independent projects. Chapter 3 reviewed mitochondria’s involvement in sleep in the literature. Chapter 4 focused on identifying common variants associated with self-reported problematic sleep in depressed individuals. Chapter 5 examined the genetic overlap between insomnia and hypersomnia in the general population and depressed individuals. Chapter 6 investigated the involvement of common variants from both the autosomal genome and mitochondrial DNA in objectively-derived sleep measures in depressed individuals. Altogether, these studies suggest the involvement of common genetic variants in sleep measures in depressed individuals, thereby suggesting that their sleep disturbances are genetically determined. This work sheds light on novel variants that may be used in the future to improve diagnosis and treatment regimens.

Keywords: Accelerometry
Genetics
Genome-wide association study
Mitochondria
Polygenic risk score
Sleep
Advisor: Kennedy, James L
Developing a Metabolically Diverse Collection of Inflammation-associated Bacteria from the Murine Gut

Gut inflammation directly impacts the growth and stability of commensal gut microbes and can lead to long-lasting changes in microbiota composition that can prolong or exacerbate disease states. While mouse models are used extensively to investigate the interplay between microbes and the inflamed state, the paucity of cultured mouse gut microbes has hindered efforts to determine causal relationships. To address this issue, I have assembled the Collection of Inflammation-Associated Mouse Intestinal Bacteria (CIAMIB). The collection comprises 41 isolates of 39 unique bacterial species, covering 4 phyla and containing 10 previously uncultivated isolates including one novel family and 7 novel genera. The collection significantly expands the number of available Muribaculaceae, Lachnospiraceae, and Atopobiaeae isolates and includes microbes from genera associated with inflammation such as Prevotella and Klebsiella. I have characterized the growth of CIAMIB isolates across a diverse range of nutritional conditions and generated draft genomes of each isolate enabling the prediction of their functional potential. Probing the growth parameters of CIAMIB strains revealed that some species are refractory to propagation in liquid media or have difficulty initiating growth from a low inoculum. Since many large-scale cultivation studies have traditionally relied on limiting dilution assays and liquid propagation steps to isolate species in pure culture, these results have important implications for future work aiming expand the phylogenetic diversity of cultured species. I have provided the first metabolic analysis of species within the Adlercreutzia and Parvibacter genera, revealing these representatives to be nitrate-reducing and severely restricted in their ability to grow on carbohydrates. Finally, using our expanded panel of cultured Muribaculaceae species, I probed their growth and biochemical properties to further our understanding on the basic physiology of this highly abundant and diverse, but understudied, family of bacteria. CIAMIB isolates are fully sequenced and available to the scientific community as a powerful tool to study host-microbiota interactions.

Keywords: Eggerthellaceae
Gut inflammation
Lachnospiraceae
Microbial growth
Microbiota
Muribaculaceae
Advisor: Navarre, William W
Cultured Cell and Knock-in Mouse Models for Studying Transmitted and Spontaneous Prion Disease

The central event driving prion-disease pathogenesis is the conversion of the cellular prion protein, PrPC, into the pathogenic prion isoform, PrPSc. Significant success has been achieved modeling spontaneous prion disorders in vivo, but the prion field has struggled to provide effective in vitro, cellular models. I selected a particularly prion-vulnerable CNS-derived line, CAD5, for CRISPR/Cas9 gene editing to remove endogenous mouse PrP and to create CAD5-PrP-/- lines that could be transfected and infected with different PrP variants. Hamster HaPrP was stably transfected to create CAD5-PrP-/-(HaPrP) lines that were susceptible to hamster prion infection, unlike comparable N2a-PrP-/-(HaPrP) lines that were derived from the mouse, neuroblastoma N2a line (Chapter 3). I supplemented the characterization of CRISPR-edited CAD5 cells with the development of cultured cell models of heritable prion disease. N2a-PrP-/- cells were stably transfected with the conformationally unstable bank vole BVPrP to create N2a-PrP-/-(BVPrP) lines. The addition of pathogenic variants to BVPrP(I109) increased PrP insolubility and turnover in my cell models, but no proteinase K-resistant PrP was identified. Although there was insufficient evidence to validate bona fide prions, it is notable that early PrPSc-like characteristics associated exclusively with mutant BVPrP, suggesting that the N2a-PrP-/-(BVPrP) paradigm is capable of partly recreating the biochemical phenotype of heritable prion disease and may spontaneously generate prions under the correct conditions (Chapter 4). I sought to complement my cellular, N2a-PrP-/-(BVPrP) models with knock-in, KI(BVPrP) mice that represented a natural progression from previously developed transgenic, Tg(BVPrP) mice. Three new lines of KI mice were generated with the BVPrP(I109) backbone: wild type KI(I109), FFI KI(D178N), and fCJD KI(E200K) lines. KI(BVPrP) mice expressing mutant but not wild type PrP developed overt neurological symptoms and enzymatically resistant PrP, characteristic of prion disease. Additionally, interactome analysis identified negligeable differences between young, asymptomatic KIs expressing mutant and wild-type BVPrP, implying that each mutation induces misfolding through alterations in PrP per se rather than its physiological binding partners (Chapter 5). Taken together, this research addresses fundamental gaps in the prion field; chiefly, the dearth of flexible cell systems that can propagate non-murine strains and the lack of faithful models of spontaneous prion disease.

Keywords: Cell line
Disease models
Knock-in mice
Neurodegenerative
Prion
Proteinopathy
Advisor: Watts, Joel C.
The Structure, Function, Kinetics, Regulation, and Interactions of β-Carboxysomal Proteins CcmM and CcaA

Carboxysomes are proteinaceous biochemical micro-compartments essential for photosynthetic CO2 fixation in cyanobacteria. These self-assembling structures encapsulate Ribulose-1,5-bisphosphate carboxylase/oxygenase and a carbonic anhydrase (CA) creating a segregated environment where both enzyme and substrate CO2 are concentrated. CO2 is generated within the β-carboxysome by the catalyzed dehydration of HCO3- mediated by two evolutionarily distinct lineages of CA, the β-class CcaA and the redox-regulated γ-class CcmM. Here we determined the components of the redox switching mechanism in CcmM from Nostoc sp. PCC7120 and provided a reaction mechanism through which the oxidation / reduction of Cys194 – Cys200 disulfide bond is transmitted to the active site to change the level of enzymatic activity between the carboxysome-localized and cytosol-localized enzyme. Switching between activity states in this trimeric protein involves residues from two adjacent protomers. Of profound importance are Pro9, Pro10, Thr11, and Trp13 from the AAPPTPWS motif and Tyr192 that forms part of a trans-protomer activation triad Tyr192’ - Asp55 – Glu56 within the active site. In a subset of cyanobacteria, however, we provide evidence that CcaA either complements or replaces CcmM as the carboxysomal CA, though the kinetic parameters, kcat and kcat / KM, are of similar magnitude. Notably, hexameric CcaA is targeted to the carboxysome through its pM binding affinity for trimeric CcmM, forming a symmetric CcmM3CcaA6CcmM3 complex. The tight association of these two structurally distinct but catalytically similar proteins has led to the evolution of distinct β-carboxysome sub-types in which CcmM is the sole CA (BDC1), CcmM and CcaA are co-contributors of CA activity (BDC2) and in which CcaA is the sole CA (BDC3). In the latter case, a catalytically inactive CcmM is retained as a structural scaffold. Through bioinformatics analysis and biochemical reconstitution experiments we demonstrate that CcmM from BDC3-strains have lost CA activity through the devolution of Cys194 - Cys200, Tyr192 and other residues of the redox activation domain. Rank ordering of the CcmM kinetic parameters within a multiple sequence alignment allowed for the identification of a minimal complement of amino acid residues required for CcmM CA activity, substitution of which leads to an enzyme in a low activity state or with no activity.

Keywords: Carbonic Anhydrase
Carboxysomes
CcmM
CO2 Concentrating Mechanism
Cyanobacteria
Photosynthesis
Advisor: Espie, George S
Image-based Characterization and Data-driven Techniques for Mitigation of Photovoltaic Reverse Osmosis Membrane Scaling

The scarcity of potable drinking water is a billion-person problem and the primary cause of health impacts in many off-grid communities of low and middle-income countries (LMICs). Decentralized, community-run photovoltaic reverse osmosis (PVRO) is one of the prospective solutions to this problem. However, fouling-related reliability issues and lack of regular supervision can lead to poor water recovery and faster degradation of PVRO membranes. Herein, the inorganic scaling of the PVRO membrane was evaluated by adopting an in-situ and hierarchical mm-to-nm scaling characterization technique. The scopes of membrane rinsing and data-driven control as interventions to minimize the scaling of ‘unsupervised’ PVRO operation were also evaluated. In-situ visualization of membrane scaling confirmed that periodic intermittency does not negatively impact the scaling growth in PVRO. The hierarchical mm-to-nm length scale characterization added new insights into the interplay between membrane active layer microstructure and scaling growth. The findings hinted at simultaneous nonclassical nucleation and surface crystallization of the dissolved minerals at the top of the active layer. In contrast, only surface or heterogeneous crystallization dominated deep inside the active layer. The hierarchical characterization framework was also used to evaluate rinsing performance as a scale mitigation intervention. 15–30 min hydraulic rinsing of PVRO membranes remarkably improved daily water production (~23%) and membrane scaling conditions, even at low-pressure conditions (mimicking low solar power availability). However, some smaller crystals within the complex folding of the active layer may remain after the rinsing cycles and act as seeding crystals for scaling growth in subsequent days of operation. Finally, short-term data-driven forecasting of permeate flux was carried out as a preliminary step towards developing a low-cost semi-automated supervisory control and data acquisition for community-run PVROs. Univariate and multivariate XGBOOST and LSTM forecasting models demonstrated reliable prediction of membrane permeability, 1-1.5 h in the future, with a mean square error (MSE) ranging from 3.8% to 11.2%. Stacking longer-term operational, characterization, and weather data (from field-scale plants) to the experimental and data-driven frameworks could fine-tune PVRO operability and make them more appropriate for off-grid applications worldwide.

Keywords: Active layer scaling
Computational imaging
Gypsum scaling
Long short-term memory
Solar desalination
Transmission electron microscopy
Advisor: Bilton, Amy
Infant Feeding Practices in a Multi-ethnic Cohort of Vulnerable Women Registered in the Canada Prenatal Nutrition Program

Background: Breastfeeding rates are lower among socio-economically vulnerable women in Canada. Improved access to skilled lactation support is one priority action to address these disparities. The Canada Prenatal Nutrition Program (CPNP) aims to support breastfeeding among vulnerable women but little is known about participants’ infant feeding practices or effective models for delivering postnatal lactation support through the CPNP. Objectives: Data from a Toronto cohort of CPNP participants were examined to determine : 1) breastfeeding intensity levels at seven time points to six months postpartum, frequency of breastmilk supplement use by time point, and intervals of exclusive breastfeeding; 2) prevalence of expressed breastmilk use and associations between its use at two weeks postpartum and breastfeeding practices (any and exclusive) to six months; 3) prevalence of household food insecurity at six months postpartum and associations with continued and exclusive breastfeeding. Methods: A pre/post study of a postnatal lactation support intervention was implemented at two CPNP sites but suspended prematurely due to the COVID-19 pandemic. Post-hoc analyses were conducted using the available dataset (study 1; n=151) and pooling it with data from a related study at another CPNP site providing free, in-home skilled lactation support (studies 2 and 3; n=337). Results: At least 60% of study 1 participants provided breastmilk for >75% of daily milk feeds at all time points and 48% exclusively breastfed for at least three consecutive months, despite only 18% doing so for six months. In the pooled cohort, in-hospital formula supplementation was common (57%) and 34% used expressed breastmilk in the first two weeks postpartum. These early feeding practices were associated with breastfeeding cessation before six months and non-exclusive breastfeeding to both four and six months. Household food insecurity was highly prevalent (44%) in the pooled cohort but was not associated with breastfeeding practices. Conclusions: Findings showed high rates of breastfeeding but a continued need for skilled lactation support from birth to improve exclusive breastfeeding. These findings inform adaptations to the redesign of the planned intervention study. Comprehensive assessment of infant feeding practices and household food insecurity in the national CPNP is also needed to shape future program delivery.

Keywords: breastfeeding
Canada Prenatal Nutrition Program
infant feeding
lactation support
Advisor: Sellen, Daniel W
O'Connor, Deborah L
Racial Ambiguity in Film: Rudolph Valentino and the International Racialized Type

When asked what he thought of Valentino’s performance in the 1921 film The Four Horsemen of the Apocalypse, silent film director D.W. Griffith stated: “is this fellow really acting or is he so perfectly the type that he doesn’t need to act?” Griffith’s comment instantiates the crux of this dissertation. The “type” to which Griffiths refers is Rudolph Valentino, a racially ambiguous Southern Italian immigrant to early 20th century America. At a time in which immigrants from the region were viewed as racially inferior and undesirable, Valentino went on to become one of the world’s biggest film stars. This dissertation argues that, while under contract to Famous Players Lasky (FPL), Valentino’s racial ambiguity was leveraged by the studio to the international film market through roles in which Valentino repeatedly played what I term the “international racialized type.” In unravelling this multi-layered issue, this dissertation places original archival research concerning FPL’s marketing of Valentino into conversation with the history and theory of scientific racism, particularly as it plays out in Michel Foucault’s writing on biopolitics. Close analysis of five Valentino films made for FPL at the height of his career together with archival records reveal FPL’s use of repetitive tropes that leveraged Valentino’s racial ambiguity both in his films and in their marketing of him internationally. Additionally, this study finds that despite his success, FPL still viewed Valentino as inferior and replaceable, all of which ultimately was destructive to his career. The intervention of this dissertation is twofold. First, original archival research conducted provides insight into marketing techniques that seek to capitalize on racial ambiguity specifically related to the formative period of early cinema; and second, it also acts as a valuable lens for viewing contemporary media culture and its marketing of race today.

Keywords: early cinema
film
Foucault
Italian
race
Valentino
Advisor: Farmer, Diane