iGEM 2022 — Team ucberkeley

{% extends "layout.html" %}
{% block title %}Notebook{% endblock %}
{% block page\_content %}

## Wetlab

---

To streamline our work in the lab, we worked with our faculty advisor to develop a series of videos, quizzes, and lab sheets that corresponded to each step in the cloning workflow. The videos were required to be watched ahead of time, to gain a familiarity with the protocol, and the quiz had to be completed. Each wetlab trainee printed out the lab sheet for the day, filled it out as the procedure was completed, and turned it in. This allowed us to document the progress of each trainee, as well as any potentially contaminated aliquots that were resulting in inconclusive results. Ultimately, the implementation of this notebook enables wetlab workflow to be compatible with an atypically large group of student researchers.

One complete experimental trial lasts about two weeks, taking 1 hour in lab each day, allowing flexibility in scheduling:

* Week 1

+ Day 1: Experiment Introduction
+ Day 2: Polymerase Chain Reaction (PCR)
+ Day 3: Gel electrophoresis confirmation and DNA purification
+ Day 4: Golden Gate assembly

* Week 2

+ Day 5: Transformation
+ Day 6: Liquid culture inoculation
+ Day 7: Miniprep and sequencing

See a detailed notebook template [here](https://docs.google.com/spreadsheets/d/1updHzk3CJ2_L7eO-Abg0cpHQleU7c8b0lbF9QmNzFWA/edit?usp=sharing).

## Computational Biology

---

The computational trainings were first developed by each of our projects, as onboarding resources and experiment design were primarily spearheaded by the project leads with iterations of feedback from the directors. This development process took about three weeks to aggregate resources, relevant papers, as well as testing software functionality and sample datasets; designed notebooks were expected to take no more than 8 hours to complete.

At the onset of the trainings, students were then given two weeks to complete their necessary notebooks, including reading the paper, walking through the introductions to their project and necessary softwares, and learning how to troubleshoot code or such software by testing it on shared data. Upon the completion of these introductory components, students were then tasked with completing their mini-experiment, generating a novel code implementation of an important task in their team’s actual project, or learning how to generate protein candidates and analyze them through a few selected quantitative tools. Project leads monitored progress and checked in with students, fostered collaborative sessions for students to work through the notebooks together, and assisted students with software downloading and setup. Because of the entirely computational, and thus drylab, nature of these notebooks, students were free to work on any components of the notebook at any point during the two weeks, break up the various components, and troubleshoot runs in groups with ease; this last component was greatly facilitated by the collaborative work sessions the project leads organized. Project leads also aggregated students’ data, code, and initial analyses, to do further benchmarking and analyze results in totality. This analytical period took place after the conclusion of the two week student sprint.

The full Jupyter notebook walkthroughs are available [on our GitHub](https://github.com/igematberkeley/compbio-training).

![](https://static.igem.wiki/teams/4394/wiki/test/loading.gif)

{% endblock %}

{% extends "layout.html" %}
{% block title %}Engineering Success{% endblock %}
{% block page\_content %}

## Wetlab

---

To enable batch-screening, we need a convenient reporter system with a signal identifiable with the human eye. To this end, we construct a standard green fluorescent protein (GFP) promoter-reporter system, as shown below, using Golden Gate Assembly.

![](https://static.igem.wiki/teams/4394/wiki/circuit.png)

It is to the consensus that the -35 and -10 boxes, the binding sites of the sigma70 factor during transcription, play a major role determining the strength of the respective promoter. Previous iGEM researchers employed saturation mutagenesis on the two regions and generated the J23119 family promoters. Among these promoters with different strengths, they concluded that J23119 from the family was the strongest [1]. Building upon this discovery, we leave the -35 and –10 consensus sequences identical to J23119 and utilize saturation mutagenesis via Golden Gate Assembly to generate randomized sequences on the sequential context surrounding the sigma factor recognition site, as shown below. This new construct is named pP6, with its design allowing us to explore strong constitutive promoters with a larger diversity in their gene sequence. Meanwhile, our hope was that altering sequential context would yield constitutive promoters even stronger than J23119.

![](https://static.igem.wiki/teams/4394/wiki/sequence.png)

The N-repeat randomized sequence is incorporated into the targeted site by polymerase chain reaction (PCR) with primers ordered from IDT (their oligonucleotide synthesis instrument interprets ‘N’ as an equal mixture of all four bases). We thus get a library of promoters with dramatically varying strengths for characterization.

Upon transformation and plating, we proceed with the brightest colonies, like the ones highlighted below, to isolate plasmid DNA and submit for sequencing.

![](https://static.igem.wiki/teams/4394/wiki/pp6-46.jpg)

For successful constructs proven by sequencing, we retransform and measure GFP signal, excited using 483 nm, at 525 nm wavelength with Tecan reader for later significance tests.

[1] http://parts.igem.org/Part:BBa\_J23119

## Computational Biology

---

In the CheRMiT notebook, students were guided through the initial setup of the cheminformatics segment of our project, which focuses on verifying if a found set of reactants and products from a sentence is actually capable of having a reaction occur between them based on cheminformatics software. Students were walked through a new cheminformatics library, RDKit, to pivot away from old Java-based libraries that the team no longer wished to use. They were then instructed to program their own reaction validator, which could apply a reaction operator (an abstract chemical operator that can take in a chemical and apply the modifications to the reactant that represent the reaction taking place, and output the modified chemicals, or the product of the reaction).

![](https://drive.google.com/uc?export=view&id=1fHQJ-vCvLATXU-eQ14HZ2SsUyOSNlaN5)

Upon outlining their code and programming an initial approach, they could then test their implementation by passing a candidate set of reactant-product pairs, as well as a canonical list of reaction operators; students were able to test their implementations and refine these implementations to improve recall, while also identifying concerns with missed reactions or the limitations of the chemical processing functionality that would be useful to address in downstream, mature software development for the project. This holistic validation and varied student approaches invited a wealth of analyses that ended up accelerating the development of the cheminformatics validation software for the project this semester, to the point where a more advanced infrastructure has been rewritten, fleshed out, and is now in the process of getting thoroughly benchmarked based on inputs from language model outputs.
![](https://drive.google.com/uc?export=view&id=1QHVP92VmkKZFobWQvxd-4P1r5zxXe0TX)

![](https://static.igem.wiki/teams/4394/wiki/test/loading.gif)

{% endblock %}

{% extends "layout.html" %}
{% block title %}Experiments{% endblock %}
{% block page\_content %}

## Wetlab

---

Our experimental procedures are detailed in our [lab sheets](https://docs.google.com/spreadsheets/d/1updHzk3CJ2_L7eO-Abg0cpHQleU7c8b0lbF9QmNzFWA/edit?usp=sharing), in an easy-to-use reproducible format for any iGEM team to replicate. This structure to us, again, enables wetlab workflow to be compatible with an atypically large group of student researchers. We thus highly recommend other undergraduate iGEM teams to incorporate this structure as they are expanding.

## Computational Biology

---

PPI:

Running Peptiderive on 11 various protein-protein interactions to generate partner-receptor candidates.
Software was run locally using Rosetta 3 on students’ computers, and candidate files were aggregated and analyzed using Rosetta binding energy tools.
For three of the 11 PPIs (1ETH, 1A01, 8FAB), which involved proteins complexes featuring multiple chains, binding energy scores were output for the best candidate from each partner-receptor chain pair. For the remaining 8 PPIs (1NW9, 4QBE, 5OEN, 7EK6, 1H59, 8E9T, 7F9D, 2p7V), the binding energy of all candidates generated by the program for each partner-receptor chain pair were assessed.

CheRMiT:

Generating chemical reaction validators and writing labeling functions. Two Jupyter notebooks were generated, as all executables generated by students were written in Python 3.8.
For the cheminformatics notebook, validator softwares were written by each student based off a canonical database list of reaction operators, RDKit functionality, and a list of sample reactions to test validators on. For the machine learning notebook, labeling functions were generated by each student to pick up on various semantic patterns within the text, at the students’ discretion; these functions were then validated based on a sample subset of annotated sentences from papers.

Hallucinating Scaffolds:

Generating alanine scans of candidate sequences and gaining familiarity with benchmarking mutant sequences. Each residue of the amino acid sequence of phytoene desaturase (PDB: 4dgk) was mutated independently to an alanine, generating 100 candidate sequences to test and benchmark. Mutated sequences were then scored for their increase in protein disorder, using IUPred, and for their change in protein secondary structure using PSIPRED.

![](https://static.igem.wiki/teams/4394/wiki/test/loading.gif)

{% endblock %}

{% extends "layout.html" %}
{% block title %}Results{% endblock %}
{% block page\_content %}

## Wetlab

---

Saturation mutagenesis is proven to be successful from the sequencing results. We successfully created a library of mutants with randomized sequences but kept the -35 and -10 boxes - TTGACA and TATAAT - untouched as showcased below.

![](https://static.igem.wiki/teams/4394/wiki/sequencing.png)

Characterizing promoter strengths we retransformed successfully cloned plasmids, plated and picked four colonies per plate for scale-up liquid culturing. Reporter signal was measured 20 hours post-culturing. GFP emission strength was measured at 525 nm after excitation at 483 nm. All data was normalized as the ratio of relative fluorescence unit to cell density at OD600. To showcase the significant variance in signal strength by just performing saturated mutagenesis on the sequential context surrounding the consensus polymerase binding boxes at the promoter, we then normalized all mutants against the white, untransformed Mach1 cells as signal fold change in comparison to wild-type.

![](https://static.igem.wiki/teams/4394/wiki/pp6.png)

The result showcased greater variance than expected. Out of the small batch of 12 hits, the standard deviation of the mean standardized fluorescence across different mutants arrived at 55%. This significant variance shed light on how mutating sequential contexts of established robust promoters might lead to the discovery of promoters that are even more potent. For future experiments, we will screen for more mutants and run characterization in comparison with the established J23199 strong constitutive promoter.

## Computational Biology

---

#### Protein-Protein Interactions (PPI)

In order to better understand the candidate generation part of our peptide inhibitor design pipeline, we examined the inhibitor candidates that Peptiderive, a program in the Rosetta software suite, output for 11 different protein-protein interactions (PPI) downloaded from the Protein Data Bank. For the three PPIs involving complex proteins with multiple chains, we compared the binding energy scores output for the best candidate from each partner-receptor chain pair. For the remaining eight PPIs, we analyzed the binding energy of all possible candidates assessed by the program for each partner-receptor chain pair.

Selected interface scores as generated from candidates for PPIs composed of multiple chains:

![](https://drive.google.com/uc?export=view&id=1Li_B7MOg2psB6cJxsNtwLHDNFb1mLyk0)

Overall, we noticed that in both of these analyses (one set of which is presented above), the candidate output by Peptiderive was not always a clear winner. This is important because the remainder of our pipeline currently revolves around optimizing the structure for the best candidate, and all other inhibitor candidates are discarded by Peptiderive. Consequently, we may be ignoring inhibitors that, once structurally optimized, outperform our current inhibitor because Peptiderive does not take into account the conformational changes that peptides undergo during the docking process. Therefore, in future updates to our pipeline, we will be exploring ways to extract and incorporate the other strong candidates from Peptiderive into the remainder of our model.

#### CheRMiT (Chemical Reaction Mining via Transformers)

Among 9 training team members, we produced several versions of chemical reaction validators and Snorkel labeling functions. Across the cheminformatics models, we processed a selection of papers, extracting possible chemicals and reacting them combinatorially with a key of known reaction motifs and looking for validated output. Similarly, we tested team members’ Snorkel labeling functions on a small set of ground truth sentences, collecting metrics and evaluating missed sentences for further analysis.

**Notebook 1 (Cheminformatics):**

Each model was benchmarked on [this paper](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5727423/).

[Model 1](https://drive.google.com/file/d/1Xi4apeWhakcPE9Y5cKmM2Kq6PSkyTzZh/view?usp=sharing): Found 60 possible reactions // Phil’s
[Model 2](https://colab.research.google.com/drive/1JRyLbRxQnZ0QhJksiSWw8vlJ78aInTzR?authuser=5): Found 1 possible reactions // Guru’s
[Model 3](https://colab.research.google.com/drive/1IL-bs5mcp85yL1tN6QZ0RN89JtiXMDpE?authuser=5): Found 2 possible reactions // Tara’s
[Model 4](https://colab.research.google.com/drive/1r-vbP_XaYiPFeK5Di0Ti4LN--lTzqWIq?authuser=5): Found 39 possible reactions // Cassie’s

The differences between these two could be explained due to the limitations of the lookup mechanism. Due to overly restrictive or under restrictive text parsing, as well as the limitations of the PubChem API lookup, there were differences in the amount of chemicals labeled. Of these reactions, there were some repeats, as well as several spurious results based on the query API returning chemicals for words where there were none. See below for a spurious returned chemical:

![](https://drive.google.com/uc?export=view&id=1FSwV9mtZhrNM3aEC6qbHpssdMn61mmrV)

This is an issue that would be fixed by inputting better quality data into the pipeline, so that it wouldn’t struggle with these types of issues. For a given dataset, about 31% of words that returned a result from the PubChem database were not chemicals, and only matched up well enough to return a hit. Therefore, more possible chemical reactions does not suggest that the model is better. A number closer to the more restrictive models is closer to what is expected for a paper.
However, it is worth noting that each of the text parsers were implemented in a very rudimentary fashion, which caused many false matches. With the addition of much cleaner data with a high likelihood of being a chemical, the accuracy would improve dramatically and the number of spurious reactions would drop to near zero.

**Notebook 2 (Natural Language Processing)**

Students made unique labeling functions:

![Phil's Labeling Function](https://drive.google.com/uc?export=view&id=1ju883_1Mxo1mF7-ov3hTGsCYDRRNqTfp)
![Guru's Labeling Function](https://drive.google.com/uc?export=view&id=1L3nnTAJnTf-p5Q0Ao83PJtvpJumJY8RJ)

An optimal Snorkel model is made up of an ensemble of labeling functions, since each individual labeling function encodes a different heuristic. In addition, most labeling functions abstain on sentences that their heuristic does not cover. Therefore, analysis of an individual labeling function using typical classification metrics does not always tell us how useful the labeling function would be in practice.

We tested labeling functions on a dataset of 405 ground truth sentences. Of these sentences, 45 were labeled as true (contains a chemical reaction) and 360 were labeled false (does not contain a reaction). This discrepancy in positive and negative data helps us simulate the proportion of positive and negative sentences within a research paper.

False/Abstain LF:

To evaluate the False/Abstain labeling function, we calculated the LF’s false negative rate (FNR), false omission rate (FOR), true negative rate (TNR), and negative likelihood ratio (NR-).

As the LF abstained on sentences it did not label false, we did not calculate metrics corresponding to true positive and false positive data. We also note that individual LFs aren’t expected to cover the entire dataset, which means that metrics such as the TNR don’t necessarily reflect usefulness.

FNR: 0.2 (FN/P)
FOR: 0.0776 (FN/PN)
TNR: 0.297 (TN/N)
LR-: 0.673 (FNR/TNR)

We also include a false negative sentence for in-depth analysis:

*Interestingly, the allele of pro1 was shown to enhance the activities of gamma-glutamyl kinase and gamma-glutamyl phosphate reductase, both of which catalyze the first two steps of l-proline synthesis from l-glutamate and which together may form a complex in vivo.*

The student’s LF failed on this sentence because it detected the word “allele”, which typically would not be found in a sentence containing a chemical reaction.

True/Abstain LFs:

To evaluate the True/Abstain labeling functions, we calculated the LFs’ true positive rate (TPR), false positive rate (FPR), false discovery rate (FDR), and positive likelihood ratio (LR+).

Guru:

TPR: 1.0 (TP/P)
FPR: 1.0 (FP/N)
FDR: 0.888 (FP/PP)
LR+: 1.0

Cassie:

TPR: 1.0 (TP/P)
FPR: 1.0 (FP/N)
FDR: 0.888 (FP/PP)
LR+: 1.0

We notice that for both of these LFs, they are actually labeling every sentence in the test set as true. This is because the test set was preprocessed so that every sentence contains multiple chemical entities, as we wanted sentences to have an explicit substrate and product.

True/False LF:

To evaluate the True/False LF, we calculate accuracy, precision, recall (TPR), and true negative rate (TNR).

Accuracy: 0.701 ((TP + TN) / (P + N))
Precision: 0.253 (TP/PP)
Recall (TPR): 0.867 (TP/P)
TNR: 0.681 (TN/N)

For accuracy, labeling negative sentences correctly is more important than labeling positive sentences correctly due to the increased amount of negative sentences. This can be confirmed by comparing the accuracy to the TNR and TPR.

We include a false positive and false negative sentence below for in-depth analysis:

FN: In addition to the racemization it also catalyzes specific elimination of l-ser to pyruvate.

FP: Newly synthesized dopamine is stored in the terminals and then released, stimulating postsynaptic dopamine receptors and mediating the antiparkinsonian action of levodopa.

For the first sentence, the student’s LF failed to recognize the reaction verb “eliminate”. For the second sentence, the student’s LF mistook the word “synthesized” for a synthesis reaction. Therefore, we can add “eliminate” and remove “synthesize” from the LF to improve it.

#### Hallucinating Scaffolds Data

See the mutation heatmaps from our Hallucinating Scaffolds team below, as well as the [relevant Jupyter notebook on our GitHub](https://github.com/igematberkeley/compbio-training/blob/main/A_scanning_4dgk.ipynb). You can also view the [PSIPRED secondary structure predictions](https://static.igem.wiki/teams/4394/wiki/secondary-structure-traces.pdf) and [IUPred disorder predictions](https://static.igem.wiki/teams/4394/wiki/disorder-traces.pdf).
Each residue of the amino acid sequence of phytoene desaturase (PDB ID: 4dgk) was mutated independently. IUPred disorder predictions and PSIPRED secondary structure predictions of all mutants were generated. The change in disorder and in secondary structure is plotted below for all mutants.
Change in disorder and secondary structure will be represented in 2 ways: traces and heat maps.
![](https://drive.google.com/uc?export=view&id=1YagXQJu2nZXEVOvpYV6wetgzeRU2W02e)
The x-axis represents the residue position. The y-axis represents new disorder/ss value - old disorder/ss value.
Heat maps sacrifice a bit of readability to compress the information in the traces. They also make it easier to observe correlations between alanine mutants.
![](https://static.igem.wiki/teams/4394/wiki/disorder-heat-map.png)
x-axis: reference amino acid sequence
y-axis: (new disorder/ss value - old disorder/ss value) across the sequence, per mutant color gradient: increasingly red for increasingly positive value changes, increasingly blue for increasingly negative value changes

Disorder:
Both the traces and heatmap show that the effect of the alanine mutants on disorder is very localized. Disorder prediction values increase/decrease slightly in a region of ~20 residues centered on the position of the mutant. Some of these 20-residue regions show particularly small changes in disorder. They correspond to areas of the protein that were already disordered (i.e., protein loops), so the alanine mutants had very little effect.
Secondary structure:
Unlike what is observed for disorder, the traces and heatmaps of the alanine mutants show that changes in secondary structure are not localized. Across all alanine mutant positions, a few regions in the protein are particularly prone to secondary structure changes. Many of these regions are the substrate-binding regions identified by Schaub, et. al (2012) (see PDB ID: 4DGK). Many of the alanine mutants appear to "break" helical regions (decreased H character and increased C character), but this is misleading since most of the substrate-binding regions are beta-strands. Oddly, the heatmaps show that some beta-strand regions have higher E character in the alanine mutants. A possible future experiment would investigate whether these regions correspond to highly-conserved domains of the protein.

![](https://static.igem.wiki/teams/4394/wiki/test/loading.gif)

{% endblock %}