iGEM 2023 — Team sjtu-software


{% extends "layout.html" %}
{% block bg %}bg{% endblock %}
{% block img\_class %}custom-image-class{% endblock %}
{% block img\_url %}https://static.igem.wiki/teams/4589/wiki/pics/background.gif{% endblock %}
{% block title %}Engineering Success{% endblock %}
{% block lead %}Research - Design - Build - Test - Learn{% endblock %}
{% block page\_content %}

[1.Project Implementation](#item-1)
[1.1 Data Collection and Preparation](#item-2)
[1.2 Model Building and Training](#item-3)
[1.3 Thermal Stability Prediction Platform](#item-4)
[2. Model Optimization](#item-5)
[2.1First Optimization](#item-6)[2.2 Second Optimization](#item-7) [2.3 Testing and
Validation](#item-8)[2.4 User Feedback](#item-9)
[3. Protein Engineering](#item-10)
[4. Conclusion](#item-11)

#### 1. Project Implementation

###### 1.1 Data Collection and Preparation

The project introduces a large language model to extract protein sequence features to support subsequent
prediction work, requiring a significant amount of protein sequence data. We used Argonaute protein data
provided by Novozymes in a public dataset on Kaggle. To extract protein sequences with downstream task
significance, we selected data within the length range of 300-800 amino acids. After the data cleaning, a
total of 13,917 valid sequences were obtained, which were randomly split into a training set and a test set in
an 8:2 ratio. Additionally, in collaboration with Professor Hong Liang's research group, we obtained Tm values
for 28 KmAgo protein mutant sequences that had been experimentally verified, which were used for validation
and subsequent work on the directed evolution of KmAgo.

###### 1.2 Model Building and Training

Protein Sequence Feature Extraction

The first language model we used was ESM-2, a protein language model that utilizes masked language modeling
(MLM) to capture amino acid dependencies in protein sequences. The protein sequence features and amino acid
dependencies obtained through ESM-2 can be used for various downstream tasks.

To optimize a part of the protein sequence extraction process, we subsequently used the TemPL language model
for feature extraction. TemPL is based on the BERT language model architecture and employs the ESM-2 model
framework. During fine-tuning, it received a significant number of temperature-related labels. As a result, we
anticipated that protein sequence features extracted using TemPL would outperform those from ESM-2. However,
the
actual results were different from our expectations.

2) Comparison of Results from Two Models

We trained Argonaute protein data using both ESM-2 and TemPL models, and the fitting results are shown below:

![](https://static.igem.wiki/teams/4589/wiki/pics/engineering1.png)
![](https://static.igem.wiki/teams/4589/wiki/pics/engineering2.png)

###### 1.3 Building the Thermal Stability Prediction Platform

Our team embarked on the development of a thermal stability prediction platform. This platform uses the
extracted protein sequence features for temperature prediction, allowing users to submit sequence data and
obtain their optimal temperature, including the one we are particularly focused on, the Tm value for Ago
proteins. The development of this platform involves a combination of front-end web design and back-end model
integration. We used Nginx for reverse proxy and created the web interface using languages such as HTML, CSS,
and JavaScript. The Flask framework in Python was used for user data upload and download.

![](
          https://static.igem.wiki/teams/4589/wiki/pics/engineeringplatform.png)

You can access our platform by clicking the button below:

[welcome to use DARWINS](http://47.103.132.204:80)

#### 2. Model Optimization

###### 2.1 First Optimization——DARWINS 2.0

After consulting literature and careful consideration, adjustments were made to the DARWINS 1.0 model. This
included changing the parameter file for feature extraction, resulting in a larger feature matrix for protein
sequence extraction (output increased from 1280 dimensions in version 1.0 to 2560 dimensions). Simultaneously,
data from 28 mutated sequences of KmAgo, obtained in collaboration with the Hong Liang research group, was
incorporated into the training set. Following training, an optimized model, DARWINS 2.0, was obtained. The
modified model aims to extract protein sequence features in a more detailed and accurate manner, exhibiting
improved performance in mutation prediction, including sequences like KmAgo. The following are the fitting
results of the DARWINS 2.0 model on the test set after training:

![](https://static.igem.wiki/teams/4589/wiki/pics/engineering3.png)

A significant improvement can be observed in the fitting of the second version model on the test set compared
to the first version (R=0.49 increased to R=0.85).

###### 2.2 Second Optimization — Binary Classification Model——DARWINS 3.0

Considering that our original training dataset mostly comprised wild-type Argonaute protein data, lacking
training support for mutation data, there might be limitations in predicting protein sequence-directed
evolution. To address this, we decided to train the model using a different mutation dataset, aiming for
higher reliability and competitiveness in predicting the thermo-stability-directed evolution of KmAgo.

After careful consideration, we chose the mutation dataset used in PremPS training. We initially filtered this
mutation dataset, retaining 3092 sequences with lengths between 100-300kb. The dataset was then split into a
7:3 ratio for training and testing.

Due to the significant influence of the wild-type on the mutant's thermo-stability, we modified the DARWINS
2.0 model, transforming it from a regression model into the binary classification model DARWINS 3.0. In other
words, inputting a mutated sequence into this model returns a result that can determine whether the
thermo-stability of the mutant sequence is increased or decreased compared to the wild-type, providing
qualitative insights for directed evolution design.
The following are the performance results of DARWINS 3.0 on the test set:

![](https://static.igem.wiki/teams/4589/wiki/pics/engineering4.png)

###### 2.3 Testing and Validation

During the platform development, it is crucial to validate the accuracy of the thermo-stability prediction
models. We compared three mainstream models—PremPS, FoldX, and dynamut2. It's worth noting that during our
replication efforts, we found that these three models only provide web-based prediction services and do not
support batch submissions. Therefore, testing their performance on a larger dataset for comparison was not
feasible. Additionally, these three models exhibit differences in how they reflect thermo-stability in their
outputs. For instance, PremPS outputs ddG, reflecting the extent that the mutant is more or less thermo-stable
compared to the wild-type. In contrast, our models, DARWINS 1.0 and DARWINS 2.0, output tm values, directly
indicating temperature. Considering these differences, we chose to perform binary classification predictions
on 28 mutated sequences of KmAgo using these three models and compared the results with our modified binary
classification model, DARWINS 3.0. The following are the prediction results of these three models on the
mutated ago dataset:

![](https://static.igem.wiki/teams/4589/wiki/pics/engineering5.png)

We also compared DARWINS 3.0 with the current three mainstream models using the original dataset of 28
sequences. The comparative results are as follows:

![](https://static.igem.wiki/teams/4589/wiki/pics/engineering6.png)

It can be observed that the DARWINS 3.0 model is generally comparable to these three models in certain aspects
of prediction. Simultaneously, there is a significant improvement in recall rate, indicating that our model
can better capture mutations that lead to an increase in thermo-stability of protein sequences, reducing the
likelihood of missing true positives. Additionally, our web platform allows for the upload and prediction of
multiple sequences, providing greater convenience for researchers compared to the web-based services of the
other three models, which is one of our advantages.

###### 2.4 User Feedback

We have provided the web-based software for testing to the Tsinghua team and senior students in our
collaborating lab. For detail please refer to [contribution page](https://2023.igem.wiki/sjtu-software/contribution)

#### 3. Protein Engineering

We use the regression models DARWINS 1.0 and DARWINS 2.0 from the three versions to predict the TM values of
KmAgo protein sequences after one round of mutation. Our goal is to identify mutation sites and related
mutated sequences that can increase the TM values, providing insights for subsequent thermo-stability-directed
evolution. Below is the visualization of TM values after mutation at some selected positions: (The complete
heatmap is not practical to display on the web due to size limitations.)

![](https://static.igem.wiki/teams/4589/wiki/pics/engineering7.png)
![](https://static.igem.wiki/teams/4589/wiki/pics/engineering8.png)

At the same time, we also performed visualizations of the mutation sites for the existing 28 mutated sequences
which we have already known the experimental results.

![](https://static.igem.wiki/teams/4589/wiki/pics/engineering9.png)
![](https://static.igem.wiki/teams/4589/wiki/pics/engineering10.png)

Through the visualized heatmaps, we can intuitively observe the variations in predicted values for each
mutated type. Simultaneously, we can identify protein sequence positions where mutations are more likely to
lead to an increase in thermo-stability. For example, in these heatmaps above, positions like N541 and Q510
show an increase in thermo-stability in the predictions of both model versions.

#### 4. Conclusion

In this project, we trained a high-precision Ago protein thermal stability prediction model and developed an
interactive online prediction platform. Users can input sequence data and receive results. Additionally, we
used this model to assist in the directed evolution of KmAgo protein, resulting in more stable protein
sequences. The test results indicate the effectiveness of our model. We have provided detailed user
instructions and plan to offer ongoing model optimization and maintenance to assist researchers in various
fields in wet lab experiments and other research activities related to Ago proteins.

To guarantee the successfulness of our project, we follow strictly with the standard engineering cycle. We
spend a lot of energy to make sure our project is successful and can generate value for the scientific
research process.

{% endblock %}

{% extends "layout.html" %}
{% block bg %}bg{% endblock %}
{% block img\_class %}custom-image-class{% endblock %}
{% block img\_url %}https://static.igem.wiki/teams/4589/wiki/background2.gif{% endblock %}
{% block title %}Design{% endblock %}
{% block lead %}The Research and Protocols Involved in Our Project{% endblock %}
{% block page\_content %}

[Motivation](#item-1)
[1.1 Importance of Argonaute Proteins](#item-2)
[1.2 Importance of KmAgo Protein](#item-3)
[1.3 Directed Evolution](#item-4)
[Design and
Planning](#item-5)

#### 1. Motivation

In the early stages of project definition, we conducted extensive literature reviews,
consultations with
multiple professors and researchers in related fields, and project planning to understand the feasibility and
significance of our project. We also formulated a project plan.

![discussing](https://static.igem.wiki/teams/4589/wiki/pics/designmotivation.jpg)

During this period, we learned about the crucial role of the conservative Argonaute protein family in gene
editing and regulation in organisms. They hold great potential to become the primary tool for gene editing.
Predicting the Tm values of Ago proteins can assist experimentalists in designing experiments, such as setting
protein annealing temperatures, significantly enhancing experimental efficiency. Furthermore, we also
discovered that directed evolution is one of the core methods for protein modification in the field of life
sciences, helping us obtain more stable Argonaute proteins under different temperature conditions.

###### 1.1 Importance of Argonaute Proteins

Argonaute proteins are a large protein family and are the principal components of the RNA-induced silencing
complex (RISC), a crucial entity in RNA interference (RNAi) technology. Some AGO proteins possess a PIWI
domain, conferring endonuclease activity, playing a critical role in binding and catalysis during the RNA
silencing process.

![demonstration of how ago works](https://static.igem.wiki/teams/4589/wiki/pics/designago.jpg)

Therefore, studying Argonaute proteins is significant for the following reasons:

1) Unraveling the molecular mechanism of RNA silencing: Argonaute proteins are key players in RNA
interference. By binding to small RNAs such as siRNA or miRNA, Argonaute can recognize, bind, and degrade
target RNA molecules. In-depth research into this molecular mechanism helps us better understand how RNA
silencing works and how precise control of gene expression can be achieved by regulating Argonaute proteins.
This has important implications for biological research and gene therapy.

2) Development and application of downstream RNA interference techniques: The study of Argonaute proteins
forms the foundation for the development and improvement of RNA interference techniques, including the
applications of siRNA and miRNA for gene silencing, gene expression regulation, and gene function research.
These techniques have wide-ranging potential in drug development, disease treatment, and biotechnology
applications.

3) Potential in antiviral defense: Argonaute proteins have significant potential in antiviral defense.
Compared to CRISPR-Cas9, Argonaute proteins more efficiently recognize and degrade viral RNA under
relatively
high-temperature conditions or in biological environments. This means that Argonaute proteins can be used to
inhibit viral infections, offering new approaches to the development of antiviral immunotherapies. This has
potential applications in pandemic control and the management of viral infections, especially when dealing
with emerging pathogens.

###### 1.2 Importance of KmAgo Protein

KmAgo is a programmable universal Argonaute ribonuclease originating from mesophilic bacteria. Its position in
the evolutionary tree is shown in the figure below. KmAgo can effectively cleave most types of nucleic acids,
as indicated by some of its cleavage sites and corresponding GC contents in the figure. KmAgo exhibits
relatively good thermal stability, making it an ideal choice for applications requiring heat resistance, as
compared to CRISPR.

###### 1.3 Directed Evolution

Our team conducted in-depth research to understand the fundamental principles of directed evolution and its
applications in protein modification. Directed evolution is a commonly used method in protein and synthetic
biology design, simulating the evolutionary process at the molecular level in the laboratory. Through random
mutations and recombination, a large number of mutant variants can be artificially generated. By applying
selective pressure, protein variants with desired features can be screened, simulating molecular-level
evolution.

![Directed Evolution Overview](https://static.igem.wiki/teams/4589/wiki/pics/designdirected.jpg)

The process includes:

1) Mutation: Introduction of random mutations into the gene to be modified, for example, through error-prone
PCR, DNA shuffling, etc.

2) Expression: Transfer of the modified gene into host cells for expression.

3) Selection: Typically achieved by rapidly evaluating optical properties such as turbidity, color,
fluorescence, or chemiluminescence.
Traditional selection methods are often laborious and susceptible to environmental factors. Therefore,
computer-assisted screening is required. This project uses a thermal stability prediction model as a screening
tool, enabling faster and more convenient selection.

#### 2. Design and Planning

After extensive research, we summarized existing data and clarified our project's objectives, including the
following three steps:

1.1 Train a high-precision thermal stability prediction model and fine-tune it for AGO proteins and mutant
strains.

1.2 Develop an online platform for thermal stability prediction, allowing users to submit sequence data and
receive Tm values along with ranking.

1.3 Employ the DARWINS model for assisted directed evolution of KmAgo to obtain a more stable and active
KmAgo at various temperatures.

The detailed design of this approach ensures the concretization and feasibility of our project.

{% endblock %}

{% extends "layout.html" %} {% block bg %}bg{% endblock %} {% block img\_class %}custom-image-class{% endblock %} {% block img\_url %}https://static.igem.wiki/teams/4589/wiki/subpage3.png{% endblock %} {% block title %}Proof of concept{% endblock %}
{% block lead %}Validation on Our Project{% endblock %}{% block page\_content %}

[Theoretical Support](#item-1)
[Dry Lab Validation](#item-2)
[Wet Lab Validation](#item-3)
[Reference](#item-4)

###### Theoretical Support

In this project, we employed Protein Language Models (PLM) to extract sequence features for thermal stability prediction. The effectiveness of features extracted by PLMs has been widely validated[1-5]. Take ESM2 for example; the features it extracts can
generate high-confidence protein structures when passed through a structural module, as published in Science in 2023. Therefore, after extensive literature research and discussions, we concluded that the use of PLMs
for assisting thermal stability prediction is a feasible approach.

###### Dry Lab Validation

We collected protein sequence data from Kaggle and PremPS. We conducted model training and effectiveness validation using various PLMs, including TempL and ESM2, and ultimately found that the model based on ESM2 yielded the best predictive performance.
During the model training process, we employed Mean Squared Error (MSE) supervision, and the loss function curve is depicted below:

![](https://static.igem.wiki/teams/4589/wiki/pics/proof1.jpg)

    As you can see, MSE converged towards the end of training, and there were no gradient anomalies during the training process. On the test set, the trained model achieved an R-squared (R2) value of 0.71 when comparing predicted values to actual values,
indicating that the model could capture the inherent thermal stability properties within the sequences.

###### Wet Lab Validation

We assessed the model's performance using experimental values for KmAgo mutants obtained from our collaborating wet lab and iterated based on the results.

    For specific results, please refer to
<https://2023.igem.wiki/sjtu-software/engineering> .

###### Reference

1. Zeming Lin et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science 379,1123-1130(2023).

2. Ruidong Wu et al. High-resolution de novo structure prediction from primary sequence. bioRxiv 2022.07.21.500999.

3. Minjie Mou et al. A Transformer-Based Ensemble Framework for the Prediction of Protein–Protein Interaction Sites, Research, 6, (2023).

4. Jiawei Xing et al. Origin and functional diversification of PAS domain, a ubiquitous intracellular sensor, Science Advances, 9, 35, (2023).

5. Ruth Nussinov et al. AlphaFold, allosteric, and orthosteric drug discovery: Ways forward, Drug Discovery Today, 28, 6, (2023).

{% endblock %}