iGEM 2023 — Team tsinghua-a

{% extends "layout.html" %}
{% block title %}Notebook{% endblock %}
{% block header %}

# {{ self.title() }}

{% endblock %}
{% block page\_content %}
![back to top](https://static.igem.wiki/teams/4771/wiki/logo-light.png)
back to top

[2022](#Header1)
[March to June 2023](#Header2)
[July to October 2023](#Header3)

# 2022

## September 2022:

Yuman Li, Jingfei Hou, Yuan Chiang, Ziqian Wang, and Zhan Shi took a course called "Interdisciplinary Research and Practice in Dept.of Automation". The theme of the course was synthetic biology. Under the guidance of Professor Gu Jin and Professors Zhen Xie, this course enriched our knowledge of biology, and we conducted extensive literature research, explored the intersection of synthetic biology and informatics.

## November 2022:

Yuman Li, Jingfei Hou, and Yuan Chiang developed a keen interest in protein structure prediction. During a literature sharing session in class, we chose to discuss the paper "Highly accurate protein structure prediction with AlphaFold," focusing on the algorithmic details of AlphaFold, such as MSA representation & pair representation, IPA & Residue Gas, Self-distillation training and Recycling.

# March to June 2023

## March 2023:

Yuman Li, Jingfei Hou, Yuan Chiang, Ziqian Wang, and Zhan Shi were interested in protein optimization and protein design. We decided to team up to participate in the iGEM competition.

Focusing on protein optimization design, Yuman Li, Jingfei Hou, and Yuan Chiang discovered significant exploration potential in enzyme optimization design. They proposed an enzyme optimization pipeline: predicting mutation sites based on MSA and HotSpot Wizard, constructing a sequence space for mutations, and predicting the activity and stability of potential sequences. Ziqian Wang and Zhan Shi attempted various algorithms, using BLOSUM encoding to construct a mutation sequence space for a specific enzyme, and used deep models to fit and predict activity and stability.

![Preliminary pipeline](https://static.igem.wiki/teams/4771/wiki/1.png)

Figure 1. Preliminary pipeline of mutation sequence space and scoring

## April 6, 2023:

We went to QiTan Tech and had a full communication with them. We not only toured the company's bioinstrumentation but also gained a deep understanding of the structure, properties, and meaningful applications of nanopore proteins. During the exchange, we realized that optimizing enzyme design is challenging and discussed the progress in de novo protein design with the company, discovering that only a few research groups worldwide can accomplish de novo protein design.

## April 22, 2023:

Focusing on protein optimization design, Yuman Li, Jingfei Hou, and Yuan Chiang discovered significant exploration potential in enzyme optimization design. They proposed an enzyme optimization pipeline: predicting mutation sites based on MSA and HotSpot Wizard, constructing a sequence space for mutations, and predicting the activity and stability of potential sequences. Ziqian Wang and Zhan Shi attempted various algorithms, using BLOSUM encoding to construct a mutation sequence space for a specific enzyme, and used deep models to fit and predict activity and stability.

## May 2023:

Inspired by Professor Xie Zhen, we decided to focus on the theme of "humanization of antibodies" and began algorithmic design for antibodies based on our previous achievements.

## May 28, 2023:

We specifically visited the Zhongguancun Life Science Park, which focuses on cutting-edge tracks such as cell and gene therapy, innovative pharmaceuticals, AI-driven drug development, AI-assisted diagnostics, and more. We visited some companies within the park, with a particular emphasis on holding discussions with Beijing Syngentech and BeiCell Biotechnology. In this exchange, we gained a greater understanding of antibody drugs.

## June 2023:

We conducted various algorithmic attempts in the design of antibody drugs, including researching machine learning methods such as Hu-mAb and BioPhi for antibody humanization. We proposed two technical routes: "CDR transplantation" and "fully human antibody design." After multiple rounds of discussion, we generally believe that there is still significant room for improvement in the method of minimal response mutation. Additionally, we also learned about the antibody structure prediction tool IgFold and deployed relevant software environments on supercomputers.

![Minimal response mutation](https://static.igem.wiki/teams/4771/wiki/2.png)

Figure 2: Minimal response mutation

# July to October 2023

## July 8-10, 2023:

Yuman Li participated in the CCiC (10th Conference of China iGEMer Community) in Hainan, China, and gained a lot of inspiration through exchanges with other iGEM teams.

## July 18, 2023:

We visited VJTbio company for the first time, whose research direction is animal antibodies. We identified the users of our project and established collaboration. In our subsequent design, we constantly consider the practical needs of the industry. Our iGEM project has expanded from humanization of antibodies to the species-specification of antibodies, aiming to provide suitable antibody drugs for a variety of species.

## July 25, 2023:

The final team roster is confirmed: Yuman Li, Jingfei Hou, Yuan Chiang, Ziqian Wang, Zhan Shi, Xiaomeng Chen.

## August 2, 2023:

We collected tens of thousands of antibody sequence data, filtered FR1-FR4 to their typical lengths, and used algorithms for species prediction, achieving an accuracy of approximately 95%. This preliminary validation demonstrates the feasibility of classifying antibodies based on the species-specific FR region sequences. It indicates that the FR regions of antibodies from different species possess sufficient specificity. Our concept of enabling more species to have access to appropriate antibody drugs is viable.

![Confusion matrix](https://static.igem.wiki/teams/4771/wiki/3.png)

Figure 3: Confusion matrix for seventeen-class classification

## August 9, 2023:

We participated in the Synbio Plus AI Conference and shared some insights regarding generalization capabilities, dataset quality, interpretability, and security.

## August 10, 2023:

We completed the construction of the Deep One Class model, extracting feature vectors for each species and providing species-specific sequence scores based on similarity measurements.

![Species-specific sequence scores](https://static.igem.wiki/teams/4771/wiki/4.png)

Figure 4: Species-specific sequence scores based on the Deep One Class model

## August 22, 2023:

We completed the part of the project that involves structural scoring based on IgFold.

![Structural scoring](https://static.igem.wiki/teams/4771/wiki/5.png)

Figure 5: Structural scoring based on IgFold

## September 2023:

We began packaging the software, summarizing the project's advantages, and reflecting on areas for improvement to make a greater impact on the world.

## October 2023:

We organized the data and code throughout the entire project process and improved the wiki.

{% endblock %}

{% extends "layout.html" %}
{% block title %}Engineering Success{% endblock %}
{% block header %}

# {{ self.title() }}

{% endblock %}
{% block page\_content %}

![back to top](https://static.igem.wiki/teams/4771/wiki/logo-light.png)
back to top

[Background](#Header1)
[Initial Model](#Header2)

* [Design](#1Design)
* [Build](#1Build)
* [Test](#1Test)
* [Learn](#1Learn)

[Version II](#Header3)

* [Design](#2Design)
* [Build](#2Build)
* [Test](#2Test)
* [Learn](#2Learn)

[Version III](#Header4)

* [Design](#3Design)
* [Build](#3Build)
* [Test](#3Test)
* [Learn](#3Learn)

[Version IV](#Header5)

* [Design](#4Design)
* [Build](#4Build)
* [Test](#4Test)
* [Learn](#4Learn)

[Version V](#Header6)

* [Design](#5Design)
* [Build](#5Build)
* [Test](#5Test)
* [Learn](#5Learn)

[Version VI](#Header7)

* [Design](#6Design)
* [Build](#6Build)
* [Test](#6Test)
* [Learn](#6Learn)

[Version VII](#Header8)

* [Design](#7Design)
* [Build](#7Build)
* [Test](#7Test)
* [Learn](#7Learn)

[Final Model](#Header9)

* [Design](#8Design)
* [Build](#8Build)
* [Test](#8Test)

[Final Learning](#Header10)

<

>

# Background

Inspired by the humanization of mouse-derived antibodies, our project starts with the other sources of antibody sequences against target species antigen (such as mouse-derived antibodies), automatically generates antibody sequences that may have better efficacy and lower immunogenicity, and performs comprehensive scoring. That is, when given specific antibody sequence from a certain species and the target species for obtaining the corresponding antibody, we hope to generate a potential antibody database for the target species. The generated antibody data should preserve the CDR region of the original antibody data as much as possible because this region is used for specific recognition and binding to antigens, thereby achieving the function of antibody drugs.

The most critical point for generating data is to ensure that it conforms as closely as possible to the characteristics of antibody data for the target species. In other words, we aim to minimize the immunogenicity of the generated antibody data for the target species.

Currently, mainstream antibody humanization tools primarily address the above issues, attempting to use machine learning or deep learning tools to provide a measure of sequence immunogenicity with respect to humans, i.e., providing homology measures of generated sequence FR segments relative to human antibody data. These algorithms have made some progress in the field of humanization of mouse antibodies, and some humanized antibody drugs have been designed. But we hope to design antibodies suitable for more species.

![overview](https://static.igem.wiki/teams/4771/wiki/model/1.png)

Figure 1. Project overview

To ensure the effectiveness and reliability of our model in antibody design, we rigorously followed engineering iteration requirements. We conducted five iterations following the **Design-Build-Test-Learn** cycle and, through various results and discussions with experts, summarized the areas where the project can be further improved.

# Initial Model

## Initial Model Design

Our project is mainly divided into three parts: potential antibody sequence generation, antibody sequence species specification scoring, and antibody structure scoring. The species specification scoring of antibody sequences extends the current mainstream tools to various species, and this part is the focus of our project. In the design part, we divide this project into three components: data encoding and processing, model construction, and training with debugging. The entire system framework for the model is a critical issue that we need to continuously iterate and explore. The iterative process presented below mainly revolves around the core content of antibody sequence species specification scoring.

We used Python as the primary programming language and experimented with multiple machine learning and deep learning libraries. In the first attempt, we chose to build the model using the basic ‘scikit-learn’ library.

## Initial Model Build

To validate the feasibility of our project, we used a relatively simple encoding method and machine learning models to complete this task.

The species specification scoring of antibody sequences represents the probability that the generated sequence belongs to a specified species. The higher the probability, the more it conforms to the antibody patterns of the specified species. In other words, we have good reason to believe that sequences with higher probabilities have lower immunogenicity for the specified species and are safer for use in the biological organism of that species.

We approached this sequence scoring task from the perspective of a natural language processing sequence multi-classification task. We modeled the species specification scoring of antibody sequences as a sequence multi-classification task and addressed it using deep learning.

After modeling the overall task, we also encoded the input data. Since we ultimately need to generate FR region segments for target species, we selected FR region segments from multiple species as input features and encoded them.

Taking the example of the heavy chain sequence of antibodies, the heavy chain segments consist of FR1, CDR1, FR2, CDR2, FR3, CDR3, and FR4. To ensure the completeness of the generated data and to consider potential interactions between different regions after sequence folding, we concatenated these four segments as input features.

![Initial scoring pipeline](https://static.igem.wiki/teams/4771/wiki/engineering/2.png)

Figure 2. Initial scoring pipeline

In the first version of the solution, we employed the commonly used one-hot encoding method in the field of bioinformatics, where each amino acid is encoded into a 20-dimensional vector.

![One-hot](https://static.igem.wiki/teams/4771/wiki/engineering/3.jpg)

Figure 3. One-hot

![One-hot](https://static.igem.wiki/teams/4771/wiki/engineering/4.png)

Figure 4. One-hot encoding example

In the classifier, we utilized the classic Support Vector Machine (SVM) algorithm in the first version of our solution. We conducted classification attempts separately on the eight species with the highest antibody counts as well as on almost all the data. Our aim was to validate the separability of the antibody data using this approach.

![SVM](https://static.igem.wiki/teams/4771/wiki/engineering/5.jpg)

Figure 5. SVM

## Initial Model Test

The experiment achieved a relatively high accuracy, demonstrating the feasibility of our initial attempt. However, one-hot encoding showed poor interpretability: while the advantage of one-hot encoding lies in its ability to represent differences between antibody data, making the data highly interpretable, our analysis revealed that it fails to capture the close relationships between similar subtypes and the distant relationships between evolutionarily distant species. It cannot learn the true features of the antibody data.

## Initial Model Learn

One-hot encoding method makes the data has high separability, but does loss valuable information. Based on this, we recognized the need for a more appropriate encoding method, one that incorporates both the similarities in the physicochemical properties of amino acids and the statistical patterns of natural genetic evolution. We investigated the mainstream amino acid coding matrices including BLOSUM matrix and PAM matrix, and improve encoding method in the next version.

# Version II

## Version II - Design

In the first version of the model, a core issue arose from the overly naive encoding method used. Therefore, we conducted research into mainstream amino acid encoding matrices, including BLOSUM matrices, PAM matrices, and Position-Specific Scoring Matrices (PSSM). Ultimately, we chose to use the BLOSUM62 substitution scoring matrix, which is a widely applied encoding method in protein analysis. The BLOSUM62 matrix is based on statistical information from protein sequence alignments, and we have good reason to believe that it incorporates both the physicochemical similarities between amino acids and statistical patterns from natural genetic evolution.

![Blosum62](https://static.igem.wiki/teams/4771/wiki/engineering/6.png)

Figure 6. Blosum62 matrix

![Blosum62 example](https://static.igem.wiki/teams/4771/wiki/model/20.png)

Figure 7. Blosum62 ecoding example

## Version II - Build

In a similar setup using the Support Vector Machine (SVM), we conducted a classification attempt on the 18 most abundant species of antibodies, employing the BLOSUM62 encoding and SVM model.

## Version II - Test

This attempt resulted in a successful classification accuracy of 0.92. From the confusion matrix provided below, it is evident that the model exhibited good performance, with some confusion occurring due to the presence of similar subtypes. For instance, there might be multiple rodent species, and such confusions actually validate the rationality of the encoding and classification methods.

![Version II test result](https://static.igem.wiki/teams/4771/wiki/engineering/8.png)

Figure 8. Version II test result

## Version II - Learn

In the end, we settled on using the BLOSUM62 encoding and shifted our focus toward the design of a more effective model.

# Version III

## Version III - Design

Once we had determined the encoding method, we employed various traditional machine learning algorithms to compare their performance with the SVM method, with the aim of improving our model.

## Version III - Build

In this phase, we conducted a horizontal comparison of three algorithms: SVM, KNN, and decision trees, to assess their performance within our model, which led to further updates and iterations.

When classifying the 18 most abundant species of antibodies, the three models showed relatively similar results. However, as the sample size increased, SVM exhibited superior performance.

## Version III - Test

Focusing on protein optimization design, Yuman Li, Jingfei Hou, and Yuan Chiang discovered significant exploration potential in enzyme optimization design. They proposed an enzyme optimization pipeline: predicting mutation sites based on MSA and HotSpot Wizard, constructing a sequence space for mutations, and predicting the activity and stability of potential sequences. Ziqian Wang and Zhan Shi attempted various algorithms, using BLOSUM encoding to construct a mutation sequence space for a specific enzyme, and used deep models to fit and predict activity and stability.

![KNN](https://static.igem.wiki/teams/4771/wiki/engineering/9.png)

Figure 9. KNN

![Comparison of three models](https://static.igem.wiki/teams/4771/wiki/engineering/10.png)

Figure 10. Comparison of three models

## Version III - Learn

Analyzing the classification performance of the three models, the encoded vectors of antibody data reached a dimensionality of 1900, which is considered high-dimensional data. SVM is well-suited for handling higher-dimensional data, while decision trees tend to generate overly complex structures when dealing with high-dimensional data, resulting in the poorest performance.

Additionally, we conducted t-SNE dimensionality reduction analysis on the antibody data to observe the separability of the 18 classes of data.

![TSNE plot](https://static.igem.wiki/teams/4771/wiki/engineering/11.png)

Figure 11. TSNE plot of 18 spieces

As the volume of data continues to increase, the performance of traditional machine learning models gradually deteriorates. Furthermore, in the collected data, there is a high proportion of small-sample data corresponding to species. Antibody data is also challenging to augment adequately. Therefore, a significant problem arises where small-sample data is overwhelmed by large-sample data. Based on this, we aim to explore deep learning models with stronger feature extraction capabilities for classification and scoring.

# Version IV

## Version IV - Design

After accumulating experience from previous model iterations, we decided to leverage the PyTorch deep learning framework and employ state-of-the-art deep learning models for designing our classifier.

## Version IV - Build

Deep learning models are widely recognized for their strong feature extraction capabilities. In the context of our antibody sequence scoring and classification task, we experimented with LSTM (Long Short-Term Memory) models and Transformer models, which are well-suited for natural language processing tasks.

![LSTM structure](https://static.igem.wiki/teams/4771/wiki/engineering/12.png)

Figure 12. LSTM structure

![Transformer structure](https://static.igem.wiki/teams/4771/wiki/engineering/13.png)

Figure 13. Transformer structure

## Version IV - Test

We attempted to use LSTM and Transformer models for the classification of antibodies from 70 different species. The LSTM model achieved a classification accuracy of 0.87, while the Transformer model achieved a classification accuracy of 0.90. The Transformer model exhibited stronger feature extraction capabilities.

![model comparison](https://static.igem.wiki/teams/4771/wiki/engineering/14.png)

Figure 14. Comparison of deep learning models

While the deep learning models achieved an accuracy of 0.90 in the classification of 70 species, there was a significant issue of imbalanced data sample sizes. This means that a few species dominated almost all of the data, while the majority of species had much smaller amounts of data.

## Version IV - Learn

The high accuracy was mainly due to the models favoring the learning of features from the species with large sample sizes and misclassifying data from species with small sample sizes into the categories with larger sample sizes.

![preference of feature learning](https://static.igem.wiki/teams/4771/wiki/engineering/15.png)

Figure 15. Confusion matrix shows preference of feature learning

Based on this, we aim to adopt new models in an effort to address the issue of small-sample data being overwhelmed by large-sample data.

# Version V

## Version V - Design

The core issue at the moment is that there are too many biological species, and the training data is not balanced. The use of neural network classification heads with a predefined number of classes not only results in insufficient learning from small-sample data but also makes it challenging to extend the model to newly added species.

Through research in the field of bioinformatics, we have learned that the concept of one-class models has shown promising practical results in studies at the cellular level. One-class models are often used to address data detection, classification, and anomaly detection when there is only one class of samples. These models are trained using only samples from a single class, learning the characteristics of that single class as thoroughly as possible. A common decision-making approach in the field of bioinformatics is to extract template feature vectors from single-class samples and use similarity calculations between new data and template feature vectors, along with a threshold, to make decisions.

One significant advantage of one-class models is their strong scalability. When new data for a new species is collected, we only need to train a one-class model for that specific species, which greatly saves computational resources and update time.

Therefore, we have decided to change the existing classification approach and explore the use of one-class models.

## Version V - Build

We conducted research on existing one-class models and found that SVM-based models, particularly the One-Class SVM, have shown superior performance in previous classifications. As a result, we primarily focused on experimenting with the One-Class SVM.

![Performance of different models](https://static.igem.wiki/teams/4771/wiki/engineering/16.png)

Figure 16. Performance of different models

We trained a separate One-Class SVM model for each class of data. The goal of the One-Class SVM model is to enclose the single-class data within the smallest possible hypersphere, with the primary parameter being the error tolerance rate. We partitioned an equal proportion of the test set for each class of data. Since all classes of data exist in the same dimensional space, we made classification decisions based on the distance between the test data and the soft classification boundary of each One-Class SVM model.

![One-Class SVM](https://static.igem.wiki/teams/4771/wiki/engineering/17.png)

Figure 17. One-Class SVM

## Version V - Test

We selected eight species, including those with both large and small data volumes. Through multiple attempts, we observed that the model's performance was significantly influenced by the error tolerance rate parameter. When adjusting this parameter separately for each species, we were able to achieve 1.0 accuracy. However, this tuning approach is not suitable for scaling up to accommodate the addition of new species.

![Performance of One-Class SVM](https://static.igem.wiki/teams/4771/wiki/engineering/18.png)

Figure 18. Performance of One-Class SVM

Based on this, we conducted a new t-SNE dimensionality reduction analysis and observed outliers in the data. We noticed the presence of significant outliers in the data, and these outliers were severely affecting the position of the soft classification planes of the One-Class SVM models.

## Version V - Learn

![TSNE clustering diagram](https://static.igem.wiki/teams/4771/wiki/engineering/19.jpg)

Figure 19. TSNE clustering diagram

It can be seen from the TSNE clustering diagram that the quality of the collected antibody sequences is uneven. They may be high-frequency somatic mutations or non-natural sequences. We need to eliminate outliers to ensure data quality and thereby reduce the immunity of the designed antibody sequences.

# Version VI

## Version VI - Design

Because we observed significant outlier data, in order to avoid these outliers causing excessive interference in the One-Class SVM model, we aim to use the One-Class SVM model to identify clear outlier data points and remove them. This will allow us to use the more compact data for each class to complete the classification and scoring tasks using SVM models.

## Version VI - Build

We added outlier detection based on our model. The updated model is as follows:

![Version VI model](https://static.igem.wiki/teams/4771/wiki/engineering/20.png)

Figure 20. Version VI model

## Version VI - Test

This approach effectively achieved the removal of outlier data points. Under this model construction, we successfully improved accuracy and achieved a classification accuracy of 1.0 for the previously selected eight classes of data.

![TSNE plot with outlier detection](https://static.igem.wiki/teams/4771/wiki/engineering/21.png)

Figure 21. TSNE plot with outlier detection

![Version VI performance](https://static.igem.wiki/teams/4771/wiki/engineering/22.png)

Figure 22. Version VI performance

## Version VI - Learn

We really need outlier detection in our model. However, as the number of species gradually increased, traditional machine learning models continued to exhibit a weakness in feature extraction capability. Therefore, we adopted the concept of one-class modeling and combined it with deep learning models.

# Version VII

## Version VII - Design

### A completely new attempt: deep one class

The advantage of one-class models is that they train on single-class data, which can result in more compact representations. However, there's no guarantee that the model extracts genuinely useful antibody features. On the other hand, deep learning multi-class methods ensure strong feature extraction capabilities but can't solve the serious misclassification problem caused by imbalanced multi-species data.

We drew inspiration from Deep One-Class models used for image anomaly detection and weighted the descriptive loss of the multi-class task along with the compactness loss of the one-class model. We used this weighted loss to update the parameters of the deep learning model, ensuring that the model has strong feature extraction capabilities while also training the single-class data to be more compact.

## Version VII - Build

![Training stage](https://static.igem.wiki/teams/4771/wiki/engineering/23.png)

Figure 23. Training stage of DOC model

![Loss funtion](https://static.igem.wiki/teams/4771/wiki/engineering/24.png)

The final training model consists of two paths, as shown in the diagram.

The lower training path takes as input the single-class antibody data for a specified species. Here, we use a Transformer Encoder model with strong feature extraction capabilities and suitability for protein sequences to extract features from the antibody data. The Transformer Encoder model does not include a classification head; its final output is the extracted feature vector. We use the variance of data within a batch as the loss function. Smaller variance indicates more compact data, ensuring that this training path trains the model to represent the specified species' data more compactly.

\(m\_i = \frac{1}{n-1} \sum\_{j\neq i} x\_j\), \(z\_i = x\_i - m\_i\)

\(l\_C = \frac{1}{nk} \sum\_{i=1}^{n} z\_i^T z\_i\)

The upper training path uses multi-class antibody data from other species and also passes through a Transformer Encoder model. This Transformer Encoder model has the same structure and shares parameters with the model in the lower training path. The extracted features are then passed through a classification layer to perform multi-class classification, with the final cross-entropy loss used as the training loss. Smaller cross-entropy indicates stronger feature extraction capabilities in the model.

The weighted loss obtained from both training paths is used to update the shared parameters of the Transformer Encoder, achieving the desired model's performance.

## Version VII - Test

We applied the above method to classify species with sample sizes ranging from 20 to 200 and achieved an accuracy of 0.92. However, this performance may not be satisfactory.

![Performance of DOC model](https://static.igem.wiki/teams/4771/wiki/engineering/25.png)

Figure 25. Performance of DOC model

Considering that small-sample species as target classes may not have enough data to support the data requirements of the Transformer Encoder model, we implemented improvements involving model pretraining and parameter transfer.

In the actual model training, due to the limited sample size for certain single-class species, using the Transformer Encoder model alone may not match sufficient data. We treated the training process described earlier as a fine-tuning step. Before performing the above training, we initially trained the Transformer Encoder and the classification layer on the entire dataset. After training, we obtained the pretrained model parameters and then transferred these pretrained parameters into the training process mentioned earlier, allowing for fine-tuning.

However, even after these efforts, the model's performance was not as expected. We suspected that the training process for the weighted model had a significant impact on the model's performance. Therefore, we explored various approaches to address this issue.

## Version VII - Learn

![Attempted fine-tuning methods](https://static.igem.wiki/teams/4771/wiki/engineering/26.png)

Figure 26. Attempted fine-tuning methods

The first training approach showed the best classification performance, but the training results of deep learning models can be sensitive to parameters. Therefore, further improvements were made:

Considering that our model uses weighted loss, the choice of weighted parameters significantly affects the model's performance. Since the compactness loss should have a relatively small weight, we tried various parameter settings and ultimately chose a compactness loss weight of 0.1 and a descriptive loss weight of 1.0.

Additionally, we set the model training to fine-tune for 10 epochs, used the Adam optimizer with a learning rate of 0.0005, and applied the corresponding learning rate decay.

These parameter choices and optimizer settings were determined after multiple iterations, experiments, and analysis to achieve the optimal results.

# Final Model

## Final Model - Design

After completing the model training, we further updated and adjusted the testing process multiple times, and the final model was determined as follows.

![Testing stage of DOC model](https://static.igem.wiki/teams/4771/wiki/engineering/27.png)

Figure 27. Testing stage of DOC model

In the field of bioinformatics, a common decision-making approach involves learning the features of single-class samples and obtaining template feature vectors for these single-class samples. When new data needs to be tested, decisions are made by calculating the similarity between the new data and template feature vectors and using a predefined threshold.

For image anomaly detection, multiple template feature vectors are used as the basis for classification.

![classification method](https://static.igem.wiki/teams/4771/wiki/engineering/28.png)

Figure 27. Classification method

In terms of the overall workflow, our project can be understood as consisting of sequence encoding, sequence generation, classification, and scoring. The scoring phase includes both sequence scoring and structural scoring as mentioned before.

After conducting literature research and team discussions, we have determined the following sequence generation strategy:

In the sequence generation part, we integrate the features of input antibody data with the statistical patterns of antibody data for the target species. We adopt a strategy of conservative mutations to determine the amino acids with a high likelihood at each position. We then generate a potential library of antibody data sequences through permutations and combinations.

Afterward, we subject the generated sequences to the model specific to the target species to obtain corresponding sequence scores.

As for the other part of scoring, the structural scoring, we will leverage models like IgFold:

In the structural scoring part, we first fold the original sequence and the generated potential sequences using the IgFold model. This helps us determine whether the generated sequences can fold into complete structures. We also visually compare the structures of the original sequence and the generated sequences to identify differences.

This comprehensive approach integrates sequence generation and structural scoring to assess the quality and potential of the generated antibody sequences.

![IgFold](https://static.igem.wiki/teams/4771/wiki/engineering/29.png)

Figure 29. IgFold antibody structure prediction

## Final Model - Build

The main sequence generation strategies employed include:

Setting a threshold at 0.9: We calculate all the amino acids at each position for the target species, where the total frequency reaches or exceeds 0.9. If certain positions are dominated by a single amino acid with a frequency of 0.9 or higher, we retain only that amino acid. For other positions, we keep two amino acids. Additionally, we take into account the amino acids at the corresponding positions in the input sequence to minimize mutations.

About sequence scoring, we attempted to randomly extract sample data and then average their output features from the model to create a template feature vector. We also experimented with both Pearson and Spearman coefficients to measure vector similarity, with Pearson coefficients showing superior results.

After obtaining the PDB (Protein Data Bank) files via IgFold, we perform a comparison of structural similarity between the original sequence and the generated sequence's PDB files using two commonly used metrics: TM-Score and RMSD (Root Mean Square Deviation). These metrics provide a reasonable structural scoring to assess the similarity between the structures of the original and generated sequences, mainly the structural similarity of the CDR regions that bind to the antigen.

![Structture scoring](https://static.igem.wiki/teams/4771/wiki/engineering/30.png)

Figure 30. Structture scoring

## Final Model - Test

![Sequence generation process](https://static.igem.wiki/teams/4771/wiki/engineering/31.png)

Figure 31. Sequence generation process

![Software output](https://static.igem.wiki/teams/4771/wiki/engineering/32.png)

Figure 32. Software output

The sequence generation process went very smoothly. We obtained a large amount of potential sequence space. We further put these sequences into the model for scoring.

In the sequence scoring part, we found that this decision-making approach, while improving the model's performance on small-class samples, had the opposite effect on larger-class data. Through dimensionality reduction analysis, we discovered that due to the distribution patterns in the data, some large-class sample test points were actually farther away from the class center than small-class sample test points. This resulted in a deterioration in the classification performance for large-class samples.

![distribution patterns](https://static.igem.wiki/teams/4771/wiki/engineering/33.jpg)

Figure 33. Distribution patterns shown in TSNE plot

To address this issue, we attempted two different approaches: using the K-Nearest Neighbors (KNN) classification decision method and applying the Pearson coefficient in combination with sample downsampling. Ultimately, we chose the latter approach, and the resulting classification and scoring performance is as follows.

![resulting performance](https://static.igem.wiki/teams/4771/wiki/engineering/34.png)

Figure 34. Resulting performance

The model's classification accuracy has reached 0.97, and any remaining confusion is primarily due to similarities between subtypes, which is reasonable. We further refined the species classification based on these similar subtypes, resulting in an accuracy of 1.0. Such a high accuracy proves that our sequence score is considerably reasonable.

![Clustering](https://static.igem.wiki/teams/4771/wiki/engineering/35.jpg)

Figure 35. Clustering

![Performance after clustring](https://static.igem.wiki/teams/4771/wiki/engineering/36.png)

Figure 36. Performance after clustring

Specifically, look at the species-specific scoring for sequences. The sequence scoring process also shown promising results, with scores of 0.9 or higher for sequences belonging to their respective species. It also performs well in classifying similar subtypes, while the species classification score for other species remains below 0.6, which shows that our sequence scoring is very reasonable.

![Sequence scoring example](https://static.igem.wiki/teams/4771/wiki/engineering/37.png)

Figure 37. Sequence scoring example

Sequences in our potential sequence space were also passed through the model to obtain sequence scores.

![Generated sequences and scores](https://static.igem.wiki/teams/4771/wiki/engineering/38.png)

Figure 38. Generated sequences and scores

Our structural scoring is also going well. First of all, from the pdb file, the sequence was successfully folded into the typical structure of the antibody, and it was relatively close to the original CDR region structure. In order to better measure the similarity, we used the structure score, which uses numbers to better characterize the score of the structure.

![PDB file generated by IgFold](https://static.igem.wiki/teams/4771/wiki/engineering/39.png)

Figure 39. PDB file generated by IgFold

![Generated sequences and structure scores](https://static.igem.wiki/teams/4771/wiki/engineering/40.png)

Figure 40. Generated sequences and structure scores

Regarding the rationality verification of our final results, you can see the wiki-Result page.

# Final Learning

### 1. Protein Encoding:

Initially, we employed a simple one-hot encoding paradigm to represent amino acids as 20-dimensional vectors. However, this approach led to overly sparse representations in the feature space and overlooked the physicochemical properties of amino acids. Based on the results of our initial tests, we recognized the need for a more suitable encoding method that would capture both the similarities in physicochemical properties among amino acids and the statistical patterns arising from natural genetic evolution. Therefore, we opted for encoding using the BLOSUM62 matrix, which is based on sequence alignment information for proteins. We had good reason to believe that it encapsulated the similarity in physicochemical properties of amino acids and the statistical patterns arising from natural genetic evolution. Indeed, its performance proved to be excellent.

### 2. Choice of Classifier Model:

In a task characterized by a high-dimensional feature space and complex calculations with limited samples, traditional machine learning models did show promise. We experimented with three such models, and each had some level of success. However, through testing, we discovered that these traditional models did not perform as well as deep learning methods when it came to fitting a large number of data samples and facilitating data expansion. Consequently, we turned to deep neural networks, particularly models built on the Transformer architecture, which ultimately yielded superior results.

### 3. Scalability and Real-World Applicability:

As a system designed for antibody design across multiple species, it was imperative to ensure that our model possessed sufficient scalability. We encountered limitations when using traditional neural networks, especially with standard classification layers, as they did not easily adapt to our needs. Our task couldn't be simplified to a mere generation, classification, or scoring paradigm. After conducting thorough research, we ultimately adopted an iterative one-class approach, and through testing, we identified the impact of outlier data on experimental outcomes. In a subsequent iteration, we successfully employed outlier detection to mitigate this impact. Following comparative experiments, we firmly established the core of our model as "deep one class," continuously fine-tuning hyperparameters and decision-making processes based on experimental results. This approach has allowed us to continually incorporate new training samples in response to real-world application scenarios and accommodate the design of antibodies for novel species.

### Areas for Improvement:

Currently, the system is presented in the form of Python engineering files, which may require users to have some programming knowledge to use effectively. Therefore, the next step is to develop a user-friendly interactive software that caters to end-to-end sequence generation requirements and can display the structural and scoring aspects based on user preferences.

{% endblock %}

{% extends "layout.html" %}
{% block title %}Experiments{% endblock %}
{% block header %}

# {{ self.title() }}

{% endblock %}
{% block page\_content %}
![back to top](https://static.igem.wiki/teams/4771/wiki/logo-light.png)
back to top

[Overview](#Header1)
[Environments](#Header2)
[BLAST Data Mining](#Header3)
[Software Experimental Procedure](#Header4)
[t-SNE](#Header5)

# Overview

![Experiments overview](https://static.igem.wiki/teams/4771/wiki/experiments/1.png)

Figure 1: Experiments overview

# Environments

In this project, our experiments were done mainly on the computer. The main part of the computer experiment files are written in python language, and the experiment environment is a virtual environment installed under Anaconda3. For the machine learning part of it, the framework used is pytorch. meanwhile, in order to accelerate the computation and easy access, the experiment uses Beijing Cloud Supercomputer.

Below are all the dependency packages and their versions installed in the experimental environment:

```
              biopython==1.79
              certifi==2016.2.28
              cycler==0.10.0
              iminuit==1.5.4
              matplotlib==2.0.2
              numpy==1.12.1
              pandas==0.20.3
              patsy==0.4.1
              pyparsing==2.2.0
              python-dateutil==2.6.1
              pytz==2017.2
              scikit-learn==0.19.0
              scipy==0.19.1
              seaborn==0.8
              six==1.10.0
              sklearn-contrib-lightning==0.4.0
              statsmodels==0.8.0
              tmscoring==0.4.post0
              wincertstore==0.2
```

For the igfold section used in structural scoring, you can refer to the installation method in the official igfold documentation: [github.com/Graylab/IgFold](https://github.com/Graylab/IgFold)

# BLAST Data Mining

We used BLAST for data mining because it can collect a richer number of species compared to other people's organized databases and can filter sequences with very low homology scores to ensure data security.

The dataframe collected by BLAST contains the following columns: id number; name (e.g., Ig heavy chain variable VDJ region), whether heavy chain or light chain; species type (e.g., Sus scrofa), accession time, sequence, sequence length, and the length of the sequence, as well as the number of sequences of FR1, CDR1, FR2, CDR2 and so on.

![BLAST workflow](https://static.igem.wiki/teams/4771/wiki/experiments/3.png)

Figure 2: BLAST workflow

The workflow of BLAST is as above, and the collected data overview is as follows:

For heavy chains, we have initially collected a total of 36,180 valid sequences in the FR-CDR region, involving 253 species names, 22 species with 100 or more data; 70 species with 10 or more data; and 100 species with 3 or more data. It can be seen that unlike the huge data on humans and mice, the existing data on many other species are very sparse. We further filtered out some core antibody data of species, and the obtained statistics are as follows:

![Heavy chain data's species distribution](https://static.igem.wiki/teams/4771/wiki/experiments/4.png)

Figure 3: Heavy chain data's species distribution

The word cloud diagram of species whose number of collected antibody sequences is not in the top ten is as follows:

![Heavy chain data's species distribution](https://static.igem.wiki/teams/4771/wiki/experiments/5.png)

Figure 4: Word cloud diagram of species with smaller data

Sequence statistics for each species are analyzed below:

First, the figure below shows **Species FR Region Portraits-Sequence Site Distribution**:

![Species FR Region Portraits-Sequence Site Distribution](https://static.igem.wiki/teams/4771/wiki/experiments/8-1.png)

![Species FR Region Portraits-Sequence Site Distribution](https://static.igem.wiki/teams/4771/wiki/experiments/8-2.png)

![Species FR Region Portraits-Sequence Site Distribution](https://static.igem.wiki/teams/4771/wiki/experiments/8-3.png)

Figure 5: Species FR Region Portraits-Sequence Site Distribution

Then we also counted the proportion of amino acids at each site, that is, **Species FR Region Portraits-Site Amino Acid Ratio**:

![Amino Acid Ratio](https://static.igem.wiki/teams/4771/wiki/experiments/9.png)

![Amino Acid Ratio](https://static.igem.wiki/teams/4771/wiki/experiments/12.png)

![Amino Acid Ratio](https://static.igem.wiki/teams/4771/wiki/experiments/10.png)

![Amino Acid Ratio](https://static.igem.wiki/teams/4771/wiki/experiments/11.png)

Here we only select the statistics of some species for display. Using this data, we can do a better job of completing projects.

# Software Experimental Procedure

Our project is mainly divided into three parts: potential antibody sequence generation, antibody sequence speciation scoring and antibody structure scoring, and among them, antibody sequence speciation scoring is to expand the humanized scoring of the current mainstream tools to multiple species, which is the focus of our project, and it mainly includes three parts: data encoding and processing, model construction and selection, and model debugging.

We attempted to classify the 18 species with the highest number of antibodies using BLOSUM62 coding as well as SVM modeling and successfully achieved a classification accuracy of 0.92.

The traditional one class model of machine learning has poor feature extraction ability for low sample and high dimensional data, based on this, we adopt the deep one class model, i.e., deep learning model+one class, and use deep learning models with strong feature extraction ability such as transformer to extract features of high dimensional data The deep learning model+one class uses a deep learning model with strong feature extraction capability, such as transformer, to extract features from the high-dimensional data, obtain the feature vector corresponding to one class, and then use the similar method as above to make category judgment or score.

The advantage of the One Class model is that it only trains one class of data, which makes it more compact, but there is no guarantee that the model extracts truly useful antibody features. The deep learning multi-classification approach can guarantee strong enough feature extraction, but it cannot solve the serious misclassification problem caused by the uneven samples of multi-species data.

We borrow the Deep One Class model for image anomaly detection, weight the descriptive loss of the multi-classification task and the compactness loss of the One Class model, and update the parameters of the deep learning model with the weighted loss to ensure that the model has sufficiently strong feature extraction capability while also being able to train the single class data more compactly.

![Model testing stage](https://static.igem.wiki/teams/4771/wiki/experiments/2.png)

Figure 10: Model testing stage

In the actual training of the model, considering that the sample size of some single-class species is small and the use of Transformer Encoder model cannot match the sufficient amount of data, we regard the above training process as the Fine-tuning process of the model. That is, before the above model training, we first train the Transformer Encoder and the classification layer with all the data, get the pre-training parameters of the model after training, and migrate the pre-training parameters to the above training process to realize the fine-tuning of the model.

The parameters of the model training were set to set the fine-tuning process to 10 epochs, and the optimizer was chosen to be Adam, with a learning rate of 0.0005, and the corresponding learning rate decay was set.

The above parameters and optimizer selection is the optimal result we got after many times adjusting the model to try and analyze the comparison.

The classification accuracy of the model has reached 97%, and the confusion that occurs is basically entirely due to similar subtypes, which is justified by the fact that we have reclassified the species according to similar subtypes, at which point the accuracy has reached 100%.

# t-SNE

Also, in order to be able to better analyze the coding performance of the model and to visualize the morphology of our coding in the high-dimensional space for analysis and debugging, we used tsne to downscale and visualize the coding space.

t-SNE (t-Distributed Stochastic Neighbor Embedding) is a commonly used technique for downscaling and visualization. It is based on the principle of constructing a representation in a low-dimensional space based on the similarity between high-dimensional data points.

t-SNE achieves dimensionality reduction through two main steps. First, it computes the similarity between data points in the high-dimensional space. This is done by measuring the Euclidean distance or other similarity metrics between data points. Then, for each data point, t-SNE constructs a probability distribution based on its similarity, i.e., the probability of data points that are similar to it in the high-dimensional space.

In the second step, t-SNE tries to reconstruct these probability distributions in the low-dimensional space in order to minimize the difference between the high-dimensional space and the low-dimensional space. It uses optimization techniques such as gradient descent to minimize this difference and map the data points into the low-dimensional space.

The t-SNE transforms the similarity between data points into conditional probabilities, where the similarity of data points in the original space is represented by a Gaussian joint distribution, and the similarity of data points in the embedding space is represented by a Student's t-distribution.

The goodness of the embedding effect is assessed by the KL scatterof the joint probability distribution in the original and embedding spaces.That is, the function about the KL scatter is used as a loss function, and the loss function is minimized by a gradient descent algorithm to finally obtain the convergence result.

First of all, t-SNE can help us visualize high-dimensional datasets in low-dimensional space, thus revealing the intrinsic structure and relationships among the data. By mapping the high-dimensional data points into two- or three-dimensional space, we can observe the distribution of data points, clustering, and similarities between data points. This helps us discover important information such as patterns, trends and outliers in the data.

Secondly, t-SNE performs better in maintaining the local structure between data points. Compared with other dimensionality reduction methods, t-SNE is better at preserving the relative distance relationship between data points in the high-dimensional space. This means that similar data points are closer and dissimilar data points are more separated in the visualization results, thus better presenting the similarities and differences of the data.

In addition, t-SNE visualization can help us discover hidden features in the data. By observing clusters or distributions in the low-dimensional space, we can infer some potential features or attributes present in the data. This is valuable for data analysis and machine learning tasks and can provide clues for subsequent analysis and modeling.

![TSNE plot](https://static.igem.wiki/teams/4771/wiki/experiments/6.png)

Figure 11: TSNE plot for 18 species

Using tSNE, we have a better view of the distributional information of the model's encoding results and can help us analyze the cases where errors occur, such as outliers and confounded classes, so that we can better iterate the model.

![TSNE plot](https://static.igem.wiki/teams/4771/wiki/experiments/7.png)

Figure 12: TSNE plot with outliers detection

{% endblock %}

{% extends "layout.html" %}
{% block title %}Results{% endblock %}
{% block header %}

# {{ self.title() }}

{% endblock %}
{% block page\_content %}
![back to top](https://static.igem.wiki/teams/4771/wiki/logo-light.png)
back to top

[Overview](#Header1)
[Core Contributions](#Header2)
[Result](#Header3)
[Validation](#Header4)

* [Sequence validation](#Sequence validation)
* [Structural validation](#Structural validation)

# Overview

The project we have developed is a comprehensive automated monoclonal antibody drug design system. Currently, we have 55 species covered by our heavy chain design and 9 commonly used species covered by our light chain design. Inspired by the AI tools related to humanization of murine antibodies, we have innovated and extended them to more species. Our project starts with the other sources of antibody sequences against target species antigen (such as mouse-derived antibodies), automatically generates antibody sequences that may have better efficacy and lower immunogenicity, and performs comprehensive scoring.

Users can enter an antibody data based on existing experimental results, or select the antibody data with a specific number for a specific species from the antibody database we provide. At the same time, the user is required to provide the name of the target species for which the antibody is desired. Our model will provide the user with multiple potential antibody data. These antibody data preserve the specific binding ability of the original sequence to a particular antigen, while minimizing the likelihood of eliciting an immune response from the target species. We will also provide multiple scoring metrics for these antibody sequences as well as the original sequence.

# Core Contributions

1. Database construction: Through independent collection and screening of antibody data, a database covering multiple species was constructed.

2. Model application: The Deep One Class model in image anomaly detection was used to score antibody data by species, solving the problem of sample data imbalance.

3. Structural scoring interface: A structural scoring interface was introduced to call the IgFold model and use TM-Score and RMSD to score the similarity of protein PDB files.

In addition to the immunogenicity-related species specification scores at the core of the project, our generation and evaluation can be divided into two main parts:

In the sequence generation part, we synthesized the characteristics of the input antibody data as well as the statistical patterns of the antibody data of the target species, and adopted the strategy of conservative mutation to determine the amino acids with higher probability at each site, and then generated a library of potential antibody data sequences by means of permutations and combinations.

In the structure scoring part, we folded the original sequences and the generated potential sequences through the IGFOLD model to determine whether the generated sequences can be folded into a complete structure, and visualized the differences between the structures of the original sequences and those of the generated sequences.

# Result

Here is an example of what our software would get.

First, in the sequence generation phase, we obtained a large amount of potential sequence space.

![Sequence generation](https://static.igem.wiki/teams/4771/wiki/result/1.png)

Figure 1: Sequence generation

![software output](https://static.igem.wiki/teams/4771/wiki/result/2.png)

Figure 2: Software output

We further put these sequences into the model for sequence scoring.

![sequence scoring](https://static.igem.wiki/teams/4771/wiki/result/3.png)

Figure 3: Sequence scoring

Further, we will perform structural scoring. First of all, from the pdb file, the sequence was successfully folded into the typical structure of the antibody, and it was relatively close to the original CDR region structure. In order to better measure the similarity, we used the structure score, which uses numbers to better characterize the score of the structure.

![Antibody PDB file](https://static.igem.wiki/teams/4771/wiki/result/4.png)

Figure 4: Antibody PDB file

![Structure scores](https://static.igem.wiki/teams/4771/wiki/result/5.png)

Figure 5: Structure scores

# Validation

### 1. Sequence validation:

Because existing tools can only provide humanization scores, we adapted our model for multi-species scoring to score human sequences and validated it by comparison with existing tools. In order to verify that our generated sequences are plausible and to validate the reliability of our species specification scores, we analyzed the generated sequences in terms of available humanization tools and antibody structures, respectively.

We attempted to validate the results of our project in conjunction with existing antibody humanization tools.

The first core of the validation is whether the sequences generated based on statistical regularity and conserved mutations satisfy the mutability and conservatism of the existing humanization tools in the humanization process sites.

>

We chose a murine antibody sequence with a complete heavy chain sequence of:

```
              EVMLVESGGGLVMPGGSLKLSCAASGFTFSNYAMSWVRQIPEKRLEWVATISIGGHFTYFPDSVKGRFTISRDNAKNTLYLRMSSLRSEDTAMYYCVRHEGYGRPYFDYWGQGTTLTVSS
```

We used the current mainstream antibody humanization tool BioPhi for humanization and obtained the following results.

![ BioPhi results](https://static.igem.wiki/teams/4771/wiki/result/6.png)

Figure 6: BioPhi heavy chain results

The possible amino acids for each site of the FR sequence we generated are as follows, and we set a maximum of two possible amino acids for each site in order to limit the size of the generated sequence space as well as to ensure as many conserved mutations as possible.

![possible amino acids for each site of the FR sequence we generate](https://static.igem.wiki/teams/4771/wiki/result/7.png)

Figure 7: Possible amino acids for each site of the FR sequence we generate

We compare and analyze the original sequence, the humanization results given by BioPhi, and the humanization results from our project.

![Result comparison](https://static.igem.wiki/teams/4771/wiki/result/8.png)

Figure 8: Result comparison between ours and BioPhi

In the above figure, the first row shows the FR fragment of the original sequence, the second row shows the humanization results given by BioPhi, and the third and fourth rows show the humanization results given by us. The reddish colored sites in the second row of the sequence are the sites that BioPhi considered necessary to mutate in the original antibody sequence, and the reddish colored sites in the third and fourth rows of the sequence are the sites that BioPhi and we considered necessary to mutate in our results.

The comparison shows that our model offers more possibilities compared to BioPhi, and the permutations can be combined in a way that gives both data more in line with the human antibody pattern and results that are as conservative as possible in terms of mutations, and this conservatism will be reflected in the sequence scoring of our model.

Comparing the CDR region, BioPhi tends to keep the CDR region of the mouse-derived antibody, and the red position of the humanized results it gives maintains the invariance of the CDR part of the sequence as much as possible, which is the same idea as that of our antibody sequence generation to ensure that the antibody clock still maintains its proper specific recognition and binding effect after speciation.

In addition, analyzing from the scoring point of view, BioPhi gave the result of 0.69 OASis identity for the original sequence and 0.81 OASis identity score for the humanized one.

![BioPhi humanization result](https://static.igem.wiki/teams/4771/wiki/result/9.png)

Figure 9: BioPhi heavy chain humanization result

Our model, on the other hand, although scoring the overall sequence higher, still improves the score of the original sequence from 0.81 to 0.99.

Because the details of different model scoring are slightly different, but from the point of view of the score enhancement, our model gives a more reasonable result.

In the following, we validated the light chain results, and we selected a special mouse-derived antibody in order to form a distinction with the multiple mutations in the heavy chain validation, and to validate the accuracy of our model from multiple perspectives.

This antibody has already shown extremely high human origin in the BioPhi test results, and its probability of human origin is basically 100%.

![BioPhi result](https://static.igem.wiki/teams/4771/wiki/result/10.png)

Figure 10: BioPhi light chain result

After humanization, BioPhi did not make any mutations to the original sequence.

![BioPhi humanization result](https://static.igem.wiki/teams/4771/wiki/result/11.png)

Figure 11: BioPhi light chain humanization result

Our model gives just two possible outcomes for this sequence and one of them is the same as the original sequence, the first two data in the figure below are the outcomes we give with ratings of 0.96 and 0.95, and the last sequence is the original sequence, which also has a rating of 0.95.

![Light chain sequence scores generated by our model](https://static.igem.wiki/teams/4771/wiki/result/12.png)

Figure 12: Light chain sequence scores generated by our model

Our model only gives a mutation at one site, i.e., the position of Kabat L100 is mutated from G to Q. Three of the five sequences with the highest multiple human homology given by BioPhi maintain the same results as ours at and around this site.

The second core of the validation is to determine the reasonableness of the speciation scores of the antibody sequences, e.g., to determine the reasonableness between the ordering of the scores of each sequence. Considering that the existing mainstream websites only give one humanization result, while our model gives users more possibilities, we would like to compare the scoring results of our model with the scoring results of BioPhi for the similarity of the ordering.

Considering that our model can output up to 999 results, we randomly selected several sequences with different scores for comparison and analysis.

The sequences we chose are given in the figure below, the first four are the generated results given by us, the yellow marker rows are the humanized results given by BioPhi, the blue marker rows are the original sequences, and the right hand side is the sequence scores given by our model.

![Sequences sampled for comparison and analysis](https://static.igem.wiki/teams/4771/wiki/result/13.png)

Figure 13: Sequences sampled for comparison and analysis

The results of the BioPhi website ratings will be given below, one at a time, in the order in which they appear in the chart above.

![BioPhi result for the 1st sequence](https://static.igem.wiki/teams/4771/wiki/result/14.png)

Figure 14: BioPhi result for the 1st sequence

![BioPhi result for the 2nd sequence](https://static.igem.wiki/teams/4771/wiki/result/15.png)

Figure 15: BioPhi result for the 2nd sequence

![BioPhi result for the 3rd sequence](https://static.igem.wiki/teams/4771/wiki/result/16.png)

Figure 16: BioPhi result for the 3rd sequence

![BioPhi result for the 4th sequence](https://static.igem.wiki/teams/4771/wiki/result/17.png)

Figure 17: BioPhi result for the 4th sequence

![BioPhi result for the 5th sequence](https://static.igem.wiki/teams/4771/wiki/result/18.png)

Figure 18: BioPhi result for the 5th sequence

![BioPhi result for the 6th sequence](https://static.igem.wiki/teams/4771/wiki/result/19.png)

Figure 19: BioPhi result for the 6th sequence

Comparative analysis shows that the results we give are better than those given by BioPhi in terms of OASis Identity, OASis Percentile and Germline Content scores.

Moreover, the ordering of the scores given by us is consistent with the ordering of the scores given by BioPhi.

This confirms the reasonableness of our sequence generation results and sequence scoring results.

### 2. Structural validation

In the structure scoring section, we first fold the original sequence and the generated potential sequence through the IgFold model, determine whether the generated sequence can be folded into a complete structure, and visualize the difference between the structure of the original sequence and the structure of the generated sequence.

After obtaining the PDB files, we compare the similarity of the PDBs of the original and generated sequences by two commonly used PDB similarity comparison methods, TM-Score and RMSD, to give a reasonable structural score.

![Structure scoring](https://static.igem.wiki/teams/4771/wiki/result/20.png)

Figure 20: Structure scoring

It can be seen that the structures of the CDR areas of the three PDB files are relatively similar.

![Structures of the three PDB files](https://static.igem.wiki/teams/4771/wiki/result/21.png)

Figure 21: Structures of the three PDB files

The degree of structural similarity between the two generated sequences is greater. The following are structural comparisons of the original sequence and the generated sequence.

![Structures of the two sequences generated by our model](https://static.igem.wiki/teams/4771/wiki/result/22.png)

Figure 22: Structures of the two sequences generated by our model

![Structures of the original sequence and the 1st sequence generated by our model](https://static.igem.wiki/teams/4771/wiki/result/23.png)

Figure 23: Structures of the original sequence and the 1st sequence generated by our model

![Structures of the original sequence and the 2nd sequence generated by our model](https://static.igem.wiki/teams/4771/wiki/result/24.png)

Figure 24: Structures of the original sequence and the 2nd sequence generated by our model

TM-Score is above 0.99 and RMSD is about 0.2. Because the output sequence still maintains structural similarity with the original sequence in the CDR region, this means that most of the antigen-binding region is maintained, and the antigen-antibody binding ability is still high.

This means that the sequence we output can not only fold into the correct antibody structure, but also has guaranteed antigen-antibody binding ability.

{% endblock %}