Update README.md

README.md

@@ -6,15 +6,15 @@ license: apache-2.0
 ZymCTRL ([Paper presented @ Machine Learning for Structural Biology workshop - December 2022](https://www.mlsb.io/papers_2022/ZymCTRL_a_conditional_language_model_for_the_controllable_generation_of_artificial_enzymes.pdf)) 
 is a conditional language model for the generation of artificial functional enzymes. It was trained on the entire BRENDA database of enzymes, comprising over 37 M sequences. 
 Given a user-defined Enzymatic Commission (EC) number, the model generates protein sequences that fulfill that catalytic reaction. 
-The generated sequences are ordered, globular and distant to natural ones, while their intended catalytic properties match those defined by users.
+The generated sequences are ordered, globular, and distant to natural ones, while their intended catalytic properties match those defined by users.
 
-If you don't know what EC number of your protein of interest, have a look at the BRENDA webpage: https://www.brenda-enzymes.org/ecexplorer.php?browser=1
+If you don't know the EC number of your protein of interest, have a look at the BRENDA webpage: https://www.brenda-enzymes.org/ecexplorer.php?browser=1
 
-See below information about the model, how to generate sequences, and how to save and rank them by perplexity.
+See below for information about the model, how to generate sequences, and how to save and rank them by perplexity.
 
 ## **Model description**
 ZymCTRL is based on the [CTRL Transformer](https://arxiv.org/abs/1909.05858) architecture (which in turn is very similar to ChatGPT) and contains 36 layers
-with a model dimensionality of 1280, totalling 738 million parameters. 
+with a model dimensionality of 1280, totaling 738 million parameters. 
 
 ZymCTRL is a decoder-only transformer model pre-trained on the BRENDA database
 (version July 2022). The pre-training was done on the raw sequences without FASTA headers,
@@ -22,10 +22,10 @@ with the EC classes prepended to each sequence. The databases will be uploaded s
 
 ZymCTRL was trained with an autoregressive objective, i.e., the model learns to predict
 the next token given a sequence context. Because the first tokens on each sequence encode the EC numbers,
-the model learns the dependencies among EC classes and their corresponding sequences, and is able to _speak_ the enzyme language.
+the model learns the dependencies among EC classes and their corresponding sequences and is able to _speak_ the enzyme language.
 
-There are stark differences in the number of members among EC classes, and for this reason we also tokenized the EC numbers.
-In this manner, EC numbers '2.7.1.1' and '2.7.1.2' share the first three tokens (six including separators) and hence the model can infer that
+There are stark differences in the number of members among EC classes, and for this reason, we also tokenized the EC numbers.
+In this manner, EC numbers '2.7.1.1' and '2.7.1.2' share the first three tokens (six, including separators), and hence the model can infer that
 there are relationships between the two classes.
 
 The figure below summarizes the process of training:
@@ -40,10 +40,10 @@ Detailed installation instructions can be found here: https://huggingface.co/doc
 Since ZymCTRL has been trained on the classical language model objective on enzyme sequences with their EC annotation,
 it particularly excels at generating enzyme sequences given a user-defined EC class, such as alcohol dehydrogenases ('1.1.1.2').
 
-The model can generate in two ways: in a zero-shot fashion, i.e directly generating from the checkpoint weights; or after fine-tuning. 
-Fine-tuning allows to augment the BRENDA datasets that were using during training, for example, 
-if you have a curated internal dataset, or a set of ancestrally-reconstructed sequences. This is entirely optional. One advantage of 
-running the model in zero-shot, is that it doesn't require any further training.
+The model can generate in two ways: in a zero-shot fashion, i.e., directly generating from the checkpoint weights, or after fine-tuning. 
+Fine-tuning allows augmenting the BRENDA datasets that were used during training, for example, 
+if you have a curated internal dataset or a set of ancestrally-reconstructed sequences. This is entirely optional. One advantage of 
+running the model in zero-shot is that it doesn't require any further training.
 
 
 ### **Example 1: Generating nitrilases (EC 3.5.5.1)**  
@@ -51,19 +51,19 @@ running the model in zero-shot, is that it doesn't require any further training.
 The script below will be used for the generation of any BRENDA class in a zero-shot fashion, 
 here we showcase the generation of novel dehalogenases.
 
-To run this script you should download ZymCTRL to a local folder in your workstation. 
+To run this script, you should download ZymCTRL to a local folder in your workstation. 
 Then replace the placeholders in the script with your actual folder path.
 
 You can run it directly in the command line (once you have hugging face installed), 
-with the following command: `python generate.py`.
+with the following command: `python generate.py`
 
 The script will write each sequence in a fasta file in the folder you specify. In the fasta header,
 it will store the sequence's computed perplexity value. Perplexity is a measure of the model's confidence
 in that generation, with lower values being better. The sequences are ordered by perplexity before writing them out,
 so those that finish in *_0.fasta and *_1.fasta will be the best ones per batch. 
 
-**Given that generation runs so fast, we recommend to generate hundreds or thousands and then only pick the best 5%. 
-With the script below that would mean picking only those that finish in '_0.fasta'**
+**Given that generation runs so fast, we recommend generating hundreds or thousands and then only picking the best 5%. 
+With the script below, that would mean picking only those that finish in '_0.fasta'**
 
 ```
 import torch
@@ -73,7 +73,7 @@ from tqdm import tqdm
 import math
 
 def remove_characters(sequence, char_list):
-    "This function removes special tokens used during training"
+    "This function removes special tokens used during training."
     columns = sequence.split('<sep>')
     seq = columns[1]
     for char in char_list:
@@ -144,18 +144,18 @@ if __name__=='__main__':
 
 ## **Example 2: Fine-tuning on a set of user-defined sequences**  
 
-This alternative to the zero-shot generation allows to update ZymCTRL's weights to new sequences.
+This alternative to the zero-shot generation allows updating ZymCTRL's weights to new sequences.
 
 This strategy is not strictly necessary, in fact, we have observed good generations even for EC classes where there are 
-only 1-2 representatives in Nature. But you might have an internal set of sequences that you'd like to incorporate to the model.
+only 1-2 representatives in Nature. But you might have an internal set of sequences that you'd like to incorporate into the model.
 For example, internal datasets after protein engineering efforts, 
 ancestrally-reconstructed sets, or after searching against metagenomics databases. In these cases, it is advisable to fine-tune ZymCTRL,
 as it will learn new properties from your dataset and potentially improve the generation quality 
 (especially for poorly populated EC classes).
 
-To fine-tune ZymCTRL, you will need to process your sequences quite a bit. With the scripts below can exactly do that without many
-modifications. The only requisite is to start with an input file 'sequences.fasta' which contain all the sequences in a fasta format. 
-We recommend using at least 200 sequences to obtain best results. But we've seen it working with fewer sequences, so if you don't have
+To fine-tune ZymCTRL, you will need to process your sequences quite a bit. The scripts below can exactly do that without any
+modifications. The only requisite is to start with an input file, 'sequences.fasta' which contains all the sequences in a fasta format. 
+We recommend using at least 200 sequences to obtain the best results. But we've seen it working with fewer sequences, so if you don't have
 that many, give it still a go.
 
 
@@ -372,14 +372,14 @@ https://github.com/huggingface/transformers/blob/master/examples/pytorch/languag
 
 
 ### **Training specs**
-The model was trained on 48 NVIDIA A100 GPUs for 8 epochs, 
-using a block size of 1024, and a total batch size of 768. 
+The model was trained on 48 NVIDIA A100 GPUs for eight epochs, 
+using a block size of 1024 and a total batch size of 768. 
 The optimizer used was Adam (beta1 = 0.9, beta2 = 0.999) 
 with a learning rate of 0.8e-04.
 
 ### **Contact**
 
 We are the AI for Protein Design group at the Institute of Molecular Biology of Barcelona (https://www.aiproteindesign.com/).
-For any question post an issue in this repository so that other people can benefit from the feedback and I'll get back to you shortly.
+For any questions post an issue in this repository so that other people can benefit from the feedback, and I'll get back to you shortly.
 We are always open for collaborations, send an email to nfccri [at] ibmb [dot] csic [dot] es