Package: deepbgc
DeepBGC: Biosynthetic Gene Cluster detection and classificationDeepBGC detects BGCs in bacterial and fungal genomes using deep learning.
DeepBGC employs a Bidirectional Long Short-Term Memory Recurrent Neural Network
and a word2vec-like vector embedding of Pfam protein domains.
Product class and activity of detected BGCs is predicted using a Random Forest classifier.📌 News 📌DeepBGC 0.1.23: Predicted BGCs can now be uploaded for visualization inantiSMASHusing a JSON output fileInstall and run DeepBGC as usual based on instructions belowUploadantismash.jsonfrom the DeepBGC output folder using "Upload extra annotations" on theantiSMASHpagePredicted BGC regions and their prediction scores will be displayed alongside antiSMASH BGCsPublicationsA deep learning genome-mining strategy for biosynthetic gene cluster predictionGeoffrey D Hannigan,  David Prihoda et al., Nucleic Acids Research, gkz654,https://doi.org/10.1093/nar/gkz654Install using conda (recommended)You can install DeepBGC usingCondaor one of the alternatives (Miniconda,Miniforge).Set up Bioconda and Conda-Forge channels:condaconfig--addchannelsbioconda
condaconfig--addchannelsconda-forgeInstall DeepBGC using:# Create a separate DeepBGC environment and install dependenciescondacreate-ndeepbgcpython=3.7hmmerprodigal# Install DeepBGC into the environment using pipcondaactivatedeepbgc
pipinstalldeepbgc# Alternatively, install everything using conda (currently unstable due to conda conflicts)condainstalldeepbgcInstall dependencies manually (if conda is not available)If you don't mind installing the HMMER and Prodigal dependencies manually, you can also install DeepBGC using pip:Install Python version 3.6 or 3.7 (Note:Python 3.8 is not supporteddue to Tensorflow < 2.0 dependency)Install Prodigal and put theprodigalbinary it on your PATH:https://github.com/hyattpd/Prodigal/releasesInstall HMMER and put thehmmscanandhmmpressbinaries on your PATH:http://hmmer.org/download.htmlRunpip install deepbgcto install DeepBGCUse DeepBGCDownload models and Pfam databaseBefore you can use DeepBGC, download trained models and Pfam database:deepbgcdownloadYou can display downloaded dependencies and models using:deepbgcinfoDetection and classificationDetect and classify BGCs in a genomic sequence.
Proteins and Pfam domains are detected automatically if not already annotated (HMMER and Prodigal needed)# Show command help docsdeepbgcpipeline--help# Detect and classify BGCs in mySequence.fa using DeepBGC detector.deepbgcpipelinemySequence.fa# Detect and classify BGCs in mySequence.fa using custom DeepBGC detector trained on your own data.deepbgcpipeline--detectorpath/to/myDetector.pklmySequence.faThis will produce amySequencedirectory with multiple files and a README.txt with file descriptions.SeeTrain DeepBGC on your own datasection below for more information about training a custom detector or classifier.Example outputSee theDeepBGC Example Result Notebook.
Data can be downloaded on thereleases pageTrain DeepBGC on your own dataYou can train your own BGC detection and classification models, seedeepbgc train --helpfor documentation and examples.Training and validation data can be found inrelease 0.1.0andrelease 0.1.5. You will need:Positive (BGC) training data - In most cases, this is your own BGC training set, see "Preparing training data" section belowNegative (Non-BGC) training data - Needed for BGC detection. You can useGeneSwap_Negatives.pfam.tsvfrom releasehttps://github.com/Merck/deepbgc/releases/tag/v0.1.0Validation data - Needed for BGC detection. Contigs with annotated BGC and non-BGC regions. A working example can be downloaded fromhttps://github.com/Merck/deepbgc/releases/tag/v0.1.5Trained Pfam2vec vectors - "Vocabulary" converting Pfam IDs to meaningful numeric vectors, you can reuse previously trainedpfam2vec.csvresults fromhttps://github.com/Merck/deepbgc/releases/tag/v0.1.0JSON configuration files - See JSON section belowIf you have any questions about using or training DeepBGC, feel free to submit an issue.Preparing training dataThe training examples need to be prepared in Pfam TSV format, which can be prepared from your sequence
usingdeepbgc prepare.First, you will need to manually add anin_clustercolumn that will contain 0 for pfams outside a BGC
and 1 for pfams inside a BGC. We recommend preparing a separate negative TSV and positive TSV file,
where the column will be equal to all 0 or 1 respectively.Finally, you will need to manually add asequence_idcolumn ,
which will identify a continuous sequence of Pfams from a single sample (BGC or negative sequence).
The samples are shuffled during training to present the model with a random order of positive and negative samples.
Pfams with the samesequence_idvalue will be kept together. For example, if your training set contains multiple BGCs, thesequence_idcolumn should contain the BGC ID.! New in version 0.1.17 !You can now prepareproteinFASTA sequences into a Pfam TSV file usingdeepbgc prepare --protein.JSON model training template filesDeepBGC is using JSON template files to define model architecture and training parameters. All templates can be downloaded inrelease 0.1.0.JSON template for DeepBGC LSTMdetectorwith pfam2vec is structured as follows:{
  "type": "KerasRNN", - Model architecture (KerasRNN/DiscreteHMM/GeneBorderHMM)
  "build_params": { - Parameters for model architecture
    "batch_size": 16, - Number of splits of training data that is trained in parallel 
    "hidden_size": 128, - Size of vector storing the LSTM inner state
    "stateful": true - Remember previous sequence when training next batch
  },
  "fit_params": {
    "timesteps": 256, - Number of pfam2vec vectors trained in one batch
    "validation_size": 0, - Fraction of training data to use for validation (if validation data is not provided explicitly). Use 0.2 for 20% data used for testing.
    "verbose": 1, - Verbosity during training
    "num_epochs": 1000, - Number of passes over your training set during training. You probably want to use a lower number if not using early stopping on validation data.
    "early_stopping" : { - Stop model training when at certain validation performance
      "monitor": "val_auc_roc", - Use validation AUC ROC to observe performance
      "min_delta": 0.0001, - Stop training when the improvement in the last epochs did not improve more than 0.0001
      "patience": 20, - How many of the last epochs to check for improvement
      "mode": "max" - Stop training when given metric stops increasing (use "min" for decreasing metrics like loss)
    },
    "shuffle": true, - Shuffle samples in each epoch. Will use "sequence_id" field to group pfam vectors belonging to the same sample and shuffle them together 
    "optimizer": "adam", - Optimizer algorithm
    "learning_rate": 0.0001, - Learning rate
    "weighted": true - Increase weight of less-represented class. Will give more weight to BGC training samples if the non-BGC set is larger.
  },
  "input_params": {
    "features": [ - Array of features to use in model, see deepbgc/features.py
      {
        "type": "ProteinBorderTransformer" - Add two binary flags for pfam domains found at beginning or at end of protein
      },
      {
        "type": "Pfam2VecTransformer", - Convert pfam_id field to pfam2vec vector using provided pfam2vec table
        "vector_path": "#{PFAM2VEC}" - PFAM2VEC variable is filled in using command line argument --config
      }
    ]
  }
}JSON template for Random Forestclassifieris structured as follows:{
  "type": "RandomForestClassifier", - Type of classifier (RandomForestClassifier)
  "build_params": {
    "n_estimators": 100, - Number of trees in random forest
    "random_state": 0 - Random seed used to get same result each time
  },
  "input_params": {
    "sequence_as_vector": true, - Convert each sample into a single vector
    "features": [
      {
        "type": "OneHotEncodingTransformer" - Convert each sequence of Pfams into a single binary vector (Pfam set)
      }
    ]
  }
}Using your trained modelSince version0.1.10you can provide a direct path to the detector or classifier model like so:deepbgcpipeline\mySequence.fa\--detectorpath/to/myDetector.pkl\--classifierpath/to/myClassifier.pkl