Package: deeprank2
DeepRank2Badgesfairnesspackagedocstestsrunning onlicenseOverviewDeepRank2 is an open-source deep learning (DL) framework for data mining of protein-protein interfaces (PPIs) or single-residue variants (SRVs). This package is an improved and unified version of three previously developed packages:DeepRank,DeepRank-GNN, andDeepRank-Mut.DeepRank2 allows for transformation of (pdb formatted) molecular data into 3D representations (either grids or graphs) containing structural and physico-chemical information, which can be used for training neural networks. DeepRank2 also offers a pre-implemented training pipeline, using eitherCNNs(for grids) orGNNs(for graphs), as well as output exporters for evaluating performances.Main features:Predefined atom-level and residue-level feature typese.g. atom/residue type, charge, size, potential energyAll features' documentation is availableherePredefined target typesbinary class, CAPRI categories, DockQ, RMSD, and FNATDetailed docking scores documentation is availablehereFlexible definition of both new features and targetsFeatures generation for both graphs and gridsEfficient data storage in HDF5 formatSupport for both classification and regression (based onPyTorchandPyTorch Geometric)📚Documentation📣DiscussionsTable of contentsDeepRank2OverviewTable of contentsInstallationContainerized InstallationLocal/remote installationYML file installationManual installationTesting DeepRank2 installationContributingUsing DeepRank2Data generationDatasetsGraphDatasetGridDatasetTrainingRun a pre-trained model on new dataComputational performancesPackage developmentInstallationThere are two ways to install DeepRank2:In adockerized container. This allows you to use DeepRank2, including all the notebooks within the container (a protected virtual space), without worrying about your operating system or installation of dependencies.We recommend this installation for inexperienced users and to learn to use or test our software, e.g. using the providedtutorials. However, resources might be limited in this installation and we would not recommend using it for large datasets or on high-performance computing facilities.Local installationon your system. This allows you to use the full potential of DeepRank2, but requires a few additional steps during installation.We recommend this installation for more experienced users, for larger projects, and for (potential)contributorsto the codebase.Containerized InstallationIn order to try out the package without worrying about your OS and without the need of installing all the required dependencies, we created aDockerfilethat can be used for taking care of everything in a suitable container.For this, you first need to installDockeron your system. Then run the following commands. You may need to have sudo permission for some steps, in which case the commands below can be preceded bysudo:# Clone the DeepRank2 repository and enter its root directorygitclonehttps://github.com/DeepRank/deeprank2cddeeprank2# Build and run the Docker imagedockerbuild-tdeeprank2.
dockerrun-p8888:8888deeprank2Next, open a browser and go tohttp://localhost:8888to access the application running inside the Docker container. From there you can use DeepRank2, e.g. to run the tutorial notebooks.More details about the tutorials' contents can be foundhere. Note that in the docker container only the raw PDB files are downloaded, which needed as a starting point for the tutorials. You can obtain the processed HDF5 files by running thedata_generation_xxx.ipynbnotebooks. Because Docker containers are limited in memory resources, we limit the number of data points processed in the tutorials. Pleaseinstall the package locallyto fully leverage its capabilities.If after running the tutorials you want to remove the (quite large) Docker image from your machine, you must firststop the containerand can thenremove the image. More general information about Docker can be found on theofficial website docs.Local/remote installationLocal installation is formally only supported on the latest stable release of ubuntu, for which widespread automated testing through continuous integration workflows has been set up. However, it is likely that the package runs smoothly on other operating systems as well.Before installing DeepRank2 please ensure you haveGCCinstalled: if runninggcc --versiongives an error, runsudo apt-get install gcc.YML file installationYou can use the provided YML file for creating aconda environmentcontaining the latest stable release of DeepRank2 and all its dependencies.
This will install the CPU-only version of DeepRank2 on Python 3.10.
Note that this will not work for MacOS. Do theManual Installationinstead.# Clone the DeepRank2 repository and enter its root directorygitclonehttps://github.com/DeepRank/deeprank2cddeeprank2# Ensure you are in your base environmentcondaactivate# Create the environmentcondaenvcreate-fenv/environment.yml# Activate the environmentcondaactivatedeeprank2See instructions below totestthat the installation was succesful.Manual installationIf you want to use the GPUs, choose a specific python version, are a MacOS user, or if the YML installation was not succesful, you can install the package manually. We advise to do this inside aconda virtual environment.
If you have any issues during installation of dependencies, please refer to the official documentation for each package (linked below), as our instructions may be out of date (last tested on 19 Jan 2024):DSSP 4:conda install -c sbl dsspMSMS:conda install -c bioconda msmsHerefor MacOS with M1 chip users.PyTorch:conda install pytorch torchvision torchaudio cpuonly -c pytorchPytorch regularly publishes updates and not all newest versions will work stably with DeepRank2. Currently, the package is tested usingPyTorch 2.1.1.We support torch's CPU library as well as CUDA.PyGand its optional dependencies:torch_scatter,torch_sparse,torch_cluster,torch_spline_conv.The exact command to install pyg will depend on the version of pytorch you are using. Please refer to the source's installation instructions (we recommend using the pip installation for this as it also shows the command for the dependencies).For MacOS with M1 chip users: installthe conda version of PyTables.Finally install deeprank2 itself:pip install deeprank2.Alternatively, get the latest updates by cloning the repo and installing the editable version of the package with:gitclonehttps://github.com/DeepRank/deeprank2cddeeprank2
pipinstall-e.'[test]'Thetestextra is optional, and can be used to install test-related dependencies, useful during development.Testing DeepRank2 installationYou can check that all components were installed correctly, using pytest. We especially recommend doing this in case you installed DeepRank2 and its dependencies manually (the latter option above).The quick test should be sufficient to ensure that the software works, while the full test (a few minutes) will cover a much broader range of settings to ensure everything is correct.First runpip install pytest, if you did not install it above. Then runpytest tests/test_integration.pyfor the quick test or justpytestfor the full test (expect a few minutes to run).ContributingIf you would like to contribute to the package in any way, please seeour guidelines.Using DeepRank2The following section serves as a first guide to start using the package, using protein-protein Interface (PPI) queries as example. For an enhanced learning experience, we provide in-depthtutorial notebooksfor generating PPI data, generating SVR data, and for the training pipeline.
For more details, see theextended documentation.Data generationFor each protein-protein complex (or protein structure containing a missense variant), aQuerycan be created and added to theQueryCollectionobject, to be processed later on. Two subtypes ofQueryexist:ProteinProteinInterfaceQueryandSingleResidueVariantQuery.AQuerytakes as inputs:a.pdbfile, representing the protein-protein structure,the resolution ("residue"or"atom"), i.e. whether each node should represent an amino acid residue or an atom,the ids of the chains composing the structure, andoptionally, the correspondent position-specific scoring matrices (PSSMs), in the form of.pssmfiles.fromdeeprank2.queryimportQueryCollection,ProteinProteinInterfaceQueryqueries=QueryCollection()# Append data pointsqueries.add(ProteinProteinInterfaceQuery(pdb_path="tests/data/pdb/1ATN/1ATN_1w.pdb",resolution="residue",chain_ids=["A","B"],targets={"binary":0},pssm_paths={"A":"tests/data/pssm/1ATN/1ATN.A.pdb.pssm","B":"tests/data/pssm/1ATN/1ATN.B.pdb.pssm"}))queries.add(ProteinProteinInterfaceQuery(pdb_path="tests/data/pdb/1ATN/1ATN_2w.pdb",resolution="residue",chain_ids=["A","B"],targets={"binary":1},pssm_paths={"A":"tests/data/pssm/1ATN/1ATN.A.pdb.pssm","B":"tests/data/pssm/1ATN/1ATN.B.pdb.pssm"}))queries.add(ProteinProteinInterfaceQuery(pdb_path="tests/data/pdb/1ATN/1ATN_3w.pdb",resolution="residue",chain_ids=["A","B"],targets={"binary":0},pssm_paths={"A":"tests/data/pssm/1ATN/1ATN.A.pdb.pssm","B":"tests/data/pssm/1ATN/1ATN.B.pdb.pssm"}))The user is free to implement a custom query class. Each implementation requires thebuildmethod to be present.The queries can then be processed into graphs only or both graphs and 3D grids, depending on which kind of network will be used later for training.fromdeeprank2.featuresimportcomponents,conservation,contact,exposure,irc,surfaceareafromdeeprank2.utils.gridimportGridSettings,MapMethodfeature_modules=[components,conservation,contact,exposure,irc,surfacearea]# Save data into 3D-graphs onlyhdf5_paths=queries.process("<output_folder>/<prefix_for_outputs>",feature_modules=feature_modules)# Save data into 3D-graphs and 3D-gridshdf5_paths=queries.process("<output_folder>/<prefix_for_outputs>",feature_modules=feature_modules,grid_settings=GridSettings(# the number of points on the x, y, z edges of the cubepoints_counts=[20,20,20],# x, y, z sizes of the box in Åsizes=[1.0,1.0,1.0]),grid_map_method=MapMethod.GAUSSIAN)DatasetsData can be split in sets implementing custom splits according to the specific application. Assuming that the training, validation and testing ids have been chosen (keys of the HDF5 file/s), then theDeeprankDatasetobjects can be defined.GraphDatasetFor training GNNs the user can create aGraphDatasetinstance:fromdeeprank2.datasetimportGraphDatasetnode_features=["bsa","res_depth","hse","info_content","pssm"]edge_features=["distance"]target="binary"train_ids=[<ids>]valid_ids=[<ids>]test_ids=[<ids>]# Creating GraphDataset objectsdataset_train=GraphDataset(hdf5_path=hdf5_paths,subset=train_ids,node_features=node_features,edge_features=edge_features,target=target)dataset_val=GraphDataset(hdf5_path=hdf5_paths,subset=valid_ids,train_source=dataset_train)dataset_test=GraphDataset(hdf5_path=hdf5_paths,subset=test_ids,train_source=dataset_train)GridDatasetFor training CNNs the user can create aGridDatasetinstance:fromdeeprank2.datasetimportGridDatasetfeatures=["bsa","res_depth","hse","info_content","pssm","distance"]target="binary"train_ids=[<ids>]valid_ids=[<ids>]test_ids=[<ids>]# Creating GraphDataset objectsdataset_train=GridDataset(hdf5_path=hdf5_paths,subset=train_ids,features=features,target=target)dataset_val=GridDataset(hdf5_path=hdf5_paths,subset=valid_ids,train_source=dataset_train,)dataset_test=GridDataset(hdf5_path=hdf5_paths,subset=test_ids,train_source=dataset_train,)TrainingLet's define aTrainerinstance, using for example of the already existingGINet. BecauseGINetis a GNN, it requires a dataset instance of typeGraphDataset.fromdeeprank2.trainerimportTrainerfromdeeprank2.neuralnets.gnn.naive_gnnimportNaiveNetworktrainer=Trainer(NaiveNetwork,dataset_train,dataset_val,dataset_test)The same can be done using a CNN, for exampleCnnClassification. Here a dataset instance of typeGridDatasetis required.fromdeeprank2.trainerimportTrainerfromdeeprank2.neuralnets.cnn.model3dimportCnnClassificationtrainer=Trainer(CnnClassification,dataset_train,dataset_val,dataset_test)By default, theTrainerclass creates the folder./outputfor storing predictions information collected later on during training and testing.HDF5OutputExporteris the exporter used by default, but the user can specify any other implemented exporter or implement a custom one.Optimizer (torch.optim.Adamby default) and loss function can be defined by using dedicated functions:importtorchtrainer.configure_optimizers(torch.optim.Adamax,lr=0.001,weight_decay=1e-04)Then theTrainercan be trained and tested; the best model in terms of validation loss is saved by default, and the user can modify so or indicate where to save it using thetrain()method parameterfilename.trainer.train(nepoch=50,batch_size=64,validate=True,filename="<my_folder/model.pth.tar>")trainer.test()Run a pre-trained model on new dataIf you want to analyze new PDB files using a pre-trained model, the first step is to process and save them into HDF5 filesas we have done above.Then, theDeeprankDatasetinstance for the newly processed data can be created. Do this by specifying the path for the pre-trained model intrain_source, together with the path to the HDF5 files just created. Note that there is no need of setting the dataset's parameters, since they are inherited from the information saved in the pre-trained model. Let's suppose that the model has been trained withGraphDatasetobjects:fromdeeprank2.datasetimportGraphDatasetdataset_test=GraphDataset(hdf5_path="<output_folder>/<prefix_for_outputs>",train_source="<pretrained_model_path>")Finally, theTrainerinstance can be defined and the new data can be tested:fromdeeprank2.trainerimportTrainerfromdeeprank2.neuralnets.gnn.naive_gnnimportNaiveNetworkfromdeeprank2.utils.exportersimportHDF5OutputExportertrainer=Trainer(NaiveNetwork,dataset_test=dataset_test,pretrained_model="<pretrained_model_path>",output_exporters=[HDF5OutputExporter("<output_folder_path>")])trainer.test()For more details about how to run a pre-trained model on new data, see thedocs.Computational performancesWe measured the efficiency of data generation in DeepRank2 using the tutorials'PDB files(~100 data points per data set), averaging the results run on Apple M1 Pro, using a single CPU.
Parameter settings were: atomic resolution,distance_cutoffof 5.5 Å, radius (for SRV only) of 10 Å. Thefeatures modulesused werecomponents,contact,exposure,irc,secondary_structure,surfacearea, for a total of 33 features for PPIs and 26 for SRVs (the latter do not useircfeatures).Data processing speed[seconds/structure]Memory[megabyte/structure]PPIsgraph only:2.99(std 0.23)graph+grid:11.35(std 1.30)graph only:0.54(std 0.07)graph+grid:16.09(std 0.44)SRVsgraph only:2.20(std 0.08)graph+grid:2.85(std 0.10)graph only:0.05(std 0.01)graph+grid:17.52(std 0.59)Package developmentIf you're looking for developer documentation, gohere.