Package: gaddlemaps
GADDLE Maps: General Algorithm for Discrete Object Deformations Based on Local Exchange MapsInstallation|Quick Start|Features|FAQs|Documentation|Cite usWhat is GADDLE Maps?GADDLE Mapsis a general algorithm
to switch between structures with different resolutions (i.e.different number
of particles per object). This algorithm is completely automatic and does not
require any human input. The algorithm has been designed specifically to
transform molecules between to levels of resolution (for example from united
atom to all atom or from coarse grain to fully atomistic). However the algorithm
works for any structure than can be represented by a graph.The algorithm apply rotations and deformations to the graph (molecule) preserving the
distance between its nodes (atoms) in order to maximize the overlapping between
the graphs (molecules) in both resolutions. Once this is achieved, it generates
anexchange mapthat transforms between both resolutions. These exchange maps
use local information about the location of the neighbours to exchange between
the 2 resolutions. This allows the algorithm to adapt to multiple instances of
the same graph (molecule) in different configurations. Using this property the
algorithm is able to transform subsequent instances of the same graph (molecule)
at almost zero cost. For these reason the algorithm is specially suitable to
transform between systems in coarse grained and fully atomistic resolutions
because by only aligning one molecule of each specie the whole system can be
transformed.IMPORTANTIf you are experiencing problems during the mapping process (the
mapping gets stuck at some point, the final system looks weird...) check theFAQssection because most common mistakes are covered there.The gaddlemaps moduleIn this module we offer the reference implementation of the algorithm in python
with a series of helper classes and functions for an easy usage of the
algorithm. Moreover a command line interface and a jupyter notebook is provided
for an easy and visual way of performing the transformation.Besides a port of the algorithm to c++ with an interface with python usingcython. This port allows for a boosted performance,
specially for large molecules such as proteins, but the results are the same
than as the reference implementation. This port can be installed alongside
the module.gaddlemapswill automatically use the c++ port if available and will
fallback to the python implementation if not present. Therefore the installation
of this c++ port is completelyoptionalbut highly recommended.ContentInstallationQuick StartFrequently asked questionsInstallationHere we will describe the installation of the reference implementation ofgaddlemaps(i.e.without the c++ backend). In order to install the
backend please read the sectionconfiguring the c++
backendfirst.The installation can be done usingpiporbuilding from
source. For any of the
methods gaddlemaps needsPython3.6 or greater.pip installationThis is the simplest and prefered method to install the module. In order to
install gaddlemaps just run:pipinstallgaddlemaps--upgradeThis method will autodetect wether the requirements for the optimized c++
backend are available and, if possible, will install the module with the
optimized backend. If the dependencies are not met, it will install the native
python version of the backend. The dependencies necessary for the backend are
detailed in the sectionconfiguring the c++
backend.Building from sourcePrerequisitesIn order to build from source the following prerequisites are needed:NumpyScipymore_itertoolsJupyter Notebooknglview(optional)Installation of prerequisitesThis requisites can installed usingpipby running:pipinstallnumpyscipyjupyterIf usingMac OSthe dependencies can be installed usingbrewby
running:brewinstallnumpyscipyjupyterInUbuntuand other Debian based Linux distribution they can be installed using
theaptpackage manager:sudoaptinstallpython3-numpypython3-scipypython3-notebookIf using a different operative system or if the previous instructions did not
work please refer to the installation instructions in the documentation of each
module.Nglviewis completely optional, it is used only when representing the system
inside jupyter notebooks. If you do not want this functionality you can skip
this step. Moreover this modulecan be installed after installing
gaddlemaps. The easiest way to install the module is through pippipinstallnglview
jupyternbextensionenable--py--sys-prefixwidgetsnbextension
jupyter-nbextensionenablenglview--py--sys-prefixfor other options and problems with the installation check the documentation fornglview.Cloning and installingThe first step is cloning or downloading the git repositorygitclonehttps://github.com/txemaotero/gaddlemapscdgaddlemapsorwgethttps://github.com/txemaotero/gaddlemaps/archive/master.zip
unzipmaster.zipcdgaddlemaps-masterAfter that the module is installed by running:sudopython3setup.pyinstallThe module can also be installed without admin privileges with:python3setup.pyinstall--userAfter that the module will be installed and the downloaded folder can be safely
removed.Configuring the c++ backendIn order to install the c++ backend the following extra prerequisites are
needed:A c++ compilerArmadilloCythonC++ compilerThe installation of a c++ compiler (for example g++ or clang++) is needed. This
can be done inUbuntuand other Debian-like Linux distribution using the apt package
manager:sudoaptinstallg++If usingMac OSthe compiler can be installed by installing thedeveloper command
line tools. These tools can be installed by running the following command in a
terminal:xcode-select--installand following the instructions on the screen.ArmadilloFor the installation of the Armadillo library please refer to theirinstallation documentation.If using a Debian based Linux distribution it should be possible to install it
using apt:sudoapt-getinstalllibarmadillo-devIn order to check if the installation is in the path create a filearmadillo_test.cppwith the following content:#include<armadillo>#include<iostream>intmain(){arma::Mat<double>A(3,3,arma::fill::eye);std::cout<<"A:\n"<<A<<"\n";return0;}And try to compile and run it:g++armadillo_test.cpp-larmadillo-I/usr/local/include-L/usr/local/lib&&./a.outIf a 3⨉3 identity matrix is printed the armadillo library is installed and in a
location where gaddlemaps will be able to find it. If the compiler is not able
to find the armadillo library update the environment variables in your.bashrc(if using Linux or Mac OS prior to Catalina) or.zshrc(if using zsh instead of bash or if using Mac OS Catalina).In general adding the following lines at the end of the file will de the trick:exportCPLUS_INCLUDE_PATH="folder where armadillo.h is located"exportLD_LIBRARY_PATH="folder where libarmadillo.so  or libarmadillo.dylib"exportLIBRARY_PATH="folder where libarmadillo.so  or libarmadillo.dylib"If armadillo was installed in Mac OS usingmacportsthe following lines should work:exportCPLUS_INCLUDE_PATH="/opt/local/include"exportLIBRARY_PATH="/opt/local/lib"exportLD_LIBRARY_PATH="/opt/local/lib"CythonUsing pipThe easiest and preferred way to install cython is by using pipsudopipinstallcythonWithout pipInMac OScan be installed using brewbrewinstallcythonInUbuntucan be installed using aptsudoaptinstallcython3For other platforms and other installation method check the official
documentation forcython.After finishing the installation of the prerequisites please follow the steps in
theinstallationsection.Quick StartFiles needed to map the systemgaddlemaps uses 2 different kind of filescoordinates filesandtopology
files. The coordinates file stores the coordinates of the atoms while the
topology file stores the connections (i.e.bonds) between the atoms. Currently
the topology file can only store the information for 1 molecule, while the
coordinate file can store the whole system.For doing the whole mapping process the following information is required:The coordinate file with the whole system in the start resolution.One topology file for each specie that you want to map (it is not required to
map all the molecules in the system).For each specie that you want to map a coordinate file with the molecule in
the end resolution (it is possible to use only 1 coordinate file with all the
molecules, but it requires more intervention by the user).One topology file for each specie that you want to map in the end resolution.gaddlemaps has been developed usingGROMACSas
reference molecular dynamics simulation suite. Therefore currently the allowed
formats for the coordinate and topology files are:Coordinate Files:.groTopology files:itpHowever, it is easy to add more formats without touching any of the code in the
module by the user. If you are interested in adding support for other formats
read the FAQHow can I use an unsupported file format?Perform the mappingThe basic usage of the module to transform systems between coarse grained and
fully atomistic resolutions can be seen in the exampleexamples/bmim_bf4/map_bmim_bf4.py.For a more visual representation there is a jupyter notebook that works as
example and can be easily modified to transform interactively any system. The
notebook is located inexamples/notebook/Interactive-example.ipynb.
In order to use this notebook thenglviewmodule is required. In order to run the notebook run the following command:jupyternotebookexamples/notebook/Interactive-example.ipynb--NotebookApp.iopub_data_rate_limit=10000000If you do not haveJupyterinstalled you can take a look
to its markdown preview inexamples/notebook/Interactive-example.md.If you are looking for a more flexible way of performing the mappings that
allows you to have more control in each step you can find another Jupyter notebookexample(and the correspondingpreview) which will take you around the
module to explore more options during the mapping process. This example will
also tell you how you can reuse the results from old mappings to afford
computational time.CLIAnother option to perform a mapping is using the command line interface that is
available after the module installation. This tool is accessible through the
commandgaddlemapsin a terminal. The documentation of this command can be
checked running:gaddlemaps-hA basic usage of it would look like this:gaddlemapssystem_initial.gro--molmolecule1_initial.itpmolecule1_final.gromolecule1_final.itp--molmolecule2_initial.itpmolecule2_final.gromolecule2_final.itp--scale0.5-osystem_final.growheresystem_initial.grois the file with the coordinates of the atoms of the
system to be mapped in the initial resolution. After each--molflag all the
files for mapping each desired type of molecule must be provided (topology in
the initial resolution, atom coordinates in the final resolution and topology of
the molecule in the final resolution). The--scaleflag specifies the scaling
factor to reduce the final atomic coordinates to avoid molecular overlaps.
Finally, the-ooption specified the path to write the output file with the
coordinates in the final resolution.There is another way to run a mapping with this tool that automatically detects
all the files needed for each molecule. For example, imagine that in the current
directory you have all the files needed for the mapping we did in with the
previous command and more (for instances files to map a "molecule3" that we do
not want to map). In this scenario we could run the same mapping as before
running the following command:gaddlemapssystem_initial.gro--auto*--excludemolecule3--scale0.5-osystem_final.groNote that the identification of the molecules is based on their names in the
".itp" files and also in the possibility of molecule object initiation so use
this option with caution. The--excludeflag is used to tell the command to
not include the molecule3 (the molecule name in the .itp file) in the mapping
process (useful for example to exclude solvent molecules).FAQsWhat are those restraints? Wasn't gaddlemaps an automatic algorithm?GADDLE Maps is an algorithm that does not need any external input. However for
molecules that are very symmetrical (for example molecules that are completely
linear) the algorithm may need a "hint" to know which extreme is which. When
the molecule have ramification these hints are not usually needed. These hints
are what we call restraints. They can be also used in order to have a faster aligment.How do I know if gaddle maps is using the c++ backend?There is a function that returns wether the c++ backend is available or not. In
order to check it run:fromgaddlemapsimportcheck_backend_installedifcheck_backend_installed():print("The backend is correctly installed")else:print("The backend is not accessible")If it is not installed refer to the sectionConfiguring the c++ backend.I have a big protein and it takes ages to align. Is this behavior normal?First of all check wether you are using or not the c++ backendHow do I know if
gaddlemaps is using the c++
backend?. However even
without the c++ backend the mapping should not take more than a day even for
really big molecules.The time that takes to align a molecule greatly increase with the number of
atoms. However, for proteins an approximation is done. Instead of aligning all
the atoms of the protein the overlapping is only calculated between atoms in
residues  with the same name. If the alignment is taken too long chances are
that the residues in both resolutions have different names or are in different
order.This problem can be solved by adding restraints between all the atoms of the
molecule but by far the easiest way to accelerate the alignment is to set the
same sequence of residues in both files.Why the final mapped system looks so weird?By default, when the molecular alignments are extrapolated to all the
configurations in the simulation box, the coordinates of the atoms in the
molecules in the final resolution are scaled down (see the original paper for
a more detailed description of this reduction) to avoid different molecules
to overlap in the final system. If this is not done, every simulation that
takes that mapped system as input would crash in the first steps. A short energy
minimization is enough to fix this scale of the atomic coordinates.NOTE: Before trying to resolvate or ionize any of the systems obtained
with this tool is highly recommended to perform a short energy minimization to
relax the molecular structures.I have some other problem or an unexpected behavior of the code. How should I report it?If you find that the code does not work as expected please check if there is anissue in githubwith the same
problem. If there is not,open a new
issueand we will
address it as soon as possible.How can I use an unsupported file format?gaddlemaps allows the enduser to extend the compatible file formats without
touching a single line of the module. For such purpose gaddlemaps have 2 base
classes (one for the coordinate files and another for the topology files) from
which the enduser can inherit. Any class that inherit from these classes will be
added automatically to the supported formats.Any class that inherits fromgaddlemaps.parsers.CoordinatesParserwill be
added to the compatible parsers for coordinate files. On the other any class
that inherits fromgaddlemaps.parsers.TopolgyParserwill be added to the
supported parsers for topology files. Below you can see an example for a simple
coordinate parser for thepdbformat (thepdbformat is not currently
supported as the parser below is not guarantee to work for the wholepdb
specification).NOTE:The following snippet is just a quick and dirty implementation of a
parser. It is just an example of how a quick parser can be written. In its
current form quality would not be good enough to be included as part of the
module.fromgaddlemaps.parsersimportCoordinatesParserimportnumpyasnpclassSimplePDB(CoordinatesParser):# We must define wich extensions will the parser work withEXTENSIONS=("pdb",)# The following methods must be defineddef__init__(self,path,mode='r'):self.open_file=open(path,mode=mode)self.mode=modeself._comment=""self._box=np.array([[0,0,0],[0,0,0],[0,0,0]])self._atom_info=[]self._atom_index=0ifmode=="r":self._load_info()defseek_atom(self,index):# Instead of moving the cursor to the desired atom we can read all the# atomic information first and use an index to seek the atom. Beware# that this will increase the needed ram and can give problems for the# bigger systems.self._atom_index=indexdefnext(self):ifself._atom_index>=self.natoms:raiseStopIterationinfo=self._atom_info[self._atom_index]self._atom_index+=1returninfodefwriteline(self,atomlist):# Instead of actually writing the file on the go it is possible to# store the information and save it all just before closingself._atom_info.append(atomlist)defclose(self):ifself.mode=="w":self.open_file.write(f"TITLE{self.comment}\n")self.open_file.write("REMARK    THIS IS A SIMULATION BOX\n")self.open_file.write("MODEL 1\n")foratomlistinself._atom_info:atom_index=atomlist[3]atom_name=atomlist[2]residue_name=atomlist[1]residue_index=atomlist[0]x_position=atomlist[4]*10# The original value is in nmy_position=atomlist[5]*10z_position=atomlist[6]*10line=(f"ATOM{atom_index:6d}{atom_name}{residue_name}{residue_index:6d}""{x_position:8.3f}{y_position:8.3f}{z_position:8.3f}\n")self.open_file.write(line)self.open_file.write("TER\n")self.open_file.write("ENDMDL\n")self.open_file.close()# We must define some properties also@propertydefnatoms(self):returnlen(self._atom_info)@propertydefbox_matrix(self):returnself._box@box_matrix.setterdefbox_matrix(self,new_box):self._box=np.array(new_box)@propertydefcomment(self):returnself._comment@comment.setterdefcomment(self,newcomment):self._comment=newcomment# You can add as many auxiliar methods as you wantdef_load_info(self):forlineinself.open_file:line=line.strip()ifline.startswith("TITLE"):self.comment=line[5:].strip()elifline.startswith("ATOM"):line=line.split()self._atom_info.append((int(line[4]),line[3],line[2],int(line[1]),float(line[5])/10,float(line[6])/10,float(line[7])/10))else:passJust having this class imported orpastedin your script will allow gaddlemaps
to be used with.pdbcoordinate files.