Package: ndexncipidloader
NDEx NCI-PID content loaderPython application that loads NCI-PID data intoNDExThis tool downloadsOWLfiles containing NCI-PID data from:ftp://ftp.ndexbio.org/NCI_PID_BIOPAX_2016-06-08-PC2v8-API/and performs the following operations:1)OWL files are converted to extendedSIFformat usingPaxtoolsand theSIFfile is loaded into a network2)A node attribute namedtypeis added to each node and is set to one of the followingby extracting its value fromPARTICIPANT_TYPEcolumn inSIFfile:protein(originally ProteinReference)smallmolecule(originally SmallMoleculeReference)proteinfamily(set if node name hasfamilyand was aprotein)RnaReference(original value)ProteinReference;SmallMoleculeReference(original value)3)A node attribute namedaliasis added to each node and is loaded fromUNIFICATION_XREFcolumn inSIFfile which is split by;into a list. Each element of this list is prefixed withuniprot:and t first element is set as therepresentsvalue in node and removed from thealiasattribute. If after
removal, thealiasattribute value is empty, it is removed.4)InSIFfileINTERACTION_TYPEdefines edge interaction type andINTERACTION_PUBMED_IDdefine
value ofcitationedge attribute. The values incitationedge attribute are
prefixed withpubmed:Once loaded redundant edges are removed
following these conventions:neighbor-ofedges are removedcontrols-state-ofedges are removed if another edge connecting same nodes has one of the following interactions:controls-state-change-of, controls-transport-of, controls-phosphorylation-of, controls-expression-ofNOTE:If above results in orphaned nodes, those nodes are removed as well5)An edge attribute nameddirectedis set toTrueif edge interaction type is one of the following (otherwise its set toFalse)controls-state-change-of
controls-transport-of
controls-phosphorylation-of
controls-expression-of
catalysis-precedes
controls-production-of
controls-transport-of-chemical
chemical-affects
used-to-produce6)If node name matchesrepresentsvalue in node (withuniprot:prefix added) then the node name is replaced with gene symbol fromgene_symbol_mapping.json7)If node name starts withCHEBIthen node name is replaced with value ofPARTICIPANT_NAMEfromSIFcolumn8)If noderepresentsvalue starts withchebi:CHEBIthechebi:is removed9)If_HUMANinSIFfilePARTICIPANT_NAMEcolumn for a given node then this value is replaced by doing a lookup ingene_symbol_mapping.json, unless value in lookup is-in which case original name is left10)Any node withfamilynode name is changed as follows if a lookup of node name againstgene_symbol_mapping.jsonreturns one or more genesNode attribute namedmemberis added and set to list of genes found in lookup ingene_symbol_mapping.jsonNode attribute namedtypeis changed toproteinfamily11)Changed in 5.0.0. For each network allproteinfamilynodes are examined and if anymembersexistas separate nodes, those nodes are removed and their edges are shifted to the correspondingproteinfamilynode. Duplicate edges are removed and other edges are merged ifinteractionanddirectedvalues are the
same. In the case of a mergecitationfield values are merged into a new unique list.12)The following network attributes are setnameset to name ofOWLfile with.owl.gzsuffix removed except forPathwayCommons.8.NCI_PID.BIOPAXwhich is renamed toNCI PID - Complete Interactionsauthor(fromCurated Bycolumn innetworkattributes.tsv)labels(fromPIDcolumn innetworkattributes.tsv)organismis pulled fromorganismattribute ofstyle.cxprov:wasGeneratedByis set to html link to this repo with text ndexncipidloader <VERSION> (example: ndexncipidloader 1.2.0)prov:wasDerivedFromis set to full path toOWLfile on ftp sitereviewers(fromReviewed Bycolumn innetworkattributes.tsv)versionis set to Abbreviated month-year (example: MAY-2019)descriptionis pulled fromdescriptionattribute ofstyle.cxexcept forNCI PID - Complete Interactionswhich has a hardcoded description set toThis network includes all interactions of the individual NCI-PID pathways.networkTypeis set to list of string with single entrypathwayexcept forNCI PID - Complete Interactionswhich also includesinteractome__iconurlis set to value of–iconurlflag (currently defaulting tohttp://search.ndexbio.org/static/media/ndex-logo.04d7bf44.svg)__normalizationversionis set to 0.113)By default each network is made public with full indexed and showcased (visible in user’s home network list page)NOTE:gene_symbol_mapping.jsonwas originally extracted fromherebut the gene families were updated by callingndexloadncipid.py –getfamilies sifdir/which callshttps://mygene.infoviabiothingsPython clientDependenciesndex2ndexutilbiothings_clientrequestspandaspy4cytoscapeCompatibilityPython 3.6+Installationgit clone https://github.com/ndexcontent/ndexncipidloader
cd ndexncipidloader
make dist
pip install dist/ndexncipidloader*whlConfigurationThendexloadncipid.pyrequires a configuration file in the following format be created.
The default path for this configuration is~/.ndexutils.confbut can be overridden with--confflag.Format of configuration file[<value in --profile (default ndexncipidloader)>]

user = <NDEx username>
password = <NDEx password>
server = <NDEx server(omit http) ie public.ndexbio.org>Example configuration file[ncipid_dev]

user = joe123
password = somepassword123
server = dev.ndexbio.orgRequired external toolPaxtools is needed to convert the OWL files to SIF format.Please downloadpaxtools.jar(http://www.biopax.org/Paxtools/)
(requires Java 8+) and put in current working directoryOr specify path topaxtools.jarwith--paxtoolsflag onloadnexncipidloader.pyUsageFor more information invokendexloadncipid.py -hExample usageThis example assumes a valid configuration file with paxtools.jar in the working directory.ndexloadncipid.py sifExample usage with sif files already downloadedThis example assumes a valid configuration file and the SIF files are located insif/directoryndexloadncipid.py --skipdownload sifCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History5.0.1 (2021-05-25)Switched default layout (can be overridden with–layoutflag) toforce-directedsinceforce-directed-clmay not work on all machines.5.0.0 (2021-05-20)Fixed duplicate node issue by removing nodes and edges from a network if a family node, contains
the node in itsmemberlist. Any edges are shifted to the family node with duplicates
merged where possible.4.0.0 (2020-11-04)New default behavior:force-directed-cllayout is now applied on
networks via py4cytoscape library and a running instance of Cytoscape.
Alternate Cytoscape layouts and the networkx “spring” layout can be
run by setting appropriate value via the new–layoutflag3.1.1 (2020-10-16)Removed NODE_LABEL_POSITION discrete mapping from style since it is
not compatible with CX 2.03.1.0 (2019-09-11)Added–disableshowcaseflag that lets caller disable showcasing ofNEWLYadded networks which is enabled by default.Added–indexlevelflag that lets caller set type of indexing performed onNEWLYadded networks. Default is full indexing (all).3.0.0 (2019-08-02)Renamed command line tool fromloadndexncipidloader.pytondexloadncipid.pyto be more consistent with other loaders. Since this is a breaking change bumped to version 3.0.0Added–visibilityflag which lets caller dictate whether newly added networks are set to PUBLIC (default) or PRIVATERemoved parameter–disablcitededgemergesince the changes in 2.0.0 causes this to no longer have any effectSet default for–paxtoolsflag to bepaxtools.jarwhich assumes the tool is in current working directory2.0.0 (2019-07-16)Spring layout adjusted by increasing iterationsCode now removes all neighbor-of edges with NO data migration. controls-state-change-of
edges are removed if more informative edges exist. Any orphaned nodes resulting from
the removal of these edges are also removed1.6.0 (2019-07-09)Added__iconurlnetwork attribute to all networksAddedinteractometonetworkType*network attribute for ‘NCI PID - Complete Interactions’ network1.5.1 (2019-07-09)Renamed network attributetypetonetworkTypeto adhere to normalization specification1.5.0 (2019-06-28)Fixed style.cx by removing view aspects that was causing networks to not render properly in cytoscape1.4.0 (2019-06-13)Network PathwayCommons.8.NCI_PID.BIOPAX is now renamed
to ‘NCI PID - Complete Interactions’ with alternate description.1.3.0 (2019-06-12)Improved description in style.cx file (JIRA ticket UD-362)1.2.0 (2019-06-11)Code now adds a citation attribute to every edge even if there is no value
in which case an empty list is set (JIRA ticket UD-360)Added type network attribute and set it to [‘pathway’] following normalization
guidelines1.1.0 (2019-06-10)Adjusted network layout to be more compact by reducing number of iterations in
spring layout algorithm as well as lowering the value of scale (JIRA ticket UD-360)1.0.2 (2019-05-24)Removed view references from cyVisualProperties aspect of style.cx file cause it was causing issues with loading in cytoscapeSet directed edge attribute type to boolean cause it was incorrectly defaulting to a string1.0.1 (2019-05-18)Renamed incorrect attribute name prov:wasDerivedBy to prov:wasDerivedFrom
to adhere to normalization document requirements1.0.0 (2019-05-16)Massive refactoring and first release where code attempts to behave as defined in README.rst0.1.1 (2019-02-15)Updated data/style.cx by renaming Protein to protein and SmallMolecule
to smallmolecule to match the new normalization conventions0.1.0 (2019-02-15)First release