Package: ndexutil
Warning: This repository is for development and features may change.
Please use this at your own risk.Dependenciesndex2networkxbiothings_clientrequestsrequests-toolbeltpandasmygenejsonschemaurllib3ijsonOptional DependenciesInstalled ifpip install ndexutil[cytoscape]is runpy4cytoscapeCompatibilityPython 3.4+Installationgit clone https://github.com/ndexbio/ndexutils
cd ndexutils
make dist
pip install dist/ndexutil*whlOR viaPyPIpip install ndexutilndexmisctools.pyWARNING:Please consider this tool alpha and probably contains errors/bugs and could cause data corruption or loss. You have been warnedndexmisctools.pylets caller perform some operations onNDExvia the
command line.
This tool follows format ofndexmisctools.py <COMMAND> <args>For more information runndexmisctools.py--helpandndexmisctools.py <COMMAND>--helpCOMMANDS:addnodeattrib- adds an attribute to all nodes in a networkCredentials toNDExserver must be stored in the configuration (default~/.ndexutils.conf)
and be formatted as follows:[addprofile]
user=bob
password=6ea8f0ab0b2e
server=public.ndexbio.orgThe following command addsfoonode attribute with valuemy new valueto all nodes in the network9025480b-6fbc-4efe-9cd8-b575ce49dfdandexmisctools.py --profile addprofile addnodeattrib --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --name foo --value 'my new value'WarningThis command changes the network in place, so it is a destructive call with NO undocopynetwork- copiesNDExnetwork between accounts and even serversFor copying the source and destination credentials must be stored in the configuration (default~/.ndexutils.conf)
and be formatted as follows:[mycopyprofile]
source_user=bob
source_password=6ea8f0ab0b2e
source_server=public.ndexbio.org
dest_user=smith
dest_password=4efe9cd8
dest_server=public.ndexbio.orgThe following command copies the network9025480b-6fbc-4efe-9cd8-b575ce49dfdafrom source credentials defined in configuration to destndexmisctools.py --profile mycopyprofile copynetwork --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfdacytoscapelayout- updates layout for network inNDExusing CytoscapeNOTE:This tool requires installation ofpy4cytoscapeand Cytoscape to be activately running (tested with Cytsocape 3.8+)This command requires four positional parameters. The first is the layout algorithm
to use followed by (username,password, andserver) which are credentials forNDExserver to update the layout on the network.Any of these credential fields set to‘-’will
force this tool to obtain the information from (default~/.ndexutils.conf) configuration file
under the profile specified by the--profilefield in this format:[<value of --profile>]
user = <NDEx username>
password = <NDEx password>
server = <NDEx server ie public.ndexbio.org>Using credentials frommyattribprofile, the following command adds grid
layout to the network9025480b-6fbc-4efe-9cd8-b575ce49dfdandexmisctools.py --profile myattrib networkxlayout grid - - - --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfdaTo get a list of available layouts printed to standard out, replace the layout withlistlayout. Just put
fake values in for credentials and any text for the –uuid flag:ndexmisctools.py cytoscapelayout listlayout x y z --uuid ignoreddeletenetwork- deletes a network or all networks under a given networkset fromNDExCredentials toNDExserver must be stored in the configuration (default~/.ndexutils.conf)
and be formatted as follows:[deleteprofile]
user=bob
password=6ea8f0ab0b2e
server=public.ndexbio.orgThe following command deletes the network with UUID9025480b-6fbc-4efe-9cd8-b575ce49dfdafromNDExndexmisctools.py --profile deleteprofile deletenetwork --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfdauploadnetwork- uploads network toNDExThis command uploads a network toNDExCredentials must be stored in the configuration (default~/.ndexutils.conf)
and be formatted as follows:[myattrib]
user=bob
password=6ea8f0ab0b2e
server=public.ndexbio.orgThe following command uploadsfoo.cxnetwork toNDExndexmisctools.py --profile myattrib uploadnetwork foo.cxfeaturednetworkreport- generates report about featurend networks onNDExThis command requires one positional parameterserver.
For production usepublic.ndexbio.orgIf the positional parameter is set to‘-’, it will
force this tool to obtain the information from (default~/.ndexutils.conf) configuration file
under the profile specified by the--profilefield in this format:[<value of --profile>]
server = <NDEx server ie public.ndexbio.org>The command below queries the production server for the featured networks json file
and then queriesNDExfor all networks that are part of the featured networks.ndexmisctools.py featurednetworkreport public.ndexbio.org --output foo.csvNOTE:If a networkset contains more then 500 networks, only the first 500 are examinednetworkattribupdate- updates network attributes on network inNDExWARNING:Currentlyname, version, and descriptionCANNOT be updated with this command.Credentials must be stored in the configuration (default~/.ndexutils.conf)
and be formatted as follows:[myattrib]
user=bob
password=6ea8f0ab0b2e
server=public.ndexbio.orgThe following command updatesfoonetwork attribute on the network9025480b-6fbc-4efe-9cd8-b575ce49dfdandexmisctools.py --profile myattrib networkattribupdate --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --name foo --type string --value 'my new value'networkxlayout- updates layout for network inNDExusing networkx libraryThis command requires four positional parameters. The first is the layout algorithm
to use followed by (username,password, andserver) which are credentials forNDExserver to update the layout on the network.Any of these credential fields set to‘-’will
force this tool to obtain the information from (default~/.ndexutils.conf) configuration file
under the profile specified by the--profilefield in this format:[<value of --profile>]
user = <NDEx username>
password = <NDEx password>
server = <NDEx server ie public.ndexbio.org>By default this tool only updates thecartesianLayoutaspect. To update the full network
use add the--updatefullnetworkflag.Using credentials frommyattribprofile, the following command adds spring
layout to the network9025480b-6fbc-4efe-9cd8-b575ce49dfdandexmisctools.py --profile myattrib networkxlayout spring - - - --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfdaremovenodeattrib- removes an attribute from all nodes in a networkCredentials toNDExserver must be stored in the configuration (default~/.ndexutils.conf)
and be formatted as follows:[removeprofile]
user=bob
password=6ea8f0ab0b2e
server=public.ndexbio.orgThe following command removesfoonode attribute from all nodes
in the network9025480b-6fbc-4efe-9cd8-b575ce49dfdandexmisctools.py --profile removeprofile removenodeattrib --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --name fooWarningThis command changes the network in place, so it is a destructive call with NO undosystemproperty- updates showcase, visibility, and indexing for single network or all networks in networkset inNDExNOTE:--showcasehas no effect if network visibility isprivateCredentials must be stored in the configuration (default~/.ndexutils.conf)
and be formatted as follows:[myattrib]
user=bob
password=6ea8f0ab0b2e
server=public.ndexbio.orgThe following command enables showcase and sets indexing tometafor network with id9025480b-6fbc-4efe-9cd8-b575ce49dfdandexmisctools.py --profile myattrib systemproperty --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --showcase --indexlevel metaThe following command sets visibility topublicfor all networks in networkset with ide9580d43-ec14-4be8-9977-9de88e1d410andexmisctools.py --profile myattrib systemproperty --networksetid e9580d43-ec14-4be8-9977-9de88e1d410a --visibility publicstyleupdate- update style of network inNDExCredentials must be stored in the configuration (default~/.ndexutils.conf)
and be formatted as follows:[styleprofile]
user=bob
password=6ea8f0ab0b2e
server=public.ndexbio.orgThe following command updates style of network with UUID9025480b-6fbc-4efe-9cd8-b575ce49dfdawith style from network specified by this UUIDe9580d43-ec14-4be8-9977-9de88e1d410andexmisctools.py --profile styleprofile styleupdate --uuid 9025480b-6fbc-4efe-9cd8-b575ce49dfda --styleuuid e9580d43-ec14-4be8-9977-9de88e1d410atsvloader- Loads TSV files as networks intoNDExThetsvloadercommand loads an edge list file in tab separated format (hence TSV) and using a load plan, loads that data as a network intoNDEx.
This tool attempts to mimic behavior of the oldertsv_uploader.pyscript located here:https://github.com/ndexbio/load-contentThis new version uses the more memory efficient StreamTSVLoader.This command requires five positional parameters.
The first three (username,password, andserver) are credentials forNDExserver to upload the network.Any of these first three credential fields set to‘-’will
force this tool to obtain the information from (default~/.ndexutils.conf) configuration file
under the profile specified by the--profilefield in this format:[<value of --profile>]
user = <NDEx username>
password = <NDEx password>
server = <NDEx server ie public.ndexbio.org>The forth positional parametertsv_file(seeTSV Loadersection below) should be
set to edge list file in tab separated format and the
fifth or last positional parameterload_planshould be
set to the load plan. The load plan is a JSON formatted text
file that maps the columns to nodes, edges, and attributes
in the network.By default this tool does not generate much output to
standard out/error. For more verbosity add three to five-vparameters
to left of command nametsvloaderas seen in examples below.NOTE:If using--layoutto run a Cytoscape layout,py4cytoscapemust be installed and Cytoscape to be activately running (tested with Cytsocape 3.8+)In example below the- - -tells the program to read the credentials
from the configuration filendexmisctools.py -vvvv tsvloader - - - datafile.tsv load.planHere is an example where the name and description of the network is set
and the-tspecifies a template network used to get style and in this case
since--copyattribsis set the network attributes (minus@context) are
also copied to the new networkndexmisctools.py -vvv tsvloader bob xx public.ndexbio.org \
                 datafile.tsv loadplan.json --uppercaseheader  \
                 -t dafe07ca-0676-11ea-93e0-525400c25d22 \
                 --name mynetwork --description 'some text' --copyattribsIn this example an alternate header is prepended via the--headerflag.NOTE:The--headerflag doesNOTremove an existing headerndexmisctools.py -vv --profile foo tsvloader - - public.ndexbio.org \
                 datafile.tsv loadplan.json \
                 --header 'col1   col2    col3' \
                 -t some_cx_file.cx \
                 -u 48a26aa0-0677-11ea-93e0-525400c25d22If successful0is returned otherwise there was an error.TSV LoaderThis module contains the Tab Separated Variable Loader (TSV Loader, seendexutil/streamtsvloader.pymodule) which generates
anNDEx CXfile from a tab separated
text file of edge data and attributes.To load data a load plan must be created. This plan tells the loader how to map the
columns in the file to nodes, and edges. This load plan needs to validate againstthis load plan JSON schemaExample TSV fileSOURCE  TARGET  WEIGHT
ABCD    AAA1    0.555
GGGG    BBBB    0.305SOURCEis the source node,TARGETis target nodeA schema that could be:{
"source_plan":
    {
        "node_name_column": "SOURCE"
    },
    "target_plan":
    {
        "node_name_column": "TARGET"
    },
    "edge_plan":
    {
        "default_predicate": "unknown",
        "property_columns": [
          {
            "column_name": "WEIGHT",
            "attribute_name": "weight",
            "data_type": "double"
          }
        ]
    }
}Example below assumes the following:./loadplan.jsonis the load plan in JSON format./style.cxis aNDEx CXwith a style.import ndex2
from ndexutil.tsv.streamtsvloader import StreamTSVLoader

# using ndex2 client library read CX file as NiceCXNetwork object
style_network = ndex2.create_nice_cx_from_file('./style.cx')

loader = StreamTSVLoader('./loadplan.json', style_network)
with open('./input.tsv', 'r') as tsvfile:
    with open('./output.cx', 'w') as outfile:
        loader.write_cx_network(tsvfile, outfile)CreditsHistory0.14.0 (2022-09-03)tqdmandijsonare now required packagesAdded--indexlevel,--showcase,--visibility,and--destnetworksettondexmisctools.pycopynetworkcommand to allow caller to set copied
networks accessibility and to add copied network to a networkset. This command
also stores the original NDEx network UUID in__copynewtork_orig_ndex_uuidnetwork
attributeAdded new command tondexmisctools.pynameduploadnetworkthat lets caller upload
networks to NDExAdded following methods tondexutil/ndex.py:get_network_summary(),get_nice_cx_from_server(),set_index_and_showcase(),save_network_to_ndex()0.13.1 (2020-11-11)Fixed bug where--styleuuidwas failing to update network inndexmisctools.pystyleupdatecommand0.13.0 (2020-11-03)Added Cytoscape layouts tondexmisctools.pytsvloadervia--layoutflagPassing-for layout tondexmisctools.pycytoscapelayoutwill runforce-directed-cllayout in Cytoscapendexmisctools.pycytoscapelayoutby default updates only the layout aspect
on network in NDEx. This can be changed to perform a full network update with
new--updatefullnetworkflag0.12.0 (2020-10-30)Added--layoutflag tondexmisctools.pytsvloadercommand to enable layouts
to be applied to network generated by loader0.11.0 2020-10-26Added new command tondexmisctools.pynamednetworkxlayoutthat lets caller apply networkx layouts to network on NDExAdded new command tondexmisctools.pynamedcytoscapelayoutthat lets caller apply Cytoscape layouts to network on NDExAdded new command tondexmisctools.pynamedfeaturednetworkreportthat generates a report about featured networks on NDEx0.10.0 2020-02-19Added new command tondexmisctools.pynamedaddnodeattribthat lets caller add an attribute to all nodes in a networkAdded new command tondexmisctools.pynamedremovenodeattribthat lets caller remove an attribute from all nodes in a network0.9.0.post1 2019-12-11Updated this history file adding a note about addition ofstyleupdatecommand tondexmisctools.pyin 0.9.0 release0.9.0 2019-12-11Added–typeonlyflag tonetworkattribupdatecommand inndexmisctools.pythat lets caller just update the data type for a given network attributeAdded new command tondexmisctools.pynameddeletenetworkthat lets caller delete a network or delete all networks in a given networkset from NDExAdded new command tondexmisctools.pynamedstyleupdatethat lets caller update the style of a network or all networks in a given set on NDEx0.8.1 2019-11-15Fixed bug wherendexmisctools.pyfailed to run due to argparse bug in Python versions below 3.70.8.0 2019-11-15Fixed bug in StreamTSVLoader where 2 @context network attributes were added to network if @context was set in load plan and @context was passed in the network attributes parameter forwrite_cx_network()Added new command tondexmisctools.pynamedtsvloaderthat lets caller load TSV files as networks into NDEx0.7.0 2019-09-11Added new command tondexmisctools.pynamedsystempropertythat lets
caller update showcase, visibility, and indexing for a single network or
all networks in a networkset in NDEx0.6.1 2019-07-12Fixed bug wherendexmisctools.pynetworkattributeupdatewas creating
duplicate network attributes for name, description, and value. This is
due to server bug. To deal with this the code removes those entries when
doing the network attribute update and forbids caller from trying to
update those attributes0.6.0 2019-07-10Added new commandline utilityndexmisctools.pywhich lets caller
copy a network from one NDEx account to another and lets one update
network attributes of a network in NDEx.
WARNING: THIS IS AN UNTESTED ALPHA RELEASE AND MAY CONTAIN ERRORS0.5.0 2019-06-06Added GeneSymbolSearcher class to loaderutils.py moduleMinor bug fix in NetworkIssueReport get_fullreport_as_string() where
issue text is wrapped in str() in case its not a string0.4.0 2019-05-23Added loaderutils.py module with two new classes, NetworkIssueReport and
an abstract class NetworkUpdator0.3.0 2019-05-14tsv2nicecx2.convert_pandas_to_nice_cx_with_load_plan now loads @context
data into a @context network attribute instead of a separate aspect0.2.0 2019-04-01StreamTSVLoader class added which enables loading of TSV data into
in streaming format to handle loading of large networks