Package: pecanpy
PecanPy: A parallelized, efficient, and acceleratednode2vec(+)in PythonLearning low-dimensional representations (embeddings) of nodes in large graphs is key to applying machine learning on massive biological networks.Node2vecis the most widely used method for node embedding. PecanPy is a fast, parallelized, memory efficient, and cache optimized Python implementation ofnode2vec. It uses cache-optimized compact graph data structures and precomputing/parallelization to result in fast, high-quality node embeddings for biological networks of all sizes and densities. Detailed source code documentation can be foundhere.The details of implementation and the optimizations, along with benchmarks, are described in the application notePecanPy: a fast, efficient and parallelized Python implementation of node2vec, which is published inBioinformatics. The benchmarking results presented in the preprint can be reproduced using the test scripts provided in the companionbenchmarks repo.v2 update: PecanPy is now equipped withnode2vec+, which is a natural extension ofnode2vecand handles weighted graph more effectively. For more information, seeAccurately Modeling Biased Random Walks on Weighted Graphs Using Node2vec+. The datasets and test scripts for reproducing the presented results are available in thenode2vec+ benchmarks repo.InstallationInstall from the latest release with:$pipinstallpecanpyInstall latest version (unreleassed) in development mode with:$gitclonehttps://github.com/krishnanlab/pecanpy.git
$cdpecanpy
$pipinstall-e.where-emeans "editable" mode so you don't have to reinstall every time you make changes.PecanPy installs a command line utilitypecanpythat can be used directly.UsagePecanPy operates in three different modes –PreComp,SparseOTF, andDenseOTF– that are optimized for networks of different sizes and densities;PreCompfor networks that are small (≤10k nodes; any density),SparseOTFfor networks that are large and sparse (>10k nodes; ≤10% of edges), andDenseOTFfor networks that are large and dense (>10k nodes; >10% of edges). These modes appropriately take advantage of compact/dense graph data structures, precomputing transition probabilities, and computing 2nd-order transition probabilities during walk generation to achieve significant improvements in performance.ExampleTo runnode2vecon Zachary's karate club network usingSparseOTFmode, execute the following command from the project home directory:pecanpy--inputdemo/karate.edg--outputdemo/karate.emb--modeSparseOTFNode2vec+To enablenode2vec+, specify the--extendoption.pecanpy--inputdemo/karate.edge--outputdemo/karate_n2vplus.emb--modeSparseOTF--extendNote:node2vec+is only beneficial for embeddingweightedgraphs. For unweighted graphs,node2vec+is equivalent tonode2vec. The above example only serves as a demonstration of enablingnode2vec+.DemoExecute the following command for full demonstration:shdemo/run_pecanpyModeAs mentioned above, PecanPy contains three main modes for generating node2vec random walks,
each of which is better optimized for different network sizes/densities:ModeNetwork size/densityOptimizationPreComp<10k nodes, <0.1% edgesPrecompute second order transition probabilities, using CSR graphSparseOTF(default)(≥10k nodes, ≥0.1% and <20% of edges) or (<10k nodes, ≥0.1% edges)Transition probabilites computed on-the-fly, using CSR graphDenseOTF>20% of edgesTransition probabilities computed on-the-fly, using dense matrixCompatibility and recommendationsModeWeightedp,q!=1Node2vec+SpeedUse this ifPreComp:white_check_mark::white_check_mark::white_check_mark::dash::dash:The graph is small and sparseSparseOTF:white_check_mark::white_check_mark::white_check_mark::dash:The graph is sparse but not necessarily smallDenseOTF:white_check_mark::white_check_mark::white_check_mark::dash:The graph is extremely densePreCompFirstOrder:white_check_mark::x::x::dash::dash:Run withp = q = 1on weighted graphFirstOrderUnweighted:x::x::x::dash::dash::dash:Run withp = q = 1on unweighted graphOptionsCheck out the full list of options available using:pecanpy--helpInputThe supported input is a network file as an edgelist.edgfile (node id could be int or string):node1_id node2_id <weight_float, optional>Another supported input format (only forDenseOTF) is the numpy array.npzfile. Run the following command to prepare a.npzfile from a.edgfile.pecanpy--input$input_edgelist--output$output_npz--tasktodenseThe default delimiter for.edgis tab space (\t), you many change this by passing in the--delimiteroption.OutputThe output file hasn+1lines for graph withnvertices, with a header line of the following format:num_of_nodes dim_of_representationThe following  nextnlines are the representations of dimensiondfollowing the corresponding node ID:node_id dim_1 dim_2 ... dim_dDevelopment NoteRunblack src/pecanpy/to automatically follow black code formatting.
Runtox -e flake8and resolve suggestions before committing to ensure consistent code style.Additional InformationDocumentationDetailed documentation for PecanPy is availablehere.SupportFor support, please consider opening a GitHub issue and we will do our best to reply in a timely manner.
Alternatively, if you would like to keep the conversation private, feel free to contactRemy Liuatliurenmi@msu.edu.LicenseThis repository and all its contents are released under theBSD 3-Clause License; SeeLICENSE.md.CitationIf you use PecanPy, please cite:
Liu R, Krishnan A (2021)PecanPy: a fast, efficient, and parallelized Python implementation ofnode2vec.Bioinformaticshttps://doi.org/10.1093/bioinformatics/btab202If you findnode2vec+useful, please cite:
Liu R, Hirn M, Krishnan A (2023)Accurately modeling biased random walks on weighted graphs usingnode2vec+.Bioinformaticshttps://doi.org/10.1093/bioinformatics/btad047AuthorsRenming Liu, Arjun Krishnan**General correspondence should be addressed to AK atarjun.krishnan@cuanschutz.edu.FundingThis work was primarily supported by US National Institutes of Health (NIH) grants R35 GM128765 to AK and in part by MSU start-up funds to AK.AcknowledgementsWe thankChristopher A. Mancuso,Anna Yannakopoulos, and the rest of theKrishnan Labfor valuable discussions and feedback on the software and manuscript. Thanks toCharles T. Hoytfor making the softwarepipinstallable and for an extensive code review.ReferencesOriginalnode2vecGrover, A. and Leskovec, J. (2016) node2vec: Scalable Feature Learning for Networks. ArXiv160700653 Cs Stat.
Originalnode2vecsoftware and networkshttps://snap.stanford.edu/node2vec/contains the original software and the networks (PPI, BlogCatalog, and Wikipedia) used in the original study (Grover and Leskovec, 2016).Other networksStark, C. et al. (2006) BioGRID: a general repository for interaction datasets. Nucleic Acids Res., 34, D535–D539.BioGRID human protein-protein interactions.Szklarczyk, D. et al. (2015) STRING v10: protein–protein interaction networks, integrated over the tree of life. Nucleic Acids Res., 43, D447–D452.STRING predicted human gene interactions.Greene, C.S. et al. (2015) Understanding multicellular function and disease with human tissue-specific networks. Nat. Genet., 47, 569–576.GIANT-TN is a generic genome-scale human gene network. GIANT-TN-c01 is a sub-network of GIANT-TN where edges with edge weight below 0.01 are discarded.BioGRID (Stark et al., 2006), STRING (Szklarczyk et al., 2015), and GIANT-TN (Greene et al., 2015) are available fromhttps://doi.org/10.5281/zenodo.3352323.Law, J.N. et al. (2019) Accurate and Efficient Gene Function Prediction using a Multi-Bacterial Network. bioRxiv, 646687.SSN200 is a cross-species network of proteins from 200 species with the edges representing protein sequence similarities. Downloaded fromhttps://bioinformatics.cs.vt.edu/~jeffl/supplements/2019-fastsinksource/.