Package: ngdi
Data Intelligence Suite with 4 line code to run Graph Algo on NebulaGraphDocumentation:https://github.com/wey-gu/nebulagraph-di#documentationSource Code:https://github.com/wey-gu/nebulagraph-diNebulaGraph Data Intelligence Suite for Python (ngdi) is a powerful Python library that offers APIs for data scientists to effectively read, write, analyze, and compute data in NebulaGraph.With the support of single-machine engine(NetworkX), or distributed computing environment using Spark we could perform Graph Analysis and Algorithms on top of NebulaGraph in less than 10 lines of code, in unified and intuitive API.Quick Start in 5 MinutesSetup env with Nebula-Up followingthis guide.Install ngdi with pip from the Jupyter Notebook withhttp://localhost:8888(password:nebula).Open the demo notebook and run cells one by one.Check theAPI ReferenceInstallationpipinstallngdiUsageCall from nGQLSee more details indocsRETURNngdi("pagerank",["follow"],["degree"],"spark",{space:"basketballplayer",max_iter:10},{write_mode:"insert"})Spark Engine ExamplesSee also:examples/spark_engine.ipynbRun Algorithm on top of NebulaGraph:Note, there is also query mode, refer toexamplesordocsfor more details.fromngdiimportNebulaReader# read data with spark engine, scan modereader=NebulaReader(engine="spark")reader.scan(edge="follow",props="degree")df=reader.read()# run pagerank algorithmpr_result=df.algo.pagerank(reset_prob=0.15,max_iter=10)Write back to NebulaGraph:fromngdiimportNebulaWriterfromngdi.configimportNebulaGraphConfigconfig=NebulaGraphConfig()properties={"louvain":"cluster_id"}writer=NebulaWriter(data=df_result,sink="nebulagraph_vertex",config=config,engine="spark")writer.set_options(tag="louvain",vid_field="_id",properties=properties,batch_size=256,write_mode="insert",)writer.write()Then we could query the result in NebulaGraph:MATCH(v:louvain)RETURNid(v),v.louvain.cluster_idLIMIT10;NebulaGraph Engine Examples(not yet implemented)Basically the same as Spark Engine, but withengine="nebula".- reader = NebulaReader(engine="spark")+ reader = NebulaReader(engine="nebula")DocumentationEnvironment SetupAPI ReferenceHow it worksngdi is an unified abstraction layer for different engines, the current implementation is based on Spark, NetworkX, DGL and NebulaGraph, but it's easy to extend to other engines like Flink, GraphScope, PyG etc.┌───────────────────────────────────────────────────┐
          │   Spark Cluster                                   │
          │    .─────.    .─────.    .─────.    .─────.       │
          │   ;       :  ;       :  ;       :  ;       :      │
       ┌─▶│   :       ;  :       ;  :       ;  :       ;      │
       │  │    ╲     ╱    ╲     ╱    ╲     ╱    ╲     ╱       │
       │  │     `───'      `───'      `───'      `───'        │
  Algo Spark                                                  │
    Engine└───────────────────────────────────────────────────┘
       │  ┌────────────────────────────────────────────────────┬──────────┐
       └──┤                                                    │          │
          │   NebulaGraph Data Intelligence Suite(ngdi)        │ ngdi-api │◀─┐
          │                                                    │          │  │
          │                                                    └──────────┤  │
          │     ┌────────┐    ┌──────┐    ┌────────┐   ┌─────┐            │  │
          │     │ Reader │    │ Algo │    │ Writer │   │ GNN │            │  │
 ┌───────▶│     └────────┘    └──────┘    └────────┘   └─────┘            │  │
 │        │          │            │            │          │               │  │
 │        │          ├────────────┴───┬────────┴─────┐    └──────┐        │  │
 │        │          ▼                ▼              ▼           ▼        │  │
 │        │   ┌─────────────┐ ┌──────────────┐ ┌──────────┐┌──────────┐   │  │
 │     ┌──┤   │ SparkEngine │ │ NebulaEngine │ │ NetworkX ││ DGLEngine│   │  │
 │     │  │   └─────────────┘ └──────────────┘ └──────────┘└──────────┘   │  │
 │     │  └──────────┬────────────────────────────────────────────────────┘  │
 │     │             │        Spark                                          │
 │     │             └────────Reader ────────────┐                           │
 │  Spark                   Query Mode           │                           │
 │  Reader                                       │                           │
 │Scan Mode                                      ▼                      ┌─────────┐
 │     │  ┌───────────────────────────────────────────────────┬─────────┤ ngdi-udf│◀─────────────┐
 │     │  │                                                   │         └─────────┤              │
 │     │  │  NebulaGraph Graph Engine         Nebula-GraphD   │   ngdi-GraphD     │              │
 │     │  ├──────────────────────────────┬────────────────────┼───────────────────┘              │
 │     │  │                              │                    │                                  │
 │     │  │  NebulaGraph Storage Engine  │                    │                                  │
 │     │  │                              │                    │                                  │
 │     └─▶│  Nebula-StorageD             │    Nebula-Metad    │                                  │
 │        │                              │                    │                                  │
 │        └──────────────────────────────┴────────────────────┘                                  │
 │                                                                                               │
 │    ┌───────────────────────────────────────────────────────────────────────────────────────┐  │
 │    │ RETURN ngdi("pagerank", ["follow"], ["degree"], "spark", {space: "basketballplayer"}) │──┘
 │    └───────────────────────────────────────────────────────────────────────────────────────┘
 │  ┌─────────────────────────────────────────────────────────────┐
 │  │ from ngdi import NebulaReader                               │
 │  │                                                             │
 │  │ # read data with spark engine, scan mode                    │
 │  │ reader = NebulaReader(engine="spark")                       │
 │  │ reader.scan(edge="follow", props="degree")                  │
 └──│ df = reader.read()                                          │
    │                                                             │
    │ # run pagerank algorithm                                    │
    │ pr_result = df.algo.pagerank(reset_prob=0.15, max_iter=10)  │
    │                                                             │
    └─────────────────────────────────────────────────────────────┘Spark Engine PrerequisitesSpark 2.4, 3.0(not yet tested)NebulaGraph 3.4+NebulaGraph Spark Connector 3.4+NebulaGraph Algorithm 3.1+NebulaGraph Engine PrerequisitesNebulaGraph 3.4+NebulaGraph Python Client 3.4+NetworkXLicenseThis project is licensed under the terms of the Apache License 2.0.