Project: GraphScope (App Definition and Development / Database)
File: getting_started.md

This tutorial provides a quick overview of GraphScope's features. To begin, we will install GraphScope on your local machine using Python. Although most examples in this guide are based on local Python installation, it also works on a Kubernetes cluster. You can easily install GraphScope through pip: ```bash python3 -m pip install graphscope -U ``` ````{note} We recommend you to install GraphScope in a clean Python virtual environment with Python 3.9 with or . Take `venv` for an example, there's a step by step instruction to create a virtual environment, activate the environment and install GraphScope: ```bash python3.9 -m venv tutorial-env source tutorial-env/bin/activate python3.9 -m pip install graphscope python3.9 >> import graphscope as gs >> ...... ``` ```` We will use a walking-through example to demonstrate how to use GraphScope to process various graph computation tasks in a one-stop manner. The example targets node classification on a citation network. ogbn-mag is a heterogeneous network composed of a subset of the Microsoft Academic Graph. It contains 4 types of entities (i.e., papers, authors, institutions, and fields of study), as well as four types of directed relations connecting two entities. Given the heterogeneous ogbn-mag data, the task is to predict the class of each paper. Node classification can identify papers in multiple venues, which represent different groups of scientific work on different topics. We apply both the attribute and structural information to classify papers. In the graph, each paper node contains a 128-dimensional word2vec vector representing its content, which is obtained by averaging the embeddings of words in its title and abstract. The embeddings of individual words are pre-trained. The structural information is computed on-the-fly. GraphScope models graph data as property graph, in which the edges/vertices are labeled and have many properties. Taking ogbn-mag as an example, the figure below shows the model of the property graph. :::{figure-md} <img src="../images/sample_pg.png" alt="Sample of property graph." width="80%"> Sample of property graph ::: This graph has four kinds of vertices, labeled as paper, author, institution, and fieldofstudy. There are four kinds of edges connecting them, each kind of edge has a label and specifies the vertex labels for its two ends. For example, cites edges connect two vertices labeled paper. Another example is writes, it requires the source vertex is labeled author and the destination is a paper vertex. All the vertices and edges may have properties. e.g., paper vertices have properties like features, publish year, subject label, etc. ````{dropdown} Import GraphScope and load a graph To load this graph to GraphScope with our retrieval module, please use these code. ```python import graphscope from graphscope.dataset import loadogbnmag g = loadogbnmag() ``` ```` Interactive queries enable users to explore, examine, and present graph data in a flexible and in-depth manner, allowing them to find specific information

GraphScope enhances the presentation of interactive queries and ensures efficient execution of these queries on a large scale by providing support for the popular query languages and . ````{dropdown} Run interactive queries with Gremlin and Cypher In this example, we use graph traversal to count the number of papers two given authors have co-authored. To simplify the query, we assume the authors can be uniquely identified by ID 2 and 4307, respectively. ```python interactive = graphscope.interactive(g) papers = interactive.execute("g.V().has('author', 'id', 2).out('writes').where(.in('writes').has('id', 4307)).count()").one() papers = interactive.execute( \ "MATCH (n1:author)-[:writes]->(p:paper)<-[:writes]-(n2:author) \ WHERE n1.id = 2 AND n2.id = 4307 \ RETURN count(DISTINCT p)", \ lang="cypher") ``` ```` Graph analytics is widely used in the real world. Many algorithms, like community detection, paths and connectivity, and centrality, have proven to be very useful in various businesses. GraphScope comes with a set of built-in algorithms, enabling users to easily analyze their graph data. ````{dropdown} Run analytical algorithms on the graph Continuing our example, we first derive a subgraph by extracting publications within a specific time range from the entire graph (using Gremlin!). Then, we run k-core decomposition and triangle counting to generate the structural features of each paper node. Please note that many algorithms may only work on homogeneous graphs. Therefore, to evaluate these algorithms on a property graph, we need to project it into a simple graph first. ```python sub_graph = interactive.subgraph("g.V().has('year', gte(2014).and(lte(2020))).outE('cites')") simpleg = subgraph.project(vertices={"paper": []}, edges={"cites": []}) ret1 = graphscope.kcore(simpleg, k=5) ret2 = graphscope.triangles(simple_g) subgraph = subgraph.add_column(ret1, {"kcore": "r"}) subgraph = subgraph.add_column(ret2, {"tc": "r"}) ``` ```` Graph neural networks (GNNs) combines superiority of both graph analytics and machine learning. GNN algorithms can compress both structural and attribute information in a graph into low-dimensional embedding vectors on each node. These embeddings can be further fed into downstream machine learning tasks. ````{dropdown} Prepare data and engine for learning In our example, we train a supervised GraphSAGE model to classify the nodes (papers) into 349 categories, each representing a venue (e.g., pre-print or conference). To accomplish this, we first launch a learning engine and construct a graph with features, following the previous step. ```python paperfeatures = [f"feat{i}" for i in range(128)] paper_features.append("kcore") paper_features.append("tc") lg = graphscope.graphlearn(subgraph, nodes=[("paper", paperfeatures)], edges=[("paper", "cites", "paper")], gen_labels=[ ("train", "paper", 100, (0, 75)), ("val", "paper", 100, (75, 85)), ("test", "paper", 100, (85, 100)) ]) ``` ```` Then we define the training process, and run it. ````{dropdown} Define the training process and run it ```python try: import tensorflow.compat.v1 as tf tf.disablev2behavior() except ImportError: import tensorflow as tf import graphscope.learning from graphscope.learning.examples import EgoGraphSAGE from graphscope.learning.examples import EgoSAGESupervisedDataLoader from graphscope.learning.examples.tf.trainer import LocalTrainer def trainsage(graph, nodetype, edgetype, classnum, features_num, hopsnum=2, nbrsnum=[25, 10], epochs=2, hiddendim=256, indroprate=0.5, learningrate=0.01, ): graphscope.learning.resetdefaulttf_graph() dimensions = [featuresnum] + [hiddendim] * (hopsnum - 1) + [classnum] model = EgoGraphSAGE(dimensions, actfunc=tf.nn.relu, dropout=indrop_rate) train_data = EgoSAGESupervisedDataLoader( graph,

nodetype=nodetype, edgetype=edgetype, nbrsnum=nbrsnum, hopsnum=hopsnum, ) trainembedding = model.forward(traindata.src_ego) trainlabels = traindata.src_ego.src.labels loss = tf.reduce_mean( tf.nn.sparsesoftmaxcrossentropywith_logits( labels=trainlabels, logits=trainembedding, ) ) optimizer = tf.train.AdamOptimizer(learningrate=learningrate) test_data = EgoSAGESupervisedDataLoader( graph, graphscope.learning.Mask.TEST, nodetype=nodetype, edgetype=edgetype, nbrsnum=nbrsnum, hopsnum=hopsnum, ) testembedding = model.forward(testdata.src_ego) testlabels = testdata.src_ego.src.labels testindices = tf.math.argmax(testembedding, 1, output_type=tf.int32) test_acc = tf.div( tf.reducesum(tf.cast(tf.math.equal(testindices, test_labels), tf.float32)), tf.cast(tf.shape(test_labels)[0], tf.float32), ) trainer = LocalTrainer() trainer.train(train_data.iterator, loss, optimizer, epochs=epochs) trainer.test(testdata.iterator, testacc) trainsage(lg, nodetype="paper", edge_type="cites", class_num=349, # output dimension features_num=130, # input dimension, 128 + kcore + triangle count ) ``` ```` The installed `graphscope` package includes everything you need to analyze a graph on your local machine. If you have a graph analytical job that needs to run iterative algorithms, it works well with `graphscope`. ````{dropdown} Example: Running iterative algorithm (SSSP) in GraphScope ```python import graphscope as gs from graphscope.dataset.moderngraph import loadmodern_graph gs.setoption(showlog=True) graph = loadmoderngraph() ret = gs.lpa(graph) print(ret.to_dataframe(selector={'id': 'v.id', 'label': 'r'})) pg = graph.project(vertices={'person': None}, edges={'knows': ['weight']}) ret = gs.sssp(pg, src=1) print(ret.to_dataframe(selector={'id': 'v.id', 'distance': 'r'}) ``` ```` With the `graphscope` package already installed, you can effortlessly engage with a graph on your local machine. You simply need to create the `interactive` instance to serve as the conduit for submitting Gremlin or Cypher queries. ````{dropdown} Example: Run Interactive Queries in GraphScope ```python import graphscope as gs from graphscope.dataset.moderngraph import loadmodern_graph gs.setoption(showlog=True) graph = loadmoderngraph() g = gs.interactive(graph) q1 = g.execute('g.V().count()') print(q1.all().result()) # should print [6] q2 = g.execute('g.V().hasLabel(\'person\')') print(q2.all().result()) # should print [[v[2], v[3], v[0], v[1]]] q3 = g.execute("MATCH (n:person) RETURN count(n)", lang="cypher", routing_=RoutingControl.READ) print(q3.records) # should print 6 ``` ```` GNN model training with GraphScope is easy and straightforward. You can use the `graphscope` package to train a GNN model on your local machine. Note that `tensorflow` is required to run the following example. ````{dropdown} Example: Training GraphSAGE Model in GraphScope ```python import graphscope from graphscope.dataset import loadogbnmag g = loadogbnmag() paperfeatures = [f"feat{i}" for i in range(128)] lg = graphscope.graphlearn(g, nodes=[("paper", paper_features)], edges=[("paper", "cites", "paper")], gen_labels=[ ("train", "paper", 100, (0, 75)), ("val", "paper", 100, (75, 85)), ("test", "paper", 100, (85, 100)) ]) try: import tensorflow.compat.v1 as tf tf.disablev2behavior() except ImportError: import tensorflow as tf import graphscope.learning from graphscope.learning.examples import EgoGraphSAGE from graphscope.learning.examples import EgoSAGESupervisedDataLoader from graphscope.learning.examples.tf.trainer import LocalTrainer def trainsage(graph, nodetype, edgetype, classnum, features_num, hopsnum=2, nbrsnum=[25, 10], epochs=2, hiddendim=256, indroprate=0.5, learningrate=0.01, ): graphscope.learning.resetdefaulttf_graph() dimensions = [featuresnum] + [hiddendim] * (hopsnum - 1) + [classnum] model = EgoGraphSAGE(dimensions, actfunc=tf.nn.relu, dropout=indrop_rate) train_data = EgoSAGESupervisedDataLoader( graph, graphscope.learning.Mask.TRAIN, nodetype=nodetype, edgetype=edgetype, nbrsnum=nbrsnum, hopsnum=hopsnum, ) trainembedding = model.forward(traindata.src_ego) trainlabels = traindata.src_ego.src.labels loss = tf.reduce_mean( tf.nn.sparsesoftmaxcrossentropywith_logits( labels=trainlabels, logits=trainembedding, ) ) optimizer = tf.train.AdamOptimizer(learningrate=learningrate) test_data = EgoSAGESupervisedDataLoader( graph, graphscope.learning.Mask.TEST, nodetype=nodetype, edgetype=edgetype, nbrsnum=nbrsnum, hopsnum=hopsnum, ) testembedding = model.forward(testdata.src_ego) testlabels = testdata.src_ego.src.labels testindices = tf.math.argmax(testembedding, 1, output_type=tf.int32) test_acc = tf.div( tf.reducesum(tf.cast(tf.math.equal(testindices, test_labels), tf.float32)), tf.cast(tf.shape(test_labels)[0], tf.float32), ) trainer = LocalTrainer() trainer.train(train_data.iterator, loss, optimizer, epochs=epochs) trainer.test(testdata.iterator, testacc) trainsage(lg, nodetype="paper", edge_type="cites", class_num=349, # output dimension features_num=128, # input dimension ) ``` ````