Build a graph database with Amazon Neptune Part 3 - Query and Visualize Graphs
Published: 2022-09-15 21:21:43

## Introduction 

From previous [blog](https://dev.to/aws-builders/build-a-graph-database-with-amazon-neptune-part-1-launch-aws-cloudformation-stack-43k6) Amazon CloudFormation was used in a production environment to create a Neptune cluster. The second blog, provided a brief overview of graph data modelling which will be explored here.

### Learning Outcomes

* You will learn how to use Graph notebooks to query and visualize your data

### High-level architecture

The following solution overview was created by me to explore the ETL process to transform data and the end-to-end process of creating a well-architected data analytics framework to query and visualize data.

![solution](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d4uw1irli2m9ho27jzkq.jpg)

### Use Case - Social Network Analysis

Popular use cases for network analysis includes fraud detection, customer journeys and 'friend of friends'.

In this blog we explore social network analysis using AWS Community Builder's Directory for builders located in Asia Pacific and Japan (APJ).

### Amazon Neptune Pricing - Free Tier


If you have never used Amazon Neptune before, your organization may be able to access 750 hours of usage for the first 30 days with EC2 instances **db.t3.medium** or **db.t4g.medium**.

Select your region and you may manually provision the suggested  instances.

Note: If you use Amazon CloudFormation template to help you provision many AWS resources, the default EC2 instance in my case was r5.2xlarge.

You may read information on pricing [here](https://aws.amazon.com/neptune/pricing/#Get_started_with_Amazon_Neptune_for_free)

### Graph Notebooks

You may access the AWS Graph Notebooks in jupyter notebooks attached to Sagemaker from the Neptune DB instance that you may incorporate into your own project.

Step 1: Navigate to the Amazon Neptune console and then on the left hand-side click **Notebooks** to launch a notebook that is attached to a Neptune db instance.


![notebook](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ooeseopch4r3nhezetc7.jpg)


Step 2: Select the radio-dial on Neptune DB instance and click **Open Notebook**


![open](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/u2c10glxlkgtg8zjxou6.jpg)

Step 3: You will see a folder called 'Neptune' under the Jupyter notebook homepage, double-click this folder.


![jupyter](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jim9qhd78zh4rt4ba2hb.jpg)

Step 4: Select '01-Getting-Started'


![get start](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/34ttrohu4d3d8f0407an.jpg)

Step 5: Click on the second option *Using-Gremlin_to_Access_the_Graph.ipynb*

![aws notebooks](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wpt89nwz7q4nk1a1xocy.jpg)

### Data Modelling - Amazon Neptune

We will explore Gremlin statements in the Neptune data model in greater detail with [four components](https://docs.aws.amazon.com/neptune/latest/userguide/gremlin-explain-background-statements.html):

* subject (S) - relationship of two vertices (ie.nodes)
* predicate (P) - edge label
* object (O) - target vertex (i.e. node)
* graph (G)


**Creating Gremlin Vertex Label Statements**

![vertex statement](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/16htopv2ulczcanm95lr.jpg)

**Creating Gremlin Edge Statements**


![edge](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xxkocasrrk40bagd3tv4.jpg)

### Neptune Workbench

You may explore how to query and visualize graphs with sample data and graph notebooks from the Amazon database blog [here](https://aws.amazon.com/blogs/database/visualize-query-results-using-the-amazon-neptune-workbench/)

You can verify the version of the graph notebook and also the health of the Neptune cluster:

![versionn](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/29xle7aj96d8nvmpkdhn.jpg)

You may write a single statement including vertices and edge statements to create a small social network graph with gremlin:


![single statement](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d3scps3v7yvjeuet6ix4.jpg)


![edge code](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/orrrivsfpjz5rrhfyh9d.jpg)


### Data Visualization

I created an interactive network graph using Flourish to highlight interesting insights from the AWS Community Builders located in Asia Pacific and Japan, just hover your mouse over the data points. 

Link: https://public.flourish.studio/visualisation/11069877/


![flourish](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jvx7pirrr3rqambf2p59.jpg)

The dataset created from Amazon Athena was set up in Flourish with *Source* and *Target*


![flourish data](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8ww4o63diw5g0myvsb9j.jpg)

Most of our AWS Community Builders are involved in Containers as at 12 August 2022.


![Bar](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tbdqw3otbj9bhk24fmm8.jpg)

### Clean Up Resources - Delete Neptune DB Instance

Best practice is to delete the Amazon Neptune database instance if is no longer required to avoid unexpected or 'surprise' charges at the end of the month. You may follow the steps here:

1. Step 1: Navigate to the Amazon Neptune console, click on **Databases** on the left-hand menu.

You may follow the Amazon documentation for instructions [here](https://docs.aws.amazon.com/neptune/latest/userguide/manage-console-instances-delete.html)

You may take a final snaphot before you delete the Neptune DB cluster.

Select the radio-dial and select your Neptune DB instance and click **delete**

![delete Neptune](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/wh9mgu0qed8jheuykf2s.jpg)

### Clean Up Resources - Detach an EBS volume before you delete an EC2 instance

Since I used PuTTy to SSH into a Windows EC2 instance, I have to detach the EBS Volume before I delete the Windows EC2 instance

Amazon documentation for instructions are provided [here](https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/ebs-detaching-volume.html)

### Clean Up Resources - Delete EC2 Instance

Delete your EC2 instances from the **Amazon EC2 console** after you have completed your project.

![delete neptune](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0prlxn331s52789osudg.jpg)

### References

* [Github - AWS Graph Notebook](https://github.com/aws/graph-notebook)

* [Loading data into Neptune DB instance](https://docs.aws.amazon.com/neptune/latest/userguide/bulk-load-data.html)

* [Gremlin data load format](https://docs.aws.amazon.com/neptune/latest/userguide/bulk-load-tutorial-format-gremlin.html)

* [Query results in Amazon Neptune workbench](https://aws.amazon.com/blogs/database/visualize-query-results-using-the-amazon-neptune-workbench/)

* [Amazon Neptune User Guide](https://docs.aws.amazon.com/neptune/latest/userguide/graph-get-started.html)

* [Amazon Neptune Resources](https://aws.amazon.com/neptune/developer-resources/)

* [Amazon Neptune documentation](https://docs.aws.amazon.com/neptune/latest/userguide/intro.html)

* [Getting started with Amazon Neptune](https://aws.amazon.com/neptune/?c=db&sec=srv)

* [Introducing Amazon Neptune Global Database](https://aws.amazon.com/blogs/database/introducing-amazon-neptune-global-database/)


### Additional Resources

* [Amazon Neptune features](https://docs.aws.amazon.com/neptune/latest/userguide/feature-overview.html)

* [AWS Graph Notebooks](https://github.com/aws/graph-notebook)

* [About Amazon Neptune](https://aws.amazon.com/neptune/)

* [Learning Path for Amazon Neptune](https://pages.awscloud.com/AWS-Learning-Path-Getting-Started-with-Amazon-Neptune_2020_LP_0009-DAT.html)

* [Keep up to date withe the latest news from AWS Database Blogs for Amazon Neptune](https://aws.amazon.com/blogs/database/category/database/amazon-neptune/)

* [Amazon Neptune simplifies graph analytics and machine learning workflows with Python integration](https://aws.amazon.com/about-aws/whats-new/2022/06/amazon-neptune-simplifies-graph-analytics-machine-learning-workflows-python-integration/)

### Register for Australia's biggest data engineering conference DataEngBytes:

* Melbourne: 27 September 2022
* Sydney: 29 September 2022

Register here: https://dataengconf.com.au/


![Data Eng Bytes](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qzzqs2s8tte2hzqrf6n8.jpg)

###  AWS Innovate Online Conference APJ - 23 & 25 August 2022


![AWS Innovate](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/st6r8cnixap7o4log7yw.jpg)

Register at this [link](https://aws.amazon.com/events/aws-innovate/apj/data/)


### 🚀 Hot off the press: Introducing Amazon Neptune Global Database 

From 27 July 2022, AWS Neptune Global Database can build graph relationships in multiple regions including US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Europe (Ireland), Europe (London), and Asia Pacific (Tokyo) Regions.

You can read about this announcement from the AWS Database Blog written by author Navtanay Sinha [here](https://aws.amazon.com/blogs/database/introducing-amazon-neptune-global-database/).

Until the next lesson, happy learning ! 😁