How to Automate Neo4j Deploys on Azure
By: David Allen
Tags: Graph, Arm, Neo4j, Azure, Deployment

Neo4j already provides some documentation on its site for how to do deployments of Neo4j to common clouds, including on Azure. But in this article, I’ll provide sample shell scripts that can do this automatically for you.

These are useful when you want to integrate Neo4j into your CI/CD pipeline and be able to create/destroy instances temporarily, and also just to spin up a sample instance. But really, if you can automate Neo4j deployment, then any other piece of software can make a Neo4j instance whenever it needs, which is extremely handy.

If you have any feedback or questions on these approaches, drop by the Neo4j Community site on this thread and share!

Neo4j on Azure using Azure Resource Manager (ARM) templates

Requirements

Before we begin, you’ll need the az command line interface program, which you can download and install with directions here. The az CLI is the main way you can automate all things with Azure. You’ll also need the jq tool for working with JSON responses, and other common linux utilities.

It will also be necessary to authenticate your az CLI, to make sure it can interact with your resource groups, and that it is configured to use the right subscription by default. You can change the subscription the tool uses by consulting the documentation.

Azure Resource Manager

Neo4j provides Azure Resource Manager (ARM) templates for Neo4j Causal Cluster (highly available clusters), and VM images for Neo4j Enterprise stand-alone. So first thing’s first, pick which one you would like to deploy. We’ll cover both in this article.

ARM templates are really just a recipe that tells Azure how to deploy a whole set of interrelated resources. By deploying all of this as a stack we can keep all of our resources together, and delete just one thing when we’re done. In this method, we will be creating a new resource group for everything that we need for our Neo4j setup, so that we can manage the entire instance by just the resource group.

Approach

How to deploy a cluster is very simple: we just submit a new ARM Deployment job, pointing to the right template URL and providing some parameters in a JSON file. To keep things simple and contained, we’ll create an Azure resource group for our deploy to group everything we need.

We’ll need to specify several common parameters, which you’ll see in the scripts below. Here’s an explanation of what they are.

VM Size : This is the Azure VM type you want to launch, which controls how much hardware you’ll be using.

: This is the Azure VM type you want to launch, which controls how much hardware you’ll be using. Disk Size/Type : you can use these parameters to control whether Neo4j uses standard spinning magnetic platters or SSD disks as well as how many GB of storage you want to allocate.

: you can use these parameters to control whether Neo4j uses standard spinning magnetic platters or SSD disks as well as how many GB of storage you want to allocate. Location : Where in the world you want to deploy Neo4j. The template supports any Azure location, but in our example we’ll use East US.

: Where in the world you want to deploy Neo4j. The template supports any Azure location, but in our example we’ll use Authentication details: specifically, the administrative username and password for access to the VMs. It’s possible to set up the administrative login via either SSH key or simple username/password. We’ll do it with a password because there are no extra pre-requisites to set up.

You can also choose other considerations like static/dynamic IP addresses, and naming in the script.

Let’s get started. Simply run any of these scripts, and it will result in a running Neo4j instance you can use immediately!

Deploying Neo4j Enterprise Causal Cluster

Because this is a clustered deploy, take note that we’re also using a parameter for “Cores” and “Read Replicas” to control how many nodes are in our cluster. This is in addition to the parameters we described above.

The way ARM works, we’ll create a simple JSON file with all of the parameters we want submitted to the deployment, then create the deployment pointing to that JSON file. The ARM template is like a small function: it takes some inputs we’ll provide here, and produces a set of infrastructure as an output.

#!/bin/bash export CORE_NODES=3

export READ_REPLICAS=0

export NEO4J_PASSWORD=s00pers3cR3T:

export ADMIN_AUTH_TYPE=password

export USERNAME=graph-hacker

export ADMIN_PASSWORD=s00pers3cR3T:

export VM_SIZE=Standard_B2ms

export DISK_TYPE=StandardSSD_LRS

export DISK_SIZE=256

export IP_ALLOCATION=Dynamic

export SEED=$(head -c 3 /dev/urandom | base64 | sed 's/[^a-zA-Z0-9]/X/g')

export RESOURCE_GROUP="neo4j-RG-${SEED}"

export CLUSTERNAME="neo4j-${SEED}"

export DEPLOYMENT=neo4j-bmdeploy

export LOCATION="East US" # The ARM template to deploy

export TEMPLATE_BASE=http://neo4j-arm.s3.amazonaws.com/3.5.5/causal-cluster/

export TEMPLATE_URL=${TEMPLATE_BASE}mainTemplate.json echo $(cat <<JSON

{

"ClusterName": { "value": "${CLUSTERNAME}" },

"CoreNodes": { "value": ${CORE_NODES} },

"ReadReplicas": { "value": ${READ_REPLICAS} },

"VmSize": { "value": "${VM_SIZE}" },

"DataDiskType": { "value": "${DISK_TYPE}" },

"DataDiskSizeGB": { "value": ${DISK_SIZE} },

"AdminUserName": { "value": "${USERNAME}" },

"AdminAuthType": { "value": "${ADMIN_AUTH_TYPE}" },

"AdminCredential": { "value": "${ADMIN_PASSWORD}" },

"PublicIPAllocationMethod": { "value": "${IP_ALLOCATION}" },

"Neo4jPassword": { "value": "${NEO4J_PASSWORD}" },

"_artifactsLocation": { "value": "${TEMPLATE_BASE}" }

}

JSON

) > "${RESOURCE_GROUP}.json" echo "Creating resource group named ${RESOURCE_GROUP}" if ! az group create --name "${RESOURCE_GROUP}" --location "${LOCATION}"; then

echo STACK_NAME=$RESOURCE_GROUP

echo "Failed to create necessary resource group ${RESOURCE_GROUP}"

exit 1

fi echo "Creating deployment" az group deployment create \

--template-uri "$TEMPLATE_URL" \

--parameters @./${RESOURCE_GROUP}.json \

--resource-group "${RESOURCE_GROUP}" \

--name "${DEPLOYMENT}" if [ $? -ne 0 ] ; then

echo STACK_NAME=$RESOURCE_GROUP

echo "Stack deploy failed"

exit 1

fi # JSON Path to server response where IP address is.

ADDR_FIELD=".[].virtualMachine.network.publicIpAddresses[0].ipAddress" IP_ADDRESS=$(az vm list-ip-addresses --resource-group "${RESOURCE_GROUP}" | jq -r "$ADDR_FIELD" | head -n 1) echo STACK_NAME=$RESOURCE_GROUP

echo NEO4J_URI=bolt+routing://$IP_ADDRESS:7687

At the end of this, a new resource group is created with all of the assets inside of it, and you get a URI of a bolt endpoint you can use. Alternatively, you could go to https://$IP_ADDRESS:7473/ to access Neo4j Browser for your new clustered instance.

Example of a deployed resource group

Deploying Neo4j Enterprise Stand Alone

This will create a single instance of Neo4j without high-availability failover capabilities, but it’s a very fast way to get started. For this deploy, we don’t use ARM but just create a simple VM and configure its firewall/security rules.

Neo4j provides the VM through an Azure marketplace offer. To refer to the right VM image, you need to know the publisher (that’s Neo4j), the “offer” (Neo4j 3.5 series) and the SKU (which is the particular version of Neo4j you’ll use).

Because we’re not using ARM for this one, this also provides an example of polling and waiting until the VM service comes up, and then changing the Neo4j default password when it does. At the top, you can choose a different password for the neo4j user as for the system administrator.

Make sure to customize the SUBSCRIPTION variable to make this work.

#!/bin/bash export LOCATION=eastus

export SUBSCRIPTION=My-Subscription-Name

export RG=neo4j-standalone-RG

export NAME=neo4j-standalone

export ADMIN_USERNAME=graph-hacker

export ADMIN_PASSWORD=ch00se:A@PASSw0rd

export NEO4J_PASSWORD=ch00se:A@PASSw0rd

export NETWORK_SECURITY_GROUP=neo4j-nsg

export VM_SIZE=Standard_D2_v3 # Options: https://azure.microsoft.com/en-us/pricing/details/virtual-machines/ export VM_SIZE=Standard_D2_v3 # Can change this to static if desired

export ADDRESS_ALLOCATION=dynamic # Configuration bits of what you're launching

# Publisher:Offer:Sku:Version

export PUBLISHER=neo4j

export OFFER=neo4j-enterprise-3_5

export SKU=neo4j_3_5_5_apoc

export VERSION=latest

export IMAGE=$PUBLISHER:$OFFER:$SKU:$VERSION echo "Creating resource group named $RG"

az group create --location $LOCATION \

--name $RG \

--subscription $SUBSCRIPTION echo "Creating Network Security Group named $NETWORK_SECURITY_GROUP"

az network nsg create \

--resource-group $RG \

--location $LOCATION \

--name $NETWORK_SECURITY_GROUP echo "Assigning NSG rules to allow inbound traffic on Neo4j ports..." prio=1000

for port in 7473 7474 7687; do

az network nsg rule create \

--resource-group $RG \

--nsg-name "$NETWORK_SECURITY_GROUP" \

--name neo4j-allow-$port \

--protocol tcp \

--priority $prio \

--destination-port-range $port

prio=$(($prio+1))

done echo "Creating Neo4j VM named $NAME"

az vm create --name $NAME \

--resource-group $RG \

--image $IMAGE \

--vnet-name $NAME-vnet \

--subnet $NAME-subnet \

--admin-username "$ADMIN_USERNAME" \

--admin-password "$ADMIN_PASSWORD" \

--public-ip-address-allocation $ADDRESS_ALLOCATION \

--size $VM_SIZE if [ $? -ne 0 ] ; then

echo "VM creation failed"

exit 1

fi echo "Updating NIC to have our NSG"

# Uses default assigned NIC name

az network nic update \

--resource-group "$RG" \

--name "${NAME}VMNic" \

--network-security-group "$NETWORK_SECURITY_GROUP" # Get the IP address of our instance

IP_ADDRESS=$(az vm list-ip-addresses -g "$RG" -n "$NAME" | jq -r '.[0].virtualMachine.network.publicIpAddresses[0].ipAddress') export NEO4J_URI=bolt://$IP_ADDRESS # Change password

echo "Checking if Neo4j is up and changing password...."

while true; do

if curl -s -I http://$IP_ADDRESS:7474 | grep "200 OK"; then

echo "Neo4j is up; changing default password" 2>&1

curl -v -H "Content-Type: application/json" \

-XPOST -d '{"password":"'$NEO4J_PASSWORD'"}' \

-u neo4j:neo4j \

http://$IP_ADDRESS:7474/user/neo4j/password 2>&1 echo "Password reset, signaling success" 2>&1

break

fi echo "Waiting for neo4j to come up" 2>&1

sleep 1

done echo NEO4J_URI=$NEO4J_URI

exit 0

Cleanup

To clean up and delete these deployment when you’re done, simply delete the entire resource group. Here’s a simple shell script which will get the job done.

In both cases (cluster and standalone) this works nicely because we put all resources together into a single resource group. So by dropping the group, there’s no individual resource cleanup, the entire setup goes away.

#!/bin/bash

if [ -z $1 ] ; then

echo "Usage: call me with deployment name"

exit 1

fi STACK_NAME=$1 if [ -f "$STACK_NAME.json" ] ; then

rm -f "$STACK_NAME.json"

fi az group delete -n "$STACK_NAME" --no-wait --yes exit $?

This will schedule the entire group to be deleted and the script will exit immediately. Actually deleting all the resources may take some minutes, and if you log into your Azure console you’ll see the progress in the notifications area.