Project: M3 (Observability and Analysis / Observability)
File: binaries_cluster.md

linktitle: "Binaries" title: Creating an M3 Cluster with Binaries weight: 2 This guide shows you the steps involved in creating an M3 cluster using M3 binaries, typically you would automate this with infrastructure as code tools such as Terraform or . {{% notice note %}} This guide assumes you have read the , and builds upon the concepts in that guide. {{% /notice %}} {{< fileinclude file="cluster-architecture.md" >}} You can download the latest release as . Inside the expanded archive are binaries for `m3dbnode`, which combines a coordinator and storage node, and a binary for `m3coordinator`, which is a standalone coordinator node. and run `make m3dbnode` to generate a binary for a combination coordinator and storage node, or `make m3coordinator` to generate a binary for a standalone coordinator node. M3 in production can run on local or cloud-based VMs, or bare-metal servers. M3 supports all popular Linux distributions (Ubuntu, RHEL, CentOS), and if you have any issues with your preferred distribution. {{% notice tip %}} If you use AWS or GCP, we recommend you use static IPs so that if you need to replace a host, you don't have to update configuration files on all the hosts, but decommission the old seed node and provision a new seed node with the same host ID and static IP that the old seed node had. If you're using AWS you can use an on a Virtual Private Cloud (VPC) and for GCP you can use an . {{% /notice %}} This example creates three static IP addresses for three storage nodes, using the embedded coordinator. This guide assumes you have host names configured, i.e., running `hostname` on a host in the cluster returns the host ID you use when creating the cluster placement. {{% notice tip %}} When using GCP the name of your instance is the host name. When you create an instance, click Management, disks, networking, SSH keys, under Networking, click the default interface, click the Primary internal IP drop down, select Reserve a static internal IP address, give it an appropriate name and description, and use Assign automatically. {{% /notice %}} {{% notice tip %}} When using AWS, you can use the host name supplied for the provisioned VM as your host ID, or use the `environment` host ID resolver and pass the host ID when launching the database process with an environment variable. {{% /notice %}} For example, if you used `M3DBHOSTID` for the environment variable name, use the following in your configuration: ```yaml hostID: resolver: config value: ${M3DBHOSTID:""} ``` Then start the `m3dbnode` process with: {{< tabs name="start_container" >}} {{% tab name="Pre-built binary" %}} ```shell M3DBHOSTID=m3db001 m3dbnode -f <config-file.yml> ``` {{% notice info %}} Depending on your operating system setup, you might need to prefix the command with `sudo`. {{% /notice %}} {{% /tab %}} {{% tab name="Output" %}} <!-- TODO: Perfect image, pref with terminalizer --> <!-- TODO: Update image --> {{% /tab %}} {{< /tabs >}} Depending on the default limits of your bare-metal machine or VM, M3 may need some Kernel tweaks to run as efficiently as possible, and before running M3 in production. You configure each M3 component by passing the location of a YAML file with the `-f`

The file groups configuration into `coordinator` or `db` sections that represent the `M3Coordinator` and `M3DB` instances of single-node cluster. {{% notice tip %}} You can find more information on configuring M3DB in the . {{% /notice %}} {{% notice note %}} The steps in this guide have the following 3 seed nodes, you need to change your configuration to suit the details of yours, including the details of an etcd cluster in the `m3dbCluster` > `endpoints` section of the M3 configuration file. {{% /notice %}} m3db001 (Region=us-east1, Zone=us-east1-a, Static IP=10.142.0.1) m3db002 (Region=us-east1, Zone=us-east1-b, Static IP=10.142.0.2) m3db003 (Region=us-east1, Zone=us-east1-c, Static IP=10.142.0.3) and change it to suit your cluster. The example below connects to an etcd instance in a zone called `eu-1` This example updates the `service` and `seedNodes` sections to match the node details above: <!-- TODO: Add more details on config items here --> ```yaml config: discovery: type: m3db_cluster m3dbCluster: env: default_env endpoints: 10.142.0.1:2379 10.142.0.2:2379 10.142.0.3:2379 ``` Start each seed node in the cluster using the same configuration file, and adjusting the `M3DBHOSTID` value to match the host name. ```shell M3DBHOSTID=m3db001 m3dbnode -f <config-file.yml> ``` {{% notice tip %}} You can daemon-ize the node startup process using your favorite utility such as systemd, init.d, or supervisor. {{% /notice %}} <!-- TODO: Again partialise and include across guides when complete --> <!-- TODO: Probably take this example up a notch for the how to guides --> This guide uses the {{% apiendpoint %}}database/create endpoint that creates a namespace, and the placement if it doesn't already exist based on the `type` argument. You can create and separately if you need more control over their settings. In the example below, the configuration for each host matches the details outlined above for the three nodes used. `isolationGroup` specifies how the cluster places shards to avoid more than one replica of a shard appearing in the same replica group. You should use at least as many isolation groups as your replication factor. This example uses the availability zones `us-east1-a`, `us-east1-b`, `us-east1-c` as the isolation groups which matches our replication factor of 3. . {{< tabs name="database_create" >}} {{% tab name="Command" %}} <!-- TODO: create partial --> ```shell curl -X POST {{% apiendpoint %}}database/create -d '{ "type": "cluster", "namespaceName": "default", "retentionTime": "48h", "numShards": "1024", "replicationFactor": "3", "hosts": [ { "id": "m3db001", "isolationGroup": "us-east1-a", "zone": "embedded", "weight": 100, "address": "10.142.0.1", "port": 9000 }, { "id": "m3db002", "isolationGroup": "us-east1-b", "zone": "embedded", "weight": 100, "address": "10.142.0.2", "port": 9000 }, { "id": "m3db003", "isolationGroup": "us-east1-c", "zone": "embedded", "weight": 100, "address": "10.142.0.3", "port": 9000 } ] }' ``` {{% /tab %}} {{% tab name="Output" %}} ```shell 20:10:12.911218[I] updating database namespaces [{adds [default]} {updates []} {removals []}] 20:10:13.462798[I] node tchannelthrift: listening on 0.0.0.0:9000 20:10:13.463107[I] cluster tchannelthrift: listening on 0.0.0.0:9001 20:10:13.747173[I] node httpjson: listening on 0.0.0.0:9002 20:10:13.747506[I] cluster httpjson: listening on 0.0.0.0:9003 20:10:13.747763[I] bootstrapping shards for range starting ... ... 20:10:13.757834[I] bootstrap finished [{namespace metrics} {duration 10.1261ms}] 20:10:13.758001[I] bootstrapped 20:10:14.764771[I] successfully updated topology to 3 hosts ``` {{% /tab %}} {{< /tabs >}} If you need to setup multiple namespaces, you can run the command above multiple times with different namespace configurations. {{< fileinclude file="cluster-common-steps.md" >}} <!-- ## Next Steps This quickstart covered getting a single-node M3DB cluster running, and writing and querying metrics to the cluster. Some next steps are: one two -->