Observabilidade de microsserviços com OpenTelemetry e Amazon OpenSearch [Lab Session]
Tags: community, aws, o11y
Published: 2024-01-29 18:58:44

> OpenTelemetry is a collection of tools, APIs, and SDKs. Use it to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) to help you analyze your software’s performance and behavior.
> [https://opentelemetry.io/](https://opentelemetry.io/)

![Topologia](https://cdn-images-1.medium.com/max/2000/1*eDLd5Xkvw6vPbUM89ka0EQ.png)
> Referências/Reference:
> [https://catalog.us-east-1.prod.workshops.aws/workshops/1abb648b-2ef8-442c-a731-efbcb69c1e1e/en-US](https://catalog.us-east-1.prod.workshops.aws/workshops/1abb648b-2ef8-442c-a731-efbcb69c1e1e/en-US)
> [https://github.com/aws-samples/observability-with-amazon-opensearch.git](https://github.com/aws-samples/observability-with-amazon-opensearch.git)
> [https://github.com/idealo/terraform-aws-opensearch.git](https://github.com/idealo/terraform-aws-opensearch.git)

---

[Código terraform aqui.](https://github.com/paulofponciano/lab-microservices-o11y-amazon-opensearch.git)

### Variáveis (VPC)

```tf
# REGIAO DO PROVISIONAMENTO
variable "aws_region" {
  default = "us-east-2"
}

# PRIMEIRA ZONA DE DISPONIBILIDADE
variable "az1" {
  default = "us-east-2a"
}

# SEGUNDA ZONA DE DISPONIBILIDADE
variable "az2" {
  default = "us-east-2b"
}

# USAR AQUI A IDENTIFICACAO DO AMBIENTE
variable "customer_env" {
  default = "lab-o11y"
}

# INFORME O NOME QUE IRA UTILIZAR NO CLUSTER EKS # IMPORTANTE PARA RESOURCE K8S SHARING
variable "cluster_eks_name" {
  default = "lab-o11y"
}

# CIDR VPC
variable "vpc_cidr_block" {
  default = "172.29.0.0/16"
}

# SUBNET PUBLICA AZ1
variable "subnet_public_1_cidr" {
  default = "172.29.0.0/20"
}

# SUBNET PUBLICA AZ2
variable "subnet_public_2_cidr" {
  default = "172.29.16.0/20"
}

# SUBNET PRIVADA AZ1
variable "subnet_private_1_cidr" {
  default = "172.29.32.0/20"
}

# SUBNET PRIVADA AZ2
variable "subnet_private_2_cidr" {
  default = "172.29.48.0/20"
}
```

### Variáveis (EKS) 

```tf
# Project info
    
cluster_name = "lab-o11y"
project      = "o11y with otel and aos"
project_env  = "dev"
region       = "us-east-2"
k8s_version  = "1.24"

# VPC and Subnets

vpc_id         = "vpc-0c496e66aaff25697"
vpc_cidr_block = "172.29.0.0/16"

public_access_cidrs = [
  "0.0.0.0/0",
]

subnets = [ # Private subnets for eks cluster
  "subnet-05c30cc378277b402",
  "subnet-03e9fd8df45175919"
]

public_subnets = [ # Public subnets for nlb
  "subnet-09881eca61aeeb25a",
  "subnet-0300707fbef49d07d"
]

# Autoscaling configuration

instance_type = "t3.medium"
desired_size  = "2"
min_size      = "1"
max_size      = "4"

# Disk

disk_size = "30"

# Endpoint access

endpoint_private_access = false
endpoint_public_access  = true

# Logging

enabled_cluster_log_types = [
  "api",
  "audit",
  "controllerManager"
]
```

### Variáveis (OpenSearch)

Abaixo apenas a variáveis que precisam ser informadas neste lab:

```tf
variable "cluster_name" {
  description = "The name of the OpenSearch cluster."
  type        = string
  default     = "lab-o11y"
}

variable "cluster_version" {
  description = "The version of OpenSearch to deploy."
  type        = string
  default     = "2.3"
}

variable "master_user" {
  description = "The master user of the OpenSearch cluster."
  type        = string
  default     = "workshop"
}

variable "master_password" {
  description = "The master password of the OpenSearch cluster."
  type        = string
  default     = "Workshop@2023"
}
```

> Sequência de execução terraform:
> 1. VPC: [https://github.com/paulofponciano/lab-microservices-o11y-amazon-opensearch/tree/main/vpc](https://github.com/paulofponciano/lab-microservices-o11y-amazon-opensearch/tree/main/vpc)
> 2. EKS: [https://github.com/paulofponciano/lab-microservices-o11y-amazon-opensearch/tree/main/eks](https://github.com/paulofponciano/lab-microservices-o11y-amazon-opensearch/tree/main/eks)
> 3. OpenSearch (fork): [https://github.com/paulofponciano/lab-microservices-o11y-amazon-opensearch/tree/main/terraform-aws-opensearch/examples/minimal](https://github.com/paulofponciano/lab-microservices-o11y-amazon-opensearch/tree/main/terraform-aws-opensearch/examples/minimal)

### Construção

- Stack VPC:

`terraform init`
`terraform plan`
`terraform apply`

Output:

![](https://cdn-images-1.medium.com/max/2000/1*FW3li1x7ZLXpAuHjLgNptg.png)

- Stack EKS:

`terraform init`
`terraform plan --var-file variables.tfvars`
`terraform apply --var-file variables.tfvars`

Output:

![](https://cdn-images-1.medium.com/max/2000/1*jNyw6hNAk-wKVRY2w3cdbQ.png)

- Stack OpenSearch:

`terraform init`
`terraform plan`
`terraform apply`

Não é preciso informar os dois *inputs* solicitados abaixo no _apply_:

![](https://cdn-images-1.medium.com/max/2116/1*T374x5EQt3EgBrgJTXN9Tg.png)

Output (Console)

![](https://cdn-images-1.medium.com/max/3184/1*S3dbFsdQr7GMV_Vd7_ykyg.png)

- Informe o _domain endpoint_, _user_ e _password_ do AOS no arquivo _yaml_ que está em _data-prepper.yaml_ — [https://github.com/paulofponciano/lab-microservices-o11y-amazon-opensearch/blob/main/eks/samples/apps-ms/01-data-prepper/kubernetes/](https://github.com/paulofponciano/lab-microservices-o11y-amazon-opensearch/blob/main/eks/samples/apps-ms/01-data-prepper/kubernetes/data-prepper.yaml)

![data-prepper.yaml](https://cdn-images-1.medium.com/max/2030/1*voDTWa8cfik_dAZeHE-iXQ.png)

- Conecte no _cluster_ EKS:

`aws eks --region us-east-2 update-kubeconfig --name lab-o11y`

- Faça o _deploy_ dos microsserviços executando o _script_ _app-deploy.sh_ — [https://github.com/paulofponciano/lab-microservices-o11y-amazon-opensearch/tree/main/eks/samples/apps-ms](https://github.com/paulofponciano/lab-microservices-o11y-amazon-opensearch/tree/main/eks/samples/apps-ms):

`chmod +x app-deploy.sh`
`./app-deploy.sh`

![Deploy](https://cdn-images-1.medium.com/max/2000/1*1ArXE6HU2lFHmyn35ck1TQ.png)

- Acesse a aplicação do _AWS workshop_ e gere dados — É possível utilizar a _url_ do _Network Load Balancer_ que foi criado na _stack_ do EKS:

![](https://cdn-images-1.medium.com/max/3118/1*Krkxn10O3KqOiq8nsvwcbw.png)

- Utilizando a _url_ do Amazon OpenSearch Dashboards, faça login com _user_ e password _master_ informados na criação da stack OpenSearch:

![](https://cdn-images-1.medium.com/max/2880/1*JJWa1LuNqBa_n_sOBf6rqQ.png)

- No OpenSearch Dashboards, agora é possível explorar os _traces_ recebidos através do OpenTelemetry:

Observability — Trace analytics — Traces

![](https://cdn-images-1.medium.com/max/3820/1*ioa27w6yVLyNXrBe2Rn9Lg.png)

![](https://cdn-images-1.medium.com/max/3406/1*H13HkEvCYs7lGrb-l1Hu3A.png)

![](https://cdn-images-1.medium.com/max/3410/1*d-aTs7ZKx7lsynhVJlufuQ.png)

![](https://cdn-images-1.medium.com/max/3402/1*AKZZMzfxGxXVtm1w6brtRg.png)

![](https://cdn-images-1.medium.com/max/3650/1*pa-YJWxW_mUikSQQlwEZGA.png)

- Crie um novo _index pattern_ para os _logs_ do kubernetes que são recebidos através do fluent-bit:

Stack Management — Index patterns

![](https://cdn-images-1.medium.com/max/3432/1*a9Xh9CIwpU7ta02DCkeKAQ.png)

- Em 'Discover' é possível explorar os _logs_ conforme o _index pattern_ criado anteriormente:

![](https://cdn-images-1.medium.com/max/3832/1*UEjX8ygjmCKeVWhPaKBrjA.png)

Happy building!