Keda com SQS e HTTP Scaler no EKS [Lab Session]
Tags: community, kubernetes, keda, aws
Published: 2024-02-13 20:13:24



> KEDA is a **Kubernetes-based Event Driven Autoscaler**. With KEDA, you can drive the scaling of any container in Kubernetes based on the number of events needing to be processed.
> KEDA is a single-purpose and lightweight component that can be added into any Kubernetes cluster. KEDA works alongside standard Kubernetes components like the Horizontal Pod Autoscaler and can extend functionality without overwriting or duplication. With KEDA you can explicitly map the apps you want to use event-driven scale, with other apps continuing to function. This makes KEDA a flexible and safe option to run alongside any number of any other Kubernetes applications or frameworks.
> https://keda.sh/

[**Keda Architecture**](https://keda.sh/docs/2.13/concepts/#architecture)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/66yjoekesmpp66c6zh3c.png)

---
**Repositórios**

* [Repositório de infraestrutura (terraform) - EKS-Istio-Karpenter](https://github.com/paulofponciano/EKS-Istio-Karpenter.git)
* [Repositório keda](https://github.com/paulofponciano/keda.git)
* [Repositório K6](https://github.com/paulofponciano/k6.git)

---

**Lab architecture**

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ylzygib6uvf0y26ou4iv.png)

---

**Deploy do EKS**

`git clone https://github.com/paulofponciano/EKS-Istio-Karpenter.git`
`cd EKS-Istio-Karpenter`
`tofu init`
`tofu plan --var-file variables.tfvars`
`tofu apply --var-file variables.tfvars`

**Utilizando SQS Scaler**

- **Criar fila SQS e IAM policy:**

```sh
aws sqs create-queue --queue-name keda-sqs --region REGION
```

```sh
aws iam create-policy --policy-name keda-sqs-policy \
  --policy-document file://policy-sqs.json
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a3mgp8ce6cj0rep7nvz8.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5jbeigwc0q7jzw8chipj.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dwo8j5lqc3ew7281t7fy.png)

- **Criar IAM role para service account (IRSA):**

```sh
eksctl create iamserviceaccount --name keda-operator \
  --namespace keda \
  --cluster pegasus \
  --attach-policy-arn arn:aws:iam::ACCOUNTID:policy/keda-sqs-policy \
  --region REGION \
  --profile default \
  --approve
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xsxpxg90gewlsqe93ej8.png)

`kubectl describe sa keda-operator -n keda`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0sqwnz7emcz2oi1xvf62.png)

- **Keda deploy via helm:**

```sh
helm repo add kedacore https://kedacore.github.io/charts
helm repo update
helm install keda kedacore/keda --namespace keda \
  --set serviceAccount.create=false \
  --set serviceAccount.name=keda-operator
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yysbupvqfs6yo3ict0nf.png)

Veja que definimos _'create service account'_ como _false_, pois já criamos anteriormente e iremos utilizar aquela.

`kubectl get pods -n keda`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7z834vgqa9slfkfn71qg.png)

- **Deployment de teste e ScaledObject:**

`kubectl apply -f deployment.yaml`

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx-deployment
spec:
  replicas: 0
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx
```

`kubectl apply -f scaledobject.yaml`

```yaml
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: aws-sqs-queue-scaledobject
  namespace: default
spec:
  scaleTargetRef:
    name: nginx-deployment
  minReplicaCount: 0
  maxReplicaCount: 5
  pollingInterval: 5
  cooldownPeriod:  25
  triggers:
  - type: aws-sqs-queue
    metadata:
      queueURL: https://sqs.REGION.amazonaws.com/ACCOUNTID/keda-sqs
      queueLength: "15"
      awsRegion: "REGION"
      identityOwner: operator
```

Podemos ver que ainda não existe nenhum _pod_ criado pelo _deployment_ que fizemos. Isso porque o numero de réplicas é definido em '0' e a fila SQS está vazia, assim o keda não irá acionar a criação:

`kubectl get pods | grep nginx`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/plb23b6on6i7obkci1ah.png)

- **Colocando mensagens na fila SQS:**

```sh
for i in `seq 35`; do
  aws sqs send-message --queue-url 'https://sqs.REGION.amazonaws.com/ACCOUNTID/keda-queue' \
    --message-body "testmessage" \
    --region REGION \
    --no-cli-pager \
    --output text
done
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/raf1u3z11z1wbjh0g8ig.png)

Agora temos mensagens na fila. No parâmetro **_'queueLength'_** do **_ScaledObject_** está definido '15', assim o keda já deve escalar nosso _deployment_:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fubo9g7zo1yeieqmbqie.png)

`kubectl get pods | grep nginx`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eq30hjdrbwehw8f2mz3p.png)

Assim temos o keda trabalhando no kubernetes de acordo com a demanda na fila SQS.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/u2uq6fynj4jbionfozmp.png)

**Utilizando HTTP Scaler**

- **Deploy addon http:**

`helm install http-add-on kedacore/keda-add-ons-http --namespace keda`
`kubectl get pods -n keda`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ao4hjkolae00zhu9b5jm.png)

- **Deployment de teste com ingress e ScaledObject:**

`kubectl apply -f deployment.yaml`

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx-2
  name: nginx-deployment-2
spec:
  replicas: 0
  selector:
    matchLabels:
      app: nginx-2
  template:
    metadata:
      labels:
        app: nginx-2
    spec:
      containers:
      - image: nginx
        name: nginx-2
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service-2
  labels:
    app: nginx-2
    service: nginx-service-2
spec:
  ports:
  - port: 80
    name: http
  selector:
    app: nginx-2
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: nginx-gateway
spec:
  selector:
    istio: ingressgateway # use istio default controller
  servers:
  - port:
      number: 443
      name: http
      protocol: HTTP
    hosts:
    - "demo-ingress.pauloponciano.digital"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx
spec:
  hosts:
  - "demo-ingress.pauloponciano.digital"
  gateways:
  - nginx-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: keda-add-ons-http-interceptor-proxy.keda.svc.cluster.local
        port:
          number: 8080
```

`kubectl apply -f http_scaledobject.yaml`

```yaml
kind: HTTPScaledObject
apiVersion: http.keda.sh/v1alpha1
metadata:
    name: nginx-http-scaledobject
spec:
    hosts:
    - demo-ingress.pauloponciano.digital
    targetPendingRequests: 1
    scaleTargetRef:
        deployment: nginx-deployment-2
        service: nginx-service-2
        port: 80
    replicas:
        min: 0
        max: 5
```

- **Gerando requests com k6:**

_request.js_

```js
import { check } from 'k6';
import http from 'k6/http';

export const options = {
  scenarios: {
    constant_request_rate: {
      executor: 'constant-arrival-rate',
      rate: 100,
      timeUnit: '1s', // 100 iterations per second, i.e. 100 RPS
      duration: '120s',
      preAllocatedVUs: 100, // how large the initial pool of VUs would be
      maxVUs: 100, // if the preAllocatedVUs are not enough, we can initialize more
    },
  },
};

export function test(params) {
  const res = http.get('http://demo-ingress.pauloponciano.digital');
  check(res, {
    'is status 200': (r) => r.status === 200,
  });
}

export default function () {
  test();
}
```

Executamos o _script_ e podemos observar o keda fazendo o _scaling_ do nosso _deployment_ de acordo com as requisições e com o que foi definido no **_ScaledObject_**:

`k6 run request.js --vus=1000 --duration=60m`

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tr5s7666phcwl0afiggh.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vdet2tmw0nklpkx408fv.png)

---

Happy building!