Secure Microservice using Istio w/ JWT
Tags: istio, zta, auth0, microservices
Published: 2022-07-22 20:33:00

Istio [security pillar](https://istio.io/latest/docs/concepts/security/) provides a comprehensive solution to secure your workloads with strong identity (with optional integration with [SPIRE](https://spiffe.io/docs/latest/spire-about/) since v1.14), authentication, and authorization (with optional integration with [OPA](https://www.openpolicyagent.org/docs/latest/envoy-tutorial-istio/)). Istio sidecar (service proxy) and ingressgateway (edge proxy) serve as the [PEP](https://csrc.nist.gov/glossary/term/pep) for the entire mesh.

According to [NIST SP 800-207](https://csrc.nist.gov/publications/detail/sp/800-207/final) that **no resource is inherently trusted** that 1) every asset (service) MUST have its security posture evaluated via a PEP before a request is granted ... 2) and the evaluation should be continual for as long as the session lasts. Istio typically operates at L7. And in order to implement Defense-in-Depth strategy we also need to complement it with L3/4 enforcement through [NetworkPolicy](https://kubernetes.io/docs/concepts/services-networking/network-policies/). Please check out [additional security considerations](https://istio.io/latest/docs/tasks/traffic-management/egress/egress-gateway/#additional-security-considerations), and we won't talk about it. Instead, let me walk you through how to protect the workload using Istio with [JWT](https://jwt.io/).

You can acquire a free JWT token through [auth0](https://auth0.com/). We'll use it for the [demo](https://github.com/feiyao/istio_authn-authz_jwt#setup-auth0).

Obviously, we need to enforce mTLS using PeerAuthentication. By default, Istio `enableAutoMtls` is set to `true` but it was in `PERMISSIVE` mode.
```
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: istio-system
spec:
  mtls:
    mode: STRICT
```

We need to make sure the request MUST be denied at the edge (istio-ingressgateway) either without JWT token or invalid JWT token with the combination of `RequestAuthentication` and `AuthorizationPolicy`.

`RequesetAuthentication` CRD will make sure JWT token adheres to the `jwtRules` specified.
```
apiVersion: security.istio.io/v1beta1
kind: RequestAuthentication
metadata:
  name: jwt-authn-gw
  namespace: istio-gateway
spec:
  selector:
    matchLabels:
      istio: ingressgateway
  jwtRules:
  - issuer: "https://dev-wl5b26zy.us.auth0.com/"
    jwksUri: "https://dev-wl5b26zy.us.auth0.com/.well-known/jwks.json"
    audiences:
    - "https://httpbin/api"
    forwardOriginalToken: true
```

JWT Token typically uses RS256(RSA Signature with SHA-256) as the asymmetric signing algorithm. Istio will make sure the token is indeed valid and tamper-proof by verifying the digital signature through `jwksUri`. When a request comes in it goes through various [HTTP filters](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/http/http_filters), and one of them is `envoy.filters.http.jwt_authn`. To be more efficient the JWKS will be "cached" in `localJwks` to improve the performance instead of every time having outbound call to the `jwksUri` endpoint.

```
...
{
    "name": "envoy.filters.http.jwt_authn",
    "typedConfig": {
        "@type": "type.googleapis.com/envoy.extensions.filters.http.jwt_authn.v3.JwtAuthentication",
        "providers": {
            "origins-0": {
                "issuer": "https://dev-wl5b26zy.us.auth0.com/",
                "audiences": [
                    "https://httpbin/api"
                ],
                "localJwks": {
                    "inlineString": "..."
                },
                "forward": true,
                "payloadInMetadata": "https://dev-wl5b26zy.us.auth0.com/"
            }
        },
...
}
...
```
`istio_authn` filter is right after the `jwt_authn` filter. And it tries to verify the issuer as well:
```
{
    "name": "istio_authn",
    "typedConfig": {
        "@type": "type.googleapis.com/istio.envoy.config.filter.http.authn.v2alpha1.FilterConfig",
        "policy": {
            "origins": [
                {
                    "jwt": {
                        "issuer": "https://dev-wl5b26zy.us.auth0.com/"
                    }
                }
            ],
            "originIsOptional": true,
            "principalBinding": "USE_ORIGIN"
        },
        "skipValidateTrustDomain": true
    }
}
```

Just with `RequesetAuthentication` will not be enough since [a request without any authentication credentials will be accepted but will not have any authenticated identity](https://istio.io/latest/docs/reference/config/security/request_authentication/). That's why it's quintessential to have `AuthorizationPolicy` in place to enforce.

First, it stipulates that the request will be denied at istio-ingressgateway if it doesn't have the `RequestPrincipal`.
```
apiVersion: "security.istio.io/v1beta1"
kind: AuthorizationPolicy
metadata:
  name: deny-jwt-gw
  namespace: istio-gateway
spec:
  selector:
    matchLabels:
      istio: ingressgateway
  action: DENY
  rules:
  - from:
    - source:
        notRequestPrincipals: ["*"]
```
Second, at the service level it has to satisfy the claims that the workload expects. In order to make it work it's important to have `forwardOriginalToken: true` to allow JWT token to be forwarded from the istio-ingressgateway to the workload.
```
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: require-jwt-httpbin
  namespace: default
spec:
  selector:
    matchLabels:
      app: httpbin
  action: ALLOW
  rules:
  - from:
    - source:
        requestPrincipals: ["https://dev-wl5b26zy.us.auth0.com//8qPXVf5npNa4yXmeyHhnGh5GDgrDK3B5@clients"]
    when:
    - key: request.auth.claims[scope]
      values: ["read:messages"]
    - key: request.auth.claims[aud]
      values: ["https://httpbin/api"]
```

These enforcement have been "encoded" into workload's sidecar EnvoyProxy HTTP Filter called `envoy.filters.http.rbac`. Any violation will cause `403 Forbidden` with the error `RBAC: access denied`.

```
{
    "name": "envoy.filters.http.rbac",
    "typedConfig": {
        "@type": "type.googleapis.com/envoy.extensions.filters.http.rbac.v3.RBAC",
        "rules": {
            "policies": {
                "ns[default]-policy[require-jwt-httpbin]-rule[0]": {
                    "permissions": [
                        {
                            "andRules": {
                                "rules": [
                                    {
                                        "any": true
                                    }
                                ]
                            }
                        }
                    ],
                    "principals": [
                        {
                            "andIds": {
                                "ids": [
                                    {
                                        "orIds": {
                                            "ids": [
                                                {
                                                    "metadata": {
                                                        "filter": "istio_authn",
                                                        "path": [
                                                            {
                                                                "key": "request.auth.principal"
                                                            }
                                                        ],
                                                        "value": {
                                                            "stringMatch": {
                                                                "exact": "https://dev-wl5b26zy.us.auth0.com//8qPXVf5npNa4yXmeyHhnGh5GDgrDK3B5@clients"
                                                            }
                                                        }
                                                    }
                                                }
                                            ]
                                        }
                                    },
...
        },
        "shadowRulesStatPrefix": "istio_dry_run_allow_"
    }
}
```

Thanks for reading, and learning with me. If you want to run it yourself here is the [source code](https://github.com/feiyao/istio_authn-authz_jwt) and instructions.

Voila!