Platform Engineering On Kubernetes Part 3: Crossplane
Tags: kubernetes, programming, devops, docker
Published: 2023-09-05 20:13:11

In Platform Engineering, the whole goal, at a high level, is to make software delivery and engineering more efficient while separating concerns between engineers and ensuring that there’s an underlying platform to carry out all of the expected tasks.

In this case, that platform is Kubernetes, but what’s the task? Or rather, how can Kubernetes carry out the task for anything outside of the cluster itself?

In part 2 of the Platform Engineering On Kubernetes series, you learned all about Cluster API along with the overlap when it comes to Operators and Cluster API. In part 3, you’ll learn about one of the top tools that truly makes Kubernetes use itself as the Platform to deploy various resources, Crossplane.

## Prerequisites

To follow along with the hands-on section, you should have access to Azure, AWS, or both. If you don’t, it would still be beneficial to read through the hands-on section as it’ll give you an idea of how Crossplane works from a declarative perspective.

## Why Crossplane

In the opening of this blog post, there was mention that Kubernetes can be the platform, but the question then becomes “How does Kubernetes manage other resources?”. Out of the box, Kubernetes can’t. Kubernetes has a very specific job - orchestrating containers. However, the way Kubernetes was designed was to give engineers the ability to extend its capabilities. That’s why Operators exist. It’s why tools like ArgoCD, Istio, etc. can give you the ability to manage environments in a declarative fashion (using the provided APIs from those tools in YAML).

Where Crossplane comes into play is it has an Operator, much like ArgoCD and Istio, but the Crossplane Operator isn’t to manage Crossplane. It’s to manage other resources and services.

For example, you can use the Crossplane Operator within Kubernetes to create an S3 bucket in AWS from Kubernetes. Literally, you can write a Kubernetes Manifest that creates an AWS S3 bucket.

In part 2 of this blog post, you learned about Cluster API, which is managing Kubernetes with Kubernetes. Crossplane is managing everything else with Kubernetes.

Crossplane gives you the ability to truly utilize Kubernetes to do anything and everything for creating, managing, and updating resources/services outside of Kubernetes. It’s arguably the definition of Platform Engineering.

## Crossplane Breakdown

Crossplane gives you a declarative method via Kubernetes to manage services, resources, and infrastructure outside of Kubernetes.

What this means is that you can do anything from creating an S3 bucket in AWS to creating virtual machines or AKS clusters. As long as the object is available in Crossplane, you can use it to create the resource/service/infrastructure.

For example, you can specify the API just like you would with any other Kubernetes Manifest

```jsx
apiVersion: compute.azure.crossplane.io/v1beta1
```

Then, you can specify the object/kind. In this case, it’s an object to create an AKS cluster.

```jsx
kind: AKSCluster
```

It doesn’t have to be a Kubernetes cluster you’re creating though. You can use the `[network.azure.crossplane.io/v1beta1](http://network.azure.crossplane.io/v1beta1)` API to create a Virtual Network in Azure.

In short, Crossplane allows you to create resources/services/infrastructure as you would in something like Terraform, except instead, you’d use a Kubernetes Manifest.

There are various Providers available, all of which can be found [here](https://marketplace.upbound.io/providers).

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8dauof8bj3nu2ujipv11.png)

## Using Crossplane

Now that you know the “how” and “why” behind Crossplane, let’s dive into the hands-on piece and see how it all works underneath the hood.

The first part will be the overall configuration. Next, you’ll see how to run workloads on both Azure and AWS.

### Configuration

First, add the Crossplane Helm repo.

```jsx
helm repo add crossplane-stable https://charts.crossplane.io/stable
```

Ensure that the Helm Chart is up to date.

```jsx
helm repo update
```

Lastly, install Crossplane. The Helm Chart for installation is straightforward in the sense that it isn’t mandatory to configure a lot of values.

```jsx
helm install crossplane \
crossplane-stable/crossplane \
--namespace crossplane-system \
--create-namespace
```

### AWS

Once Crossplane is installed, you can start going through the actual configuration of using it.

To use Crossplane, regardless of which Provider you’re using (Azure, AWS, GCP, etc.), you’ll need two key components:

1. The Provider itself
2. Credentials for authentication

First, set up the Provider. You’ll see that the below code configures the AWS provider.

```jsx
cat <<EOF | kubectl apply -f -
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-aws
spec:
  package: xpkg.upbound.io/upbound/provider-aws-s3:v0.37.0
EOF
```

To ensure that the provider was installed properly, run the following command.

```jsx
kubectl get providers
```

Next, it’s time to configure the credentials. Generate an access key and secret key from AWS IAM. Then, insert them into the below and save it to a file called `creds.txt`

```jsx
[default]
aws_access_key_id = 
aws_secret_access_key =
```

Create a new Kubernetes secret with the IAM permissions.

```jsx
kubectl create secret \
generic aws-secret \
-n crossplane-system \
--from-file=creds=./creds.txt
```

Create a Provider Config to customize Crossplane to use the secret you just created.

```jsx
cat <<EOF | kubectl apply -f -
apiVersion: aws.upbound.io/v1beta1
kind: ProviderConfig
metadata:
  name: creds
spec:
  credentials:
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: aws-secret
      key: creds
EOF
```

You’re now ready to use Crossplane. Below is a configuration to create an S3 bucket.

```jsx
kubectl apply -f -
apiVersion: s3.aws.upbound.io/v1beta1
kind: Bucket
metadata:
  name: mjlbucket929211
spec:
  forProvider:
    region: us-east-1
  providerConfigRef:
    name: creds
EOF
```

Below is another configuration to create an EKS cluster.

```jsx
apiVersion: eks.aws.upbound.io/v1beta1
kind: Cluster
metadata:
  name: ekscluster
  labels:
    example: "true"
spec:
  forProvider:
    region: us-east-1
    roleArnRef:
      name: role
    resourcesVpcConfig:
      endpointPrivateAccess: true
      endpointPublicAccess: true
      subnetIdRefs:
        - name: subnet1
        - name: subnet2
      securityGroupIdRefs:
        - name: cluster-sg
    version: "1.26"
  writeConnectionSecretToRef:
    name: cluster
    namespace: default
  providerConfigRef:
    name: provider-aws
```

### Azure

When working with Azure, you’ll see that the setup and process is essentially the same.

First, add the Azure Provider.

```jsx
cat <<EOF | kubectl apply -f -
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: upbound-provider-azure
spec:
  package: xpkg.upbound.io/upbound/provider-azure:v0.29.0
EOF
```

Ensure that it was installed.

```jsx
kubectl get providers
```

Generate RBAC credentials.

Run the following, copy the output, and save it to a file called `azure.json`

```jsx
az ad sp create-for-rbac \
--sdk-auth \
--role Owner \
--scopes /subscriptions/your_sub_id
```

Create a Kubernetes Secret from the credentials above.

```jsx
kubectl create secret \
generic azure-secret \
-n crossplane-system \
--from-file=creds=./azure.json
```

Add the secret to the provider config

```jsx
cat <<EOF | kubectl apply -f -
apiVersion: azure.upbound.io/v1beta1
metadata:
  name: default
kind: ProviderConfig
spec:
  credentials:
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: azure-secret
      key: creds
EOF
```

Start working with Crossplane! Below is an example of creating a new vNet.

```jsx
cat <<EOF | kubectl create -f -
apiVersion: network.azure.upbound.io/v1beta1
kind: VirtualNetwork
metadata:
  name: vnet
spec:
  forProvider:
    addressSpace:
      - 10.0.0.0/16
    location: "US East"
    resourceGroupName: devrel-as-a-service
EOF
```

As you can see, working with Crossplane is a straightforward process, which makes your environment far easier to manage. You no longer have to have multiple tools and platforms to configure resources, services, and infrastructure. You only need one.