CircleCI Split Config Orb!
Tags: circleci, cue
Published: 2022-07-21 16:24:00

I created a CircleCI Orb named Split Config. This is my private fun coding :)

* https://circleci.com/developer/orbs/orb/bufferings/split-config

## Do you want to split your configuration file?

Have you ever thought if you could split the `config.yml` file of your CircleCI projects? Especially it could be bigger when the repository is a type of MonoRepo which includes multiple services in one repository.

Good news! Split Config Orb is here for you!

## Split Config Orb

You can split your `config.yml` into multiple configuration files. This Orb merges the split configs into one YAML when CircleCI starts and execute a pipeline with the merged configuration.

## Example

Let's say you have multiple services in your repo like this:

![File tree](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2rdqf0y8ubxemrjth7q3.png)

Well, it's empty but please imagine the source codes of each services are there :)

Each service has a `config.yml` for the service. For example. the config for service1 is as follows:

```yaml
version: 2.1

jobs:
  service1-say-hello:
    docker:
      - image: cimg/base:stable
    steps:
      - checkout
      - run:
          name: "Say hello"
          command: "echo Hello, World!1"

workflows:
  service1-say-hello-workflow:
    jobs:
      - common-say-hello
      - service1-say-hello
```

service2:

```yaml
version: 2.1

jobs:
  service2-say-hello:
    docker:
      - image: cimg/base:stable
    steps:
      - checkout
      - run:
          name: "Say hello"
          command: "echo Hello, World!2"

workflows:
  service2-say-hello-workflow:
    jobs:
      - common-say-hello
      - service2-say-hello
```

etc...

The Orb merges these config files and it will be like this:

```yaml
version: 2.1
jobs:
  common-say-hello:
    docker:
      - image: cimg/base:stable
    steps:
      - checkout
      - run:
          name: Say hello
          command: echo Hello, World! common
  service1-say-hello:
    docker:
      - image: cimg/base:stable
    steps:
      - checkout
      - run:
          name: Say hello
          command: echo Hello, World!1
  service2-say-hello:
    docker:
      - image: cimg/base:stable
    steps:
      - checkout
      - run:
          name: Say hello
          command: echo Hello, World!2
  service3-say-hello:
    docker:
      - image: cimg/base:stable
    steps:
      - checkout
      - run:
          name: Say hello
          command: echo Hello, World!3
workflows:
  service1-say-hello-workflow:
    jobs:
      - common-say-hello
      - service1-say-hello
  service2-say-hello-workflow:
    jobs:
      - common-say-hello
      - service2-say-hello
  service3-say-hello-workflow:
    jobs:
      - common-say-hello
      - service3-say-hello
```

## How to use

It's super simple. You just need to prepare this `config.yml`

```yaml
version: 2.1

setup: true

orbs:
  split-config: bufferings/split-config@0.1.0

workflows:
  generate-config:
    jobs:
      - split-config/generate-config:
          find-config-regex: .*/\.circleci/config\.yml
```

Then the Orb finds the config files according to the regex you specified, and merge them into one YAML file. After that, the Orb executes CircleCI pipeline with the generated YAML file.

It uses the [Dynamic Config](https://circleci.com/docs/dynamic-config) feature of CircleCI to start a new pipeline. By default, the feature is disabled, so please [enable the feature](https://circleci.com/docs/dynamic-config#getting-started-with-dynamic-config-in-circleci) to get started.

Then you will see something like this:

![Example execution picture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kin2j0oo2qj34mv0vf8b.png)

That's all!

Are you interested in how it merges the configs?

## How does it merge the configs?

The Split Config Orb uses CUE to merge the YAML files.

* CUE https://cuelang.org/

CUE can import YAML file, convert them into CUE format, and merge them into one CUE config. In addition, it has a feature to export CUE config as YAML.

I used these CUE features to merge your YAML configs. CUE is so powerful but I only use a small part of it. If you want to know more about CUE or how it merges multiple CUE configs, please check CUE specification!

## For more details

Please check the README in GitHub:
https://github.com/bufferings/orb-split-config

I wrote some usecases with working examples! Enjoy smaller configuration files