Introducing the CircleCI Config SDK
Tags: circleci, cicd, sdk, pipelines
Published: 2022-11-02 17:45:09

---
title: Introducing the CircleCI Config SDK
published: true
date: 2022-09-19 18:00:00 UTC
tags: circleci,cicd,sdk,pipelines
canonical_url: https://circleci.com/blog/config-sdk/
cover_image: https://production-cci-com.imgix.net/blog/media/2022-09-19-config-sdk-blog.png
---

We are excited to announce the new CircleCI Config SDK is now available as an open-source TypeScript library. Developers can now write and manage their CircleCI `config.yml` files using TypeScript and JavaScript.

- [NPM Package](https://www.npmjs.com/package/@circleci/circleci-config-sdk)
- [GitHub Repository](https://github.com/CircleCI-Public/circleci-config-sdk-ts)
- [Docs](https://github.com/CircleCI-Public/circleci-config-sdk-ts/wiki)

For developers used to the ecosystem and flexibility of a full-fledged programming language, sometimes YAML can feel limiting or intimidating. With the Config SDK you can define and generate your YAML config from type-safe and annotated JavaScript. You can even take advantage of package management to modularize any portion of your config code for reuse.

When paired with CircleCI’s [dynamic configuration](https://circleci.com/docs/dynamic-config), the Config SDK makes it possible to dynamically build your CI configuration at run-time, allowing you to choose what you want to execute based on any number of factors, such as the status of your Git repo, an external API, or just the day of the week.

## Getting started

For our example, let’s imagine we manage several Node.js projects that are all built using the same framework and generally require the same CI configuration. So we decided that we want to build a config “template” that all of our projects will use, and which can be centrally managed and updated. We’ll create and publish an NPM package that will generate the perfect config file for all of our Node projects.

![Generating a config](https://circleci.com/blog/media/2022-09-19-config-sdk.gif)

Let’s build the config template package, and then build the pipelines that will use it.

### Setup

We’re going to start by creating a standard NPM package. You can use TypeScript or JavaScript, but we’ll use JavaScript in this example for the sake of speed. The example shown here is based on[this page from the repo’s wiki](https://github.com/CircleCI-Public/circleci-config-sdk-ts/wiki/Write-Package).

Begin by initializing a JavaScript project in a new directory.

```
mkdir <your-package-name>

cd <your-package-name>

npm init -y

npm i --save @circleci/circleci-config-sdk

```

The `@circleci/circleci-config-sdk` package will allow us to define a CircleCI config file with JavaScript. While we could simply define a config and export it, we can also take advantage of dynamic config and export a function instead. In our example, we’ll keep it simple and create a config generation function that will take a `tag` parameter for our deployments, and a `path` parameter to choose where the config file will be exported to.

## Create the app

Create an `index.js` file and import the CircleCI Config SDK package and Node’s `fs` package so we can write the config to a file.

```
const CircleCI = require("@circleci/circleci-config-sdk");
const fs = require('fs');

```

Next we’ll start building up the components of our config file using the Config SDK. You’ll notice because we are working with a TypeScript-based library, we are able to receive code hints, type definitions, documentation and auto-completion.

### Create executor

Given we are building a config for our Node.js projects, we’ll begin by defining the [Docker executor](https://circleci.com/docs/using-docker) our jobs will use. You can pass in the Docker image, resource class, and any other parameters you may want to configure.

```
// Node executor
const dockerNode = new CircleCI.executors.DockerExecutor(
  "cimg/node:lts"
);

```

### Create jobs

We are building up to a workflow that will test our application on every commit, and deploy it when we provide a certain tag. Like our executor, we’ll define these two jobs, both using the executor we just defined and each with a unique set of steps for their respective purposes.

```
// Test Job
const testJob = new CircleCI.Job("test", dockerNode);
testJob.addStep(new CircleCI.commands.Checkout());
testJob.addStep(new CircleCI.commands.Run({ command: "npm install && npm run test" }));

//Deploy Job
const deployJob = new CircleCI.Job("deploy", dockerNode);
deployJob.addStep(new CircleCI.commands.Checkout());
deployJob.addStep(new CircleCI.commands.Run({ command: "npm run deploy" }));

```

Jobs can be instantiated with steps or dynamically added to an existing job like shown above. In this overly simplified example, we lack a [caching](https://circleci.com/docs/caching) step, but you can see how we build up elements of our configuration.

## Create a workflow

With our jobs defined, it’s time to implement them in a workflow and define how they should run. We mentioned earlier that we want the `test` job to run on all commits, and the `deploy` job to only run on the given tag.

Now that we are working with top-level components of our config file, let’s finally define a new CircleCI config object, and name the workflow that we will add our jobs to.

```
//Instantiate Config and Workflow
const nodeConfig = new CircleCI.Config();
const nodeWorkflow = new CircleCI.Workflow("node-test-deploy");
nodeConfig.addWorkflow(nodeWorkflow);

```

We are not adding any parameters to our testing job because we want to run it on all commits, so we can directly add it to our config object.

```
nodeWorkflow.addJob(testJob);

```

For the deploy job, we need to first define the workflow job so that we can add filters to it. We are going to add one filter now which tells CircleCI to ignore this job for all branches, so it doesn’t execute on every commit. We’ll deal with enabling it for a tag in a moment.

```
const wfDeployJob = new CircleCI.workflow.WorkflowJob(deployJob, {requires: ["test"], filters: {branches: {ignore: ".*"}}});
nodeWorkflow.jobs.push(wfDeployJob);

```

## Export the config generator function

Now that we have everything defined we are ready to create and export the final piece. We are going to create a function which takes in the tag and path parameters we mentioned earlier and will write what we have defined to a new file.

```
/**
* Exports a CircleCI config for a node project
*/
export default function writeNodeConfig(deployTag, configPath) {
 // next step
}

```

In the newly created `writeNodeConfig` function, we’ll add the tag filter being passed in here to the deploy job in our workflow and finally write the config to the file supplied by the `path` parameter using the `generate` function on the config object.

```
/**
* Exports a CircleCI config for a node project
*/
export default function writeNodeConfig(deployTag, configPath) {
  wfDeployJob.parameters.filters.tags = {only: deployTag}
  fs.writeFile(configPath, nodeConfig.generate(), (err) => {
    if (err) {
      console.error(err);
      return
    }
  })
}

```

Here is the full source code, which you can also find [in the wiki](https://github.com/CircleCI-Public/circleci-config-sdk-ts/wiki/Write-Package):

```
const CircleCI = require("@circleci/circleci-config-sdk");
const fs = require('fs');

// Node executor
const dockerNode = new CircleCI.executors.DockerExecutor(
  "cimg/node:lts"
);

// Test Job
const testJob = new CircleCI.Job("test", dockerNode);
testJob.addStep(new CircleCI.commands.Checkout());
testJob.addStep(new CircleCI.commands.Run({ command: "npm install && npm run test" }));

//Deploy Job
const deployJob = new CircleCI.Job("deploy", dockerNode);
deployJob.addStep(new CircleCI.commands.Checkout());
deployJob.addStep(new CircleCI.commands.Run({ command: "npm run deploy" }));

//Instantiate Config and Workflow
const nodeConfig = new CircleCI.Config();
const nodeWorkflow = new CircleCI.Workflow("node-test-deploy");
nodeConfig.addWorkflow(nodeWorkflow);

//Add Jobs. Add filters to deploy job
nodeWorkflow.addJob(testJob);
const wfDeployJob = new CircleCI.workflow.WorkflowJob(deployJob, {requires: ["test"], filters: {branches: {ignore: ".*"}}});
nodeWorkflow.jobs.push(wfDeployJob);

/**
* Exports a CircleCI config for a node project
*/
export default function writeNodeConfig(deployTag, configPath) {
  wfDeployJob.parameters.filters.tags = {only: deployTag};
  fs.writeFile(configPath, nodeConfig.generate(), (err) => {
    if (err) {
      console.error(err);
      return
    }
  });
}

```

## Publish the package

With your `index.js` file completed with the `writeNodeConfig` function exported, it’s time to [publish the package](https://docs.npmjs.com/creating-and-publishing-unscoped-public-packages) to your package repository of choice, such as NPM or GitHub.

When complete, you should be able to import your package in other projects, just like we imported `@circleci/circleci-config-sdk` earlier.

## Create a CI pipeline

You now have a published NPM package that can generate a CircleCI config file. We can use CircleCI’s dynamic configuration to pull in this package at run-time and dynamically run our generated config file. We can replicate this basic template across our many similar NodeJS projects, and when we want to update or change our config, we will be able to simply update the package we created.

### Create config.yml

As usual, our CircleCI project will require a `.circleci` directory and a `config.yml` file inside. The config file in this case will be a basic template used in all of our projects which simply tells CircleCI to enable the dynamic configuration feature for the current pipeline, generate the new config file, and run it. We will also create a `dynamic` directory that we will use later.

```
└── .circleci/
    ├── dynamic/
    └── config.yml` \

```

Use the following [example configuration file](https://github.com/CircleCI-Public/circleci-config-sdk-ts/blob/main/sample/01-dynamic-workflow-javascript/.circleci/config.yml):

```
version: 2.1
orbs:
  continuation: circleci/continuation@0.3.1
  node: circleci/node@5.0.2
setup: true
jobs:
  generate-config:
    executor: node/default
    steps:
      - checkout
      - node/install-packages:
          app-dir: .circleci/dynamic
      - run:
          name: Generate config
          command: node .circleci/dynamic/index.js
      - continuation/continue:
          configuration_path: ./dynamicConfig.yml
workflows:
  dynamic-workflow:
    jobs:
      - generate-config

```

You can find this config file and other examples in the [Wiki](https://github.com/CircleCI-Public/circleci-config-sdk-ts/wiki/Write-Dynamic-Config-App) on GitHub.

This config is a boilerplate responsible for executing our package which contains the “real” logic we want to execute.

Notice the `setup` key is set to `true`, enabling dynamic config for the pipeline. Using the Node orb, we install a Node app located in `.circleci/dynamic` (we’ll come back to this), and run the `.circleci/dynamic/index.js` in Node. This is what will use the package we wrote earlier and create a new config file at `./dynamicConfig.yml`, which will finally be executed by the `continuation` orb.

We will use a config like this one in all of our Node projects, and it is unlikely to need to be updated or changed often. Rather than modifying this config, we update the package we created earlier.

### Create the config app

The last thing to do is build our “Config application”. This is the app responsible for implementing our package and acts as the source of our “real” config we plan on executing. Because in this example we have outsourced the majority of the logic of our config to an external package, in this example, our config app is mostly boilerplate as well.

Change directory into `.circleci/dynamic` where we will set up our application. Initialize a new repository and install the package you published previously.

```
npm init -y

npm i <your-package-name>

```

After running these two commands, you should have a `package.json` file with a dependency showing your package.

```
"dependencies": {
  "my-circleci-node-config": "^1.0.0",
}

```

You can [modify the semantic version string](https://docs.npmjs.com/about-semantic-versioning#using-semantic-versioning-to-specify-update-types-your-package-can-accept) to dictate what version of your package should be pulled at run time. This means, you could update your `my-circleci-node-config` package and, if you choose, have all of your CircleCI projects that utilize this package immediately pick up these changes the next time your CI pipeline is triggered.

**Not Recommended:** Say you always wanted to pull the latest version of the custom dependency you have created, you could use:

```
"dependencies": {
  "my-circleci-node-config": "x",
},

```

To be safer, pull in only minor and patch updates, not major releases.

```
"dependencies": {
  "my-circleci-node-config": "1.x",
},

```

Finally, utilize your custom package in the `.circleci/dynamic.index.js.`

Our package exports a function named `writeNodeConfig` which takes in the value of the `tag` we want for triggering deployments, and the `path` we want to export the config to. We know the path from earlier in our config.yml, we set to be `./dynamicConfig.yml`, because we are in the `dynamic` directory, we will prepend `..`. For the tag, we’ll use a generic regex string `v.*`

```
import writeNodeConfig from '<your/package>';

writeNodeConfig("v.*", "../dynamicConfig.yml")

```

That is our entire application. We simply need to pull whichever version of our package we desire and invoke it to generate our config file from the template we created in the package.

## Running the pipeline

To recap, we have a boilerplate `config.yml` file which instructs CircleCI to enable dynamic configuration, and uses Node.js to build a new config file at run-time. The Node app responsible for building our new config uses a package dependency which contains the “template” of our desired config. We can now use this package in many different projects and update it centrally. Our projects can either pull in the latest version of this package by specifying an `x` in the package.json, or we can use tools like dependabot to open a pull request automatically to all of our projects that use this package when it is updated.

With dynamic configuration and the Config SDK together, the possibilities are endless. The tutorial above was based on [this page from the wiki](https://github.com/CircleCI-Public/circleci-config-sdk-ts/wiki/Write-Package) on GitHub. Check out the rest of our [wiki](https://github.com/CircleCI-Public/circleci-config-sdk-ts/wiki) and [documentation](https://circleci-public.github.io/circleci-config-sdk-ts/) for even more examples of interesting things you can do with the Config SDK.

We’d love to hear from you and see how you utilize the Config SDK in your own pipelines. Connect with us on [Twitter](https://twitter.com/CircleCI), say hello in our [discussion forum](https://discuss.circleci.com/t/circleci-config-sdk-now-g-a/45457), or chat with us on [Discord](https://discord.gg/5dUX4FstW2).