Project: CDK for Kubernetes (CDK8s) (Provisioning / Automation & Configuration)
File: 1272-helm-import.md

Original Author(s):* @vinayak-kukreja Tracking Issue:* https://github.com/cdk8s-team/cdk8s/issues/1271 API Bar Raiser:* @iliapolo Users are now able to import helm charts into their cdk8s app with `cdk8s import` command. You can import a helm chart to your cdk8s application. If the helm chart contains a `values.schema.json`, then you would have typed `values` property within the construct properties. And if the schema file is not present, then the values property would not be typed. For example, you can run the following to import a chart. A `values.schema.json` file is present in this chart. ``` cdk8s import helm:https://charts.bitnami.com/bitnami/mysql@9.10.10 ``` The format for the url is: `helm:<repo-url>/<chart-name>@<chart-version>`: Repository URL: `https://charts.bitnami.com/bitnami/` Chart name: `mysql` Chart version: `9.10.10` You can use this chart within your application: ```typescript import { Construct } from 'constructs'; import { App, Chart, ChartProps } from 'cdk8s'; import { Mysql, MysqlArchitecture } from './imports/mysql'; export class MyChart extends Chart { constructor(scope: Construct, id: string, props: ChartProps = { }) { super(scope, id, props); new Mysql(this, 'demo', { values: { architecture: MysqlArchitecture.STANDALONE, common: { anyProp: 'foo', }, global: { anyGlobalProp: 'bar', } } }); } } const app = new App(); new MyChart(app, 'Typescript-App'); app.synth(); ``` Since a `values.schema.json` file was present in the chart, the `values` properties are strongly typed. For instance, `architecture` property is of type `MysqlArchitecture`. `global` here is for any and `common` here is a dependency() of the `mysql` parent chart. For instance, this is how some of the generated file would look like for `mysql` helm chart: ```typescript // generated by cdk8s import { Helm } from 'cdk8s'; import { Construct } from 'constructs'; export interface MysqlValuesProps extends SchemaGeneratedValues { readonly common?: { [key: string]: any }; readonly global?: { [key: string]: any }; } export interface MysqlProps { readonly namespace?: string; readonly releaseName?: string; readonly helmExecutable?: string; readonly helmFlags?: string[]; readonly values?: MysqlValuesProps; } export class Mysql extends Helm { public constructor(scope: Construct, id: string, props: MysqlProps = {}) { const finalProps = { chart: 'mysql', repo: 'https://charts.bitnami.com/bitnami', version: '9.10.10', ...props, }; super(scope, id, { ...finalProps, }); } } / @schema mysql */ export interface SchemaGeneratedValues { / Allowed values: `standalone` or `replication` * @schema mysql#architecture */ readonly architecture?: MysqlArchitecture; / @schema mysql#auth */ readonly auth?: MysqlAuth; / @schema mysql#primary */ readonly primary?: MysqlPrimary; / @schema mysql#secondary */ readonly secondary?: MysqlSecondary; } ``` Note: You would need `helm` to be installed on your machine for using this feature. For accessing private helm repositories, you must be authenticated to the repository in a way that the `helm pull` command recognizes. Ticking the box below indicates that the public API of this RFC has been signed-off by the API bar raiser (the `api-approved` label was applied to the RFC pull request): ``` [ ] Signed-off by API Bar Raiser @iliapolo ``` We have added a new feature to the `cdk8s import` command which allows users to import helm charts into their cdk8s app. You should use this feature if you would like to use helm charts within your cdk8s app. This feature would allow you to add values for helm chart in a type safe manner if a `values.schema.json` file is present within the chart being

Our current support for including helm charts to cdk8s app adds churn for the user to setup the and pass in values to the construct. This feature would have two advantages over the current solution: It would make it easier for the user to setup a construct for helm charts. They would just need to run the `cdk8s import` command with a valid url and we would auto generate the construct for them. If a `values.schema.json` file is present in the helm chart that the user is referring, then our generated construct would have type safe values properties. This is not supported in our current implementation. As mentioned above, we do currently have a ) for including helm charts in cdk8s app. The proposed feature is definitely an enhancement but would take up developers time and effort to implement and maintain. The following would be the format for a helm url being passed into the import command: ``` cdk8s import helm:<repo-url>/<chart-name>@<chart-version> ``` An example of a valid input for importing a helm chart into a cdk8s app: ``` 'cdk8s import helm:https://charts.bitnami.com/bitnami/mysql@9.10.10' ``` Here, `helm:`: Is how we will identify if the url being passed in supposed to be a helm chart. `https://charts.bitnami.com/bitnami/mysql@9.10.10`: Is the helm chart url. `https://charts.bitnami.com/bitnami/mysql`: Is the helm repo. `mysql`: Is the name of the helm chart. `9.10.10`: Is the helm chart version. We would create a new class for importing helm charts called as `ImportHelm` which would extends . In this class we will, Validate and get chart information from the url If the url is valid, we can then get the information about the helm chart like, chart name, repo url and chart version from the url. This can be obtained based on the syntax `helm:<repo-url>/<chart-name>@<chart-version>`. Get `values.schema.json` file When the user would run the import command, we would need to identify if a `values.schema.json` is present in the helm chart being mentioned. To do so, we can create a temporary directory and pull in and extract the chart in that directory. We can do this with the help of `helm pull` command. ``` helm pull chart-name --repo chart-url --version chart-version --untar --untardir working-directory ``` ```typescript const args = new Array<string>(); args.push('pull'); args.push(chartName); args.push('--repo', chartUrl); args.push('--version', chartVersion); args.push('--untar'); args.push('--untardir', workdir); const command = 'helm'; const helm = spawnSync(command, args, { maxBuffer: MAXHELMBUFFER, }); ``` So after extraction, we would look for the `values.schema.json` in the root of the helm chart. ImportBase class has abstract methods and . As part of the new class `ImportHelm` we also need to define these. Because, when the function is ), it invokes the function. `moduleNames` This method is used to set the name of the generated file. ```typescript public get moduleNames() { return [this.chartName]; } ``` `generateTypeScript` This method would generate the typescript code for the helm chart. ```typescript protected async generateTypeScript(code: CodeMaker) { emitHelmHeader(code); let schema: JSONSchema4 | undefined; if (this.chartSchemaPath !== undefined) { JSON.parse(fs.readFileSync(this.chartSchemaPath, 'utf-8')); } else { schema = undefined; } const types = new TypeGenerator({ definitions: schema?.definitions, }); generateHelmConstruct(types, { schema: schema, chartName: this.chartName, chartUrl: this.chartUrl, chartVersion: this.chartVersion, fqn: this.chartName, }); code.line(types.render()); cleanup(this.tmpDir); } ``` In the prior code snippet, for code generation, we utilize . `emitHelmHeader(code)` The function would be responsible for adding any relevant imports for the generated construct. ```typescript / Emit imports for generated helm construct @param code CodeMaker instance */ export function emitHelmHeader(code: CodeMaker) { code.line('// generated by cdk8s');

{ Helm } from \'cdk8s\';'); code.line('import { Construct } from \'constructs\';'); code.line(); } ``` Type generation If a `values.schema.json` file was present in the helm chart, the definitions of the schema are passed in to the . TypeGenerator makes it possible to emit structs for a schema and also enables to emit custom types. We then invoke `generateHelmConstruct`(NEW) function. In this function, we are emitting: A custom type. This would be our helm construct. A `values` property interface which extends schema generated `struct` if a schema file was present in the root of the chart. Otherwise, it would be `{ [key: string]: any }` i.e properties will not be typed. If it is typed, it would also have `global` and `dependencies or sub-charts` as properties within the values interface. Currently, these will be not strongly typed. An interface representing properties of the construct. ```typescript export function generateHelmConstruct(typegen: TypeGenerator, def: HelmObjectDefinition) { const chartName = TypeGenerator.normalizeTypeName(def.chartName); const schema = def.schema; const repoUrl = def.chartUrl; const chartVersion = def.chartVersion; // Create custom type typegen.emitCustomType(chartName, code => { const valuesInterface = `${chartName}ValuesProps`; if (schema !== undefined) { // Interface for schema generated props let schemaGenValuesInterface: string = 'SchemaGeneratedValues'; schemaGenValuesInterface = typegen.emitType(schemaGenValuesInterface, schema, def.fqn); // Creating values interface emitValuesInterface(); function emitValuesInterface() { code.openBlock(`export interface ${valuesInterface} extends ${schemaGenValuesInterface}`); // Sub charts or dependencies for (const dependency of def.chartDependencies) { code.line(`readonly ${dependency}?: { [key: string]: any };`); } // Global values code.line('readonly global?: { [key: string]: any };'); code.closeBlock(); } } // Creating construct properties emitPropsInterface(); code.line(); // Creating construct for helm chart emitConstruct(); function emitPropsInterface() { code.openBlock(`export interface ${chartName}Props`); code.line('readonly namespace?: string;'); code.line('readonly releaseName?: string;'); code.line('readonly helmExecutable?: string;'); code.line('readonly helmFlags?: string[];'); if (schema === undefined) { code.line('readonly values?: { [key: string]: any };'); } else { code.line(`readonly values?: ${valuesInterface};`); } code.closeBlock(); } function emitConstruct() { code.openBlock(`export class ${chartName} extends Helm`); emitInitializer(); code.closeBlock(); } function emitInitializer() { code.openBlock(`public constructor(scope: Construct, id: string, props: ${chartName}Props = {})`); code.open('const finalProps = {'); code.line(`chart: \'${def.chartName}\',`); code.line(`repo: \'${repoUrl}\',`); code.line(`version: \'${chartVersion}\',`); code.line('...props,'); code.close('};'); code.open('super(scope, id, {'); code.line('...finalProps,'); code.close('});'); code.closeBlock(); } }); ``` The generated construct would just be invoking constructor and that would use command to generate the manifest with the values that the user has passed in. Note: The interface being generated here is similar to with the difference being in `chart` and `values` properties. Unfortunately, is not supported by jsii and hence the code repetition. There is a by `mgrain` to support this use case but has not been added to jsii yet. After the code is successfully generated in typescript, we transpile it to other languages in our class using . There are no changes required here for this feature. Note: The language naming conventions are followed when typescript code is transpiled. This results in casing changes for the properties of the generated construct. But, this does not impact the manifest generation since as part of transpiling the casing corelation between properties and transpiled properties is added. For instance, following is some of the properties mapping for `mysql` helm chart in python: ```python @jsii.data_type( jsii_type="mysql.MysqlAuth", jsiistructbases=[], name_mapping={ "password": "password", "username": "username", "create_database": "createDatabase", "database": "database", "replication_password": "replicationPassword", "replication_user": "replicationUser", "root_password": "rootPassword", }, ) ``` cdk8s import Demo:=helm:https://charts.bitnami.com/bitnami/mysql@9.10.10* This would work as expected and will add "Demo" as the . cdk8s import

This would fail the regex since no chart version is being mentioned in the url. cdk8s import helm:https://charts.bitnami.com/* This would fail since no chart has been mentioned in the url. cdk8s import helm:www.google.com/mysql@9.10.10* This would fail since the url would not lead to a helm chart. ```mermaid title: cdk8s import helm:https://charts.bitnami.com/bitnami/mysql@9.10.10 flowchart TD A([cdk8s import command executed]) B{is helm chart url} A --> B B --no--> C(check for other \n import and resume import) B --yes--> D{is valid url} D --no--> E(error) D --yes--> F(get chart details from url) F --> G(create temporary directory) G --> H(helm pull chart \n in tmp dir) H --> I{is \n values.schema.json \n present} I --no--> J(schema path is set \n to undefined) I --yes--> K(schema path is set) L(import continues) J --> L K --> L L --> M(generate typescript) M --> N(emit 'values' struct) N --> O(emit construct props) O --> P(emit construct) P --> Q(render to file and save) Q --> R{is language typescript} R --yes--> S(import finishes) R --no--> T(transpile to target language) T --> S C --> S ``` This is not a breaking change. This is adding new functionality to the cdk8s cli import command. Using omit utility type* In the proposed solution we are recreating a similar interface to . This is unideal since it adds redundancy. The reason for doing this is because: We already would know the `chart` value and it would not make sense to ask the user for it which is required in HelmProps. `values` could have data types associated with its properties which is not supported in HelmProps. I initially used and it works but just for typescript. When trying to transpile it to other languages it gives out an error due to jsii limitation. --helm as a flag* Instead of using `cdk8s import helm:url`, I initially thought of using a flag `--helm` where if present, it would mean that the url is supposed to be leading to a helm chart. This adds a flag to the import command and keeping it as `helm:` makes it similar to what we have for `github:`. Reusing ApiObjectDefinition* In the proposed solution, I have added `HelmObjectDefinition` as an interface instead of reusing and adding to . I think the two interface represent different things and it would not make sense to combine them for this feature. Interface similar to HelmProps* As mentioned in the previous sections, there is an interface similar to HelmProps in the proposed solution due to a jsii limitation. We can use `mgrain's` to reduce code but the emitted interface would still be similar to HelmProps. Implementing the feature and adding tests for it. Create a blog post about this feature demonstrating an end to end development workflow for the user. During my research, I encountered couple of errors when processing different `values.schemas.json` for code generation. Taking an example of Airflow helm chart, some of the errors that I encountered were: `unsupported array type undefined`* * Impacting property in schema: `for "properties", if "type" is specified it has to be an "object"`* * Impacting properties in schema: * * Before releasing the feature, we need to invest into testing the solution with different schemas and add sanitization to properties where possible and callout schema patterns that cannot be supported. Here is a of current implementation for Helm Import.