Repository: garden-io/garden

Issue #4882: 0.13: [Bug]: ModuleTemplate failed to identify variable with default value pass in schema.json file
Created: 2023-07-21T12:10:45Z
Garden Bonsai (0.13) Bug
Current Behavior
Garden fails with the error:
Failed resolving one or more modules:

name: Invalid template string (-Djib.from.image=${var.dockerRepo}…): Could not find key jdkVersion under inputs. Available keys: (none).
Expected behavior
No errors, validation should pass.
Reproducible example
Run garden validate with the configuration below.
Garden configuration.
kind: Project
apiVersion: garden.io/v1
name: repro
environments:
  - name: default
providers:
  - name: local-kubernetes
    environments: [ default ]
variables:
  dockerRepo: "hub.docker.io"
  baseImageJava8: eclipse-temurin:8u345-b01-jre-ubuntu23
  baseImageJava11: distroless-os-images:debian11-java11.0.18

---
kind: Module
type: templated
template: k8s-container
name: name

---
kind: ModuleTemplate
name: k8s-container
inputsSchemaPath: schema.json

modules:
  - kind: Module
    type: container
    name: ${parent.name}  # Now possible as it doesn't conflict with the RenderTemplate type
    extraFlags:
      - "-Djib.from.image=${var.dockerRepo}/${inputs.jdkVersion == '11' ? var.baseImageJava11 : var.baseImageJava8}"
schema.json
{
    "type": "object",
    "properties": {
      "excludeModules": {
        "type": "array",
        "items": {
            "type": "string"
        }
      },
      "jdkVersion": {
        "type": "string",
        "default": "8"
      }
    }
}
Dummy Dockerfile
FROM busybox:1.36.0
ENTRYPOINT ["tail", "-f", "/dev/null"]
Workaround
Use action-based template:
---
kind: ModuleTemplate
name: k8s-container
inputsSchemaPath: schema.json

configs:
  - kind: Build
    type: container
    name: ${parent.name}  # Now possible as it doesn't conflict with the RenderTemplate type
    spec:
      extraFlags:
        - "-Djib.from.image=${var.dockerRepo}/${inputs.jdkVersion == '11' ? var.baseImageJava11 : var.baseImageJava8}"

Duplicate issue #4880: Non-existing variable breaks the behaviour of `$merge` operator
Created: 2023-07-21T10:46:36Z
Bug
Current Behavior
When $merge operator encounters an undefined variable as an argument it never gets resolved properly. It is rendered as JSON object instead.
Assume we have variables defined in the project-level config:
variables:
  example-env:
    FIELD1: hi
    FIELD2: bye
If we use $merge operator like this
  env:
    $merge: ${var.example-env}
    HELLO: GOODBYE

then we get a valid output:
          env:
            - name: FIELD1
              value: hi
            - name: FIELD2
              value: bye
            - name: HELLO
              value: GOODBYE
But if we change $merge: ${var.example-env} to ``$merge: ${var.empty || var.example-env}` then the output is invalid:
          env:
            - name: $merge
              value:
                FIELD1: hi
                FIELD2: bye
            - name: HELLO
              value: GOODBYE
Here $merge is not considered a valid helper function and is interpreted as a JSON object key.
Expected behavior
The missing variable can be interpreted as an empty one to produce a valid output:
          env:
            - name: FIELD1
              value: hi
            - name: FIELD2
              value: bye
            - name: HELLO
              value: GOODBYE
Or an error can be thrown because of the reference to a missing variable. To be confirmed. Need to check the expected behavior in the docs.
Reproducible example


Create the project from the files below:


bug-demo.garden.yml
---
kind: Module
type: container
name: bug-module
description: Container Module
---
kind: Module
type: templated
template: manifests-local
name: manifests
inputs:
  env:
    $merge: ${var.empty || var.example-env}
    HELLO: GOODBYE
---
kind: ModuleTemplate
name: manifests-local
inputsSchemaPath: schema.json
modules:
  - type: kubernetes
    name: bug-service
    description: ${parent.name} manifests
    build:
      dependencies:
        - bug-module
    variables:
      env:
        # put non-secrets into the correct format.
        - $concat:
            $forEach: ${inputs.env || []}
            $return:
              name: ${item.key}
              value: ${item.value}
    files: [.manifests-template.yml]
    generateFiles:
      - sourcePath: manifests-template.yml
        targetPath: .manifests-template.yml
project.garden.yml
kind: Project
name: example-project
variables:
  example-env:
    FIELD1: hi
    FIELD2: bye
environments:
  - name: default
    defaultNamespace: ${project.name}
providers:
  - name: local-kubernetes
    environments: [ default ]
    namespace: ${environment.namespace}
manifests-template.yml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: bug-demo
  labels: my-label
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ${parent.name}
      service: ${parent.name}
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      labels: my-label
    spec:
      containers:
        - name: ${parent.name}
          imagePullPolicy: IfNotPresent
          securityContext:
            allowPrivilegeEscalation: false
          env:
${indent(yamlEncode(var.env), 12)}
schema.json
{
  "type": "object"
}
Dockerfile
FROM nginxdemos/hello


Run garden validate
Check the spec.template.containers.spec section in the generated .manifests-template.yml file, it will have the following env values (which are correct):

          env:
            - name: FIELD1
              value: hi
            - name: FIELD2
              value: bye
            - name: HELLO
              value: GOODBYE

Modify bug-demo.garden.yml file by rewriting the inputs section as follows (change $merge: ${var.example-env} to $merge: ${var.empty || var.example-env}, the variable empty is not defined in any scope):

inputs:
  env:
    $merge: ${var.empty || var.example-env}
    HELLO: GOODBYE


Rerun garden validate
Check the re-generated .manifests-template.yml again, the env values will be:

          env:
            - name: $merge
              value:
                FIELD1: hi
                FIELD2: bye
            - name: HELLO
              value: GOODBYE

Workaround
Unknown
Suggested solution(s)
Additional context
This might not be a module-template-specific issue. Need to check it with a simpler example if possible.

Your environment
garden version 0.13.9

Closing comment (2023-07-24T07:20:11Z): One more issue reported by user:

The templated module also does not properly allow its own variables to be assigned to the inputs:

---
kind: Module
type: templated
template: manifests-local
name: manifests
variables:
  replicas: 1
inputs:
  replicas: ${var.replicas}
---
kind: ModuleTemplate
name: manifests-local
inputsSchemaPath: schema.json
modules:
  - type: kubernetes
    name: bug-service
    description: ${parent.name} manifests
    template-replicas: ${inputs.replicas}

It gives the error:

bug-service: Invalid template string (${var.replicas}): Could not find key replicas under var. Available keys: example-env.
varfiles also seem to not work here.
This might be related to #4882.