How to Run Multiple Long Commands in Docker Compose
Tags: docker, dockercompose, yaml
Published: 2022-02-21 09:28:54

---
title: How to Run Multiple Long Commands in Docker Compose
published: true
date: 2022-02-03 00:00:00 UTC
tags: docker, dockercompose, yaml
canonical_url: https://tech.serhatteker.com/post/2022-01/multiple-long-commands-docker-compose/
---

## Use Case

I want to run multiple/long commands for my service(s) in my `docker-compose` file, like below:


```yaml
---
# docker-compose.yaml

services:
  server: &server
    build:
      context: .
      dockerfile: ./compose/staging/server/Dockerfile
    image:  someproject_slug_staging_server:staging_shortsha_unixtime
    depends_on:
      - postgres
    env_file:
      - ./.envs/.staging/.server
      - ./.envs/.staging/.postgres
    command: python /app/manage.py collectstatic --no-input
    command: python /app/manage.py migrate
    command: gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app

  postgres:
    build:
      context: .
      dockerfile: ./compose/staging/postgres/Dockerfile
    image:  someproject_slug_staging_postgres:staging_shortsha_unixtime
    volumes:
      - staging_postgres_data:/var/lib/postgresql/data:Z
      - staging_postgres_data_backups:/backups:z
    env_file:
      - ./.envs/.staging/.postgres
```

However it won't work since it's not possible.


## Solutions


### Script File

One solution is that putting all your commands in a script file -let's say `start.sh`, put it into your image, build with it and call it directly.

As an example here is a `docker compose` file:

```yaml
---
# staging.yaml

version: '3.7'

volumes:
  staging_postgres_data: {}
  staging_postgres_data_backups: {}
  staging_traefik: {}
  staging_redis: {}

services:
  server: &server
    build:
      context: .
      dockerfile: ./compose/staging/server/Dockerfile
    image:  someproject_slug_server:staging_shortsha_unixtime
    depends_on:
      - postgres
      - redis
    env_file:
      - ./.envs/.staging/.server
      - ./.envs/.staging/.postgres
      - ./.envs/.staging/.redis
    command: /start

  postgres:
    build:
      context: .
      dockerfile: ./compose/staging/postgres/Dockerfile
    image:  someproject_slug_postgres:staging_shortsha_unixtime
    volumes:
      - staging_postgres_data:/var/lib/postgresql/data:Z
      - staging_postgres_data_backups:/backups:z
    env_file:
      - ./.envs/.staging/.postgres

  traefik:
    build:
      context: .
      dockerfile: ./compose/staging/traefik/Dockerfile
    image:  someproject_slug_traefik:staging_shortsha_unixtime
    depends_on:
      - server
    volumes:
      - staging_traefik:/etc/traefik/acme:z
    ports:
      - "0.0.0.0:80:80"
      - "0.0.0.0:443:443"
      - "0.0.0.0:5555:5555"

  redis:
    image: docker.io/bitnami/redis:6.2
    env_file:
      - ./.envs/.staging/.redis
    ports:
      - "6379:6379" # WARNING: In production never bind 0.0.0.0 wildcard adress of the host
    volumes:
      - staging_redis:/bitnami/redis/data:z

  celeryworker:
    <<: *server
    command: /start-celeryworker

  celerybeat:
    <<: *server
    command: /start-celerybeat

  celeryflower:
    <<: *server
    command: /start-celeryflower

  aws-backup:
    build:
      context: .
      dockerfile: ./compose/staging/aws/Dockerfile
    env_file:
      - ./.envs/.staging/.server
    volumes:
      - staging_postgres_data_backups:/backups:z
```

> **WARNING**
> 
> Don't bind your host's `0.0.0.0` wildcard adress in production environment. (Actually instead of deploying via `docker-compose`in production  you should consider to use more reliable, and robust distributed systems like `Kubernetes` but this is a topic for another post.)
> 
> `6379:6379` means `0.0.0.0:6379:6379`, so you open your redis instance to whole open world. And if you don't use `AUTH` or don't take required security precaution -like **TLS** etc. you may get some headache. So use it only if you
know the related consequences.
>
> For more redis security topic look at this: [Redis Security](https://redis.io/topics/security)


And as an example here is content of `start` from `server` service:

```bash
#!/usr/env/bin bash
# start

set -o errexit
set -o pipefail
set -o nounset


python /app/manage.py collectstatic --noinput
python /app/manage.py migrate   # WARNING: Not a good idea for prod
gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app
```

> **WARNING**
>
> Migration is not a command you want to run every container start but since it's staging environment let's assume it's OK.

### Multiline Scalar

Running in a script file is OK however maybe it's *local* or *staging* environment which means you need to debug and tweak these commands one by one; maybe some flags are not working etc. and you don't want to *re-build* the image every time.

How are we gonna do this?

The answer lines in the multiline capability of `yaml` syntax:

```yaml
command: >
  bash -c "python /app/manage.py collectstatic --noinput
  && python /app/manage.py migrate
  && gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app"
```

This is called **Folded Style**. The indention in each line will be ignored. A line break (`\n`) will be inserted at the end.

Or you can use **Literal Style**:

```yaml
command:
  - /bin/bash
  - -c
  - |
  python /app/manage.py collectstatic --noinput
  python /app/manage.py migrate
  gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app
```

This turns every newline within the string into a literal newline, and adds one at the end.

More on these styles:

#### Folded Style

```yaml
Key: >
  this is my very very very
  long command
```

Output:

`this is my very very very long command\n`

Mind new line-`\n` (end of line-`EOL`)

#### Literal Style

```yaml
Key: |
  this is my very very very
  long command
```

Output:

`this is my very very very\nlong command\n`

Mind new lines-`\n`, and compare to it above output.

Fore more detailed style and `yaml` syntax you can look at this amazing explanation in this [link].

These were the solutions for running multiple or long commands in your docker-compose file.

All done!



[link]: https://stackoverflow.com/a/21699210