Repository: ZcashFoundation/zebra
Pull request #9344: refactor(docker): Enhance Zebra configuration options and entrypoint logic
Author: login: gustavovalverde
Labels: name: C-bug
description: Category: This is a bug
name: A-devops
description: Area: Pipelines, CI/CD and Dockerfiles
name: I-integration-fail
description: Continuous integration fails, including build and test failures
name: I-usability
description: Zebra is hard to understand or use
name: P-High 🔥
Created: 2025-03-24T22:50:27Z | Merged: 2025-04-16T12:27:32Z

## Motivation

We had some issues with how we were dealing with configuration files:
- If the user mounted a configuration file, zebra would (try to) overwrite it. This could cause odd behaviors like:
  - #9325 
- When mounting a configuration file using the [docker compose configs](https://docs.docker.com/reference/compose-file/configs/) this would cause an error as the mounted directory is _read-only_
- We should be able to differentiate between a _user-provided_ configuration file, our _default_ configuration file (mainly used with docker compose), and if there's no configuration file provided, so we can build it _on-the-fly_ with the provided variables by the user.

A possible approach to fix this was using a temporal directory, and using the resulting config from that directory instead from the original mounted dir, but this was also problematic if the temporal directory was mounted as read-only from the host, which also led to the decision of not using `sed` and trying to replace the content of the file or creating a new file in a location that could potentially be `read-only` by external host limitations.

Fixes #9325 

## Solution

**Key changes:**
- Do not mount a default configuration file in our Dockerfile (`default-zebra-config.toml`), to allow more flexibility: default configuration build, default config file mounting, user-provided config file mounting with a custom `$ZEBRA_CONF_PATH`
  - Validate if the user is mounting its own file (this requires them to set `$ZEBRA_CONF_PATH` to a valid mounted file)
  - If the user doesn't set `ZEBRA_CONF_PATH` then validate if the default `${HOME}/.config/zebrad.toml` is mounted (commonly provided with our docker compose https://github.com/ZcashFoundation/zebra/blob/ac25192afcb1c3280ff32d2bb2e75f91f1d9ceb0/docker/docker-compose.yml#L23)
  - Otherwise, create a zebra configuration file based on the variables set by the user (or using the default values)
- Do not try to overwrite mounted files, dynamic configuration should only happen `on-the-fly` and must be avoided if a configuration file was provided (this changes `sed` to `cat` as using `sed` is no longer required)

**Other (required) changes**
- `exec_as_user` must validate if the actual user is root, otherwise running `gosu` will fail when that's not the case
- If cache or other directories were configured with the environment variables, make sure they exists and change their ownership to the user running `zebrad`.
- Do not mount the `.cookies` directory in the same directory as the `.cache` as mounting a cached state would override the cookies directory.

### Tests

All tests should be passing, and building with the docker compose, with our without a mount file, should give the expected configuration results.

### Follow-up Work

Validate which other configurations should be added using variables

### PR Checklist

<!-- Check as many boxes as possible. -->

- [x] The PR name is suitable for the release notes.
- [x] The solution is tested.
- [x] The documentation is up to date.
- [x] The PR has a priority label.
- [ ] If the PR shouldn't be in the release notes, it has the
      `C-exclude-from-changelog` label.


Linked issue #9325: Using custom config in Docker fails
Running 

```bash
docker run --rm -it --mount type=bind,src=/path/to/zebra/docker/default-zebra-config.toml,dst=/home/zebra/.config/zebrad.toml zebra:local
```

fails with an error. The issue is that the entrypoint tries to overwrite the mounted file on the host.

This is a release blocker.

One solution would be that the entrypoint would copy the mounted file to `/tmp` in the container and adjust it there before passing it to Zebra. This assumes that the `/tmp` directory is available in the container.
Linked issue comments: user: login: mpguerra
created_at: 2025-03-31T14:37:48Z
body: blocking #8999 
author_association: CONTRIBUTOR

Diff:
diff --git a/book/src/user/docker.md b/book/src/user/docker.md
index 90dad34a2d7..991c8bdefa7 100644
--- a/book/src/user/docker.md
+++ b/book/src/user/docker.md
@@ -59,49 +59,47 @@ All available Cargo features are listed at
 
 ## Configuring Zebra
 
-To configure Zebra, edit the `docker/default-zebra-config.toml` config file and
-uncomment the `configs` mapping in `docker/docker-compose.yml` so your config
-takes effect. You can see if your config works as intended by looking at Zebra's
-logs.
-
-Alternatively, you can configure Zebra by setting the environment variables in
-the `docker/.env` file. Note that the config options of this file are limited to
-the variables already present in the commented out blocks in it and adding new
-ones will not be effective. Also note that the values of the variables take
-precedence over the values set in the `docker/default-zebra-config.toml` config
-file. The `docker/.env` file serves as a quick way to override the most commonly
-used settings for Zebra, whereas the `docker/default-zebra-config.toml` file
-provides full config capabilities.
+To configure Zebra using Docker, you have a few options, processed in this order:
+
+1. **Provide a specific config file path:** Set the `ZEBRA_CONF_PATH` environment variable to point to your config file within the container.
+2. **Use the default config file:** By default, the `docker-compose.yml` file mounts `./default-zebra-config.toml` to `/home/zebra/.config/zebrad.toml` using the `configs:` mapping. Zebra will use this file if `ZEBRA_CONF_PATH` is not set. To use environment variables instead, you must **comment out** the `configs:` mapping in `docker/docker-compose.yml`.
+3. **Generate config from environment variables:** If neither of the above methods provides a config file (i.e., `ZEBRA_CONF_PATH` is unset *and* the `configs:` mapping in `docker-compose.yml` is commented out), the container's entrypoint script will *automatically generate* a default configuration file at `/home/zebra/.config/zebrad.toml`. This generated file uses specific environment variables (like `NETWORK`, `ZEBRA_RPC_PORT`, `ENABLE_COOKIE_AUTH`, `MINER_ADDRESS`, etc.) to define the settings. Using the `docker/.env` file is the primary way to set these variables for this auto-generation mode.
+
+You can see if your config works as intended by looking at Zebra's logs.
+
+Note that if you provide a configuration file using methods 1 or 2, environment variables from `docker/.env` will **not** override the settings within that file. The environment variables are primarily used for the auto-generation scenario (method 3).
 
 ### RPC
 
-Zebra's RPC server is disabled by default. To enable it, you need to set its RPC
-port. You can do that either in the `docker/default-zebra-config.toml` file or
-`docker/.env` file, as described in the two paragraphs above.
+Zebra's RPC server is disabled by default. To enable it, you need to define the RPC settings in Zebra's configuration. You can achieve this using one of the configuration methods described above:
 
-When connecting to Zebra's RPC server, your RPC clients need to provide an
-authentication cookie to the server or you need to turn the authentication off
-in Zebra's config. By default, the cookie file is stored at
-`/home/zebra/.cache/zebra/.cookie` in the container. You can print its contents
-by running
+* **Using a config file (methods 1 or 2):** Add or uncomment the `[rpc]` section in your `zebrad.toml` file (like the one provided in `docker/default-zebra-config.toml`). Ensure you set the `listen_addr` (e.g., `"0.0.0.0:8232"` for Mainnet).
+* **Using environment variables (method 3):** Set the `ZEBRA_RPC_PORT` environment variable (e.g., in `docker/.env`). This tells the entrypoint script to include an enabled `[rpc]` section listening on `0.0.0.0:<ZEBRA_RPC_PORT>` in the auto-generated configuration file.
 
-```bash
-docker exec zebra cat /home/zebra/.cache/zebra/.cookie
-```
+**Cookie Authentication:**
 
-when the `zebra` container is running. Note that Zebra generates the cookie file
-only if the RPC server is enabled, and each Zebra instance generates a unique
-one. To turn the authentication off, either set `ENABLE_COOKIE_AUTH=false` in
-`docker/.env` or set
+By default, Zebra uses cookie-based authentication for RPC requests (`enable_cookie_auth = true`). When enabled, Zebra generates a unique, random cookie file required for client authentication.
 
-```toml
-[rpc]
-enable_cookie_auth = false
-```
+* **Cookie Location:** The entrypoint script configures Zebra to store this file at `/home/zebra/.cache/zebra/.cookie` inside the container.
+* **Viewing the Cookie:** If the container is running and RPC is enabled with authentication, you can view the cookie content using:
+
+    ```bash
+    docker exec <container_name> cat /home/zebra/.cache/zebra/.cookie
+    ```
+
+    (Replace `<container_name>` with your container's name, typically `zebra` if using the default `docker-compose.yml`). Your RPC client will need this value.
+* **Disabling Authentication:** If you need to disable cookie authentication (e.g., for compatibility with tools like `lightwalletd`):
+  * If using a **config file** (methods 1 or 2), set `enable_cookie_auth = false` within the `[rpc]` section:
+
+    ```toml
+    [rpc]
+    # listen_addr = ...
+    enable_cookie_auth = false
+    ```
+
+  * If using **environment variables** for auto-generation (method 3), set `ENABLE_COOKIE_AUTH=false` in your `docker/.env` file.
 
-in `docker/default-zebra-config.toml` and mount this config file into the
-container's filesystem in `docker/docker-compose.yml` as described at the
-beginning of this section.
+Remember that Zebra only generates the cookie file if the RPC server is enabled *and* `enable_cookie_auth` is set to `true` (or omitted, as `true` is the default).
 
 ## Examples
 
@@ -121,8 +119,8 @@ Note that Docker will run Zebra with the RPC server enabled and the cookie
 authentication mechanism disabled since Lightwalletd doesn't support it. Instead
 of configuring Zebra via the recommended config file or `docker/.env` file, we
 configured the RPC server by setting environment variables directly in the
-`docker/docker-compose.lwd.yml` file. This is a shortcut we can take when we are
-familiar with the `docker/.env` file.
+`docker/docker-compose.lwd.yml` file. This takes advantage of the entrypoint
+script's auto-generation feature (method 3 described above).
 
 ### Running Zebra with Prometheus and Grafana
 
diff --git a/docker/.env b/docker/.env
index b4e8bade406..629d7d6b9c7 100644
--- a/docker/.env
+++ b/docker/.env
@@ -1,5 +1,11 @@
 # Configuration variables for running Zebra in Docker
 
+# Sets the path to a custom Zebra config file. If not set, Zebra will look for a config at
+# ${HOME}/.config/zebrad.toml or generate one using environment variables below.
+# ! Setting ZEBRA_CONF_PATH will make most of the following environment variables ineffective.
+#
+# ZEBRA_CONF_PATH="/path/to/your/custom/zebrad.toml"
+
 # Sets the network Zebra runs will run on.
 #
 # NETWORK=Mainnet
@@ -7,14 +13,17 @@
 # Zebra's RPC server is disabled by default. To enable it, set its port number.
 #
 # ZEBRA_RPC_PORT=8232   # Default RPC port number on Mainnet.
-# ZEBRA_RPC_PORT=18323  # Default RPC port number on Testnet.
+# ZEBRA_RPC_PORT=18232  # Default RPC port number on Testnet.
 
 # To disable cookie authentication, set the value below to false.
 #
 # ENABLE_COOKIE_AUTH=true
 
-# Sets a custom directory for the state and network caches. Zebra will also
-# store its cookie authentication file in this directory.
+# Sets a custom directory for the cookie authentication file.
+#
+# ZEBRA_COOKIE_DIR="/home/zebra/.config/cookie"
+
+# Sets a custom directory for the state and network caches.
 #
 # ZEBRA_CACHE_DIR="/home/zebra/.cache/zebra"
 
@@ -25,6 +34,11 @@
 #
 # FEATURES=""
 
+# Sets the listen address and port for Prometheus metrics.
+#
+# METRICS_ENDPOINT_ADDR="0.0.0.0"
+# METRICS_ENDPOINT_PORT=9999
+
 # Logging to a file is disabled by default. To enable it, uncomment the line
 # below and alternatively set your own path.
 #
@@ -41,6 +55,12 @@
 #
 # USE_JOURNALD=true
 
+# Sets the listen address and port for the tracing endpoint.
+# Only active when the 'filter-reload' feature is enabled.
+#
+# TRACING_ENDPOINT_ADDR="0.0.0.0"
+# TRACING_ENDPOINT_PORT=3000
+
 # If you are going to use Zebra as a backend for a mining pool, set your mining
 # address.
 #
diff --git a/docker/Dockerfile b/docker/Dockerfile
index 8da67cb20e2..4b24b004b13 100644
--- a/docker/Dockerfile
+++ b/docker/Dockerfile
@@ -72,6 +72,7 @@ ENV GID=${GID}
 ARG USER
 ENV USER=${USER}
 ARG HOME
+ENV HOME=${HOME}
 
 RUN addgroup --quiet --gid ${GID} ${USER} && \
     adduser --quiet --gid ${GID} --uid ${UID} --home ${HOME} ${USER} --disabled-password --gecos ""
@@ -119,10 +120,6 @@ COPY --from=electriccoinco/lightwalletd:latest /usr/local/bin/lightwalletd /usr/
 # and allow to change permissions for mounted cache directories
 COPY --from=tianon/gosu:bookworm /gosu /usr/local/bin/
 
-# Use the same default config as in the production environment.
-ENV ZEBRA_CONF_PATH="${HOME}/.config/zebrad.toml"
-COPY --chown=${UID}:${GID} ./docker/default-zebra-config.toml ${ZEBRA_CONF_PATH}
-
 # As the build has already run with the root user,
 # we need to set the correct permissions for the home and cargo home dirs owned by it.
 RUN chown -R ${UID}:${GID} "${HOME}" && \
@@ -139,7 +136,9 @@ ENTRYPOINT [ "entrypoint.sh", "test" ]
 # `test` stage. The resulting zebrad binary is used in the `runtime` stage.
 FROM deps AS release
 
+# Set the working directory for the build.
 ARG HOME
+WORKDIR ${HOME}
 
 RUN --mount=type=bind,source=tower-batch-control,target=tower-batch-control \
     --mount=type=bind,source=tower-fallback,target=tower-fallback \
@@ -191,20 +190,14 @@ ENV GID=${GID}
 ARG USER
 ENV USER=${USER}
 ARG HOME
+ENV HOME=${HOME}
 
 RUN addgroup --quiet --gid ${GID} ${USER} && \
     adduser --quiet --gid ${GID} --uid ${UID} --home ${HOME} ${USER} --disabled-password --gecos ""
 
 WORKDIR ${HOME}
-
-# We set the default locations of the conf and cache dirs according to the XDG
-# spec: https://specifications.freedesktop.org/basedir-spec/latest/
-
 RUN chown -R ${UID}:${GID} ${HOME}
 
-ARG ZEBRA_CONF_PATH="${HOME}/.config/zebrad.toml"
-ENV ZEBRA_CONF_PATH=${ZEBRA_CONF_PATH}
-
 # We're explicitly NOT using the USER directive here.
 # Instead, we run as root initially and use gosu in the entrypoint.sh
 # to step down to the non-privileged user. This allows us to change permissions
@@ -214,7 +207,6 @@ ENV ZEBRA_CONF_PATH=${ZEBRA_CONF_PATH}
 # Copy the gosu binary to be able to run the entrypoint as non-root user
 COPY --from=tianon/gosu:bookworm /gosu /usr/local/bin/
 COPY --from=release /usr/local/bin/zebrad /usr/local/bin/
-COPY --chown=${UID}:${GID} ./docker/default-zebra-config.toml ${ZEBRA_CONF_PATH}
 COPY --chown=${UID}:${GID} ./docker/entrypoint.sh /usr/local/bin/entrypoint.sh
 
 ENTRYPOINT [ "entrypoint.sh" ]
diff --git a/docker/docker-compose.yml b/docker/docker-compose.yml
index a30e248b020..b561312fe27 100644
--- a/docker/docker-compose.yml
+++ b/docker/docker-compose.yml
@@ -15,13 +15,12 @@ services:
       - zebrad-cache:/home/zebra/.cache/zebra
     # Having `tty` set to true makes Zebra use colored logs.
     tty: true
-    #! Uncomment the `configs` mapping below to make your custom configuration
-    #! take effect.
-    #
-    # configs:
-    #   - source: zebra-config
-    #     target: /home/zebra/.config/zebrad.toml
-    #
+    # ! Comment out the `configs` mapping below to use the environment variables in the
+    # ! `.env` file, instead of the default configuration file.
+    configs:
+      - source: zebra-config
+        target: /home/zebra/.config/zebrad.toml
+
     # Uncomment the `ports` mapping below to map ports between the container and
     # host.
     #
diff --git a/docker/entrypoint.sh b/docker/entrypoint.sh
index 1f4f868c1be..e73ad84e086 100755
--- a/docker/entrypoint.sh
+++ b/docker/entrypoint.sh
@@ -6,11 +6,12 @@
 #
 # ## Notes
 #
-# - `$ZEBRA_CONF_PATH` must point to a Zebra conf file.
+# - `$ZEBRA_CONF_PATH` can point to an existing Zebra config file, or if not set,
+#   the script will look for a default config at ${HOME}/.config/zebrad.toml,
+#   or generate one using environment variables.
 
 set -eo pipefail
 
-
 # These are the default cached state directories for Zebra and lightwalletd.
 #
 # They are set to `${HOME}/.cache/zebra` and `${HOME}/.cache/lwd`
@@ -18,101 +19,122 @@ set -eo pipefail
 # `ZEBRA_CACHE_DIR` and `LWD_CACHE_DIR` environment variables.
 : "${ZEBRA_CACHE_DIR:=${HOME}/.cache/zebra}"
 : "${LWD_CACHE_DIR:=${HOME}/.cache/lwd}"
-
-# Exit early if `ZEBRA_CONF_PATH` does not point to a file.
-if [[ ! -f "${ZEBRA_CONF_PATH}" ]]; then
-  echo "ERROR: No Zebra config file found at ZEBRA_CONF_PATH (${ZEBRA_CONF_PATH})."
-  echo "Please ensure the file exists or mount your custom config file and set ZEBRA_CONF_PATH accordingly."
-  exit 1
-fi
+: "${ZEBRA_COOKIE_DIR:=${HOME}/.cache/zebra}"
 
 # Use gosu to drop privileges and execute the given command as the specified UID:GID
 exec_as_user() {
-  exec gosu "${UID}:${GID}" "$@"
+  user=$(id -u)
+  if [[ ${user} == '0' ]]; then
+    exec gosu "${UID}:${GID}" "$@"
+  else
+    exec "$@"
+  fi
 }
 
-# Modifies the existing Zebra config file at ZEBRA_CONF_PATH using environment variables.
+# Modifies the Zebra config file using environment variables.
 #
-# The config options this function supports are also listed in the "docker/.env" file.
+# This function generates a new config file from scratch at ZEBRA_CONF_PATH
+# using the provided environment variables.
 #
-# This function modifies the existing file in-place and prints its location.
+# It creates a complete configuration with network settings, state, RPC,
+# metrics, tracing, and mining sections based on environment variables.
 prepare_conf_file() {
-  # Set a custom network.
-  if [[ -n "${NETWORK}" ]]; then
-    sed -i '/network = ".*"/s/".*"/"'"${NETWORK//\"/}"'"/' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Enable the RPC server by setting its port.
-  if [[ -n "${ZEBRA_RPC_PORT}" ]]; then
-    sed -i '/# listen_addr = "0.0.0.0:18232" # Testnet/d' "${ZEBRA_CONF_PATH}"
-    sed -i 's/ *# Mainnet$//' "${ZEBRA_CONF_PATH}"
-    sed -i '/# listen_addr = "0.0.0.0:8232"/s/^# //; s/8232/'"${ZEBRA_RPC_PORT//\"/}"'/' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Disable or enable cookie authentication.
-  if [[ -n "${ENABLE_COOKIE_AUTH}" ]]; then
-    sed -i '/# enable_cookie_auth = true/s/^# //; s/true/'"${ENABLE_COOKIE_AUTH//\"/}"'/' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Set a custom state, network and cookie cache dirs.
-  #
-  # We're pointing all three cache dirs at the same location, so users will find
-  # all cached data in that single location. We can introduce more env vars and
-  # use them to set the cache dirs separately if needed.
-  if [[ -n "${ZEBRA_CACHE_DIR}" ]]; then
-    mkdir -p "${ZEBRA_CACHE_DIR//\"/}"
-    chown -R "${UID}:${GID}" "${ZEBRA_CACHE_DIR//\"/}"
-    sed -i 's|_dir = ".*"|_dir = "'"${ZEBRA_CACHE_DIR//\"/}"'"|' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Set a custom lightwalletd cache dir.
-  if [[ -n "${LWD_CACHE_DIR}" ]]; then
-    mkdir -p "${LWD_CACHE_DIR//\"/}"
-    chown -R "${UID}:${GID}" "${LWD_CACHE_DIR//\"/}"
-  fi
-
-  # Enable the Prometheus metrics endpoint.
-  if [[ "${FEATURES}" == *"prometheus"* ]]; then
-    sed -i '/# endpoint_addr = "0.0.0.0:9999" # Prometheus/s/^# //' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Enable logging to a file by setting a custom log file path.
-  if [[ -n "${LOG_FILE}" ]]; then
-    mkdir -p "$(dirname "${LOG_FILE//\"/}")"
-    chown -R "${UID}:${GID}" "$(dirname "${LOG_FILE//\"/}")"
-    sed -i 's|# log_file = ".*"|log_file = "'"${LOG_FILE//\"/}"'"|' "${ZEBRA_CONF_PATH}"
-  fi
+  # Base configuration
+  cat >"${ZEBRA_CONF_PATH}" <<EOF
+[network]
+network = "${NETWORK:=Mainnet}"
+listen_addr = "0.0.0.0"
+cache_dir = "${ZEBRA_CACHE_DIR}"
+
+[state]
+cache_dir = "${ZEBRA_CACHE_DIR}"
+
+$( [[ -n ${ZEBRA_RPC_PORT} ]] && cat <<-SUB_EOF
+
+[rpc]
+listen_addr = "${RPC_LISTEN_ADDR:=0.0.0.0}:${ZEBRA_RPC_PORT}"
+enable_cookie_auth = ${ENABLE_COOKIE_AUTH:=true}
+$( [[ -n ${ZEBRA_COOKIE_DIR} ]] && echo "cookie_dir = \"${ZEBRA_COOKIE_DIR}\"" )
+SUB_EOF
+)
+
+$( [[ " ${FEATURES} " =~ " prometheus " ]] && cat <<-SUB_EOF
+
+[metrics]
+endpoint_addr = "${METRICS_ENDPOINT_ADDR:=0.0.0.0}:${METRICS_ENDPOINT_PORT:=9999}"
+SUB_EOF
+)
+
+$( [[ -n ${LOG_FILE} || -n ${LOG_COLOR} || -n ${TRACING_ENDPOINT_ADDR} || -n ${USE_JOURNALD} ]] && cat <<-SUB_EOF
+
+[tracing]
+$( [[ -n ${USE_JOURNALD} ]] && echo "use_journald = ${USE_JOURNALD}" )
+$( [[ " ${FEATURES} " =~ " filter-reload " ]] && echo "endpoint_addr = \"${TRACING_ENDPOINT_ADDR:=0.0.0.0}:${TRACING_ENDPOINT_PORT:=3000}\"" )
+$( [[ -n ${LOG_FILE} ]] && echo "log_file = \"${LOG_FILE}\"" )
+$( [[ ${LOG_COLOR} == "true" ]] && echo "force_use_color = true" )
+$( [[ ${LOG_COLOR} == "false" ]] && echo "use_color = false" )
+SUB_EOF
+)
+
+$( [[ -n ${MINER_ADDRESS} ]] && cat <<-SUB_EOF
+
+[mining]
+miner_address = "${MINER_ADDRESS}"
+SUB_EOF
+)
+EOF
+
+# Ensure the config file itself has the correct ownership
+#
+# This is safe in this context because prepare_conf_file is called only when
+# ZEBRA_CONF_PATH is not set, and there's no file mounted at that path.
+chown "${UID}:${GID}" "${ZEBRA_CONF_PATH}" || exit_error "Failed to secure config file: ${ZEBRA_CONF_PATH}"
 
-  # Enable or disable colored logs.
-  if [[ -n "${LOG_COLOR}" ]]; then
-    sed -i '/# force_use_color = true/s/^# //' "${ZEBRA_CONF_PATH}"
-    sed -i '/use_color = true/s/true/'"${LOG_COLOR//\"/}"'/' "${ZEBRA_CONF_PATH}"
-  fi
+}
 
-  # Enable or disable logging to systemd-journald.
-  if [[ -n "${USE_JOURNALD}" ]]; then
-    sed -i '/# use_journald = true/s/^# //; s/true/'"${USE_JOURNALD//\"/}"'/' "${ZEBRA_CONF_PATH}"
-  fi
+# Helper function
+exit_error() {
+  echo "$1" >&2
+  exit 1
+}
 
-  # Set a mining address.
-  if [[ -n "${MINER_ADDRESS}" ]]; then
-    sed -i '/# miner_address = ".*"/{s/^# //; s/".*"/"'"${MINER_ADDRESS//\"/}"'"/}' "${ZEBRA_CONF_PATH}"
+# Creates a directory if it doesn't exist and sets ownership to specified UID:GID.
+# Also ensures the parent directories have the correct ownership.
+#
+# ## Parameters
+#
+# - $1: Directory path to create and own
+create_owned_directory() {
+  local dir="$1"
+  # Skip if directory is empty
+  [[ -z ${dir} ]] && return
+
+  # Create directory with parents
+  mkdir -p "${dir}" || exit_error "Failed to create directory: ${dir}"
+
+  # Set ownership for the created directory
+  chown -R "${UID}:${GID}" "${dir}" || exit_error "Failed to secure directory: ${dir}"
+
+  # Set ownership for parent directory (but not if it's root or home)
+  local parent_dir
+  parent_dir="$(dirname "${dir}")"
+  if [[ "${parent_dir}" != "/" && "${parent_dir}" != "${HOME}" ]]; then
+    chown "${UID}:${GID}" "${parent_dir}"
   fi
-
-  # Trim all comments and empty lines.
-  sed -i '/^#/d; /^$/d' "${ZEBRA_CONF_PATH}"
-
-  echo "${ZEBRA_CONF_PATH}"
 }
 
+# Create and own cache and config directories
+[[ -n ${ZEBRA_CACHE_DIR} ]] && create_owned_directory "${ZEBRA_CACHE_DIR}"
+[[ -n ${LWD_CACHE_DIR} ]] && create_owned_directory "${LWD_CACHE_DIR}"
+[[ -n ${ZEBRA_COOKIE_DIR} ]] && create_owned_directory "${ZEBRA_COOKIE_DIR}"
+[[ -n ${LOG_FILE} ]] && create_owned_directory "$(dirname "${LOG_FILE}")"
+
 # Runs cargo test with an arbitrary number of arguments.
 #
 # Positional Parameters
 #
-# - '$1' must contain
-#   - either cargo FEATURES as described here:
-#     https://doc.rust-lang.org/cargo/reference/features.html#command-line-feature-options,
-#   - or be empty.
+# - '$1' must contain cargo FEATURES as described here:
+#   https://doc.rust-lang.org/cargo/reference/features.html#command-line-feature-options
 # - The remaining params will be appended to a command starting with
 #   `exec_as_user cargo test ... -- ...`
 run_cargo_test() {
@@ -255,12 +277,39 @@ run_tests() {
 }
 
 # Main Script Logic
+#
+# 1. First check if ZEBRA_CONF_PATH is explicitly set or if a file exists at that path
+# 2. If not set but default config exists, use that
+# 3. If neither exists, generate a default config at ${HOME}/.config/zebrad.toml
+# 4. Print environment variables and config for debugging
+# 5. Process command-line arguments and execute appropriate action
+if [[ -n ${ZEBRA_CONF_PATH} ]]; then
+  if [[ -f ${ZEBRA_CONF_PATH} ]]; then
+    echo "ZEBRA_CONF_PATH was set to ${ZEBRA_CONF_PATH} and a file exists."
+    echo "Using user-provided config file"
+  else
+    echo "ERROR: ZEBRA_CONF_PATH was set and no config file found at ${ZEBRA_CONF_PATH}."
+    echo "Please ensure a config file exists or set ZEBRA_CONF_PATH to point to your config file."
+    exit 1
+  fi
+else
+  if [[ -f "${HOME}/.config/zebrad.toml" ]]; then
+    echo "ZEBRA_CONF_PATH was not set."
+    echo "Using default config at ${HOME}/.config/zebrad.toml"
+    ZEBRA_CONF_PATH="${HOME}/.config/zebrad.toml"
+  else
+    echo "ZEBRA_CONF_PATH was not set and no default config found at ${HOME}/.config/zebrad.toml"
+    echo "Preparing a default one..."
+    ZEBRA_CONF_PATH="${HOME}/.config/zebrad.toml"
+    create_owned_directory "$(dirname "${ZEBRA_CONF_PATH}")"
+    prepare_conf_file
+  fi
+fi
 
-prepare_conf_file "${ZEBRA_CONF_PATH}"
 echo "INFO: Using the following environment variables:"
 printenv
 
-echo "Prepared the following Zebra config:"
+echo "Using Zebra config at ${ZEBRA_CONF_PATH}:"
 cat "${ZEBRA_CONF_PATH}"
 
 # - If "$1" is "--", "-", or "zebrad", run `zebrad` with the remaining params.


Reviews:
user: login: upbqdn
body: @gustavovalverde, have you tested the solution? It looks like some parts don't work. For example, setting the RPC port.

If I set `ZEBRA_RPC_PORT=28232`, the script configures Zebra to use

```toml
[rpc]
listen_addr = "0.0.0.0:8232"
```

which is not the desired port.
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-01T14:36:27Z
user: login: gustavovalverde
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-08T13:00:54Z
user: login: gustavovalverde
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-08T13:01:19Z
user: login: gustavovalverde
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-08T13:01:24Z
user: login: gustavovalverde
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-08T13:01:31Z
user: login: gustavovalverde
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-08T13:03:04Z
user: login: gustavovalverde
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-08T13:03:10Z
user: login: upbqdn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-11T12:02:40Z
user: login: gustavovalverde
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-11T15:12:08Z
user: login: upbqdn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-14T22:08:44Z
user: login: upbqdn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T08:58:37Z
user: login: upbqdn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T09:01:44Z
user: login: gustavovalverde
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T15:26:27Z
user: login: gustavovalverde
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T15:26:32Z
user: login: gustavovalverde
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T15:28:42Z
user: login: gustavovalverde
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T15:28:59Z
user: login: upbqdn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T21:35:50Z
user: login: upbqdn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T21:49:14Z
user: login: upbqdn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T21:52:42Z
user: login: gustavovalverde
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-15T21:52:49Z
user: login: gustavovalverde
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T06:14:28Z
user: login: upbqdn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-16T09:22:16Z
user: login: upbqdn
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-16T11:05:42Z

Review comments:
diff_hunk: @@ -23,7 +23,7 @@ cache_dir = "/home/zebra/.cache/zebra"
 # listen_addr = "0.0.0.0:8232" # Mainnet
 # listen_addr = "0.0.0.0:18232" # Testnet
 
-cookie_dir = "/home/zebra/.cache/zebra"
+cookie_dir = "/home/zebra/.config/cookie"
path: docker/default-zebra-config.toml
user: login: upbqdn
body: This is a breaking change. We need to update the existing docs and add new since the location differs from where Zebra stores the cookie by default.
created_at: 2025-04-01T09:50:10Z
author_association: MEMBER
diff_hunk: @@ -59,19 +59,22 @@ All available Cargo features are listed at
 
 ## Configuring Zebra
 
-To configure Zebra, edit the `docker/default-zebra-config.toml` config file and
-uncomment the `configs` mapping in `docker/docker-compose.yml` so your config
-takes effect. You can see if your config works as intended by looking at Zebra's
-logs.
-
-Alternatively, you can configure Zebra by setting the environment variables in
-the `docker/.env` file. Note that the config options of this file are limited to
-the variables already present in the commented out blocks in it and adding new
-ones will not be effective. Also note that the values of the variables take
-precedence over the values set in the `docker/default-zebra-config.toml` config
-file. The `docker/.env` file serves as a quick way to override the most commonly
-used settings for Zebra, whereas the `docker/default-zebra-config.toml` file
-provides full config capabilities.
+To configure Zebra, you have three options:
+
+1. Edit the `docker/default-zebra-config.toml` config file and uncomment the `configs` mapping in `docker/docker-compose.yml` so your config takes effect.
+2. Provide a path to your own config file via the `ZEBRA_CONF_PATH` environment variable.
+3. Let Zebra generate a default config file automatically from environment variables (if neither of the above methods is used).
+
+The entrypoint script follows this sequence:
+
+1. If `ZEBRA_CONF_PATH` is set and a file exists at that path, it uses that file
+2. If `ZEBRA_CONF_PATH` is not set but a default config exists at `${HOME}/.config/zebrad.toml`, it uses that file
path: book/src/user/docker.md
user: login: upbqdn
body: It won't be the default config if users mount their own.

```suggestion
2. If `ZEBRA_CONF_PATH` is not set but a file exists at `${HOME}/.config/zebrad.toml`, it uses that file
```
created_at: 2025-04-01T10:40:59Z
author_association: MEMBER
diff_hunk: @@ -6,113 +6,125 @@
 #
 # ## Notes
 #
-# - `$ZEBRA_CONF_PATH` must point to a Zebra conf file.
+# - `$ZEBRA_CONF_PATH` can point to an existing Zebra config file, or if not set,
+#   the script will look for a default config at ${HOME}/.config/zebrad.toml,
path: docker/entrypoint.sh
user: login: upbqdn
body: ```suggestion
#   the script will look for a config at ${HOME}/.config/zebrad.toml,
```
created_at: 2025-04-01T10:47:15Z
author_association: MEMBER
diff_hunk: @@ -255,12 +267,39 @@ run_tests() {
 }
 
 # Main Script Logic
+#
+# 1. First check if ZEBRA_CONF_PATH is explicitly set or if a file exists at that path
+# 2. If not set but default config exists, use that
+# 3. If neither exists, generate a default config at ${HOME}/.config/zebrad.toml
+# 4. Print environment variables and config for debugging
+# 5. Process command-line arguments and execute appropriate action
+if [[ -n ${ZEBRA_CONF_PATH} ]]; then
+  if [[ -f ${ZEBRA_CONF_PATH} ]]; then
+    echo "ZEBRA_CONF_PATH was set to ${ZEBRA_CONF_PATH} and a file exists."
+    echo "Using user-provided config file"
+  else
+    echo "ERROR: ZEBRA_CONF_PATH was set and no config file found at ${ZEBRA_CONF_PATH}."
+    echo "Please ensure a config file exists or set ZEBRA_CONF_PATH to point to your config file."
+    exit 1
+  fi
+else
+  if [[ -f "${HOME}/.config/zebrad.toml" ]]; then
+    echo "ZEBRA_CONF_PATH was not set."
+    echo "Using default config at ${HOME}/.config/zebrad.toml"
path: docker/entrypoint.sh
user: login: upbqdn
body: ```suggestion
    echo "Using the config at ${HOME}/.config/zebrad.toml"
```
created_at: 2025-04-01T10:58:43Z
author_association: MEMBER
diff_hunk: @@ -255,12 +267,39 @@ run_tests() {
 }
 
 # Main Script Logic
+#
+# 1. First check if ZEBRA_CONF_PATH is explicitly set or if a file exists at that path
+# 2. If not set but default config exists, use that
+# 3. If neither exists, generate a default config at ${HOME}/.config/zebrad.toml
+# 4. Print environment variables and config for debugging
+# 5. Process command-line arguments and execute appropriate action
+if [[ -n ${ZEBRA_CONF_PATH} ]]; then
+  if [[ -f ${ZEBRA_CONF_PATH} ]]; then
+    echo "ZEBRA_CONF_PATH was set to ${ZEBRA_CONF_PATH} and a file exists."
+    echo "Using user-provided config file"
+  else
+    echo "ERROR: ZEBRA_CONF_PATH was set and no config file found at ${ZEBRA_CONF_PATH}."
+    echo "Please ensure a config file exists or set ZEBRA_CONF_PATH to point to your config file."
+    exit 1
+  fi
+else
+  if [[ -f "${HOME}/.config/zebrad.toml" ]]; then
+    echo "ZEBRA_CONF_PATH was not set."
+    echo "Using default config at ${HOME}/.config/zebrad.toml"
+    ZEBRA_CONF_PATH="${HOME}/.config/zebrad.toml"
+  else
+    echo "ZEBRA_CONF_PATH was not set and no default config found at ${HOME}/.config/zebrad.toml"
path: docker/entrypoint.sh
user: login: upbqdn
body: ```suggestion
    echo "ZEBRA_CONF_PATH was not set and no config found at ${HOME}/.config/zebrad.toml"
```
created_at: 2025-04-01T11:06:58Z
author_association: MEMBER
diff_hunk: @@ -1,30 +1,60 @@
 # Configuration variables for running Zebra in Docker
 
+# Sets the path to a custom Zebra config file. If not set, Zebra will look for a config at
+# ${HOME}/.config/zebrad.toml or generate one using environment variables below.
+# ! Setting ZEBRA_CONF_PATH will make most of the following environment variables ineffective.
+# TODO: zebrad should be able to read from both the config file and environment variables.
+#
+# ZEBRA_CONF_PATH="/path/to/your/custom/zebrad.toml"
+
 # Sets the network Zebra runs will run on.
 #
 # NETWORK=Mainnet
 
+# Sets the listen address for Zebra's peer-to-peer network connections.
+#
+# ZEBRA_LISTEN_ADDR="0.0.0.0"
+
 # Zebra's RPC server is disabled by default. To enable it, set its port number.
 #
 # ZEBRA_RPC_PORT=8232   # Default RPC port number on Mainnet.
-# ZEBRA_RPC_PORT=18323  # Default RPC port number on Testnet.
+# ZEBRA_RPC_PORT=18232  # Default RPC port number on Testnet.
+
+# Sets the listen address and port for Zebra's RPC server.
+# RPC_PORT will be used if ZEBRA_RPC_PORT is not set.
+#
+# RPC_LISTEN_ADDR="0.0.0.0"
+# RPC_PORT=8232   # Default RPC port number on Mainnet
 
 # To disable cookie authentication, set the value below to false.
 #
 # ENABLE_COOKIE_AUTH=true
 
-# Sets a custom directory for the state and network caches. Zebra will also
-# store its cookie authentication file in this directory.
+# Sets a custom directory for the cookie authentication file.
+#
+# ZEBRA_COOKIE_DIR="/home/zebra/.config/cookie"
+
+# Sets a custom directory for the state and network caches.
 #
 # ZEBRA_CACHE_DIR="/home/zebra/.cache/zebra"
 
+# Sets a custom directory for the lightwalletd cache.
+#
+# LWD_CACHE_DIR="/home/zebra/.cache/lwd"
+
 # Sets custom Cargo features. Available features are listed at
 # <https://docs.rs/zebrad/latest/zebrad/index.html#zebra-feature-flags>.
 #
 # Must be set at build time.
 #
 # FEATURES=""
 
+# Sets the listen address and port for Prometheus metrics.
+# Only active when the 'prometheus' feature is enabled.
path: docker/.env
user: login: upbqdn
body: We'll make the `prometheus` feature enabled by default soon.

```suggestion
```
created_at: 2025-04-01T13:43:12Z
author_association: MEMBER
diff_hunk: @@ -1,30 +1,60 @@
 # Configuration variables for running Zebra in Docker
 
+# Sets the path to a custom Zebra config file. If not set, Zebra will look for a config at
+# ${HOME}/.config/zebrad.toml or generate one using environment variables below.
+# ! Setting ZEBRA_CONF_PATH will make most of the following environment variables ineffective.
+# TODO: zebrad should be able to read from both the config file and environment variables.
+#
+# ZEBRA_CONF_PATH="/path/to/your/custom/zebrad.toml"
+
 # Sets the network Zebra runs will run on.
 #
 # NETWORK=Mainnet
 
+# Sets the listen address for Zebra's peer-to-peer network connections.
+#
+# ZEBRA_LISTEN_ADDR="0.0.0.0"
+
path: docker/.env
user: login: upbqdn
body: I don't think users should be presented with this option since using any other address will make the endpoint inaccessible outside Docker.
created_at: 2025-04-01T13:51:57Z
author_association: MEMBER
diff_hunk: @@ -1,30 +1,60 @@
 # Configuration variables for running Zebra in Docker
 
+# Sets the path to a custom Zebra config file. If not set, Zebra will look for a config at
+# ${HOME}/.config/zebrad.toml or generate one using environment variables below.
+# ! Setting ZEBRA_CONF_PATH will make most of the following environment variables ineffective.
+# TODO: zebrad should be able to read from both the config file and environment variables.
+#
+# ZEBRA_CONF_PATH="/path/to/your/custom/zebrad.toml"
+
 # Sets the network Zebra runs will run on.
 #
 # NETWORK=Mainnet
 
+# Sets the listen address for Zebra's peer-to-peer network connections.
+#
+# ZEBRA_LISTEN_ADDR="0.0.0.0"
+
 # Zebra's RPC server is disabled by default. To enable it, set its port number.
 #
 # ZEBRA_RPC_PORT=8232   # Default RPC port number on Mainnet.
-# ZEBRA_RPC_PORT=18323  # Default RPC port number on Testnet.
+# ZEBRA_RPC_PORT=18232  # Default RPC port number on Testnet.
+
+# Sets the listen address and port for Zebra's RPC server.
+# RPC_PORT will be used if ZEBRA_RPC_PORT is not set.
+#
+# RPC_LISTEN_ADDR="0.0.0.0"
+# RPC_PORT=8232   # Default RPC port number on Mainnet
 
 # To disable cookie authentication, set the value below to false.
 #
 # ENABLE_COOKIE_AUTH=true
 
-# Sets a custom directory for the state and network caches. Zebra will also
-# store its cookie authentication file in this directory.
+# Sets a custom directory for the cookie authentication file.
+#
+# ZEBRA_COOKIE_DIR="/home/zebra/.config/cookie"
+
+# Sets a custom directory for the state and network caches.
 #
 # ZEBRA_CACHE_DIR="/home/zebra/.cache/zebra"
 
+# Sets a custom directory for the lightwalletd cache.
+#
+# LWD_CACHE_DIR="/home/zebra/.cache/lwd"
+
path: docker/.env
user: login: upbqdn
body: Is there any use-case for `LWD_CACHE_DIR`?

```suggestion
```
created_at: 2025-04-01T13:58:40Z
author_association: MEMBER
diff_hunk: @@ -1,30 +1,60 @@
 # Configuration variables for running Zebra in Docker
 
+# Sets the path to a custom Zebra config file. If not set, Zebra will look for a config at
+# ${HOME}/.config/zebrad.toml or generate one using environment variables below.
+# ! Setting ZEBRA_CONF_PATH will make most of the following environment variables ineffective.
+# TODO: zebrad should be able to read from both the config file and environment variables.
path: docker/.env
user: login: upbqdn
body: Since this is a user-facing config file, I expect the TODO will be confusing.

```suggestion
```
created_at: 2025-04-01T14:05:38Z
author_association: MEMBER
diff_hunk: @@ -59,19 +59,22 @@ All available Cargo features are listed at
 
 ## Configuring Zebra
 
-To configure Zebra, edit the `docker/default-zebra-config.toml` config file and
-uncomment the `configs` mapping in `docker/docker-compose.yml` so your config
-takes effect. You can see if your config works as intended by looking at Zebra's
-logs.
-
-Alternatively, you can configure Zebra by setting the environment variables in
-the `docker/.env` file. Note that the config options of this file are limited to
-the variables already present in the commented out blocks in it and adding new
-ones will not be effective. Also note that the values of the variables take
-precedence over the values set in the `docker/default-zebra-config.toml` config
-file. The `docker/.env` file serves as a quick way to override the most commonly
-used settings for Zebra, whereas the `docker/default-zebra-config.toml` file
-provides full config capabilities.
+To configure Zebra, you have three options:
+
+1. Edit the `docker/default-zebra-config.toml` config file and uncomment the `configs` mapping in `docker/docker-compose.yml` so your config takes effect.
+2. Provide a path to your own config file via the `ZEBRA_CONF_PATH` environment variable.
+3. Let Zebra generate a default config file automatically from environment variables (if neither of the above methods is used).
+
+The entrypoint script follows this sequence:
+
+1. If `ZEBRA_CONF_PATH` is set and a file exists at that path, it uses that file
+2. If `ZEBRA_CONF_PATH` is not set but a default config exists at `${HOME}/.config/zebrad.toml`, it uses that file
path: book/src/user/docker.md
user: login: gustavovalverde
body: The user should mount their file and set `ZEBRA_CONF_PATH` if they want to use their own custom configuration, otherwise the default config (`default-zebra-config.toml`) will be used if it's mounted. That's now the default approach in the `docker-compose.yml`.

Certainly, the user can set `ZEBRA_CONF_PATH` and mount their file to the default location`${HOME}/.config/zebrad.toml`, but that case would fall into the _user-provided_ condition.

Recent change to make this clearer: [`be37ecb` (#9344)](https://github.com/ZcashFoundation/zebra/pull/9344/commits/be37ecb853637ec7494d9202060a50d112383ba3)
created_at: 2025-04-08T13:00:54Z
author_association: MEMBER
diff_hunk: @@ -6,113 +6,125 @@
 #
 # ## Notes
 #
-# - `$ZEBRA_CONF_PATH` must point to a Zebra conf file.
+# - `$ZEBRA_CONF_PATH` can point to an existing Zebra config file, or if not set,
+#   the script will look for a default config at ${HOME}/.config/zebrad.toml,
path: docker/entrypoint.sh
user: login: gustavovalverde
body: https://github.com/ZcashFoundation/zebra/pull/9344#discussion_r2033143765
created_at: 2025-04-08T13:01:19Z
author_association: MEMBER
diff_hunk: @@ -255,12 +267,39 @@ run_tests() {
 }
 
 # Main Script Logic
+#
+# 1. First check if ZEBRA_CONF_PATH is explicitly set or if a file exists at that path
+# 2. If not set but default config exists, use that
+# 3. If neither exists, generate a default config at ${HOME}/.config/zebrad.toml
+# 4. Print environment variables and config for debugging
+# 5. Process command-line arguments and execute appropriate action
+if [[ -n ${ZEBRA_CONF_PATH} ]]; then
+  if [[ -f ${ZEBRA_CONF_PATH} ]]; then
+    echo "ZEBRA_CONF_PATH was set to ${ZEBRA_CONF_PATH} and a file exists."
+    echo "Using user-provided config file"
+  else
+    echo "ERROR: ZEBRA_CONF_PATH was set and no config file found at ${ZEBRA_CONF_PATH}."
+    echo "Please ensure a config file exists or set ZEBRA_CONF_PATH to point to your config file."
+    exit 1
+  fi
+else
+  if [[ -f "${HOME}/.config/zebrad.toml" ]]; then
+    echo "ZEBRA_CONF_PATH was not set."
+    echo "Using default config at ${HOME}/.config/zebrad.toml"
path: docker/entrypoint.sh
user: login: gustavovalverde
body: https://github.com/ZcashFoundation/zebra/pull/9344#discussion_r2033143765
created_at: 2025-04-08T13:01:24Z
author_association: MEMBER
diff_hunk: @@ -255,12 +267,39 @@ run_tests() {
 }
 
 # Main Script Logic
+#
+# 1. First check if ZEBRA_CONF_PATH is explicitly set or if a file exists at that path
+# 2. If not set but default config exists, use that
+# 3. If neither exists, generate a default config at ${HOME}/.config/zebrad.toml
+# 4. Print environment variables and config for debugging
+# 5. Process command-line arguments and execute appropriate action
+if [[ -n ${ZEBRA_CONF_PATH} ]]; then
+  if [[ -f ${ZEBRA_CONF_PATH} ]]; then
+    echo "ZEBRA_CONF_PATH was set to ${ZEBRA_CONF_PATH} and a file exists."
+    echo "Using user-provided config file"
+  else
+    echo "ERROR: ZEBRA_CONF_PATH was set and no config file found at ${ZEBRA_CONF_PATH}."
+    echo "Please ensure a config file exists or set ZEBRA_CONF_PATH to point to your config file."
+    exit 1
+  fi
+else
+  if [[ -f "${HOME}/.config/zebrad.toml" ]]; then
+    echo "ZEBRA_CONF_PATH was not set."
+    echo "Using default config at ${HOME}/.config/zebrad.toml"
+    ZEBRA_CONF_PATH="${HOME}/.config/zebrad.toml"
+  else
+    echo "ZEBRA_CONF_PATH was not set and no default config found at ${HOME}/.config/zebrad.toml"
path: docker/entrypoint.sh
user: login: gustavovalverde
body: https://github.com/ZcashFoundation/zebra/pull/9344#discussion_r2033143765
created_at: 2025-04-08T13:01:31Z
author_association: MEMBER
diff_hunk: @@ -1,30 +1,60 @@
 # Configuration variables for running Zebra in Docker
 
+# Sets the path to a custom Zebra config file. If not set, Zebra will look for a config at
+# ${HOME}/.config/zebrad.toml or generate one using environment variables below.
+# ! Setting ZEBRA_CONF_PATH will make most of the following environment variables ineffective.
+# TODO: zebrad should be able to read from both the config file and environment variables.
+#
+# ZEBRA_CONF_PATH="/path/to/your/custom/zebrad.toml"
+
 # Sets the network Zebra runs will run on.
 #
 # NETWORK=Mainnet
 
+# Sets the listen address for Zebra's peer-to-peer network connections.
+#
+# ZEBRA_LISTEN_ADDR="0.0.0.0"
+
 # Zebra's RPC server is disabled by default. To enable it, set its port number.
 #
 # ZEBRA_RPC_PORT=8232   # Default RPC port number on Mainnet.
-# ZEBRA_RPC_PORT=18323  # Default RPC port number on Testnet.
+# ZEBRA_RPC_PORT=18232  # Default RPC port number on Testnet.
+
+# Sets the listen address and port for Zebra's RPC server.
+# RPC_PORT will be used if ZEBRA_RPC_PORT is not set.
+#
+# RPC_LISTEN_ADDR="0.0.0.0"
+# RPC_PORT=8232   # Default RPC port number on Mainnet
 
 # To disable cookie authentication, set the value below to false.
 #
 # ENABLE_COOKIE_AUTH=true
 
-# Sets a custom directory for the state and network caches. Zebra will also
-# store its cookie authentication file in this directory.
+# Sets a custom directory for the cookie authentication file.
+#
+# ZEBRA_COOKIE_DIR="/home/zebra/.config/cookie"
+
+# Sets a custom directory for the state and network caches.
 #
 # ZEBRA_CACHE_DIR="/home/zebra/.cache/zebra"
 
+# Sets a custom directory for the lightwalletd cache.
+#
+# LWD_CACHE_DIR="/home/zebra/.cache/lwd"
+
path: docker/.env
user: login: gustavovalverde
body: Fixed with [`9e336f6` (#9344)](https://github.com/ZcashFoundation/zebra/pull/9344/commits/9e336f65d5647c8d0fb6901d6df1e5fc697275c2) 
created_at: 2025-04-08T13:03:04Z
author_association: MEMBER
diff_hunk: @@ -1,30 +1,60 @@
 # Configuration variables for running Zebra in Docker
 
+# Sets the path to a custom Zebra config file. If not set, Zebra will look for a config at
+# ${HOME}/.config/zebrad.toml or generate one using environment variables below.
+# ! Setting ZEBRA_CONF_PATH will make most of the following environment variables ineffective.
+# TODO: zebrad should be able to read from both the config file and environment variables.
+#
+# ZEBRA_CONF_PATH="/path/to/your/custom/zebrad.toml"
+
 # Sets the network Zebra runs will run on.
 #
 # NETWORK=Mainnet
 
+# Sets the listen address for Zebra's peer-to-peer network connections.
+#
+# ZEBRA_LISTEN_ADDR="0.0.0.0"
+
path: docker/.env
user: login: gustavovalverde
body: Fixed with [`9e336f6` (#9344)](https://github.com/ZcashFoundation/zebra/pull/9344/commits/9e336f65d5647c8d0fb6901d6df1e5fc697275c2) 
created_at: 2025-04-08T13:03:10Z
author_association: MEMBER
diff_hunk: @@ -15,13 +15,12 @@ services:
       - zebrad-cache:/home/zebra/.cache/zebra
     # Having `tty` set to true makes Zebra use colored logs.
     tty: true
-    #! Uncomment the `configs` mapping below to make your custom configuration
-    #! take effect.
-    #
-    # configs:
-    #   - source: zebra-config
-    #     target: /home/zebra/.config/zebrad.toml
-    #
+    # ! Comment out the `configs` mapping below to not use our default configuration file.
+    # ! This will use the environment variables in the `.env` file.
path: docker/docker-compose.yml
user: login: upbqdn
body: From the user's perspective, it's not clear to me what _"This"_ refers to. Is it the comment above or the configs below?
created_at: 2025-04-11T12:01:03Z
author_association: MEMBER
diff_hunk: @@ -15,13 +15,12 @@ services:
       - zebrad-cache:/home/zebra/.cache/zebra
     # Having `tty` set to true makes Zebra use colored logs.
     tty: true
-    #! Uncomment the `configs` mapping below to make your custom configuration
-    #! take effect.
-    #
-    # configs:
-    #   - source: zebra-config
-    #     target: /home/zebra/.config/zebrad.toml
-    #
+    # ! Comment out the `configs` mapping below to not use our default configuration file.
+    # ! This will use the environment variables in the `.env` file.
path: docker/docker-compose.yml
user: login: gustavovalverde
body: Fixed here: [chore: better comment for user understanding](https://github.com/ZcashFoundation/zebra/pull/9344/commits/63f4c6795568f676c788fb785283a6f11ad11af8)
created_at: 2025-04-11T15:12:08Z
author_association: MEMBER
diff_hunk: @@ -255,12 +258,39 @@ run_tests() {
 }
 
 # Main Script Logic
+#
+# 1. First check if ZEBRA_CONF_PATH is explicitly set or if a file exists at that path
+# 2. If not set but default config exists, use that
+# 3. If neither exists, generate a default config at ${HOME}/.config/zebrad.toml
+# 4. Print environment variables and config for debugging
+# 5. Process command-line arguments and execute appropriate action
+if [[ -n ${ZEBRA_CONF_PATH} ]]; then
+  if [[ -f ${ZEBRA_CONF_PATH} ]]; then
+    echo "ZEBRA_CONF_PATH was set to ${ZEBRA_CONF_PATH} and a file exists."
+    echo "Using user-provided config file"
+  else
+    echo "ERROR: ZEBRA_CONF_PATH was set and no config file found at ${ZEBRA_CONF_PATH}."
+    echo "Please ensure a config file exists or set ZEBRA_CONF_PATH to point to your config file."
+    exit 1
+  fi
+else
+  if [[ -f "${HOME}/.config/zebrad.toml" ]]; then
+    echo "ZEBRA_CONF_PATH was not set."
+    echo "Using default config at ${HOME}/.config/zebrad.toml"
+    ZEBRA_CONF_PATH="${HOME}/.config/zebrad.toml"
+  else
+    echo "ZEBRA_CONF_PATH was not set and no default config found at ${HOME}/.config/zebrad.toml"
+    echo "Preparing a default one..."
+    ZEBRA_CONF_PATH="${HOME}/.config/zebrad.toml"
+    mkdir -p "$(dirname "${ZEBRA_CONF_PATH}")"
+    prepare_conf_file
path: docker/entrypoint.sh
user: login: upbqdn
body: ###### non-blocking

This will create the config file and its parent dir owned by root:

```bash
root@1cbf7683b54d:~/.config# pwd
/home/zebra/.config
root@1cbf7683b54d:~/.config# ls -lah
total 4.0K
drwxr-xr-x 1 root  root   34 Apr 14 14:10 .
drwx------ 1 zebra zebra  26 Apr 14 14:10 ..
drwxr-xr-x 1 zebra zebra   0 Apr 14 14:10 cookie
-rw-r--r-- 1 root  root  149 Apr 14 14:10 zebrad.toml
```
created_at: 2025-04-14T21:54:19Z
author_association: MEMBER
diff_hunk: @@ -6,113 +6,116 @@
 #
 # ## Notes
 #
-# - `$ZEBRA_CONF_PATH` must point to a Zebra conf file.
+# - `$ZEBRA_CONF_PATH` can point to an existing Zebra config file, or if not set,
+#   the script will look for a default config at ${HOME}/.config/zebrad.toml,
+#   or generate one using environment variables.
 
 set -eo pipefail
 
-
 # These are the default cached state directories for Zebra and lightwalletd.
 #
 # They are set to `${HOME}/.cache/zebra` and `${HOME}/.cache/lwd`
 # respectively, but can be overridden by setting the
 # `ZEBRA_CACHE_DIR` and `LWD_CACHE_DIR` environment variables.
 : "${ZEBRA_CACHE_DIR:=${HOME}/.cache/zebra}"
 : "${LWD_CACHE_DIR:=${HOME}/.cache/lwd}"
-
-# Exit early if `ZEBRA_CONF_PATH` does not point to a file.
-if [[ ! -f "${ZEBRA_CONF_PATH}" ]]; then
-  echo "ERROR: No Zebra config file found at ZEBRA_CONF_PATH (${ZEBRA_CONF_PATH})."
-  echo "Please ensure the file exists or mount your custom config file and set ZEBRA_CONF_PATH accordingly."
-  exit 1
-fi
+: "${ZEBRA_COOKIE_DIR:=${HOME}/.config/cookie}"
 
 # Use gosu to drop privileges and execute the given command as the specified UID:GID
 exec_as_user() {
-  exec gosu "${UID}:${GID}" "$@"
+  user=$(id -u)
+  if [[ ${user} == '0' ]]; then
+    exec gosu "${UID}:${GID}" "$@"
+  else
+    exec "$@"
+  fi
 }
 
-# Modifies the existing Zebra config file at ZEBRA_CONF_PATH using environment variables.
+# Modifies the Zebra config file using environment variables.
 #
-# The config options this function supports are also listed in the "docker/.env" file.
+# This function generates a new config file from scratch at ZEBRA_CONF_PATH
+# using the provided environment variables.
 #
-# This function modifies the existing file in-place and prints its location.
+# It creates a complete configuration with network settings, state, RPC,
+# metrics, tracing, and mining sections based on environment variables.
 prepare_conf_file() {
-  # Set a custom network.
-  if [[ -n "${NETWORK}" ]]; then
-    sed -i '/network = ".*"/s/".*"/"'"${NETWORK//\"/}"'"/' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Enable the RPC server by setting its port.
-  if [[ -n "${ZEBRA_RPC_PORT}" ]]; then
-    sed -i '/# listen_addr = "0.0.0.0:18232" # Testnet/d' "${ZEBRA_CONF_PATH}"
-    sed -i 's/ *# Mainnet$//' "${ZEBRA_CONF_PATH}"
-    sed -i '/# listen_addr = "0.0.0.0:8232"/s/^# //; s/8232/'"${ZEBRA_RPC_PORT//\"/}"'/' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Disable or enable cookie authentication.
-  if [[ -n "${ENABLE_COOKIE_AUTH}" ]]; then
-    sed -i '/# enable_cookie_auth = true/s/^# //; s/true/'"${ENABLE_COOKIE_AUTH//\"/}"'/' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Set a custom state, network and cookie cache dirs.
-  #
-  # We're pointing all three cache dirs at the same location, so users will find
-  # all cached data in that single location. We can introduce more env vars and
-  # use them to set the cache dirs separately if needed.
-  if [[ -n "${ZEBRA_CACHE_DIR}" ]]; then
-    mkdir -p "${ZEBRA_CACHE_DIR//\"/}"
-    chown -R "${UID}:${GID}" "${ZEBRA_CACHE_DIR//\"/}"
-    sed -i 's|_dir = ".*"|_dir = "'"${ZEBRA_CACHE_DIR//\"/}"'"|' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Set a custom lightwalletd cache dir.
-  if [[ -n "${LWD_CACHE_DIR}" ]]; then
-    mkdir -p "${LWD_CACHE_DIR//\"/}"
-    chown -R "${UID}:${GID}" "${LWD_CACHE_DIR//\"/}"
-  fi
-
-  # Enable the Prometheus metrics endpoint.
-  if [[ "${FEATURES}" == *"prometheus"* ]]; then
-    sed -i '/# endpoint_addr = "0.0.0.0:9999" # Prometheus/s/^# //' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Enable logging to a file by setting a custom log file path.
-  if [[ -n "${LOG_FILE}" ]]; then
-    mkdir -p "$(dirname "${LOG_FILE//\"/}")"
-    chown -R "${UID}:${GID}" "$(dirname "${LOG_FILE//\"/}")"
-    sed -i 's|# log_file = ".*"|log_file = "'"${LOG_FILE//\"/}"'"|' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Enable or disable colored logs.
-  if [[ -n "${LOG_COLOR}" ]]; then
-    sed -i '/# force_use_color = true/s/^# //' "${ZEBRA_CONF_PATH}"
-    sed -i '/use_color = true/s/true/'"${LOG_COLOR//\"/}"'/' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Enable or disable logging to systemd-journald.
-  if [[ -n "${USE_JOURNALD}" ]]; then
-    sed -i '/# use_journald = true/s/^# //; s/true/'"${USE_JOURNALD//\"/}"'/' "${ZEBRA_CONF_PATH}"
-  fi
+  # Base configuration
+  cat >"${ZEBRA_CONF_PATH}" <<EOF
+[network]
+network = "${NETWORK:=Mainnet}"
+listen_addr = "0.0.0.0"
+cache_dir = "${ZEBRA_CACHE_DIR}"
+
+[state]
+cache_dir = "${ZEBRA_CACHE_DIR}"
+
+$( [[ -n ${ZEBRA_RPC_PORT} ]] && cat <<-SUB_EOF
+
+[rpc]
+listen_addr = "${RPC_LISTEN_ADDR:=0.0.0.0}:${ZEBRA_RPC_PORT}"
+enable_cookie_auth = ${ENABLE_COOKIE_AUTH:=true}
+$( [[ -n ${ZEBRA_COOKIE_DIR} ]] && echo "cookie_dir = \"${ZEBRA_COOKIE_DIR}\"" )
+SUB_EOF
+)
+
+$( [[ " ${FEATURES} " =~ " prometheus " ]] && cat <<-SUB_EOF
+
+[metrics]
+endpoint_addr = "${METRICS_ENDPOINT_ADDR:=0.0.0.0}:${METRICS_ENDPOINT_PORT:=9999}"
+SUB_EOF
+)
+
+$( [[ -n ${LOG_FILE} || -n ${LOG_COLOR} || -n ${TRACING_ENDPOINT_ADDR} || -n ${USE_JOURNALD} ]] && cat <<-SUB_EOF
+
+[tracing]
+$( [[ -n ${USE_JOURNALD} ]] && echo "use_journald = ${USE_JOURNALD}" )
+$( [[ " ${FEATURES} " =~ " filter-reload " ]] && echo "endpoint_addr = \"${TRACING_ENDPOINT_ADDR:=0.0.0.0}:${TRACING_ENDPOINT_PORT:=3000}\"" )
+$( [[ -n ${LOG_FILE} ]] && echo "log_file = \"${LOG_FILE}\"" )
+$( [[ ${LOG_COLOR} == "true" ]] && echo "force_use_color = true" )
+$( [[ ${LOG_COLOR} == "false" ]] && echo "use_color = false" )
+SUB_EOF
+)
+
+$( [[ -n ${MINER_ADDRESS} ]] && cat <<-SUB_EOF
+
+[mining]
+miner_address = "${MINER_ADDRESS}"
+SUB_EOF
+)
+EOF
+}
 
-  # Set a mining address.
-  if [[ -n "${MINER_ADDRESS}" ]]; then
-    sed -i '/# miner_address = ".*"/{s/^# //; s/".*"/"'"${MINER_ADDRESS//\"/}"'"/}' "${ZEBRA_CONF_PATH}"
-  fi
+# Helper function
+exit_error() {
+  echo "$1" >&2
+  exit 1
+}
 
-  # Trim all comments and empty lines.
-  sed -i '/^#/d; /^$/d' "${ZEBRA_CONF_PATH}"
+# Creates a directory if it doesn't exist and sets ownership to specified UID:GID.
+#
+# ## Parameters
+#
+# - $1: Directory path to create and own
+create_owned_directory() {
+  local dir="$1"
+  [[ -z ${dir} ]] && return
 
-  echo "${ZEBRA_CONF_PATH}"
+  mkdir -p "${dir}" || exit_error "Failed to create directory: ${dir}"
+  chown -R "${UID}:${GID}" "${dir}" || exit_error "Failed to secure directory: ${dir}"
path: docker/entrypoint.sh
user: login: upbqdn
body: ###### non-blocking

This will create parent dirs such as `.cache` owned by root:

```bash
root@1cbf7683b54d:~# pwd
/home/zebra
root@1cbf7683b54d:~# ls -lah
total 12K
drwx------ 1 zebra zebra   26 Apr 14 14:10 .
drwxr-xr-x 1 root  root    10 Apr  9 13:03 ..
-rw-r--r-- 1 zebra zebra  220 Apr  9 13:03 .bash_logout
-rw-r--r-- 1 zebra zebra 3.5K Apr  9 13:03 .bashrc
drwxr-xr-x 1 root  root    16 Apr 14 14:10 .cache
drwxr-xr-x 1 root  root    34 Apr 14 14:10 .config
-rw-r--r-- 1 zebra zebra  807 Apr  9 13:03 .profile
```
created_at: 2025-04-14T22:04:10Z
author_association: MEMBER
diff_hunk: @@ -59,19 +59,22 @@ All available Cargo features are listed at
 
 ## Configuring Zebra
 
-To configure Zebra, edit the `docker/default-zebra-config.toml` config file and
-uncomment the `configs` mapping in `docker/docker-compose.yml` so your config
-takes effect. You can see if your config works as intended by looking at Zebra's
-logs.
-
-Alternatively, you can configure Zebra by setting the environment variables in
-the `docker/.env` file. Note that the config options of this file are limited to
-the variables already present in the commented out blocks in it and adding new
-ones will not be effective. Also note that the values of the variables take
-precedence over the values set in the `docker/default-zebra-config.toml` config
-file. The `docker/.env` file serves as a quick way to override the most commonly
-used settings for Zebra, whereas the `docker/default-zebra-config.toml` file
-provides full config capabilities.
+To configure Zebra, you have three options:
+
+1. Edit the `docker/default-zebra-config.toml` config file and uncomment the `configs` mapping in `docker/docker-compose.yml` so your config takes effect.
path: book/src/user/docker.md
user: login: upbqdn
body: This documentation is outdated.
created_at: 2025-04-15T08:53:23Z
author_association: MEMBER
diff_hunk: @@ -59,19 +59,22 @@ All available Cargo features are listed at
 
 ## Configuring Zebra
 
-To configure Zebra, edit the `docker/default-zebra-config.toml` config file and
-uncomment the `configs` mapping in `docker/docker-compose.yml` so your config
-takes effect. You can see if your config works as intended by looking at Zebra's
-logs.
-
-Alternatively, you can configure Zebra by setting the environment variables in
-the `docker/.env` file. Note that the config options of this file are limited to
-the variables already present in the commented out blocks in it and adding new
-ones will not be effective. Also note that the values of the variables take
-precedence over the values set in the `docker/default-zebra-config.toml` config
-file. The `docker/.env` file serves as a quick way to override the most commonly
-used settings for Zebra, whereas the `docker/default-zebra-config.toml` file
-provides full config capabilities.
+To configure Zebra, you have three options:
+
+1. Edit the `docker/default-zebra-config.toml` config file and uncomment the `configs` mapping in `docker/docker-compose.yml` so your config takes effect.
+2. Provide a path to your own config file via the `ZEBRA_CONF_PATH` environment variable.
+3. Let Zebra generate a default config file automatically from environment variables (if neither of the above methods is used).
path: book/src/user/docker.md
user: login: upbqdn
body: Zebra doesn't generate the config. We'd use `zebrad generate` for that (and it wouldn't consider env vars).
created_at: 2025-04-15T09:01:26Z
author_association: MEMBER
diff_hunk: @@ -255,12 +258,39 @@ run_tests() {
 }
 
 # Main Script Logic
+#
+# 1. First check if ZEBRA_CONF_PATH is explicitly set or if a file exists at that path
+# 2. If not set but default config exists, use that
+# 3. If neither exists, generate a default config at ${HOME}/.config/zebrad.toml
+# 4. Print environment variables and config for debugging
+# 5. Process command-line arguments and execute appropriate action
+if [[ -n ${ZEBRA_CONF_PATH} ]]; then
+  if [[ -f ${ZEBRA_CONF_PATH} ]]; then
+    echo "ZEBRA_CONF_PATH was set to ${ZEBRA_CONF_PATH} and a file exists."
+    echo "Using user-provided config file"
+  else
+    echo "ERROR: ZEBRA_CONF_PATH was set and no config file found at ${ZEBRA_CONF_PATH}."
+    echo "Please ensure a config file exists or set ZEBRA_CONF_PATH to point to your config file."
+    exit 1
+  fi
+else
+  if [[ -f "${HOME}/.config/zebrad.toml" ]]; then
+    echo "ZEBRA_CONF_PATH was not set."
+    echo "Using default config at ${HOME}/.config/zebrad.toml"
+    ZEBRA_CONF_PATH="${HOME}/.config/zebrad.toml"
+  else
+    echo "ZEBRA_CONF_PATH was not set and no default config found at ${HOME}/.config/zebrad.toml"
+    echo "Preparing a default one..."
+    ZEBRA_CONF_PATH="${HOME}/.config/zebrad.toml"
+    mkdir -p "$(dirname "${ZEBRA_CONF_PATH}")"
+    prepare_conf_file
path: docker/entrypoint.sh
user: login: gustavovalverde
body: Fixed here: [fix(entrypoint): set permissions correctly for parent directories](https://github.com/ZcashFoundation/zebra/pull/9344/commits/e45c951b15993c83b1c73654779c83aa1292fa31)
created_at: 2025-04-15T15:26:27Z
author_association: MEMBER
diff_hunk: @@ -6,113 +6,116 @@
 #
 # ## Notes
 #
-# - `$ZEBRA_CONF_PATH` must point to a Zebra conf file.
+# - `$ZEBRA_CONF_PATH` can point to an existing Zebra config file, or if not set,
+#   the script will look for a default config at ${HOME}/.config/zebrad.toml,
+#   or generate one using environment variables.
 
 set -eo pipefail
 
-
 # These are the default cached state directories for Zebra and lightwalletd.
 #
 # They are set to `${HOME}/.cache/zebra` and `${HOME}/.cache/lwd`
 # respectively, but can be overridden by setting the
 # `ZEBRA_CACHE_DIR` and `LWD_CACHE_DIR` environment variables.
 : "${ZEBRA_CACHE_DIR:=${HOME}/.cache/zebra}"
 : "${LWD_CACHE_DIR:=${HOME}/.cache/lwd}"
-
-# Exit early if `ZEBRA_CONF_PATH` does not point to a file.
-if [[ ! -f "${ZEBRA_CONF_PATH}" ]]; then
-  echo "ERROR: No Zebra config file found at ZEBRA_CONF_PATH (${ZEBRA_CONF_PATH})."
-  echo "Please ensure the file exists or mount your custom config file and set ZEBRA_CONF_PATH accordingly."
-  exit 1
-fi
+: "${ZEBRA_COOKIE_DIR:=${HOME}/.config/cookie}"
 
 # Use gosu to drop privileges and execute the given command as the specified UID:GID
 exec_as_user() {
-  exec gosu "${UID}:${GID}" "$@"
+  user=$(id -u)
+  if [[ ${user} == '0' ]]; then
+    exec gosu "${UID}:${GID}" "$@"
+  else
+    exec "$@"
+  fi
 }
 
-# Modifies the existing Zebra config file at ZEBRA_CONF_PATH using environment variables.
+# Modifies the Zebra config file using environment variables.
 #
-# The config options this function supports are also listed in the "docker/.env" file.
+# This function generates a new config file from scratch at ZEBRA_CONF_PATH
+# using the provided environment variables.
 #
-# This function modifies the existing file in-place and prints its location.
+# It creates a complete configuration with network settings, state, RPC,
+# metrics, tracing, and mining sections based on environment variables.
 prepare_conf_file() {
-  # Set a custom network.
-  if [[ -n "${NETWORK}" ]]; then
-    sed -i '/network = ".*"/s/".*"/"'"${NETWORK//\"/}"'"/' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Enable the RPC server by setting its port.
-  if [[ -n "${ZEBRA_RPC_PORT}" ]]; then
-    sed -i '/# listen_addr = "0.0.0.0:18232" # Testnet/d' "${ZEBRA_CONF_PATH}"
-    sed -i 's/ *# Mainnet$//' "${ZEBRA_CONF_PATH}"
-    sed -i '/# listen_addr = "0.0.0.0:8232"/s/^# //; s/8232/'"${ZEBRA_RPC_PORT//\"/}"'/' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Disable or enable cookie authentication.
-  if [[ -n "${ENABLE_COOKIE_AUTH}" ]]; then
-    sed -i '/# enable_cookie_auth = true/s/^# //; s/true/'"${ENABLE_COOKIE_AUTH//\"/}"'/' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Set a custom state, network and cookie cache dirs.
-  #
-  # We're pointing all three cache dirs at the same location, so users will find
-  # all cached data in that single location. We can introduce more env vars and
-  # use them to set the cache dirs separately if needed.
-  if [[ -n "${ZEBRA_CACHE_DIR}" ]]; then
-    mkdir -p "${ZEBRA_CACHE_DIR//\"/}"
-    chown -R "${UID}:${GID}" "${ZEBRA_CACHE_DIR//\"/}"
-    sed -i 's|_dir = ".*"|_dir = "'"${ZEBRA_CACHE_DIR//\"/}"'"|' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Set a custom lightwalletd cache dir.
-  if [[ -n "${LWD_CACHE_DIR}" ]]; then
-    mkdir -p "${LWD_CACHE_DIR//\"/}"
-    chown -R "${UID}:${GID}" "${LWD_CACHE_DIR//\"/}"
-  fi
-
-  # Enable the Prometheus metrics endpoint.
-  if [[ "${FEATURES}" == *"prometheus"* ]]; then
-    sed -i '/# endpoint_addr = "0.0.0.0:9999" # Prometheus/s/^# //' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Enable logging to a file by setting a custom log file path.
-  if [[ -n "${LOG_FILE}" ]]; then
-    mkdir -p "$(dirname "${LOG_FILE//\"/}")"
-    chown -R "${UID}:${GID}" "$(dirname "${LOG_FILE//\"/}")"
-    sed -i 's|# log_file = ".*"|log_file = "'"${LOG_FILE//\"/}"'"|' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Enable or disable colored logs.
-  if [[ -n "${LOG_COLOR}" ]]; then
-    sed -i '/# force_use_color = true/s/^# //' "${ZEBRA_CONF_PATH}"
-    sed -i '/use_color = true/s/true/'"${LOG_COLOR//\"/}"'/' "${ZEBRA_CONF_PATH}"
-  fi
-
-  # Enable or disable logging to systemd-journald.
-  if [[ -n "${USE_JOURNALD}" ]]; then
-    sed -i '/# use_journald = true/s/^# //; s/true/'"${USE_JOURNALD//\"/}"'/' "${ZEBRA_CONF_PATH}"
-  fi
+  # Base configuration
+  cat >"${ZEBRA_CONF_PATH}" <<EOF
+[network]
+network = "${NETWORK:=Mainnet}"
+listen_addr = "0.0.0.0"
+cache_dir = "${ZEBRA_CACHE_DIR}"
+
+[state]
+cache_dir = "${ZEBRA_CACHE_DIR}"
+
+$( [[ -n ${ZEBRA_RPC_PORT} ]] && cat <<-SUB_EOF
+
+[rpc]
+listen_addr = "${RPC_LISTEN_ADDR:=0.0.0.0}:${ZEBRA_RPC_PORT}"
+enable_cookie_auth = ${ENABLE_COOKIE_AUTH:=true}
+$( [[ -n ${ZEBRA_COOKIE_DIR} ]] && echo "cookie_dir = \"${ZEBRA_COOKIE_DIR}\"" )
+SUB_EOF
+)
+
+$( [[ " ${FEATURES} " =~ " prometheus " ]] && cat <<-SUB_EOF
+
+[metrics]
+endpoint_addr = "${METRICS_ENDPOINT_ADDR:=0.0.0.0}:${METRICS_ENDPOINT_PORT:=9999}"
+SUB_EOF
+)
+
+$( [[ -n ${LOG_FILE} || -n ${LOG_COLOR} || -n ${TRACING_ENDPOINT_ADDR} || -n ${USE_JOURNALD} ]] && cat <<-SUB_EOF
+
+[tracing]
+$( [[ -n ${USE_JOURNALD} ]] && echo "use_journald = ${USE_JOURNALD}" )
+$( [[ " ${FEATURES} " =~ " filter-reload " ]] && echo "endpoint_addr = \"${TRACING_ENDPOINT_ADDR:=0.0.0.0}:${TRACING_ENDPOINT_PORT:=3000}\"" )
+$( [[ -n ${LOG_FILE} ]] && echo "log_file = \"${LOG_FILE}\"" )
+$( [[ ${LOG_COLOR} == "true" ]] && echo "force_use_color = true" )
+$( [[ ${LOG_COLOR} == "false" ]] && echo "use_color = false" )
+SUB_EOF
+)
+
+$( [[ -n ${MINER_ADDRESS} ]] && cat <<-SUB_EOF
+
+[mining]
+miner_address = "${MINER_ADDRESS}"
+SUB_EOF
+)
+EOF
+}
 
-  # Set a mining address.
-  if [[ -n "${MINER_ADDRESS}" ]]; then
-    sed -i '/# miner_address = ".*"/{s/^# //; s/".*"/"'"${MINER_ADDRESS//\"/}"'"/}' "${ZEBRA_CONF_PATH}"
-  fi
+# Helper function
+exit_error() {
+  echo "$1" >&2
+  exit 1
+}
 
-  # Trim all comments and empty lines.
-  sed -i '/^#/d; /^$/d' "${ZEBRA_CONF_PATH}"
+# Creates a directory if it doesn't exist and sets ownership to specified UID:GID.
+#
+# ## Parameters
+#
+# - $1: Directory path to create and own
+create_owned_directory() {
+  local dir="$1"
+  [[ -z ${dir} ]] && return
 
-  echo "${ZEBRA_CONF_PATH}"
+  mkdir -p "${dir}" || exit_error "Failed to create directory: ${dir}"
+  chown -R "${UID}:${GID}" "${dir}" || exit_error "Failed to secure directory: ${dir}"
path: docker/entrypoint.sh
user: login: gustavovalverde
body: Fixed here: [fix(entrypoint): set permissions correctly for parent directories](https://github.com/ZcashFoundation/zebra/pull/9344/commits/e45c951b15993c83b1c73654779c83aa1292fa31)
created_at: 2025-04-15T15:26:32Z
author_association: MEMBER
diff_hunk: @@ -59,19 +59,22 @@ All available Cargo features are listed at
 
 ## Configuring Zebra
 
-To configure Zebra, edit the `docker/default-zebra-config.toml` config file and
-uncomment the `configs` mapping in `docker/docker-compose.yml` so your config
-takes effect. You can see if your config works as intended by looking at Zebra's
-logs.
-
-Alternatively, you can configure Zebra by setting the environment variables in
-the `docker/.env` file. Note that the config options of this file are limited to
-the variables already present in the commented out blocks in it and adding new
-ones will not be effective. Also note that the values of the variables take
-precedence over the values set in the `docker/default-zebra-config.toml` config
-file. The `docker/.env` file serves as a quick way to override the most commonly
-used settings for Zebra, whereas the `docker/default-zebra-config.toml` file
-provides full config capabilities.
+To configure Zebra, you have three options:
+
+1. Edit the `docker/default-zebra-config.toml` config file and uncomment the `configs` mapping in `docker/docker-compose.yml` so your config takes effect.
path: book/src/user/docker.md
user: login: gustavovalverde
body: Fixed here: [chore: align documentation and implementation](https://github.com/ZcashFoundation/zebra/pull/9344/commits/2520086bd2effaefe7ee2ddf7caa684d3d897904)
created_at: 2025-04-15T15:28:42Z
author_association: MEMBER
diff_hunk: @@ -59,19 +59,22 @@ All available Cargo features are listed at
 
 ## Configuring Zebra
 
-To configure Zebra, edit the `docker/default-zebra-config.toml` config file and
-uncomment the `configs` mapping in `docker/docker-compose.yml` so your config
-takes effect. You can see if your config works as intended by looking at Zebra's
-logs.
-
-Alternatively, you can configure Zebra by setting the environment variables in
-the `docker/.env` file. Note that the config options of this file are limited to
-the variables already present in the commented out blocks in it and adding new
-ones will not be effective. Also note that the values of the variables take
-precedence over the values set in the `docker/default-zebra-config.toml` config
-file. The `docker/.env` file serves as a quick way to override the most commonly
-used settings for Zebra, whereas the `docker/default-zebra-config.toml` file
-provides full config capabilities.
+To configure Zebra, you have three options:
+
+1. Edit the `docker/default-zebra-config.toml` config file and uncomment the `configs` mapping in `docker/docker-compose.yml` so your config takes effect.
+2. Provide a path to your own config file via the `ZEBRA_CONF_PATH` environment variable.
+3. Let Zebra generate a default config file automatically from environment variables (if neither of the above methods is used).
path: book/src/user/docker.md
user: login: gustavovalverde
body: Fixed here: [chore: align documentation and implementation](https://github.com/ZcashFoundation/zebra/pull/9344/commits/2520086bd2effaefe7ee2ddf7caa684d3d897904)
created_at: 2025-04-15T15:28:59Z
author_association: MEMBER
diff_hunk: @@ -59,49 +59,47 @@ All available Cargo features are listed at
 
 ## Configuring Zebra
 
-To configure Zebra, edit the `docker/default-zebra-config.toml` config file and
-uncomment the `configs` mapping in `docker/docker-compose.yml` so your config
-takes effect. You can see if your config works as intended by looking at Zebra's
-logs.
-
-Alternatively, you can configure Zebra by setting the environment variables in
-the `docker/.env` file. Note that the config options of this file are limited to
-the variables already present in the commented out blocks in it and adding new
-ones will not be effective. Also note that the values of the variables take
-precedence over the values set in the `docker/default-zebra-config.toml` config
-file. The `docker/.env` file serves as a quick way to override the most commonly
-used settings for Zebra, whereas the `docker/default-zebra-config.toml` file
-provides full config capabilities.
+To configure Zebra using Docker, you have a few options, processed in this order:
+
+1. **Provide a specific config file path:** Set the `ZEBRA_CONF_PATH` environment variable to point to your config file within the container.
+2. **Use the default config file:** If `ZEBRA_CONF_PATH` is not set, Zebra will look for a config file at `/home/zebra/.config/zebrad.toml`. You can mount your custom config file to this location, for example, by uncommenting the `configs` mapping in `docker/docker-compose.yml`.
path: book/src/user/docker.md
user: login: upbqdn
body: The reason I mentioned the documentation was outdated was that this PR uncomments the `configs` mapping in `docker/docker-compose.yml`, but the documentation was still saying that users should uncomment the mapping. This update says it again.
created_at: 2025-04-15T21:35:17Z
author_association: MEMBER
diff_hunk: @@ -255,12 +258,39 @@ run_tests() {
 }
 
 # Main Script Logic
+#
+# 1. First check if ZEBRA_CONF_PATH is explicitly set or if a file exists at that path
+# 2. If not set but default config exists, use that
+# 3. If neither exists, generate a default config at ${HOME}/.config/zebrad.toml
+# 4. Print environment variables and config for debugging
+# 5. Process command-line arguments and execute appropriate action
+if [[ -n ${ZEBRA_CONF_PATH} ]]; then
+  if [[ -f ${ZEBRA_CONF_PATH} ]]; then
+    echo "ZEBRA_CONF_PATH was set to ${ZEBRA_CONF_PATH} and a file exists."
+    echo "Using user-provided config file"
+  else
+    echo "ERROR: ZEBRA_CONF_PATH was set and no config file found at ${ZEBRA_CONF_PATH}."
+    echo "Please ensure a config file exists or set ZEBRA_CONF_PATH to point to your config file."
+    exit 1
+  fi
+else
+  if [[ -f "${HOME}/.config/zebrad.toml" ]]; then
+    echo "ZEBRA_CONF_PATH was not set."
+    echo "Using default config at ${HOME}/.config/zebrad.toml"
+    ZEBRA_CONF_PATH="${HOME}/.config/zebrad.toml"
+  else
+    echo "ZEBRA_CONF_PATH was not set and no default config found at ${HOME}/.config/zebrad.toml"
+    echo "Preparing a default one..."
+    ZEBRA_CONF_PATH="${HOME}/.config/zebrad.toml"
+    mkdir -p "$(dirname "${ZEBRA_CONF_PATH}")"
+    prepare_conf_file
path: docker/entrypoint.sh
user: login: upbqdn
body: This solution fixes the problem only for immediate parent dirs instead of the full path.

This is not a blocking bug, but it might bite us in the future.
created_at: 2025-04-15T21:49:14Z
author_association: MEMBER
diff_hunk: @@ -255,12 +258,39 @@ run_tests() {
 }
 
 # Main Script Logic
+#
+# 1. First check if ZEBRA_CONF_PATH is explicitly set or if a file exists at that path
+# 2. If not set but default config exists, use that
+# 3. If neither exists, generate a default config at ${HOME}/.config/zebrad.toml
+# 4. Print environment variables and config for debugging
+# 5. Process command-line arguments and execute appropriate action
+if [[ -n ${ZEBRA_CONF_PATH} ]]; then
+  if [[ -f ${ZEBRA_CONF_PATH} ]]; then
+    echo "ZEBRA_CONF_PATH was set to ${ZEBRA_CONF_PATH} and a file exists."
+    echo "Using user-provided config file"
+  else
+    echo "ERROR: ZEBRA_CONF_PATH was set and no config file found at ${ZEBRA_CONF_PATH}."
+    echo "Please ensure a config file exists or set ZEBRA_CONF_PATH to point to your config file."
+    exit 1
+  fi
+else
+  if [[ -f "${HOME}/.config/zebrad.toml" ]]; then
+    echo "ZEBRA_CONF_PATH was not set."
+    echo "Using default config at ${HOME}/.config/zebrad.toml"
+    ZEBRA_CONF_PATH="${HOME}/.config/zebrad.toml"
+  else
+    echo "ZEBRA_CONF_PATH was not set and no default config found at ${HOME}/.config/zebrad.toml"
+    echo "Preparing a default one..."
+    ZEBRA_CONF_PATH="${HOME}/.config/zebrad.toml"
+    mkdir -p "$(dirname "${ZEBRA_CONF_PATH}")"
+    prepare_conf_file
path: docker/entrypoint.sh
user: login: upbqdn
body: Note that the config file is also owned by root.
created_at: 2025-04-15T21:52:42Z
author_association: MEMBER
diff_hunk: @@ -59,49 +59,47 @@ All available Cargo features are listed at
 
 ## Configuring Zebra
 
-To configure Zebra, edit the `docker/default-zebra-config.toml` config file and
-uncomment the `configs` mapping in `docker/docker-compose.yml` so your config
-takes effect. You can see if your config works as intended by looking at Zebra's
-logs.
-
-Alternatively, you can configure Zebra by setting the environment variables in
-the `docker/.env` file. Note that the config options of this file are limited to
-the variables already present in the commented out blocks in it and adding new
-ones will not be effective. Also note that the values of the variables take
-precedence over the values set in the `docker/default-zebra-config.toml` config
-file. The `docker/.env` file serves as a quick way to override the most commonly
-used settings for Zebra, whereas the `docker/default-zebra-config.toml` file
-provides full config capabilities.
+To configure Zebra using Docker, you have a few options, processed in this order:
+
+1. **Provide a specific config file path:** Set the `ZEBRA_CONF_PATH` environment variable to point to your config file within the container.
+2. **Use the default config file:** If `ZEBRA_CONF_PATH` is not set, Zebra will look for a config file at `/home/zebra/.config/zebrad.toml`. You can mount your custom config file to this location, for example, by uncommenting the `configs` mapping in `docker/docker-compose.yml`.
path: book/src/user/docker.md
user: login: gustavovalverde
body: Fixed here: [chore: clearer and aligned comments with docker-compose](https://github.com/ZcashFoundation/zebra/pull/9344/commits/c5610648a150edb74a5b10309df978d69ca23dfb)
created_at: 2025-04-15T21:52:49Z
author_association: MEMBER
diff_hunk: @@ -255,12 +258,39 @@ run_tests() {
 }
 
 # Main Script Logic
+#
+# 1. First check if ZEBRA_CONF_PATH is explicitly set or if a file exists at that path
+# 2. If not set but default config exists, use that
+# 3. If neither exists, generate a default config at ${HOME}/.config/zebrad.toml
+# 4. Print environment variables and config for debugging
+# 5. Process command-line arguments and execute appropriate action
+if [[ -n ${ZEBRA_CONF_PATH} ]]; then
+  if [[ -f ${ZEBRA_CONF_PATH} ]]; then
+    echo "ZEBRA_CONF_PATH was set to ${ZEBRA_CONF_PATH} and a file exists."
+    echo "Using user-provided config file"
+  else
+    echo "ERROR: ZEBRA_CONF_PATH was set and no config file found at ${ZEBRA_CONF_PATH}."
+    echo "Please ensure a config file exists or set ZEBRA_CONF_PATH to point to your config file."
+    exit 1
+  fi
+else
+  if [[ -f "${HOME}/.config/zebrad.toml" ]]; then
+    echo "ZEBRA_CONF_PATH was not set."
+    echo "Using default config at ${HOME}/.config/zebrad.toml"
+    ZEBRA_CONF_PATH="${HOME}/.config/zebrad.toml"
+  else
+    echo "ZEBRA_CONF_PATH was not set and no default config found at ${HOME}/.config/zebrad.toml"
+    echo "Preparing a default one..."
+    ZEBRA_CONF_PATH="${HOME}/.config/zebrad.toml"
+    mkdir -p "$(dirname "${ZEBRA_CONF_PATH}")"
+    prepare_conf_file
path: docker/entrypoint.sh
user: login: gustavovalverde
body: I changed the specific use case when a file is generated by the entrypoint and not mounted, as trying to own a mounted file has caused issues previously: [fix(entrypoint): chown the zebra config file when](https://github.com/ZcashFoundation/zebra/pull/9344/commits/94c5a3e6a5c97fe3bcc0f684bef3a821f7cdfbf7)
created_at: 2025-04-16T06:14:28Z
author_association: MEMBER
diff_hunk: @@ -23,7 +23,7 @@ cache_dir = "/home/zebra/.cache/zebra"
 # listen_addr = "0.0.0.0:8232" # Mainnet
 # listen_addr = "0.0.0.0:18232" # Testnet
 
-cookie_dir = "/home/zebra/.cache/zebra"
+cookie_dir = "/home/zebra/.config/cookie"
path: docker/default-zebra-config.toml
user: login: upbqdn
body: We reverted this change.
created_at: 2025-04-16T09:22:16Z
author_association: MEMBER

PR comments:
user: login: upbqdn
created_at: 2025-04-01T14:46:57Z
body: The issue is similar to what we had before the refactor:

Even setting `ZEBRA_RPC_PORT=catchthisbug` makes Zebra run just fine. It will be hard to debug this if we have something wrong, but things will run anyway.
author_association: MEMBER
user: login: upbqdn
created_at: 2025-04-01T16:49:30Z
body: I wonder if there's a way we could test these things. It should be quick to write a bunch of basic tests if we had a suitable framework for them. 
author_association: MEMBER
user: login: gustavovalverde
created_at: 2025-04-01T16:53:45Z
body: > I wonder if there's a way we could test these things. It should be quick to write a bunch of basic tests if we had a suitable framework for them.

I'll start a PR tomorrow just for that. There's just way too many edge-cases that I can forget and can go wrong with this changes, I'll just start building a test suite and make this PR dependent on that.

I'll start with https://github.com/ZcashFoundation/zebra/issues/9037

I'll put this in draft, and let's revisit once the test suite is good enough
author_association: MEMBER
user: login: gustavovalverde
created_at: 2025-04-08T13:11:08Z
body: I created https://github.com/ZcashFoundation/zebra/pull/9388 to be able to test different scenarios from these changes, and made the fixes here to correctly support them.

This should be ready now @upbqdn 
author_association: MEMBER
user: login: gustavovalverde
created_at: 2025-04-14T09:12:12Z
body: @upbqdn I actually didn't realize I left #9388 targeting this branch. I did it just to be able to rebase this one and test the #9388. I'm planning to revert the merge, and reopen the PR, so we can merge them individually.

This is because we're going to squash the commits, and then the title and PR content will be confusing, and unrelated to most of the changes.
author_association: MEMBER
user: login: gustavovalverde
created_at: 2025-04-14T10:31:04Z
body: @upbqdn This PR is clean now, and ready for review/approval
author_association: MEMBER
user: login: upbqdn
created_at: 2025-04-15T22:02:03Z
body: We should update the cookie path here as well: https://github.com/zcashfoundation/zebra/blob/c5610648a150edb74a5b10309df978d69ca23dfb/docker/default-zebra-config.toml#L26.
author_association: MEMBER
user: login: gustavovalverde
created_at: 2025-04-16T05:43:32Z
body: > We should update the cookie path here as well: [zcashfoundation/zebra@`c561064`/docker/default-zebra-config.toml#L26](https://github.com/zcashfoundation/zebra/blob/c5610648a150edb74a5b10309df978d69ca23dfb/docker/default-zebra-config.toml#L26).

This seems correct: `cookie_dir = "/home/zebra/.cache/zebra"`

All other locations and docs are pointing to this same dir; except for the commented out $ZEBRA_COOKIE_DIR here https://github.com/ZcashFoundation/zebra/pull/9344/files#diff-8f78f3866f02305f1df51579c939174b2c53e8295ad13bd9d2ab763176a792c1R24 which refers to a custom location, as the comment above also implies. 

There are comments referring to the cookie file itself, and those are pointing to `/home/zebra/.cache/zebra/.cookie`
author_association: MEMBER
user: login: upbqdn
created_at: 2025-04-16T10:45:29Z
body: utACK

One simplification I thought of was if we could omit the first config option, and only go with the static location at `/home/zebra/.config/zebrad.toml` we already use in the container. The script would then check if there's a file at that location, and if not, it would generate one. Omitting custom config locations would also simplify the docs quite a bit.
author_association: MEMBER
user: login: gustavovalverde
created_at: 2025-04-16T10:54:57Z
body: > Omitting custom config locations would also simplify the docs quite a bit.

I agree with this, but there are edge-cases we have to consider and we might break those https://github.com/ZcashFoundation/zebra/issues/3432

Unfortunately, this depend on the host and the security policies set on it for docker.
author_association: MEMBER