Project: EDB (App Definition and Development / Database)
File: README.md

CloudNativePG documentation is written in Markdown from the `docs` folder. The `src` folder contains the sources with the `.md` extension. We have adopted as an open source solution to build the documentation starting from Markdown format. Before you submit a pull request for the documentation, you must have gone through the following steps: local test of the documentation run through the spell checker You can locally test the documentation in two ways: using Docker using `mkdocs` directly In both cases, you should issue the commands inside the `docs` folder. With Docker, you just need to execute the following command and point your browser to `http://127.0.0.1:8000/`: ``` bash docker run --rm -v "$(pwd):$(pwd)" -w "$(pwd)" -p 8000:8000 \ minidocks/mkdocs \ mkdocs serve -a 0.0.0.0:8000 ``` If you have installed `mkdocs` directly in your workstation, you can simply run: ``` bash mkdocs serve ``` Even in this case, point your browser to `http://127.0.0.1:8000/`. Make sure you review what you have written by putting yourself in the end user's shoes. Once you are ready, proceed with the spell check and then with the pull request. Every time you work on the documentation, please run from the top directory: ``` bash make spellcheck ``` This will run a spell checker and highlight all the words that need to be either fixed or added to the `.wordlist-en-custom.txt` file. From the `docs` folder, run the following command to build the documentation and place it in the `dist` directory: ``` bash docker run --rm -v "$(pwd):$(pwd)" -w "$(pwd)" \ minidocks/mkdocs \ mkdocs build -v -d dist ``` If you added samples to `docs/src/samples` or modified existing samples, please consider if they should be included in the curated And please help keeping the samples in the curated list, as well as any samples named `cluster-example-*` in runnable condition. These can be a big help for beginners. utils`, `git`, `gpg`, `jq`, `make`, `pandoc`, `sed`, `tar`, `util-linux`, `zlib1g`. The previous list assumes that you are using a Debian-based distribution. For other distributions the name of the packages may vary. To setup a development environment you can use the same instructions discussed for GNU/Linux. Please ensure that the requirements for [kind using WSL2](https://kind.sigs.k8s.io/docs/user/using-wsl2/) are also met. We recommend to use to install the following components in your Mac OS X system: ``` bash brew install go \ kind \ golangci/tap/golangci-lint \ goreleaser ``` Please note that bash v5.0+ is required, this can be installed with: ``` bash brew install bash ``` Note: If `kind` is already installed, make sure you have the latest version: ``` bash brew upgrade kind ``` You can now proceed with installing the remaining required packages: ``` bash brew install jq \ make \ coreutils \ diffutils \ findutils \ git \ gpg \ gnu-getopt \ gnu-sed \ gnu-tar \ pandoc \ zlib ``` You can then follow the provided instructions. As you'll see, you need to add the following lines to the profile of your shell (eg `~/.bash_profile` or `~/.zprofile`): Warning: for Apple Silicon macs, the new default location for homebrew is `/opt/homebrew`, not `/usr/local`. See . ``` bash export GOPATH="${HOME}/go" HOMEBREW_PREFIX=$(brew --prefix) export PATH="${HOMEBREW_PREFIX}/opt/gettext/bin:$PATH" export PATH="${HOMEBREW_PREFIX}/opt/coreutils/libexec/gnubin:$PATH" export PATH="${HOMEBREW_PREFIX}/opt/findutils/libexec/gnubin:$PATH" export PATH="${HOMEBREW_PREFIX}/opt/gnu-getopt/bin:$PATH" export PATH="${HOMEBREW_PREFIX}/opt/gnu-sed/libexec/gnubin:$PATH" export PATH="${HOMEBREW_PREFIX}/opt/gnu-tar/libexec/gnubin:$PATH" export MANPATH="${HOMEBREW_PREFIX}/opt/coreutils/libexec/gnuman:$MANPATH" export MANPATH="${HOMEBREW_PREFIX}/opt/findutils/libexec/gnuman:$MANPATH" export MANPATH="${HOMEBREW_PREFIX}/opt/gnu-getopt/share/man:$MANPATH" export MANPATH="${HOMEBREW_PREFIX}/opt/gnu-sed/libexec/gnuman:$MANPATH" export MANPATH="${HOMEBREW_PREFIX}/opt/gnu-tar/libexec/gnuman:$MANPATH" export LDFLAGS="-L${HOMEBREW_PREFIX}/opt/zlib/lib $LDFLAGS" export LDFLAGS="-L${HOMEBREW_PREFIX}/opt/gettext/lib $LDFLAGS" export LDFLAGS="-L${HOMEBREW_PREFIX}/opt/readline/lib $LDFLAGS" export CPPFLAGS="-I${HOMEBREW_PREFIX}/opt/zlib/include $CPPFLAGS" export CPPFLAGS="-I${HOMEBREW_PREFIX}/opt/gettext/include $CPPFLAGS" export CPPFLAGS="-I${HOMEBREW_PREFIX}/opt/readline/include $CPPFLAGS" export PKGCONFIGPATH="${HOMEBREW_PREFIX}/opt/readline/lib/pkgconfig" unset HOMEBREW_PREFIX export

export GPG_TTY=$(tty) ``` If you are using Docker Desktop, please make sure you follow the instructions in the . MacOS Airplay uses port 5000 and can interfere with docker repository port. Make sure Airplay is off (System Preferences -> Share -> uncheck the Airplay checkbox). Unless you are a developer with write permissions on the main repository of , in order to propose a pull request for CloudNativePG you need to fork the repo. Please follow the , making sure you use the instead of `octocat/Spoon-Knife` - which is used in the example. Then clone it on your local environment and proceed with the next step. These instructions will guide you through building the operator from a local cloned repository, normally a fork of CloudNativePG, and deploying it in your local `kind` cluster for evaluation and testing. Enter your local clone directory and select `main` as the working branch to build and deploy: ```shell cd cloudnative-pg git checkout main make deploy-locally ``` This will build the operator based on the `main` branch content, create a `kind` cluster in your workstation with a container registry that provides the operator image that you just built. NOTE: In case of errors, make sure that you have the latest versions of the Go binaries in your system. For this reason, from time to time, we recommend you running: `make distclean`. If everything went well, you are now able to use this version of the operator in the local `kind` cluster. For example, you should be able to see the CloudNativePG operator installed with: ```shell kubectl get deploy -n cnpg-system cnpg-controller-manager ``` Now that your system has been validated, you can tear down the local cluster with: ```shell make kind-cluster-destroy ``` Congratulations, you have a suitable development environment. You are now able to contribute your patches to CloudNativePG! ter.sh` and `hack/e2e/run-e2e.sh` to create a local Kubernetes cluster and then run E2E tests on it. There is also a script to run E2E tests on an existing `local` Kubernetes cluster. It tries to detect the appropriate defaults for storage class and volume snapshot class environment variables by looking at the annotation of the default storage class and the volume snapshot class. All the current test cases are labeled with features. Which can be selected by exporting value `FEATURE_TYPE` and running any script. By default, if test level is not exported, it will select all medium test cases from the feature type provided. | Currently Available Feature Types | |--| | `disruptive` | | `performance` | | `upgrade` | | `smoke` | | `basic` | | `service-connectivity` | | `self-healing` | | `backup-restore` | | `snapshot` | | `operator` | | `observability` | | `replication` | | `plugin` | | `postgres-configuration` | | `pod-scheduling` | | `cluster-metadata` | | `recovery` | | `importing-databases` | | `storage` | | `security` | | `maintenance` | | `prometheus` | ex: ```shell export FEATURE_TYPE=smoke,basic,service-connectivity ``` This will run smoke, basic and service connectivity e2e. One or many can be passed as value with comma separation without spaces. You can test the operator locally on `kind` with: ```shell run-e2e-kind.sh ``` It will take care of creating a `kind` cluster and run the tests on it. We have also provided a shortcut to this script in the main `Makefile`: ```shell make e2e-test-kind ``` You can test the operator locally on `k3d` with: ``` bash run-e2e-k3d.sh ``` NOTE: error messages, like the example below, that will be shown during cluster creation are NOT an issue: ``` Error response from daemon: manifest for

not found: manifest unknown: manifest unknown ``` The script will take care of creating a K3d cluster and then run the tests on it. We have also provided a shortcut to this script in the main `Makefile`: ```shell make e2e-test-k3d ``` You can test the operator locally on `local` with: ``` bash run-e2e-local.sh ``` The script will try detecting the storage class and volume snapshot class to use by looking at the following annotations and environment variables: `storageclass.kubernetes.io/is-default-class: "true"` for the default storage class to use `E2ECSISTORAGE_CLASS` variable for the default CSI storage class to use. The default is `csi-hostpath-sc` `storage.kubernetes.io/default-snapshot-class: "$SNAPSHOTCLASSNAME"` for the default volume snapshot class to use with the storage class provided in the `E2ECSISTORAGE_CLASS` environment variable. The clusters created by setup-cluster.sh script will have the correct storage class and volume snapshot class detected automatically. The script will then run the tests on the existing cluster. We have also provided a shortcut to this script in the main `Makefile`: ```shell make e2e-test-local ``` In addition to the environment variables for the script, the following ones can be defined: `PRESERVE_CLUSTER`: true to prevent the script from destroying the Kubernetes cluster. Default: `false` `PRESERVE_NAMESPACES`: space separated list of namespace to be kept after the tests. Only useful if specified with `PRESERVE_CLUSTER=true` `K8S_VERSION`: the version of K8s to run. Default: `v1.26.0` `KIND_VERSION`: the version of `kind`. Defaults to the latest release `BUILD_IMAGE`: true to build the Dockerfile and load it on kind, false to get the image from a registry. Default: `false` `LOG_DIR`: the directory where the container logs are exported. Default: `_logs/` directory in the project root `run-e2e-kind.sh` forces `E2EDEFAULTSTORAGE_CLASS=standard` while `run-e2e-k3d.sh` forces `E2EDEFAULTSTORAGE_CLASS=local-path` Both scripts use the `setup-cluster.sh` script, in order to initialize the cluster choosing between `kind` or K3d engine. Additionally, if you fork the repository and want to run the tests on your fork, you can do so with the `/test` command. `/test` is used to trigger a run of the end-to-end tests in the GitHub Actions. Only users who have `write` permission to the repository can use this command. The format of `/test` `/test options` Options supported are: test_level (level or tl for short) The priority of test to run, If test level is set, all tests with that priority and higher will be triggered. Default value for the test level is 4. Available values are: 0: highest 1: high 2: medium 3: low 4: lowest depth (d for short) The type of test to run. Default value is `main`, available values are: push main pull_request schedule feature_type (type or ft for short) The label for the end-to-end test to be run, empty value means all labels. Default value is empty, available type are: log_level (ll for short) Defines the log value for cloudNativePG operator, which will be specified as the value for `--log-value` in the operator command. Default value is info and available values are: `error, warning, info, debug, trace` build_plugin (plugin or bp for short) Whether to run the build cnpg plugin job in the end-to-end test. Available values are true and false and default value is false. Example: Trigger an e2e test to run all test cases with `highest` test level, we want to cover most Kubernetes and Postgres metrics ``` /test -tl=0 d=schedule ``` Run smoke and upgrade test for the pull request ``` /test type=smoke,upgrade ``` In order to enable testing of Kubernetes volume snapshots on a local Kind Cluster, we are installing the `csi-hostpath-sc` storage class and the `csi-hostpath-snapclass` volume snapshot class.