Chapter 2. Configuring external PostgreSQL databases As an administrator, you can configure and use external PostgreSQL databases in Red Hat Developer Hub. You can use a PostgreSQL certificate file to configure an external PostgreSQL instance using the Operator or Helm Chart. Note Developer Hub supports the configuration of external PostgreSQL databases. You can perform maintenance activities, such as backing up your data or configuring high availability (HA) for the external PostgreSQL databases. By default, the Red Hat Developer Hub operator or Helm Chart creates a local PostgreSQL database. However, this configuration is not suitable for the production environments. For production deployments, disable the creation of local database and configure Developer Hub to connect to an external PostgreSQL instance instead. 2.1. Configuring an external PostgreSQL instance using the Operator You can configure an external PostgreSQL instance using the Red Hat Developer Hub Operator. By default, the Operator creates and manages a local instance of PostgreSQL in the same namespace where you have deployed the RHDH instance. However, you can change this default setting to configure an external PostgreSQL database server, for example, Amazon Web Services (AWS) Relational Database Service (RDS) or Azure database. Prerequisites You are using a supported version of PostgreSQL. For more information, see the Product life cycle page . You have the following details: db-host : Denotes your PostgreSQL instance Domain Name System (DNS) or IP address db-port : Denotes your PostgreSQL instance port number, such as 5432 username : Denotes the user name to connect to your PostgreSQL instance password : Denotes the password to connect to your PostgreSQL instance You have installed the Red Hat Developer Hub Operator. Optional: You have a CA certificate, Transport Layer Security (TLS) private key, and TLS certificate so that you can secure your database connection by using the TLS protocol. For more information, refer to your PostgreSQL vendor documentation. Note By default, Developer Hub uses a database for each plugin and automatically creates it if none is found. You might need the Create Database privilege in addition to PSQL Database privileges for configuring an external PostgreSQL instance. Procedure Optional: Create a certificate secret to configure your PostgreSQL instance with a TLS connection: cat <<EOF | oc -n my-rhdh-project create -f - apiVersion: v1 kind: Secret metadata: name: my-rhdh-database-certificates-secrets 1 type: Opaque stringData: postgres-ca.pem: |- -----BEGIN CERTIFICATE----- <ca-certificate-key> 2 postgres-key.key: |- -----BEGIN CERTIFICATE----- <tls-private-key> 3 postgres-crt.pem: |- -----BEGIN CERTIFICATE----- <tls-certificate-key> 4 # ... EOF 1 Provide the name of the certificate secret. 2 Provide the CA certificate key. 3 Optional: Provide the TLS private key. 4 Optional: Provide the TLS certificate key. Create a credential secret to connect with the PostgreSQL instance: cat <<EOF | oc -n my-rhdh-project create -f - apiVersion: v1 kind: Secret metadata: name: my-rhdh-database-secrets 1 type: Opaque stringData: 2 POSTGRES_PASSWORD: <password> POSTGRES_PORT: "<db-port>" POSTGRES_USER: <username> POSTGRES_HOST: <db-host> PGSSLMODE: <ssl-mode> # for TLS connection 3 NODE_EXTRA_CA_CERTS: <abs-path-to-pem-file> # for TLS connection, e.g. /opt/app-root/src/postgres-crt.pem 4 EOF 1 Provide the name of the credential secret. 2 Provide credential data to connect with your PostgreSQL instance. 3 Optional: Provide the value based on the required Secure Sockets Layer (SSL) mode . 4 Optional: Provide the value only if you need a TLS connection for your PostgreSQL instance. Create your Backstage custom resource (CR): cat <<EOF | oc -n my-rhdh-project create -f - apiVersion: rhdh.redhat.com/v1alpha3 kind: Backstage metadata: name: <backstage-instance-name> spec: database: enableLocalDb: false 1 application: extraFiles: mountPath: <path> # e g /opt/app-root/src secrets: - name: my-rhdh-database-certificates-secrets 2 key: postgres-crt.pem, postgres-ca.pem, postgres-key.key # key name as in my-rhdh-database-certificates-secrets Secret extraEnvs: secrets: - name: my-rhdh-database-secrets 3 # ... 1 Set the value of the enableLocalDb parameter to false to disable creating local PostgreSQL instances. 2 Provide the name of the certificate secret if you have configured a TLS connection. 3 Provide the name of the credential secret that you created. Note The environment variables listed in the Backstage CR work with the Operator default configuration. If you have changed the Operator default configuration, you must reconfigure the Backstage CR accordingly. Apply the Backstage CR to the namespace where you have deployed the Developer Hub instance. 2.2. Configuring an external PostgreSQL instance using the Helm Chart You can configure an external PostgreSQL instance by using the Helm Chart. By default, the Helm Chart creates and manages a local instance of PostgreSQL in the same namespace where you have deployed the RHDH instance. However, you can change this default setting to configure an external PostgreSQL database server, for example, Amazon Web Services (AWS) Relational Database Service (RDS) or Azure database. Prerequisites You are using a supported version of PostgreSQL. For more information, see the Product life cycle page . You have the following details: db-host : Denotes your PostgreSQL instance Domain Name System (DNS) or IP address db-port : Denotes your PostgreSQL instance port number, such as 5432 username : Denotes the user name to connect to your PostgreSQL instance password : Denotes the password to connect to your PostgreSQL instance You have installed the RHDH application by using the Helm Chart. Optional: You have a CA certificate, Transport Layer Security (TLS) private key, and TLS certificate so that you can secure your database connection by using the TLS protocol. For more information, refer to your PostgreSQL vendor documentation. Note By default, Developer Hub uses a database for each plugin and automatically creates it if none is found. You might need the Create Database privilege in addition to PSQL Database privileges for configuring an external PostgreSQL instance. Procedure Optional: Create a certificate secret to configure your PostgreSQL instance with a TLS connection: cat <<EOF | oc -n <your-namespace> create -f - apiVersion: v1 kind: Secret metadata: name: my-rhdh-database-certificates-secrets 1 type: Opaque stringData: postgres-ca.pem: |- -----BEGIN CERTIFICATE----- <ca-certificate-key> 2 postgres-key.key: |- -----BEGIN CERTIFICATE----- <tls-private-key> 3 postgres-crt.pem: |- -----BEGIN CERTIFICATE----- <tls-certificate-key> 4 # ... EOF 1 Provide the name of the certificate secret. 2 Provide the CA certificate key. 3 Optional: Provide the TLS private key. 4 Optional: Provide the TLS certificate key. Create a credential secret to connect with the PostgreSQL instance: cat <<EOF | oc -n <your-namespace> create -f - apiVersion: v1 kind: Secret metadata: name: my-rhdh-database-secrets 1 type: Opaque stringData: 2 POSTGRES_PASSWORD: <password> POSTGRES_PORT: "<db-port>" POSTGRES_USER: <username> POSTGRES_HOST: <db-host> PGSSLMODE: <ssl-mode> # for TLS connection 3 NODE_EXTRA_CA_CERTS: <abs-path-to-pem-file> # for TLS connection, e.g. /opt/app-root/src/postgres-crt.pem 4 EOF 1 Provide the name of the credential secret. 2 Provide credential data to connect with your PostgreSQL instance. 3 Optional: Provide the value based on the required Secure Sockets Layer (SSL) mode . 4 Optional: Provide the value only if you need a TLS connection for your PostgreSQL instance. Configure your PostgreSQL instance in the Helm configuration file named values.yaml : # ... upstream: postgresql: enabled: false # disable PostgreSQL instance creation 1 auth: existingSecret: my-rhdh-database-secrets # inject credentials secret to Backstage 2 backstage: appConfig: backend: database: connection: # configure Backstage DB connection parameters host: USD{POSTGRES_HOST} port: USD{POSTGRES_PORT} user: USD{POSTGRES_USER} password: USD{POSTGRES_PASSWORD} ssl: rejectUnauthorized: true, ca: USDfile: /opt/app-root/src/postgres-ca.pem key: USDfile: /opt/app-root/src/postgres-key.key cert: USDfile: /opt/app-root/src/postgres-crt.pem extraEnvVarsSecrets: - my-rhdh-database-secrets # inject credentials secret to Backstage 3 extraEnvVars: - name: BACKEND_SECRET valueFrom: secretKeyRef: key: backend-secret name: '{{ include "janus-idp.backend-secret-name" USD }}' extraVolumeMounts: - mountPath: /opt/app-root/src/dynamic-plugins-root name: dynamic-plugins-root - mountPath: /opt/app-root/src/postgres-crt.pem name: postgres-crt # inject TLS certificate to Backstage cont. 4 subPath: postgres-crt.pem - mountPath: /opt/app-root/src/postgres-ca.pem name: postgres-ca # inject CA certificate to Backstage cont. 5 subPath: postgres-ca.pem - mountPath: /opt/app-root/src/postgres-key.key name: postgres-key # inject TLS private key to Backstage cont. 6 subPath: postgres-key.key extraVolumes: - ephemeral: volumeClaimTemplate: spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi name: dynamic-plugins-root - configMap: defaultMode: 420 name: dynamic-plugins optional: true name: dynamic-plugins - name: dynamic-plugins-npmrc secret: defaultMode: 420 optional: true secretName: '{{ printf "%s-dynamic-plugins-npmrc" .Release.Name }}' - name: postgres-crt secret: secretName: my-rhdh-database-certificates-secrets 7 # ... 1 Set the value of the upstream.postgresql.enabled parameter to false to disable creating local PostgreSQL instances. 2 Provide the name of the credential secret. 3 Provide the name of the credential secret. 4 Optional: Provide the name of the TLS certificate only for a TLS connection. 5 Optional: Provide the name of the CA certificate only for a TLS connection. 6 Optional: Provide the name of the TLS private key only if your TLS connection requires a private key. 7 Provide the name of the certificate secret if you have configured a TLS connection. Apply the configuration changes in your Helm configuration file named values.yaml : helm upgrade -n <your-namespace> <your-deploy-name> openshift-helm-charts/redhat-developer-hub -f values.yaml --version 1.4.2 2.3. Migrating local databases to an external database server using the Operator By default, Red Hat Developer Hub hosts the data for each plugin in a PostgreSQL database. When you fetch the list of databases, you might see multiple databases based on the number of plugins configured in Developer Hub. You can migrate the data from an RHDH instance hosted on a local PostgreSQL server to an external PostgreSQL service, such as AWS RDS, Azure database, or Crunchy database. To migrate the data from each RHDH instance, you can use PostgreSQL utilities, such as pg_dump with psql or pgAdmin . Note The following procedure uses a database copy script to do a quick migration. Prerequisites You have installed the pg_dump and psql utilities on your local machine. For data export, you have the PGSQL user privileges to make a full dump of local databases. For data import, you have the PGSQL admin privileges to create an external database and populate it with database dumps. Procedure Configure port forwarding for the local PostgreSQL database pod by running the following command on a terminal: oc port-forward -n <your-namespace> <pgsql-pod-name> <forward-to-port>:<forward-from-port> Where: The <pgsql-pod-name> variable denotes the name of a PostgreSQL pod with the format backstage-psql-<deployment-name>-<_index> . The <forward-to-port> variable denotes the port of your choice to forward PostgreSQL data to. The <forward-from-port> variable denotes the local PostgreSQL instance port, such as 5432 . Example: Configuring port forwarding oc port-forward -n developer-hub backstage-psql-developer-hub-0 15432:5432 Make a copy of the following db_copy.sh script and edit the details based on your configuration: #!/bin/bash to_host=<db-service-host> 1 to_port=5432 2 to_user=postgres 3 from_host=127.0.0.1 4 from_port=15432 5 from_user=postgres 6 allDB=("backstage_plugin_app" "backstage_plugin_auth" "backstage_plugin_catalog" "backstage_plugin_permission" "backstage_plugin_scaffolder" "backstage_plugin_search") 7 for db in USD{!allDB[@]}; do db=USD{allDB[USDdb]} echo Copying database: USDdb PGPASSWORD=USDTO_PSW psql -h USDto_host -p USDto_port -U USDto_user -c "create database USDdb;" pg_dump -h USDfrom_host -p USDfrom_port -U USDfrom_user -d USDdb | PGPASSWORD=USDTO_PSW psql -h USDto_host -p USDto_port -U USDto_user -d USDdb done 1 The destination host name, for example, <db-instance-name>.rds.amazonaws.com . 2 The destination port, such as 5432 . 3 The destination server username, for example, postgres . 4 The source host name, such as 127.0.0.1 . 5 The source port number, such as the <forward-to-port> variable. 6 The source server username, for example, postgres . 7 The name of databases to import in double quotes separated by spaces, for example, ("backstage_plugin_app" "backstage_plugin_auth" "backstage_plugin_catalog" "backstage_plugin_permission" "backstage_plugin_scaffolder" "backstage_plugin_search") . Create a destination database for copying the data: /bin/bash TO_PSW=<destination-db-password> /path/to/db_copy.sh 1 1 The <destination-db-password> variable denotes the password to connect to the destination database. Note You can stop port forwarding when the copying of the data is complete. For more information about handling large databases and using the compression tools, see the Handling Large Databases section on the PostgreSQL website. Reconfigure your Backstage custom resource (CR). For more information, see Configuring an external PostgreSQL instance using the Operator . Check that the following code is present at the end of your Backstage CR after reconfiguration: # ... spec: database: enableLocalDb: false application: # ... extraFiles: secrets: - name: my-rhdh-database-certificates-secrets key: postgres-crt.pem # key name as in my-rhdh-database-certificates-secrets Secret extraEnvs: secrets: - name: my-rhdh-database-secrets # ... Note Reconfiguring the Backstage CR deletes the corresponding StatefulSet and Pod objects, but does not delete the PersistenceVolumeClaim object. Use the following command to delete the local PersistenceVolumeClaim object: oc -n developer-hub delete pvc <local-psql-pvc-name> where, the <local-psql-pvc-name> variable is in the data-<psql-pod-name> format. Apply the configuration changes. Verification Verify that your RHDH instance is running with the migrated data and does not contain the local PostgreSQL database by running the following command: oc get pods -n <your-namespace> Check the output for the following details: The backstage-developer-hub-xxx pod is in running state. The backstage-psql-developer-hub-0 pod is not available. You can also verify these details using the Topology view in the OpenShift Container Platform web console.

Commands:
cat <<EOF | oc -n my-rhdh-project create -f - apiVersion: v1 kind: Secret metadata: name: my-rhdh-database-certificates-secrets 1 type: Opaque stringData: postgres-ca.pem: |- -----BEGIN CERTIFICATE----- <ca-certificate-key> 2 postgres-key.key: |- -----BEGIN CERTIFICATE----- <tls-private-key> 3 postgres-crt.pem: |- -----BEGIN CERTIFICATE----- <tls-certificate-key> 4 # EOF
cat <<EOF | oc -n my-rhdh-project create -f - apiVersion: v1 kind: Secret metadata: name: my-rhdh-database-secrets 1 type: Opaque stringData: 2 POSTGRES_PASSWORD: <password> POSTGRES_PORT: "<db-port>" POSTGRES_USER: <username> POSTGRES_HOST: <db-host> PGSSLMODE: <ssl-mode> # for TLS connection 3 NODE_EXTRA_CA_CERTS: <abs-path-to-pem-file> # for TLS connection, e.g. /opt/app-root/src/postgres-crt.pem 4 EOF
cat <<EOF | oc -n my-rhdh-project create -f - apiVersion: rhdh.redhat.com/v1alpha3 kind: Backstage metadata: name: <backstage-instance-name> spec: database: enableLocalDb: false 1 application: extraFiles: mountPath: <path> # e g /opt/app-root/src secrets: - name: my-rhdh-database-certificates-secrets 2 key: postgres-crt.pem, postgres-ca.pem, postgres-key.key # key name as in my-rhdh-database-certificates-secrets Secret extraEnvs: secrets: - name: my-rhdh-database-secrets 3 #
cat <<EOF | oc -n <your-namespace> create -f - apiVersion: v1 kind: Secret metadata: name: my-rhdh-database-certificates-secrets 1 type: Opaque stringData: postgres-ca.pem: |- -----BEGIN CERTIFICATE----- <ca-certificate-key> 2 postgres-key.key: |- -----BEGIN CERTIFICATE----- <tls-private-key> 3 postgres-crt.pem: |- -----BEGIN CERTIFICATE----- <tls-certificate-key> 4 # EOF
cat <<EOF | oc -n <your-namespace> create -f - apiVersion: v1 kind: Secret metadata: name: my-rhdh-database-secrets 1 type: Opaque stringData: 2 POSTGRES_PASSWORD: <password> POSTGRES_PORT: "<db-port>" POSTGRES_USER: <username> POSTGRES_HOST: <db-host> PGSSLMODE: <ssl-mode> # for TLS connection 3 NODE_EXTRA_CA_CERTS: <abs-path-to-pem-file> # for TLS connection, e.g. /opt/app-root/src/postgres-crt.pem 4 EOF
upstream: postgresql: enabled: false # disable PostgreSQL instance creation 1 auth: existingSecret: my-rhdh-database-secrets # inject credentials secret to Backstage 2 backstage: appConfig: backend: database: connection: # configure Backstage DB connection parameters host: USD{POSTGRES_HOST} port: USD{POSTGRES_PORT} user: USD{POSTGRES_USER} password: USD{POSTGRES_PASSWORD} ssl: rejectUnauthorized: true, ca: USDfile: /opt/app-root/src/postgres-ca.pem key: USDfile: /opt/app-root/src/postgres-key.key cert: USDfile: /opt/app-root/src/postgres-crt.pem extraEnvVarsSecrets: - my-rhdh-database-secrets # inject credentials secret to Backstage 3 extraEnvVars: - name: BACKEND_SECRET valueFrom: secretKeyRef: key: backend-secret name: '{{ include "janus-idp.backend-secret-name" USD }}' extraVolumeMounts: - mountPath: /opt/app-root/src/dynamic-plugins-root name: dynamic-plugins-root - mountPath: /opt/app-root/src/postgres-crt.pem name: postgres-crt # inject TLS certificate to Backstage cont. 4 subPath: postgres-crt.pem - mountPath: /opt/app-root/src/postgres-ca.pem name: postgres-ca # inject CA certificate to Backstage cont. 5 subPath: postgres-ca.pem - mountPath: /opt/app-root/src/postgres-key.key name: postgres-key # inject TLS private key to Backstage cont. 6 subPath: postgres-key.key extraVolumes: - ephemeral: volumeClaimTemplate: spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi name: dynamic-plugins-root - configMap: defaultMode: 420 name: dynamic-plugins optional: true name: dynamic-plugins - name: dynamic-plugins-npmrc secret: defaultMode: 420 optional: true secretName: '{{ printf "%s-dynamic-plugins-npmrc" .Release.Name }}' - name: postgres-crt secret: secretName: my-rhdh-database-certificates-secrets 7 #
helm upgrade -n <your-namespace> <your-deploy-name> openshift-helm-charts/redhat-developer-hub -f values.yaml --version 1.4.2
port-forward -n <your-namespace> <pgsql-pod-name> <forward-to-port>:<forward-from-port>
port-forward -n developer-hub backstage-psql-developer-hub-0 15432:5432
#!/bin/bash to_host=<db-service-host> 1 to_port=5432 2 to_user=postgres 3 from_host=127.0.0.1 4 from_port=15432 5 from_user=postgres 6 allDB=("backstage_plugin_app" "backstage_plugin_auth" "backstage_plugin_catalog" "backstage_plugin_permission" "backstage_plugin_scaffolder" "backstage_plugin_search") 7 for db in USD{!allDB[@]}; do db=USD{allDB[USDdb]} echo Copying database: USDdb PGPASSWORD=USDTO_PSW psql -h USDto_host -p USDto_port -U USDto_user -c "create database USDdb;" pg_dump -h USDfrom_host -p USDfrom_port -U USDfrom_user -d USDdb | PGPASSWORD=USDTO_PSW psql -h USDto_host -p USDto_port -U USDto_user -d USDdb done
/bin/bash TO_PSW=<destination-db-password> /path/to/db_copy.sh 1
spec: database: enableLocalDb: false application: # extraFiles: secrets: - name: my-rhdh-database-certificates-secrets key: postgres-crt.pem # key name as in my-rhdh-database-certificates-secrets Secret extraEnvs: secrets: - name: my-rhdh-database-secrets
-n developer-hub delete pvc <local-psql-pvc-name>
get pods -n <your-namespace>