Project: DolphinScheduler (Orchestration & Management / Scheduling & Orchestration)
File: datasource-setting.md

We here use MySQL as an example to illustrate how to configure an external database: NOTE: If you use MySQL, you need to manually download (8.0.16) and move it to the libs directory of DolphinScheduler which is `api-server/libs` and `alert-server/libs` and `master-server/libs` and `worker-server/libs`. First of all, follow the instructions in `Pseudo-Cluster/Cluster Initialize the Database` section to create and initialize database Set the following environment variables in your terminal with your database address, username and password for `{address}`, `{user}` and `{password}`: ```shell export DATABASE=mysql export SPRINGPROFILESACTIVE=${DATABASE} export SPRINGDATASOURCEURL="jdbc:mysql://{address}/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false" export SPRINGDATASOURCEUSERNAME={user} export SPRINGDATASOURCEPASSWORD={password} ``` Add mysql-connector-java driver to `./standalone-server/libs/standalone-server/`, see `Pseudo-Cluster/Cluster Initialize the Database` section about where to download Start standalone-server, now you are using mysql as database and it will not clear up your data when you stop or restart standalone-server. DolphinScheduler stores metadata in `relational database`. Currently, we support `PostgreSQL` and `MySQL`. Let's walk through how to initialize the database in `MySQL` and `PostgreSQL` : If you use MySQL, you need to manually download (8.0.16) and move it to the libs directory of DolphinScheduler which is `api-server/libs` and `alert-server/libs` and `master-server/libs` and `worker-server/libs` and `tools/libs`. For mysql 5.6 / 5.7 ```shell mysql -uroot -p mysql> CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8generalci; mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'%' IDENTIFIED BY '{password}'; mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'localhost' IDENTIFIED BY '{password}'; mysql> flush privileges; ``` For mysql 8: ```shell mysql -uroot -p mysql> CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8generalci; mysql> CREATE USER '{user}'@'%' IDENTIFIED BY '{password}'; mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'%'; mysql> CREATE USER '{user}'@'localhost' IDENTIFIED BY '{password}'; mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'localhost'; mysql> FLUSH PRIVILEGES; ``` For PostgreSQL: ```shell psql postgres=# CREATE DATABASE dolphinscheduler; postgres=# CREATE USER {user} PASSWORD

postgres=# ALTER DATABASE dolphinscheduler OWNER TO {user}; postgres=#\q echo "host dolphinscheduler {user} {ip} md5" >> $PGDATA/pg_hba.conf pg_ctl reload ``` Then, set the database configurations by exporting the following environment variables, change {user} and {password} to what you set in the previous step. For MySQL: ```shell export DATABASE=${DATABASE:-mysql} export SPRINGPROFILESACTIVE=${DATABASE} export SPRINGDATASOURCEURL="jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&useSSL=false" export SPRINGDATASOURCEUSERNAME={user} export SPRINGDATASOURCEPASSWORD={password} ``` For PostgreSQL: ```shell export DATABASE=${DATABASE:-postgresql} export SPRINGPROFILESACTIVE=${DATABASE} export SPRINGDATASOURCEURL="jdbc:postgresql://127.0.0.1:5432/dolphinscheduler" export SPRINGDATASOURCEUSERNAME={user} export SPRINGDATASOURCEPASSWORD={password} ``` After the above steps done you would create a new database for DolphinScheduler, then run the Shell script to init database: ```shell bash tools/bin/upgrade-schema.sh ``` DataSource supports MySQL, PostgreSQL, Hive/Impala, Spark, ClickHouse, Oracle, SQL Server and other DataSources. Click bottom `Data Source Center -> Create Data Source` to create a new datasource. Click `Test Connection` to test whether the DataSource can connect successfully (datasource can be saved only if it passes the connection test). Some of datasource are native supported to DolphinScheduler while others need users download JDBC driver package manually, because those JDBC driver incompatible to Apache LICENSE V2 LICENSE. For this reason we have to release DolphinScheduler's distribute package without those packages, even if this will make more complicated for users. Datasource such as MySQL, Oracle, SQL Server as the examples, but we have the solution to solve this For example, if you want to use MySQL datasource, you need to download the correct JDBC driver from , and move it into directory `api-server/libs` and `worker-server/libs`. After that, you could activate MySQL datasource by restarting `api-server` and `worker-server`. Mount to container volume in the same path and restart it if you use container like Docker. Note: If you only want to use MySQL in the datasource center, there is no requirement for the version of MySQL JDBC driver. But if you want to use MySQL as the metabase of DolphinScheduler, it only supports version.