Project: DolphinScheduler (Orchestration & Management / Scheduling & Orchestration)
File: pseudo-cluster.md

The purpose of the pseudo-cluster deployment is to deploy the DolphinScheduler service on a single machine. In this mode, DolphinScheduler's master, worker, API server, are all on the same machine. If you are a new hand and want to experience DolphinScheduler functions, we recommend you install follow . If you want to experience more complete functions and schedule massive tasks, we recommend you install follow or . Pseudo-cluster deployment of DolphinScheduler requires external software support: JDKdownload (1.8+), install and configure environment variable `JAVAHOME` and append `bin` dir (included in `JAVAHOME`) to `PATH` variable. You can skip this step if it already exists in your environment. Binary package: Download the DolphinScheduler binary package at Database: (8.2.15+) or (5.7+), you can choose one of the two, such as MySQL requires JDBC Driver 8.0.16 Registry Center: (3.8.0+) Process tree analysis `pstree` for macOS `psmisc` for Fedora/Red/Hat/CentOS/Ubuntu/Debian _Note:_ DolphinScheduler itself does not depend on Hadoop, Hive, Spark, but if you need to run tasks that depend on them, you need to have the corresponding environment support. Create a deployment user, and make sure to configure `sudo` without password. Here make an example to create user `dolphinscheduler`: ```shell useradd dolphinscheduler echo "dolphinscheduler" | passwd --stdin dolphinscheduler sed -i '$adolphinscheduler ALL=(ALL) NOPASSWD: NOPASSWD: ALL' /etc/sudoers sed -i 's/Defaults requirett/#Defaults requirett/g' /etc/sudoers chown -R dolphinscheduler:dolphinscheduler apache-dolphinscheduler-*-bin chmod -R 755 apache-dolphinscheduler-*-bin ``` _NOTICE:_ Due to DolphinScheduler's multi-tenant task switch user using command `sudo -u {linux-user}`, the deployment user needs to have `sudo` privileges and be password-free. If novice learners dont understand, you can ignore this point for now. If you find the line "Defaults requirett" in the `/etc/sudoers` file, please comment the content. Since resources need to be sent to different machines during installation, SSH password-free login is required between each machine. The following shows the steps to configure password-free login: ```shell su dolphinscheduler ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa cat ~/.ssh/idrsa.pub >> ~/.ssh/authorizedkeys chmod 600 ~/.ssh/authorized_keys ``` _Notice:_ After the configuration is complete, you can run the command `ssh localhost` to test works or not. If you can login with ssh without password stands for successful. Go to the ZooKeeper installation directory, copy configure file `zoo_sample.cfg` to `conf/zoo.cfg`, and change value of dataDir in `conf/zoo.cfg` to `dataDir=./tmp/zookeeper`. ```shell

start ``` After completing the preparation of the basic environment, you need to modify the configuration file according to the environment you used. Change the environment configurations via `export <ENVNAME>=<VALUE>`. The configuration files are located in directory `bin/env` as `installenv.sh` and `dolphinscheduler_env.sh`. File `install_env.sh` describes which machines will be installed DolphinScheduler and what server will be installed on each machine. You could find this file in the path `bin/env/install_env.sh` and the detail of the configuration as below. ```shell ips="localhost" sshPort="22" masters="localhost" workers="localhost:default" alertServer="localhost" apiServers="localhost" installPath=~/dolphinscheduler deployUser="dolphinscheduler" ``` File `./bin/env/dolphinscheduler_env.sh` describes the following configurations: Database configuration of DolphinScheduler, see for detailed instructions. Some tasks which need external dependencies or libraries such as `JAVAHOME` and `SPARKHOME`. Registry center `zookeeper`. Server related configuration, such as cache type, timezone, etc. You could ignore the task external dependencies if you do not use those tasks, but you have to change `JAVA_HOME`, registry center and database related configurations based on your environment. ```shell export JAVAHOME=${JAVAHOME:-/opt/soft/java} export DATABASE=${DATABASE:-postgresql} export SPRINGPROFILESACTIVE=${DATABASE} export SPRINGDATASOURCEURL="jdbc:postgresql://127.0.0.1:5432/dolphinscheduler" export SPRINGDATASOURCEUSERNAME={user} export SPRINGDATASOURCEPASSWORD={password} export SPRINGCACHETYPE=${SPRINGCACHETYPE:-none} export SPRINGJACKSONTIMEZONE=${SPRINGJACKSONTIMEZONE:-UTC} export REGISTRYTYPE=${REGISTRYTYPE:-zookeeper} export REGISTRYZOOKEEPERCONNECTSTRING=${REGISTRYZOOKEEPERCONNECTSTRING:-localhost:2181} export HADOOPHOME=${HADOOPHOME:-/opt/soft/hadoop} export HADOOPCONFDIR=${HADOOPCONFDIR:-/opt/soft/hadoop/etc/hadoop} export SPARKHOME=${SPARKHOME:-/opt/soft/spark} export PYTHONLAUNCHER=${PYTHONLAUNCHER:-/opt/soft/python/bin/python3} export HIVEHOME=${HIVEHOME:-/opt/soft/hive} export FLINKHOME=${FLINKHOME:-/opt/soft/flink} export DATAXLAUNCHER=${DATAXLAUNCHER:-/opt/soft/datax/bin/datax.py} export PATH=$HADOOPHOME/bin:$SPARKHOME/bin:$PYTHONLAUNCHER:$JAVAHOME/bin:$HIVEHOME/bin:$FLINKHOME/bin:$DATAX_LAUNCHER:$PATH ``` Follow the instructions in `Pseudo-Cluster/Cluster Initialize the Database` section to create and initialize database Use deployment user you created above, running the following command to complete the deployment, and the server log will be stored in the logs folder. ```shell bash ./bin/install.sh ``` _Note:_ For the first time deployment, there maybe occur five times of `sh: bin/dolphinscheduler-daemon.sh: No such file or directory` in the terminal, this is non-important information that you can ignore. Access address `http://localhost:12345/dolphinscheduler/ui` and login DolphinScheduler UI. The default username and password are admin/dolphinscheduler123 ```shell bash ./bin/stop-all.sh bash ./bin/start-all.sh bash ./bin/dolphinscheduler-daemon.sh stop master-server bash ./bin/dolphinscheduler-daemon.sh start master-server bash ./bin/dolphinscheduler-daemon.sh start worker-server bash ./bin/dolphinscheduler-daemon.sh stop worker-server bash ./bin/dolphinscheduler-daemon.sh start api-server bash ./bin/dolphinscheduler-daemon.sh stop api-server bash ./bin/dolphinscheduler-daemon.sh start alert-server bash ./bin/dolphinscheduler-daemon.sh stop alert-server ``` _Note1:_: Each server have `dolphinschedulerenv.sh` file in path `<service>/conf/dolphinschedulerenv.sh` which for micro-services need. It means that you could start all servers by command `<service>/bin/start.sh` with different environment variable from `<service>/conf/dolphinschedulerenv.sh`. But it will use file `bin/env/dolphinschedulerenv.sh` overwrite `<service>/conf/dolphinscheduler_env.sh` if you start server with command `/bin/dolphinscheduler-daemon.sh start <service>`. _Note2:_: Please refer to the section of "System Architecture Design" for service usage. Python gateway service is started along with the api-server, and if you do not want to start Python gateway service please disabled it by changing the yaml config `python-gateway.enabled : false` in api-server's configuration path `api-server/conf/application.yaml`