Project: Fluid (Orchestration & Management / Scheduling & Orchestration)
File: how_to_develop.md

Git Golang (version >= 1.13) Docker (version >= 19.03) Kubernetes (version >= 1.14) GNU Make For installation of Golang, please refer to `make` is usually in a `build-essential` package in your distribution's package manager of choice. Make sure you have `make` on your machine. There're great chances that you may want to run your implementation in a real Kubernetes cluster, so probably a Docker is needed for some necessary operations like building images. See for more information. ```shell $ mkdir -p $GOPATH/src/github.com/fluid-cloudnative/ $ cd $GOPATH/src/github.com/fluid-cloudnative $ git clone https://github.com/fluid-cloudnative/fluid.git ``` NOTE: In this document, we build, run and debug under non-module environment. See for more information if some issue occurs to you. `Makefile` under project directory provides many tasks you may want to use including Test, Build, Debug, Deploy etc. You can simply get a binary by running: ```shell $ make build ``` ```shell $ make alluxioruntime-controller-build ``` By default, the binary would be put under `<fluid-path>/bin`. Set tags for images ```shell $ export DATASETCONTROLLERIMG=<your-registry>/<your-namespace>/<img-name> $ export ALLUXIORUNTIMECONTROLLERIMG=<your-registry>/<your-namespace>/<img-name> $ export CSI_IMG=<your-registry>/<your-namespace>/<csi-img-name> $ export INITUSERSIMG=<your-registry>/<your-namespace>/<csi-img-name> $ make docker-build-all $ docker run --rm --privileged multiarch/qemu-user-static --reset -p yes $ make docker-buildx-all-push ``` Before running Fluid, you need to push the built image to an accessible image registry. Login to a image registry Make sure you've login to a docker image registry that you'd like to push your image to: ```shell $ sudo docker login <docker-registry> ``` push your images: ```shell $ make docker-push-all ``` In the following steps, we assume you have properly configured `KUBECONFIG` environment variable or set up `~/.kube/config`. See for more information. Push your images to a image registry accessible to your Kubernetes cluster If your images are pushed to some private repositories, make sure your Kubernetes cluster hold credentials for accessing those repositories. Change image in the samples we provide: ```yaml ... ... containers: name: manager image: <registry>/<namespace>/<img-repo>:<img-tag> ``` ```yaml ... ... containers: name: plugins image: <registry>/<namespace>/<csi-img-name>:<csi-img-tag> ``` Install CRDs ```shell $ kubectl apply -k config/crd ``` Check CRD with: ```shell $ kubectl get crd | grep fluid alluxiodataloads.data.fluid.io 2020-08-22T03:53:46Z alluxioruntimes.data.fluid.io 2020-08-22T03:53:46Z datasets.data.fluid.io 2020-08-22T03:53:46Z ``` Install your implementation ```shell $ kubectl apply -k config/fluid ``` Check Fluid system with: ```shell $ kubectl get pod -n fluid-system NAME READY STATUS RESTARTS AGE controller-manager-7fd6457ccf-p7j2x 1/1 Running 0 84s csi-nodeplugin-fluid-pj9tv 2/2 Running 0 84s csi-nodeplugin-fluid-t8ctj 2/2 Running 0 84s ``` Run samples to verify your implementation Here is a sample provided by us, you may want to rewrite it according to your implementation. ```shell $ kubectl apply -k config/samples ``` Check sample pods: ```shell $ kubectl get pod NAME READY STATUS RESTARTS AGE cifar10-fuse-vb6l4 1/1 Running 0 6m15s cifar10-fuse-vtqpx 1/1 Running 0 6m15s cifar10-master-0 2/2 Running 0 8m24s cifar10-worker-729xz 2/2 Running 0 6m15s cifar10-worker-d6kmd 2/2 Running 0 6m15s nginx-0 1/1 Running 0 8m30s nginx-1 1/1 Running 0 8m30s ``` Check logs to verify your implementation ```shell $ kubectl logs -n fluid-system <controllermanagername> ``` Clean up ```shell $ kubectl delete -k config/samples $ kubectl delete -k config/fluid $ kubectl delete -k config/crd ``` Execute following command from project root to run basic unit tests: ```shell $ make unit-test ``` `kubebuilder` provided a integration test framework based on package. You must install `kubebuilder` before running integration tests: ```shell $ os=$(go env GOOS) $ arch=$(go env GOARCH) $ curl -L https://go.kubebuilder.io/dl/2.3.1/${os}/${arch} | tar -xz -C /tmp/ $ sudo mv

/usr/local/kubebuilder $ export PATH=$PATH:/usr/local/kubebuilder/bin ``` Next, run all unit tests (integration tests included) with: ```shell $ make test ``` NOTE: When running unit tests on a non-linux system such as macOS, if testing failed and says `exec format error`, you may need to check whether `GOOS` is consistent with your actual OS. The Fluid controller component supports local operation or debugging. The Fluid controller components include Dataset Controller, various runtime controllers, and Application Controller. Before running the controller component locally, it is necessary to configure kubeconfig in advance in the local environment (configured through the `KUBECONFIG` environment variable or through the `$HOME/.kube/config` file) and be able to access a Kubernetes cluster normally. The Fluid CSI plugin cannot run locally to interact with the Kubernetes cluster. To run such components, it is necessary to first perform image construction, manually replace the corresponding image address of `charts/fluid/fluid/values.yaml`, and then deploy it to the Kubernetes cluster. The runtime controller depends on . And related Helm Charts to function properly. Therefore, before running the Runtime Controller, execute the following command to configure the environment: Create a soft link for the helm command locally ``` $ ln -s $(which helm) /usr/local/bin/ddc-helm ``` Create a soft link directory for related Charts locally ``` $ ln -s $GOPATH/src/github.com/fluid-cloudnative/fluid/charts $HOME/charts ``` Taking the Alluxio Runtime Controller as an example, run the component locally using the following command ``` $ export ALLUXIORUNTIMEIMAGE_ENV="alluxio/alluxio-dev:2.9.0" $ export ALLUXIOFUSEIMAGE_ENV="alluxio/alluxio-dev:2.9.0 $ export DEFAULTINITIMAGE_ENV="fluidcloudnative/init-users:v0.8.0-5bb4677" $ export MOUNT_ROOT="/runtime-mnt" $ export HOME="$HOME" $ ./bin/alluxioruntime-controller start --development=true --enable-leader-election ``` The Fluid controller component supports local operation or debugging. The Fluid controller components include Dataset Controller, various runtime controllers, and Application Controller. Before running the controller component locally, it is necessary to configure kubeconfig in advance in the local environment (configured through the `KUBECONFIG` environment variable or through the `$HOME/.kube/config` file) and be able to access a Kubernetes cluster normally. The Fluid CSI plugin cannot run locally to interact with the Kubernetes cluster. To run such components, it is necessary to first perform image construction, manually replace the corresponding image address of `charts/fluid/fluid/values.yaml`, and then deploy it to the Kubernetes cluster. Ensure that go help is installed in the environment, and refer to the for the specific installation process ```shell $ dlv debug cmd/alluxio/main.go ``` If VSCode is used as the development environment, the of VSCode can be directly installed and conduct local debugging. Taking debugging the Alluxio Runtime Controller as an example, the Go code debugging task is defined in `./.vscode/launch.json` as follows: ```json { "version": "0.2.0", "configurations": [ { "name": "Alluxio Runtime Controller", "type": "go", "request": "launch", "mode": "debug", "program": "cmd/alluxio/main.go", "args": ["start", "--development=true", "--enable-leader-election"], "env": { "KUBECONFIG": "<path>/<to>/<kubeconfig>", "ALLUXIORUNTIMEIMAGE_ENV": "alluxio/alluxio-dev:2.9.0", "ALLUXIOFUSEIMAGE_ENV": "alluxio/alluxio-dev:2.9.0", "DEFAULTINITIMAGE_ENV": "fluidcloudnative/init-users:v0.8.0-5bb4677", "MOUNT_ROOT": "/runtime-mnt", "HOME": "<HOME_PATH>" } }, ] } ``` Proxy access to WebHook in the cluster to the local machine: ```shell $ curl -OL https://github.com/alibaba/kt-connect/releases/download/v0.3.7/ktctl0.3.7Linuxx8664.tar.gz $ tar zxf ktctl0.3.7Linuxx8664.tar.gz $ mv ktctl /usr/local/bin/ktctl $ ktctl --version $ ktctlexchangefluid-pod-admission-webhook --kubeconfig<path>/<to>/<kubeconfig>--namespacefluid-system--expose9443 ``` Set the debug task in `./.vscode/launch.json`: ```json { "version": "0.2.0", "configurations": [ { "name": "Fluid Webhook", "type": "go", "request": "launch", "mode": "debug", "program": "cmd/webhook/main.go", "args": ["start", "--development=true", "--full-go-profile=false", "--pprof-addr=:6060"], "env": { "TIMETRACKDEBUG": "true", "MYPODNAMESPACE": "fluid-system" } }, ] } ``` For components such as Fluid Webhook and Fluid CSI plugins, remote debugging is usually the more commonly used method. Please ensure that go help is correctly installed on both the local machine and component images. On remote host: ```shell $ dlv debug --headless --listen ":12345" --log --api-version=2 cmd/alluxio/main.go ``` This will cause the remote host's debugging program to listen to the specified port (e.g. 12345) On local machine: ```shell $ dlv connect