Project: Fluvio (App Definition and Development / Streaming & Messaging)
File: DEVELOPER.md

Table of contents: - - - - - - - - - - - - - - - - Thank you for joining the Fluvio community. The goal of this document is to provide everything you need to start developing Fluvio. Examples should work with the following platforms: macOS X Linux Other platforms such as Windows can be made to work, but we haven't tried them yet. To test and run services, you need to get access to the development Kubernetes cluster. Our guide uses Minikube as an example because it is easy to get it started, but you can use another Kubernetes cluster as well. Please see for setting up a development cluster. Please read for a technical arch and operation guide. The development environment requires the Rust toolchain and a few other build dependencies to build changes to the source code. To deploy and test the source code, Kubernetes is required because Fluvio stores its metadata in Kubernetes. Fluvio binaries can be tested locally, running outside of any k8s cluster using the `--local` parameter or they can be built into a docker image and deployed into the Kubernetes cluster. Please follow instructions to install Rust and Cargo. Most of these are required for building a docker image: it is also possible to not install these and just `cargo build` the binaries you need and test them with a local setup. make zig lld (v16) git Kubernetes is required for Fluvio to store its metadata. Please set up one of the following recommended kubernetes distros: Helm is used for installing Fluvio on Kubernetes. Please follow to install the helm. Zig and LLVM is required. For mac: You can skip LLVM if you are not building docker image. ```bash $ brew install llvm@16 $ ./actions/zig-install.sh $ export PATH="/opt/homebrew/opt/llvm@16/bin:$PATH" ``` For ubuntu: See https://apt.llvm.org for installing LLVM. LLVM up to 16 is confirmed to work. ```bash $ ./actions/zig-install.sh ``` It is recommended to use native binaries for development and testing but not for production. For production, please build docker image and run in the Kuberentes as pod. You still need to have Kubernets cluster running. CLI is required to install, manage and access the Fluvio cluster. To build CLI, run: ``` $ make build-cli ``` To build minimum CLI without Kubernetes dependencies for target such as Raspberry Pi, run: ``` $ make build-cli-minimal ``` Entire Fluvio cluster (including SC and SPU) is contained a single binary. To build the binary, run: ``` $ make build-cluster ``` By default, the build will use Rust `develop` profile. To use `release` profile, set `RELEASE` to Makefile. For example, to generate optimized binaries, run, `make build-cluster RELEASE=true`. `make clean` to completely remove all build binaries and artifacts. Fluvio uses helm chart to install and manage Kubernetes components. They are inline into Fluvio CLI binary. If there is any issue with helm chart, run `make -C k8-util/helm clean` to clean up helm artifacts. Binaries are located in `target` directory. You can run them directly or you can use following handy aliases: ``` alias flvd='target/debug/fluvio' alias flvdr='target/release/fluvio' alias flvt='target/debug/flv-test' ``` We will use the alias going forward. Use following commands to start Fluvio cluster using native binaries. ``` $ flvd cluster start --local --develop Running pre-flight checks Supported helm version 3.10.0+gce66412 is installed Kubectl active cluster rancher-desktop at: https://127.0.0.1:6443 found Supported Kubernetes server 1.22.7+k3s1 found Local Fluvio is not installed Fixed: Fluvio Sys chart

is installed All checks passed! Local Cluster initialized SC Launched Profile set Starting SPU: (1/1) / 1 SPU launched Successfully installed Local Fluvio cluster ``` Then you can create topic, produce and consume messages. ``` $ flvd topic create hello topic "hello" created $> echo "hello world" | flvd produce hello $> flvd consume hello -B Consuming records from the beginning of topic 'hello' hello world ^C ``` You can see SC and SPU running: ``` ps -ef | grep fluvio 501 61948 1 0 4:51PM ttys000 0:00.01 /tmp/fluvio/target/debug/fluvio run sc --local 501 61949 61948 0 4:51PM ttys000 0:00.24 /tmp/fluvio/target/debug/fluvio-run sc --local 501 61955 1 0 4:51PM ttys000 0:00.03 /tmp/fluvio/target/debug/fluvio run spu -i 5001 -p 0.0.0.0:9010 -v 0.0.0.0:9011 --log-base-dir /Users/myuser/.fluvio/data 501 61956 61955 0 4:51PM ttys000 0:00.27 /tmpfluvio/target/debug/fluvio-run spu -i 5001 -p 0.0.0.0:9010 -v 0.0.0.0:9011 --log-base-dir /Users/myuser/.fluvio/data 501 62035 989 0 4:52PM ttys000 0:00.00 grep fluvio ``` Since we still leverages Kubernetes CRDs, sys chart is still installed. ``` $> helm list NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION fluvio-sys default 1 2022-10-06 18:46:23.359066 -0700 PDT deployed fluvio-sys-0.9.10 0.9.34 ``` During development, it is necessary to restart SC and SPU separately. In order do so, you can kill SC or SPU and starting them individually. You can use following commands for SC ``` kill -9 <process id of fluvio-run sc> flvd run sc --local CLI Option: ScOpt { local: true, bind_public: None, bind_private: None, namespace: None, tls: TlsConfig { tls: false, server_cert: None, server_key: None, enableclientcert: false, ca_cert: None, bindnontls_public: None, }, x509authscopes: None, auth_policy: None, white_list: [], } Starting SC, platform: 0.9.34 Streaming Controller started successfully ``` You can then kill by ^C Note that this will not kill SPU. Once new SC is up, SPU will reconnect to it. For SPU, you can use following template. Note that `--log-base` should be same as the previously. ``` kill -9 <process id of fluvio-run spu> flvd run spu -i 5001 -p 0.0.0.0:9010 -v 0.0.0.0:9011 --log-base-dir ~/.fluvio/data starting spu server (id:5001) SPU Version: 0.0.0 started successfully ``` You can launch additional SPU as needed; just ensure that ports don't conflict with each other. For example, to add 2nd: ``` $ flvd cluster spu register --id 5002 --public-server 0.0.0.0:9020 --private-server 0.0.0.0:9021 $ flvd run spu -i 5002 -p 0.0.0.0:9020 -v 0.0.0.0:9021 ``` If Fluvio cluster is no longer needed, you can delete it using following command: ``` $ flvd cluster delete Current channel: stable Uninstalled fluvio kubernetes components Uninstalled fluvio local components Objects and secrets have been cleaned up ``` There are two helm charts that are installed by Fluvio CLI. Only `fluvio-sys` chart is installed when using native binaries. Other chart `fluvio-app` chart is installed when running Fluvio with docker image. ``` $> helm list NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION fluvio-sys default 1 2022-10-06 19:18:37.416564 -0700 PDT deployed fluvio-sys-0.9.10 0.9.34 ``` You can install system chart only using following command. This assume system chart is not installed. ``` $> flvd cluster start --sys-only installing sys chart, upgrade: false ``` You can set various log levels . For example, to start cluster using log level `info` using cluster start ``` flvd cluster start --local --develop --rust-log fluvio=info ``` For individual binaries, you can use RUST_LOG env variable: ``` RUST_LOG=fluvio=info flvd run sc --local ``` To remove all fluvio related objects in the Kubernetes cluster, you can use the following command: ``` $ flvd cluster delete ``` Note that when you uninstall the cluster, CLI will remove all related objects such as Topics Partitions Tls Secrets Storage etc The docker image requires first installing a cross compilation toolchain, along with other build dependencies mentioned such as

``` $ rustup target add x86_64-unknown-linux-musl $ rustup target add aarch64-unknown-linux-musl ``` This will build the Fluvio cli and then create a docker image and import it into your local k8s cluster: ``` $ make build-cli buildk8image ``` If you are not running recommended version of k8s, image may not be imported into Kubernetes cluster. This will run fluvio components as Kubernetes pods. ``` $ flvd cluster start --develop using development git hash: c540c3a6ca488261edd20cdfdb95fdf50a050483 Running pre-flight checks Kubectl active cluster rancher-desktop at: https://127.0.0.1:6443 found Supported helm version 3.10.0+gce66412 is installed Supported Kubernetes server 1.22.7+k3s1 found Fixed: Fluvio Sys chart 0.9.34 is installed Previous fluvio installation not found All checks passed! Installed Fluvio app chart: 0.9.34 Connected to SC: 192.168.50.106:30003 Profile set SPU group main launched with 1 replicas Successfully installed Fluvio ``` Then you can create topic, produce and consume messages as described above. You should see two helm chart installed. There is additional chart `fluvio` that is used for installing fluvio components. ``` $> helm list NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION fluvio default 1 2022-10-06 19:42:07.051782 -0700 PDT deployed fluvio-app-0.9.2 0.9.34 fluvio-sys default 1 2022-10-06 19:42:06.668329 -0700 PDT deployed fluvio-sys-0.9.10 0.9.34 ``` You should have two pods running: ``` $> kubectl get pods NAME READY STATUS RESTARTS AGE fluvio-sc-fc976685d-qbxg2 1/1 Running 0 4m17s fluvio-spg-main-0 1/1 Running 0 4m15s ``` And services for SC and SPG (SPU group) are running: ``` $> kubectl get service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 112d fluvio-sc-internal ClusterIP 10.43.110.7 <none> 9004/TCP 5m8s fluvio-sc-public NodePort 10.43.31.194 <none> 9003:30003/TCP 5m8s fluvio-spg-main ClusterIP None <none> 9005/TCP,9006/TCP 5m6s fluvio-spu-main-0 NodePort 10.43.88.71 <none> 9005:30004/TCP 5m6s ``` Fluvio uses `NodePort` to expose SC and SPU to the outside world. And use PVC to store data: ``` $> kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE data-fluvio-spg-main-0 Bound pvc-dff4c156-5718-4b41-a825-cee7d07fd997 10Gi RWO local-path 6m31s ``` Fluvio uses the default storage class used in the current Kubernetes but can be overridden using helm config. We have 3 types of tests: Tests run w/ `cargo` This includes unit tests and doc tests Tests run with `fluvio-test` These are integration tests executed with our `fluvio-test` test harness Build with `make build-test` Tests run with `bats` These are CLI tests written and executed with `bats-core` Run with `make cli-smoke` Bats-core is used for our CLI-based testing. Please follow the installation guide. ``` make build_smartmodules ``` This requires a running cluster. Perform smoke test using local cluster mode: ``` make smoke-test-local ``` This results in message such as: ``` Creating the topic: test topic "test" created found topic: test offset: 0 starting fetch stream for: test base offset: 0, expected new records: 1000 <<consume test done for: test >>>> consume message validated!, records: 1000 deleting cluster ``` Perform smoke test as Kubernetes objects: ``` make smoke-test-k8 ``` Perform CLI smoke test against your running cluster (Kubernetes or local) ``` make cli-fluvio-smoke ``` This guide helps users to solve issues they might face during the setup process. If you face connection issues while creating minikube image Re-build i.e. delete and restart minikube cluster ``` sh k8-util/minikube/reset-minikube.sh ``` In certain cases, partition may not be deleted correctly. In this case, you can manually force delete by: ``` kubectl patch partition <partition_name> -p '{"metadata":{"finalizers":null}}' --type merge ``` Instead of building Fluvio, you may want to prefer just to download it and get to work. You can use our one-line installation script. You can use it to install the latest release or prerelease, or install a specific version: ``` $ curl -fsS https://hub.infinyon.cloud/install/install.sh | bash # Install latest release $