Project: KubeVela (App Definition and Development / Application Definition & Image Build)
File: apply-workload-and-trait.md

Owner: Yue Wang(@captainroy-hy), Jianbo Sun(@wonderflow) Date: 01/21/2021 Status: When an ApplicationConfiguration is deployed, vela-core will create(apply) corresponding workload/trait instances and keep them stay align with the `spec` defined in ApplicationConfiguration through periodical reconciliation in AppConfig controller. In each round of reconciliation, if the configurations rendered from AppConfig are changed comparing to last round reconciliation, it's required to apply all changes to the workloads or traits. Additionally, it also allows others (anything except AppConfig controller, e.g., trait controllers) to modify workload/trait instances. Apply should handle three kinds of modification including add a field change a field remove a field by omitting it Meanwhile, Apply should have no impact on changes made by others, namely, not eliminate or override those changes UNLESS the change is made upon fields that are rendered from AppConfig originally. We employed the same mechanism as `kubectl apply`, that is, computing a 3-way diff based on target object's current state, modified state, and last-appied state. Specifically, a new annotation, `app.oam.dev/last-applied-configuration`, is introduced to record workload/trait's last-applied state. Once there's a conflict on field, both changed by AppConfig and others, AppConfig's value will always override others' assignment. Before this implementation, vela-core use `JSON-Merge` patch to apply workloads and `update` to apply traits. That brought several defects shown in below samples. This section introduced a comparison between how old mechanism and new applies workload/trait, also shows how new Apply overcomed the defects. The reason why abandon json-merge patch is that, it cannot remove a field through unsetting value in the patched manifest. For example, apply below deployment as a workload. json-merge patch cannot remove `minReadySeconds` field through applying a modified manifest with `minReadySeconds` omitted . ```yaml apiVersion: apps/v1 kind: Deployment ... spec: minReadySeconds: 60 replicas: 3 template: spec: containers: name: nginx image: nginx:1.14.2 apiVersion: apps/v1 kind: Deployment

spec: replicas: 3 template: spec: containers: name: nginx image: nginx:1.14.2 apiVersion: apps/v1 kind: Deployment ... spec: minReadySeconds: 60 # <=== not removed replicas: 3 template: spec: containers: name: nginx image: nginx:1.14.2 ``` By computing a 3-way diff, we can get a patch aware of the field set in last-applied manifest is omitted in the new modified manifest, namely, users wanna remove this field. And an annotation, `app.oam.dev/last-applied-configuration`, is used to record last-applied-state of the resource for further use in computing 3-way diff next time. ```yaml apiVersion: apps/v1 kind: Deployment metadata: annotations: # v=== record last-applied-state app.oam.dev/last-applied-configuration: | {"apiVersion":"apps/v1","kind":"Deployment","metadata":{"name":"nginx-deployment","labels":{"app":"nginx"}},"spec":{"replicas":3,"selector":{"matchLabels":{"app":"nginx"}},"template":{"metadata":{"labels":{"app":"nginx"}},"spec":{"containers":[{"name":"nginx","image":"nginx:1.14.2"}]}}}} ... spec: replicas: 3 template: spec: containers: name: nginx image: nginx:1.14.2 ``` The reasons why abandon `update` update always eliminates all fields set by others (e.g., trait controllers) if trait contains immutable field (e.g., k8s Service), update fails For example, apply below Service as a trait. ```yaml apiVersion: v1 kind: Service metadata: name: my-service spec: selector: app: myweb ports: protocol: TCP port: 80 apiVersion: v1 kind: Service ... spec: clusterIP: 172.21.7.149 # <=== immutable field ports: port: 80 protocol: TCP targetPort: 80 selector: app: myweb sessionAffinity: None type: ClusterIP status: loadBalancer: {} ``` Additionally, if a trait has no immutable field, update will eliminate all fields set by others. ```yaml kind: Bar spec: f1: v1 kind: Bar spec: f1: v1 f2: v2 # <=== newly set field kind: Bar spec: f1: v1 ``` But as described in section, we expect to keep these changes. Applying traits works in the same way as workloads. We use annotation, `app.oam.dev/last-applied-configuration`, to record last-applied manifest. Because 3-way diff will ignore the fields not touched in last-applied-state, the immutable fields will not be involved into patch data. Because 3-way diff also will ignore the fields set by others (others are not supposed to modify the field rendered from AppConfig), the changes made by others will be retained.