cwielenb/CNCF-Dataset huggingface dataset:
Project: Capsule (Orchestration & Management / Scheduling & Orchestration)
File: index.md

Capsule implements a multi-tenant and policy-based environment in your Kubernetes cluster. It is designed as a micro-services-based ecosystem with the minimalist approach, leveraging only on upstream Kubernetes. Kubernetes introduces the Namespace object type to create logical partitions of the cluster as isolated slices. However, implementing advanced multi-tenancy scenarios, it soon becomes complicated because of the flat structure of Kubernetes namespaces and the impossibility to share resources among namespaces belonging to the same tenant. To overcome this, cluster admins tend to provision a dedicated cluster for each groups of users, teams, or departments. As an organization grows, the number of clusters to manage and keep aligned becomes an operational nightmare, described as the well known phenomena of the clusters sprawl. Capsule takes a different approach. In a single cluster, the Capsule Controller aggregates multiple namespaces in a lightweight abstraction called Tenant, basically a grouping of Kubernetes Namespaces. Within each tenant, users are free to create their namespaces and share all the assigned resources. On the other side, the Capsule Policy Engine keeps the different tenants isolated from each other. Network and Security Policies, Resource Quota, Limit Ranges, RBAC, and other policies defined at the tenant level are automatically inherited by all the namespaces in the tenant. Then users are free to operate their tenants in autonomy, without the intervention of the cluster administrator.
Project: Gloo (Orchestration & Management / API Gateway)
File: version.proto.sk.md

title: "version.proto" weight: 5 <!-- Code generated by solo-kit. DO NOT EDIT. --> - ```yaml "type": .gloo.solo.io.GlooType "enterprise": bool "kubernetes": .gloo.solo.io.Kubernetes ``` | Field | Type | Description | | -- | - | -- | | `type` | | | | `enterprise` | `bool` | Whether or not this is an enterprise distribution. | | `kubernetes` | | | ```yaml "containers": []gloo.solo.io.Kubernetes.Container "namespace": string ``` | Field | Type | Description | | -- | - | -- | | `containers` | | Array of containers comprising a single distribution of gloo. | | `namespace` | `string` | namespace gloo is running in. | ```yaml "tag": string "name": string "registry": string "ossTag": string ``` | Field | Type | Description | | -- | - | -- | | `tag` | `string` | | | `name` | `string` | | | `registry` | `string` | | | `ossTag` | `string` | | ```yaml "version": string ``` | Field | Type | Description | | -- | - | -- | | `version` | `string` | | ```yaml "major": string "minor": string "gitVersion": string "buildDate": string "platform": string ``` | Field | Type | Description | | -- | - | -- | | `major` | `string` | | | `minor` | `string` | | | `gitVersion` | `string` | | | `buildDate` | `string` | | | `platform` | `string` | | ```yaml "client": .gloo.solo.io.ClientVersion "server": []gloo.solo.io.ServerVersion "kubernetesCluster": .gloo.solo.io.KubernetesClusterVersion ``` | Field | Type | Description | | -- | - | -- | | `client` | | | | `server` | | This field is an array of server versions because although there can only be 1 client version, there can potentially be many instances of gloo running on a single cluster. | | `kubernetesCluster` | | | Description: type of gloo server instance | Name | Description | | -- | -- | | Unknown | | | Gateway | | | Ingress | | | Knative | Deprecated: Will not be available in Gloo Edge 1.11 | <!-- Start of HubSpot Embed Code --> <script type="text/javascript" id="hs-script-loader" async defer src="//js.hs-scripts.com/5130874.js"></script> <!-- End of HubSpot Embed Code -->
Project: DolphinScheduler (Orchestration & Management / Scheduling & Orchestration)
File: dependent.md

Dependent nodes are dependency check nodes. For example, process A depends on the successful execution of process B from yesterday, and the dependent node will check whether process B run successful yesterday. Click`Project Management -> Project Name -> Workflow Definition`, and click the`Create Workflow`button to enter the DAG editing page. Drag from the toolbar <img src="../../../../img/tasks/icons/dependent.png" width="15"/> task node to canvas. Please refer to `Default Task Parameters` section for default parameters. | Parameter | Description | ||--| | Add Dependencies | Configure dependent upstream tasks. | | Check interval | Check the dependent upstream task status interval, the default is 10s. | | Dependency failure policy | Failure: The dependent upstream task failure and the current task directly failure; Wait: The dependent upstream task failure and the current task continues to wait; | | Dependency failure waiting time | When the dependency failure policy chooses to wait, the current task wait time. | The Dependent node provides a logical judgment function, which can detect the execution of the dependent node according to the logic. Two dependency modes are supported, including workflow-dependent and task-dependent. The task-dependent mode is divided into two cases: depend on all tasks in the workflow and depend on a single task. The workflow-dependent mode checks the status of the dependent workflow; the all-task-dependent mode checks the status of all tasks in the workflow; and the single-task-dependent mode checks the status of the dependent task. When the dependent result is success and the parameter passing option is true, the Dependent node will output the output parameters of the dependency to the downstream task. When the parameter names of multiple dependencies are the same, it involves the priority of the parameters. See also For example, process A is a weekly task, processes B and C are daily tasks, and task A requires tasks B and C to be successfully executed last week. And another example is that process A is a weekly report task, processes B and C are daily tasks, and task A requires tasks B or C to be successfully executed last week: If the weekly report A also needs to be executed successfully last Tuesday:
Project: DolphinScheduler (Orchestration & Management / Scheduling & Orchestration)
File: kubeflow.md

task type is used to create tasks on Kubeflow. The backend mainly uses the `kubectl` command to create kubeflow tasks, and continues to monitor the resource status on Kubeflow until the task is completed. Now it mainly supports creating kubeflow tasks using yaml files. If you need to publish `kubeflow pipeline` tasks, you can use the . Click `Project Management -> Project Name -> Workflow Definition`, and click the `Create Workflow` button to enter the DAG editing page. Drag <img src="../../../../img/tasks/icons/kubeflow.png" width="15"/> from the toolbar to the canvas. The task plugin picture is as follows Please refer to `Default Task Parameters` section for default parameters. NamespaceThe namespace parameter of the cluster yamlContentCRD YAML file content, for example: ```yaml apiVersion: "kubeflow.org/v1" kind: TFJob metadata: name: tfjob-simple namespace: kubeflow-user-example-com spec: tfReplicaSpecs: Worker: replicas: 2 restartPolicy: OnFailure template: metadata: annotations: sidecar.istio.io/inject: "false" spec: containers: name: tensorflow image: gcr.io/kubeflow-ci/tf-mnist-with-summaries:1.0 command: "python" "/var/tfmnist/mnistwith_summaries.py" ``` Configure Kubernetes environment Reference . Only the required fields need to be filled in, and the others do not need to be filled in. The resource management depends on the YAML file definition in the specific Job. kubectl Install , and make sure `kubectl` can submit tasks to kubeflow normally.
Project: Apache bRPC (Orchestration & Management / Remote Procedure Call)
File: newcommitter_en.md

More than 10 commits from the contributor Contributor is willing to accept the invitation to become a committer Contributor subscribe to dev@brpc.apache.org and introduce himself by sending an email to dev@apache.org The nominator sends an email to private@brpc to initiate discussion and begins to voting. If the voting is passed, it is OK (at least 3 +1, +1 >- 1). (See the ) The nominator sends a close vote email to private@brpc, the title can be subject ) The nominator sends an invitation letter to the nominee () and prompts him to submit ICLA after receiving a reply () The nominee fills in , individual contributors need to download Fill in personal information and sign, and send the electronic version to secretary@apache.org (Note: ICLA needs to fill in complete information, including mailing address and signature, otherwise it will be returned by ASF's secretary.) The personal information entry (except for the signature) can be filled in with a PDF reader or browser, and then saved for signature. Signature method support: Print pdf documents and scan them into electronic version after handwritten signature; Electronic signature using devices that support handwriting; Use 'gpg' for electronic signature, that is, to operate the pdf file filled with personal basic information (a public key/key pair matching the registered mailbox needs to be generated in advance): 'gpg -- armor -- detach sign icla. pdf'; Use 'DocuSign' to sign; The nominator sends an announcement email to dev@brpc.apache.org Add as committer () Let him set github id () Let him visit the website and get github permission() Please do these things: Hold the discussion and vote on your private@ list. This avoids any issues related to personnel, which should remain private. If the vote is successful, announce the result to the private@ list with a new email thread with subject . This makes it easier for secretary to find the result of the vote in order to request the account at the time of the filing of the ICLA. Only if the candidate accepts committership, announce the new committer on your dev@ list. Doing these things will make everyone's job easier. https://incubator.apache.org/guides/ppmc.html#votinginanewppmc_member https://community.apache.org/newcommitter.html https://incubator.apache.org/guides/ppmc.html#podlingprojectmanagementcommitteeppmc Initiate discussion in the private@brpc . If there is no objection, continue Open a Vote in private@brpc Send an email to close the voting and notify private@incubator.apache.org Announce new PPMC On private@brpc Set his authority by visiting https://whimsy.apache.org/roster/ppmc/brpc Help him subscribe to a private email group. See https://whimsy.apache.org/committers/moderationhelper.cgi
Project: BFE (Orchestration & Management / Service Proxy)
File: bal_table_status.md

The endpoint `/monitor/baltablestatus` exposes metrics about backend clusters. | Metric | Description | | | | | Balancers | State of cluster, it is map data, key is cluster name, value is cluster state | | BackendNum | Number of cluster backend | | Monitor Item | Description | | | | | SubClusters | State of sub-cluster, it is map data, key is sub-cluster name, value is number of sub-cluster backend | | BackendNum | Number of sub-cluster backend |
Project: STUNner (Orchestration & Management / Service Proxy)
File: turncat.md

`turncat` is a STUN/TURN client to open a connection through a TURN server to an arbitrary remote address/port. The main use is to open a local tunnel endpoint to any service running inside a Kubernetes cluster via STUNner. This is very similar in functionality to `kubectl port-forward`, but it uses STUN/TURN to enter the cluster. This is much faster than the TCP connection used by `kubectl`. On Linux and macOS, use to download the latest version of the `turncat` binary: ```console curl -sL https://raw.githubusercontent.com/l7mp/stunner/main/cmd/getstunner/getstunner.sh | sh - export PATH=$HOME/.l7mp/bin:$PATH ``` [!NOTE] The script installs `stunnerctl` too. Install the `turncat` binary using the standard Go toolchain and add it to `$PATH`. ```console go install github.com/l7mp/stunner/cmd/turncat@latest ``` You can also enforce a specific OS, CPU architecture, and STUNner version like below: ```console GOOS=windows GOARCH=amd64 go install github.com/l7mp/stunner/cmd/turncat@v0.17.5 ``` Building from source is as easy as it usually gets with Go: ```console cd stunner go build -o turncat cmd/turncat/main.go ``` Listen to client connections on the UDP listener `127.0.0.1:5000` and tunnel the received packets through the TURN server located at `192.0.2.1:3478` to the UDP listener located at `192.0.2.2:53`. Use the to authenticate with the TURN server and set the user/passwd to `test/test`: ```console ./turncat --log=all:INFO,turncat:DEBUG udp://127.0.0.1:5000 turn://test:test@192.0.2.1:3478 \ udp://192.0.2.2:53 ``` TLS/DTLS should also work. Below `--insecure` allows `turncat` to accept self-signed TLS certificates and `--verbose` is equivalent to setting all loggers to DEBUG mode (`-l all:DEBUG`). ```console ./turncat --verbose --insecure udp://127.0.0.1:5000 \ turn://test:test@192.0.2.1:3478?transport=tls udp://192.0.2.2:53 ``` Alternatively, you can specify the special TURN server meta-URI `k8s://stunner/udp-gateway:udp-listener` to let `turncat` parse the running STUNner configuration from the active Kubernetes cluster. The URI directs `turncat` to read the config of the STUNner Gateway called `udp-gateway` in the `stunner` namespace and connect to the TURN listener named `udp-listener`. The CLI flag `-` instructs `turncat` to listen on the standard input: anything you type in the terminal will be sent via STUNner to the peer `udp://10.0.0.1:9001` (after you press Enter). The CLI flag `-v` will enable verbose logging. ```console ./turncat -v - k8s://stunner/udp-gateway:udp-listener udp://10.0.0.1:9001 ``` Note that the standard `kubectl` command line flags are available. For instance, the below will use the context `prod-europe` from the kubeconfig file `kube-prod.conf`: ```console ./turncat --kubeconfig=kube-prod.conf --context prod-europe -v - k8s://... udp://... ``` Copyright 2021-2023 by its authors. Some rights reserved. See . MIT License - see for full text. Initial code adopted from and .
Project: Hango (Orchestration & Management / API Gateway)
File: bug_report.md

name: Bug report about: Create a report to help us improve title: '' labels: '' assignees: '' Describe the bug A clear and concise description of what the bug is. To Reproduce Steps to reproduce the behavior: kube apply '...' curl '....' See error Expected behavior A clear and concise description of what you expected to happen. Additional context Add any other context about the problem here, e.g. Kubernetes version OS
Project: hami (Orchestration & Management / Scheduling & Orchestration)
File: protocol.md

<img src="./imgs/protocol_register.png" width = "600" /> HAMi needs to know the spec of each AI devices in the cluster in order to schedule properly. During device registration, device-plugin needs to keep patching the spec of each device into node annotations every 30 seconds, in the format of the following: ``` hami.io/node-handshake-{device-type}: Reported{devicenodecurrenttimestamp} hami.io/node-register-{deivce-type}: {Device 1}:{Device2}:...:{Device N} ``` The definiation of each device is in the following format: ``` {Device UUID},{device split count},{device memory limit},{device core limit},{device type},{device numa},{healthy} ``` An example is shown below: ``` hami.io/node-handshake-nvidia: Reported 2024-01-23 04:30:04.434037031 +0000 UTC m=+1104711.777756895 hami.io/node-handshake-mlu: Requesting_2024.01.10 04:06:57 hami.io/node-mlu-register: MLU-45013011-2257-0000-0000-000000000000,10,23308,0,MLU-MLU370-X4,0,false:MLU-54043011-2257-0000-0000-000000000000,10,23308,0, hami.io/node-nvidia-register: GPU-00552014-5c87-89ac-b1a6-7b53aa24b0ec,10,32768,100,NVIDIA-Tesla V100-PCIE-32GB,0,true:GPU-0fc3eda5-e98b-a25b-5b0d-cf5c855d1448,10,32768,100,NVIDIA-Tesla V100-PCIE-32GB,0,true: ``` In this example, this node has two different AI devices, 2 Nvidia-V100 GPUs, and 2 Cambircon 370-X4 MLUs Note that a device node may become unavailable due to hardware or network failure, if a node hasn't registered in last 5 minutes, scheduler will mark that node as 'unavailable'. Since system clock on scheduler node and 'device' node may not align properly, scheduler node will patch the following device node annotations every 30s ``` hami.io/node-handshake-{device-type}: Requesting{schedulernodecurrenttimestamp} ``` If hami.io/node-handshake annotations remains in "Requesting_xxxx" and {scheduler current timestamp} > 5 mins + {scheduler timestamp in annotations}, then this device on that node will be marked "unavailable" in scheduler. <img src="./imgs/protocol_pod.png" width = "400" /> HAMi scheduler needs to patch schedule decisions into pod annotations, in the format of the following: ``` hami.io/devices-to-allocate:{ctr1 request}:{ctr2 request}:...{Last ctr request}: hami.io/device-node: {schedule decision node} hami.io/device-schedule-time: {timestamp} ``` each container request is in the following format: ``` {device UUID},{device type keywork},{device memory request}:{device core request} ``` for example: A pod with 2 containers, first container requests 1 GPU with 3G device Memory, second container requests 1 GPU with 5G device Memory, then the patched annotations will be like the ``` hami.io/devices-to-allocate: GPU-0fc3eda5-e98b-a25b-5b0d-cf5c855d1448,NVIDIA,3000,0:GPU-0fc3eda5-e98b-a25b-5b0d-cf5c855d1448,NVIDIA,5000,0: hami.io/vgpu-node: node67-4v100 hami.io/vgpu-time: 1705054796 ```
Project: Karmada (Orchestration & Management / Scheduling & Orchestration)
File: clusteroverridepolicy_test.md

| Test Case | E2E Describe Text | Comments | |--||--| | Check if the ClusterOverridePolicy will update the namespace label value | Namespace labelOverride testing | | | Test Case | E2E Describe Text | Comments | |--|-|-| | Check if the ClusterOverridePolicy with nil resourceSelector will update the deployment image value | deployment imageOverride testing | |
Project: Capsule (Orchestration & Management / Scheduling & Orchestration)
File: oidc-keycloak.md

Keycloak realm for Rancher Rancher OIDC authentication provider These instructions is specific to a setup made with Keycloak as an OIDC identity provider. Add to userinfo Group Membership type, claim name `groups` Add to userinfo Audience type, claim name `client audience` Add to userinfo, full group path, Group Membership type, claim name `fullgrouppath` More on this on the . Configure an OIDC authentication provider, with Client with issuer, return URLs specific to the Keycloak setup. Use old and Rancher-standard paths with `/auth` subpath (see issues below). Add custom paths, remove `/auth` subpath in return and issuer URLs. In Rancher, configure OIDC authentication with Keycloak to use . In Keycloak, Create a Group in the rancher Realm: capsule.clastix.io. In Keycloak, Create a User in the rancher Realm, member of capsule.clastix.io Group. In the Kubernetes target cluster, update the `CapsuleConfiguration` by adding the `"keycloakoidc_group://capsule.clastix.io"` Kubernetes `Group`. Login to Rancher with Keycloak with the new user. In Rancher as an administrator, set the user custom role with `get` of Cluster. In Rancher as an administrator, add the Rancher user ID of the just-logged in user as Owner of a `Tenant`. (optional) configure `proxySettings` for the `Tenant` to enable tenant users to access cluster-wide resources.
Project: APISIX (Orchestration & Management / API Gateway)
File: ip-restriction.md

title: ip-restriction keywords: Apache APISIX API Gateway Plugin IP restriction ip-restriction description: This document contains information about the Apache APISIX ip-restriction Plugin. <!-- --> The `ip-restriction` Plugin allows you to restrict access to a Service or a Route by either whitelisting or blacklisting IP addresses. Single IPs, multiple IPs or even IP ranges in CIDR notation like `10.10.10.0/24` can be used. | Name | Type | Required | Default | Valid values | Description | |--||-||--|-| | whitelist | array[string] | False | | | List of IPs or CIDR ranges to whitelist. | | blacklist | array[string] | False | | | List of IPs or CIDR ranges to blacklist. | | message | string | False | "Your IP address is not allowed" | [1, 1024] | Message returned when the IP address is not allowed access. | :::note Either one of `whitelist` or `blacklist` attribute must be specified. They cannot be used together. ::: You can enable the Plugin on a Route or a Service as shown below: :::note You can fetch the `admin_key` from `config.yaml` and save to an environment variable with the following command: ```bash adminkey=$(yq '.deployment.admin.adminkey[0].key' conf/config.yaml | sed 's/"//g') ``` ::: ```shell curl http://127.0.0.1:9180/apisix/admin/routes/1 -H "X-API-KEY: $admin_key" -X PUT -d ' { "uri": "/index.html", "upstream": { "type": "roundrobin", "nodes": { "127.0.0.1:1980": 1 } }, "plugins": { "ip-restriction": { "whitelist": [ "127.0.0.1", "113.74.26.106/24" ] } } }' ``` To return a custom message when an IP address is not allowed access, configure it in the Plugin as shown below: ```json "plugins": { "ip-restriction": { "whitelist": [ "127.0.0.1", "113.74.26.106/24" ], "message": "Do you want to do something bad?" } } ``` After you have configured the Plugin as shown above, when you make a request from the IP `127.0.0.1`: ```shell curl http://127.0.0.1:9080/index.html -i ``` ```shell HTTP/1.1 200 OK ... ``` But if you make requests from `127.0.0.2`: ```shell curl http://127.0.0.1:9080/index.html -i --interface 127.0.0.2 ``` ``` HTTP/1.1 403 Forbidden ... {"message":"Your IP address is not allowed"} ``` To change the whitelisted/blacklisted IPs, you can update the Plugin configuration. The changes are hot reloaded and there is no need to restart the service. ```shell curl http://127.0.0.1:9180/apisix/admin/routes/1 -H "X-API-KEY: $admin_key" -X PUT -d ' { "uri": "/index.html", "upstream": { "type": "roundrobin", "nodes": { "127.0.0.1:1980": 1 } }, "plugins": { "ip-restriction": { "whitelist": [ "127.0.0.2", "113.74.26.106/24" ] } } }' ``` To remove the `ip-restriction` Plugin, you can delete the corresponding JSON configuration from the Plugin configuration. APISIX will automatically reload and you do not have to restart for this to take effect. ```shell curl http://127.0.0.1:9180/apisix/admin/routes/1 -H "X-API-KEY: $admin_key" -X PUT -d ' { "uri": "/index.html", "plugins": {}, "upstream": { "type": "roundrobin", "nodes": { "127.0.0.1:1980": 1 } } }' ```
Project: SpiceDB (App Definition and Development / Database)
File: README.md

This directory contains the definitions of [Protocol Buffers] used by SpiceDB. [Buf] is used to lint and distribute these definitions and generate source code from them. The `version` field found in various buf YAML configuration is actually schema of the YAML of the file and is not related to the version of the definitions. `buf build` and `buf generate` do entirely different things. Building compiles definitions and ensures semantic validity. Generate builds and then produces actual source code according to `buf.gen.yaml`. All of the following commands can be done at the root of the project thanks to the `buf.work.yaml` file. Edit the various `.proto` definition files and run the following to validate them: ```sh buf build && buf lint ``` Once valid and ready to be used in source code, you can generate code by executing `buf.gen.yaml`: ```sh ./buf.gen.yaml ```