cwielenb/CNCF-Dataset huggingface dataset:
Project: Volcano (Orchestration & Management / Scheduling & Orchestration)
File: how_to_use_job_ttl.md

Similar to a standard `Job` resource, VolcanoJobs can be configured to be automatically garbage collected after they finish execution (either Complete or Failed). This is configured by setting `spec.ttlSecondsAfterFinished` which limits the lifetime of a Job. `ttlSecondsAfterFinished` is an optional parameter that can be configured on VolcanoJobs which defaults to `nil`. The value of `ttlSecondsAfterFinished` must be a positive integer and indicates the number of seconds after a job finishes executing (either Complete or Failed) before it becomes eligible for garbage collection. If `ttlSecondsAfterFinished` is unset or set to `nil`, the job will remain indefinitely. If set to zero (`0`), the job will become eligible for garbage collection immediately upon completion. If set to a positive integer, `N`, the job will become eligible for garbage collection `N` seconds after the job has completed. While this uses a custom garbage collector, this operates nearly identically to `ttlSecondsAfterFinished` from a standard `batch.v1.job` resource. The [official Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/controllers/ttlafterfinished/) has some useful tips describing how mutating webhooks can be used to take greater advantage of `ttlSecondsAfterFinished`. The manifest below creates a job that will be eligible for garbage collection 10 minutes after it either completes or fails. ```yaml apiVersion: batch.volcano.sh/v1alpha1 kind: job metadata: generateName: test-job- spec: minAvailable: 1 schedulerName: volcano queue: testing ttlSecondsAfterFinished: 600 policies: event: PodEvicted action: RestartJob tasks: replicas: 1 name: sleeper policies: event: TaskCompleted action: CompleteJob template: spec: restartPolicy: Never imagePullPolicy: IfNotPresent containers: name: sleeper image: debian:buster command: /bin/bash -c | for i in {0..5}; do echo "sleeping" sleep 1 done ```
Project: kcp (Orchestration & Management / Scheduling & Orchestration)
File: storage-to-rest-patterns.md

kcp promises to support 1 mln of workspaces. A workspace is like a Kubernetes endpoint, i.e. an endpoint usual Kubernetes client tooling (client-go, controller-runtime and others) and user interfaces (kubectl, helm, web console, ...) can talk to, just like to a Kubernetes cluster. Thus creating a workspace must be efficient, both in terms of storage and compute. It also must provide isolation, just like regular clusters. etcd is the primary datastore used by kcp. It stores data in a key-value store. The stores logical view is a flat binary key space. The key space has a lexically sorted index on byte string keys. In order to create a logical hierarchy, keys are usually mixed with `/` e.g. `/company/branch/location` To make it more concrete lets create a company acme with two branches, marketing, and sales in some locations. ``` etcdctl put /acme/marketing/london foo etcdctl put /acme/marketing/gdansk foo2 etcdctl put /acme/sales/warszawa foo3 ``` In order to get all branches of acme company, all we have to do is to ask the database to return all keys matching `/acme` prefix ``` etcdctl get --prefix /acme /acme/marketing/gdansk foo2 /acme/marketing/london foo /acme/sales/warszawa foo3 ``` In order to see all locations of marketing teams, we as for records matching /acme/marketing prefix ``` etcdctl get --prefix /acme/marketing /acme/marketing/gdansk foo2 /acme/marketing/london foo ``` Note that those queries are based on byte comparisons. We didn't create an `/acme` key. Everything matching the prefix will be returned when using the --prefix parameter. This is the key idea behind workspaces in kcp. Creating a workspace on the storage layer is very efficient because it boils down to concatenating a string. It also provides isolation on the lowest possible level. Data is filtered by the database engine. kcp is based on the generic apiserver library provided by Kubernetes. The central type provided by the library that interacts with a storage layer is called the generic registry. It connects an API endpoint (REST) with a database layer. Almost all API types make use of it. The generic apiserver library keeps an in-memory representation of the store for each resource in an API group. For example, `secrets` resources in the `core` API group gets their own storage. From the perspective of this document, we can assume that the most important feature of generic storage is to compute the key that is passed to the database to find the data the user wants. When the server starts it precomputes the `ResourcePrefix` with a group and a resource name. Everything else, like a workspace name, is added dynamically. For example, a request with a URL of `/clusters/acme/core/secrets` finds a storage responsible for `core/secrets` resources and adds `acme` to the `ResourcePrefix`. The new string becomes a key that is passed to etcd to find only resources for `acme` cluster.
Project: Knative (Orchestration & Management / Scheduling & Orchestration)
File: knative-encryption.md

Warning: not everything is implemented, as Knative Serving encryption is still a work-in-progress. Tracking project: https://github.com/orgs/knative/projects/63/views/1 There are three parts to Knative Serving encryption (1) HTTPS on the ingress layer external to the cluster (cluster external domain, like `myapp-<namespace>.example.com`). (2) HTTPS on the ingress layer internal to the cluster (cluster local domains, like `myapp.<namespace>.svc.cluster.local`). (3) HTTPS between Knative internal components (`ingress-controller`, `activator`, `queue-proxy`). Note: currently all control-plane traffic (including Kubernetes PreStopHooks and metadata like metrics) are not (yet) encrypted. The different parts are independent of each other and (can) use different Certificate Authorities to sign the necessary certificates. Certificate CN/SAN contains the external domain of a Knative Service, e.g. `myapp-<namespace>.example.com`. The certificates are hosted using SNI by the external endpoint of the ingress-controller. The caller has to trust the (external) CA that signed the certificates (this is out of the scope of Knative). These certificates are either or by using an implementation to the . Currently, we support the implementation. Please refer to the for more information. Certificate CN/SAN contains the cluster-local domain of a Knative Service, e.g. `myapp.namespace.svc.cluster.local`, `myapp.namespace.svc`, `myapp.namespace`. The certificates are hosted using SNI by the cluster-local endpoint of the ingress-controller. The caller has to trust the CA that signed the certificates (this is out of the scope of Knative). One option to do this is using from cert-manager. To create the certificates, Knative relies on and our bridging component . They need to be installed and configured for the feature to work. Knative internal system components (`ingress-controller`, `activator`, `queue-proxy`) are hosting TLS endpoints when this configuration is enabled. To get the certificates, Knative relies on and our bridging component . They need to be installed and configured for the feature to work. Specific SANs are used to verify each connection. Each component needs to trust the CA (possibly the full chain) that signed the certificates. For this, Knative system components will consume and trust a provided `CABundle`. The CA bundle needs to be provided by the cluster administrator, possibly using from cert-manager.
Project: Istio (Orchestration & Management / Service Mesh)
File: REAME.md

This sample demonstrates Istio's Open Telemetry and sending logs to . Run the following script to install Istio with an Open Telemetry ALS provider: ```bash istioctl install -f iop.yaml -y ``` Run the following script to install `Loki`: ```bash kubectl apply -f ../../addons/loki.yaml -n istio-system ``` First, create an `otel-collector` backend with a simple configuration. ```bash kubectl apply -f otel.yaml -n istio-system ``` With the following configuration, otel-collector-contrib will create a grpc receiver on port `4317`, and output to stdout. You can find more details . ```yaml receivers: otlp: protocols: grpc: http: processors: batch: attributes: actions: action: insert key: loki.attribute.labels value: podName, namespace,cluster,meshID exporters: loki: endpoint: "http://loki.istio-system.svc:3100/loki/api/v1/push" logging: loglevel: debug extensions: health_check: service: extensions: health_check pipelines: logs: receivers: [otlp] processors: [attributes] exporters: [loki, logging] ``` Next, add a Telemetry resource that tells Istio to send access logs to the OpenTelemetry collector. ```bash kubectl apply -f telemetry.yaml ``` Following this , start the `fortio` and `httpbin` services. Run the following script to request `httpbin` from `fortio`. ```bash kubectl exec -it deploy/fortio -- fortio curl httpbin:8000/ip ``` Run the following script to view ALS output. ```bash kubectl logs -l app=opentelemetry-collector -n istio-system --tail=-1 ``` You can also check the Grafana logs: ```bash istioctl dashboard grafana ``` Learn how to use Loki with Grafana . ```bash kubectl delete -f otel.yaml -n istio-system kubectl delete telemetry mesh-logging -n istio-system istioctl uninstall --purge -y ```
Project: Apache Mesos (Orchestration & Management / Scheduling & Orchestration)
File: disk-xfs.md

title: Apache Mesos - XFS Disk Isolator in Mesos Containerizer layout: documentation The `disk/xfs` isolator uses XFS project quotas to track the disk space used by each container sandbox and to enforce the corresponding disk space allocation. When quota enforcement is enabled, write operations performed by tasks exceeding their disk allocation will fail with an `EDQUOT` error. The task will not be terminated by the containerizer. To enable the XFS Disk isolator, append `disk/xfs` to the `--isolation` flag when starting the agent. The XFS Disk isolator supports the `--enforcecontainerdisk_quota` flag. If enforcement is enabled, the isolator will set both the hard and soft quota limit. Otherwise, no limits will be set, Disk usage accounting will be performed but the task will be allowed to exceed its allocation. The XFS Disk isolator requires the sandbox directory to be located on an XFS filesystem that is mounted with the `pquota` option. There is no need to configure or files. The range of project IDs given to the `--xfsprojectrange` must not overlap any project IDs allocated for other uses. The command can be used to show the current allocation of project IDs and quota. For example: $ xfs_quota -x -c "report -a -n -L 5000 -U 10000" To show which project a file belongs to, use the command to display the `fsxattr.projid` field. For example: $ xfs_io -r -c stat /mnt/mesos/ Project IDs are not reclaimed until the sandboxes they were assigned to are garbage collected. The XFS Disk isolator periodically checks if sandboxes of terminated containers still exist and deallocates project IDs of the ones that were removed. Such checks are performed at intervals specified by the flag. Current number of available project IDs and total number of project IDs used by the isolator can be tracked using `containerizer/mesos/disk/projectidsfree` and `containerizer/mesos/disk/projectidstotal` metrics. The XFS Disk isolator flag `--xfskillcontainers` will create container quotas that have a gap between the soft and hard limits. The soft limit is equal to the limit requested for the `disk` resource and the hard limit is 10MB higher. If a container violates the soft limit then it will be killed. The isolator polls for soft limit violations at the interval specified by the `--containerdiskwatch_interval` flag. Note that the `--containerdiskwatch_interval` flag only applies to the XFS Disk isolator when `--xfskillcontainers` is set to true.
Project: Envoy (Orchestration & Management / Service Proxy)
File: cache_filter_plugins.md

The HTTP Cache Filter handles most of the complexity of HTTP caching semantics, but delegates the actual storage of HTTP responses to implementations of the HttpCache interface. These implementations can cover all points on the spectrum of persistence, performance, and distribution, from local RAM caches to globally distributed persistent caches. They can be fully custom caches, or wrappers/adapters around local or remote open-source or proprietary caches. If you write a new cache storage implementation, please add it to the Envoy repository if possible. This is the only way to make sure it stays up-to-date with Envoy changes, and lets other developers contribute fixes and improvements. As you read this, also read the example implementation in `simplehttpcache.h/.cc`. You need to write implementations of four small interfaces: Example Implementation: `SimpleHttpCache` `HttpCache` represents an actual cache of responses stored somewhere. It provides methods to set up cache lookups and inserts, and to update the headers of cached responses. Example Implementation: `SimpleHttpCacheFactory` `HttpCacheFactory` does what it sounds like: it creates HttpCache implementations, based on a name that came from the cache filter's config. Example Implementation: `SimpleLookupContext` `LookupContext` represents a single lookup operation; this is a good place to store whatever per-lookup state you may need during the lookup process. Example Implementation: `SimpleInsertContext` `LookupContext` represents a single insert operation; this is a good place to store whatever per-insert state you may need during the lookup process. To initiate a lookup on an HttpCache implementation, the cache filter calls `HttpCache::makeLookupContext`, which should return a `LookupContextPtr`. The cache filter will call `LookupContext::getHeaders` to find out if there's a cached response. If a result is found, the `LookupContext` implementation must call `LookupRequest::makeLookupResult`, and pass the result to the callback. The cache filter will then make a series of `getBody` requests followed by `getTrailers` (if needed). If the `LookupResult` in the callback indicates that a response wasn't found, the cache filter will let the request pass upstream. If the origin replies with a cacheable response, the filter will call `HttpCache::makeInsertContext`, and use its methods to insert the response. The following diagram shows a potential GET request for a 5M resource that is present and fresh in the cache, with no trailers. In the case of a synchronous in-memory cache, this all happens within `CacheFilter::decodeHeaders`. Solid arrows denote synchronous function calls, while dashed arrows denote asynchronous function calls or their callbacks. Objects that are part of the cache implementation (`HttpCache` and `LookupContext`) are blue. (Other objects are part of the cache filter, or of Envoy.
Project: gRPC (Orchestration & Management / Remote Procedure Call)
File: perf_notes.md

Generally, each write operation (Write(), WritesDone()) implies a syscall. gRPC will try to batch together separate write operations from different threads, but currently cannot automatically infer batching in a single stream. If message k+1 in a stream does not rely on responses from message k, it's possible to enable write batching by passing a WriteOptions argument to Write with the buffer_hint set: ~~~{.cpp} streamwriter->Write(message, WriteOptions().setbuffer_hint()); ~~~ The write will be buffered until one of the following is true: the per-stream buffer is filled (controllable with the channel argument GRPCARGHTTP2WRITEBUFFER_SIZE) - this prevents infinite buffering leading to OOM a subsequent Write without buffer_hint set is posted the call is finished for writing (WritesDone() called on the client, or Finish() called on an async server stream, or the service handler returns for a sync server stream) Right now, the best performance trade-off is having numcpu's threads and one completion queue per thread.
Project: Open Cluster Management (Orchestration & Management / Scheduling & Orchestration)
File: fuzzy_mode_convert_table.md

| json type \ dest type | bool | int | uint | float |string| | | | | |--|--| | number | positive => true <br/> negative => true <br/> zero => false| 23.2 => 23 <br/> -32.1 => -32| 12.1 => 12 <br/> -12.1 => 0|as normal|same as origin| | string | empty string => false <br/> string "0" => false <br/> other strings => true | "123.32" => 123 <br/> "-123.4" => -123 <br/> "123.23xxxw" => 123 <br/> "abcde12" => 0 <br/> "-32.1" => -32| 13.2 => 13 <br/> -1.1 => 0 |12.1 => 12.1 <br/> -12.3 => -12.3<br/> 12.4xxa => 12.4 <br/> +1.1e2 =>110 |same as origin| | bool | true => true <br/> false => false| true => 1 <br/> false => 0 | true => 1 <br/> false => 0 |true => 1 <br/>false => 0|true => "true" <br/> false => "false"| | object | true | 0 | 0 |0|originnal json| | array | empty array => false <br/> nonempty array => true| [] => 0 <br/> [1,2] => 1 | [] => 0 <br/> [1,2] => 1 |[] => 0<br/>[1,2] => 1|original json|
Project: Gloo (Orchestration & Management / API Gateway)
File: proxy_endpoint.proto.sk.md

title: "proxy_endpoint.proto" weight: 5 <!-- Code generated by solo-kit. DO NOT EDIT. --> ```yaml "namespace": string "name": string "selector": map<string, string> "source": string ``` | Field | Type | Description | | -- | - | -- | | `namespace` | `string` | Optional. The namespace to look for proxies. If this is omitted, all namespaces will be considered. | | `name` | `string` | Optional. The name of the proxy to look up. If this is provided, a namespace must be included as well. | | `selector` | `map<string, string>` | Optional. Selector to use to filter returned proxies. This will be ignored if a name is provided. | | `source` | `string` | Optional. The translator source that produced the requested Proxy. Available values are: 1. edge-gw: If the Proxy was created using the Edge Gateway resources 2. k8s-gw: If the Proxy was created using the K8s Gateway resources If this is omitted, all sources will be considered. | ```yaml "proxies": []gloo.solo.io.Proxy ``` | Field | Type | Description | | -- | - | -- | | `proxies` | | List of proxies returned from the gloo pod. | <!-- Start of HubSpot Embed Code --> <script type="text/javascript" id="hs-script-loader" async defer src="//js.hs-scripts.com/5130874.js"></script> <!-- End of HubSpot Embed Code -->
Project: Contour (Orchestration & Management / Service Proxy)
File: deploy-aws-nlb.md

title: Deploying Contour on AWS with NLB This is an advanced deployment guide to configure Contour on AWS with the . This configuration has several advantages: NLBs are often cheaper. This is especially true for development. Idle LBs do not cost money. There are no extra network hops. Traffic goes to the NLB, to the node hosting Contour, and then to the target pod. Source IP addresses are retained. Envoy (running as part of Contour) sees the native source IP address and records this with an `X-Forwarded-For` header. We run Envoy as a DaemonSet across the cluster and Contour as a deployment The Envoy pod runs on host ports 80 and 443 on the node Host networking means that traffic hits Envoy without transitioning through any other fancy networking hops Contour also binds to 8001 for Envoy->Contour config traffic. and cd into the repo Edit the Envoy service (`02-service-envoy.yaml`) in the `examples/contour` directory: Remove the existing annotation: `service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp` Add the following annotation: `service.beta.kubernetes.io/aws-load-balancer-type: nlb` Run `kubectl apply -f examples/contour` This creates the `projectcontour` Namespace along with a ServiceAccount, RBAC rules, Contour Deployment and an Envoy DaemonSet. It also creates the NLB based loadbalancer for you. You can get the address of your NLB via: ``` $ kubectl get service envoy --namespace=projectcontour -o jsonpath='{.status.loadBalancer.ingress[0].hostname}' ``` You can now test your NLB. Install a workload (see the kuard example in the ). Look up the address for your NLB in the AWS console and enter it in your browser. Notice that Envoy fills out `X-Forwarded-For`, because it was the first to see the traffic directly from the browser.
Project: KEDA (Orchestration & Management / Scheduling & Orchestration)
File: ROADMAP.md

KEDA uses to track open work items with to plan for upcoming releases. This document provides insights to the community on how we use it and what to expect. You can find our roadmap . As part of our , we are planning to release a new KEDA version every 3 months and ship on Thursdays. Here is an overview of our current release estimations: | Version | Estimated Release Date | |:--|:--| | v2.15 | July 9th, 2024 | | v2.16 | Oct 3rd, 2024 | | v2.17 | Jan 9th, 2025 | Here is an overview of our previous releases: | Version | Release Date | Links | |:--|:|:--| | v2.14 | April 25th, 2024 | | | v2.13 | Jan 19th, 2024 | | | v2.12 | Sept 28th, 2023 | | | v2.11 | June 22nd, 2023 | | | v2.10 | March 9th, 2023 | | | v2.9 | Dec 9th, 2022 | | | v2.8 | August 10th, 2022 | | | v2.7 | May 5th, 2022 | | Here is some guidance on how to use our roadmap. As we work towards our next release, we are planning and tracking work as part of the next release cycle. You can find an overview of the items in our upcoming release: As list with the respective categories () As a board with the current status () All newly created issues are automatically added to the roadmap and waiting to be triaged by a maintainer. You can find an overview of all issues pending to be triaged .
Project: Gloo (Orchestration & Management / API Gateway)
File: context_params.proto.sk.md

title: "context_params.proto" weight: 5 <!-- Code generated by solo-kit. DO NOT EDIT. --> Additional parameters that can be used to select resource variants. These include any global context parameters, per-resource type client feature capabilities and per-resource type functional attributes. All per-resource type attributes will be `xds.resource.` prefixed and some of these are documented below: `xds.resource.listening_address`: The value is "IP:port" (e.g. "10.1.1.3:8080") which is the listening address of a Listener. Used in a Listener resource query. ```yaml "params": map<string, string> ``` | Field | Type | Description | | -- | - | -- | | `params` | `map<string, string>` | | <!-- Start of HubSpot Embed Code --> <script type="text/javascript" id="hs-script-loader" async defer src="//js.hs-scripts.com/5130874.js"></script> <!-- End of HubSpot Embed Code -->