cwielenb/CNCF-Dataset huggingface dataset:
Project: Volcano (Orchestration & Management / Scheduling & Orchestration)
File: pdb-plugin.md

When users apply jobs to volcano, they may limit the number of Pod copies that are destroyed at the same time. This restriction is often limited by the created by the user. So PDB plugin is provided to meet the PBD constraints set by the user during the Volcano scheduling process. Firstly, please ensure that your k8s version is at least 1.21 and that you have created the PDB resource. Secondly, user can open `PDB Plugin` via `volcano-scheduler-configmap` in following format: ```yaml actions: "reclaim, preempt, shuffle" tiers: plugins: name: pdb ``` PDB Plugin can be used in any one action of reclaim, preempt, shuffle The plugin register , `ReclaimableFn`, `PreemptableFn` and `VictimTasksFn` in `Reclaim`, `Preempt` and `Shuffle` action to filter out tasks that violate the PDB constraint. The execution flow chart of filter process of `pdb` plugin is shown as below: `DisruptedPods` contains information about pods whose eviction was processed by the API server eviction subresource handler but has not yet been observed by the PodDisruptionBudget controller. The PDB plugin first registers `informerFactory.Policy().V1().PodDisruptionBudgets()` in `newSchedulerCache()` function. Then the plugin adds `v1.PodDisruptionBudgetLister` variable to type definition and inits it with `informerFactory.Policy().V1().PodDisruptionBudgets().Lister()` in `New()` function. Finally the plugin uses this `Lister` to list the PDBs from the informer cache. By now we only need 1 argument `sla-waiting-time`, so I add it into annotations for simplicity and invocation, but when `sla` plugin is extended with more arguments, a better way to invoke this plugin may be job plugin like `svc` and `ssh`.
Project: BFE (Orchestration & Management / Service Proxy)
File: capture_packet.md

Use packet capture and analysis tools to locate and analyze complex network problems tcpdump example ```bash ``` Use wireshark to open packet capture file. For TLS-based encrypted traffic, you can use modkeylog and wireshark for analysis. Step1: Enable modkeylog module and save the TLS session key to key.log file Notemodify bfe.conf and enable modkeylog, See module configuration for details ```ini [Server] Modules = modkeylog ``` Step2: Set the path of the Master-Secret log file to key.log in wireshark NoteEditPreferencesProtocolsSSL(Pre)-Master-Secret log filename Step3: Use wireshark to open and decrypt the captured data
Project: Kuma (Orchestration & Management / Service Mesh)
File: 002-psat.md

Status: accepted Technical Story: https://github.com/kumahq/kuma/issues/4121 Kuma uses Kubernetes service account tokens (SAT) to identify data plane proxies to the control plane during bootstrapping. These tokens have no expiry. If the user wants to rotate the token, Kuma won't reload the newer token for authentication and will use the old one to reconnect. Currently, `kuma-dp` loads the token only once at the startup and does not detect the token's change. That is the problem and we want to support the case where the token is rotated and later used for authentication with control-plane. Use GoogleGRPC configuration in Envoy Implement support for reading token from files in EnvoyGRPC Chosen option: "Use GoogleGRPC configuration in Envoy", because it works and does not require additional implementation in Envoy which would do the same. At the beginning we weren't sure if it works because of , but after validation, it seems that the problem described there doesn't occur anymore. should we add grpc connection timeout ? (if we are keeping the connection `forever/to the failure` we won't use token to reauthenticate) support for rotation token won't be visible in config_dump, only file path if we add connection timeout we might observe some logs about breaking connection `kuma-dp` allows providing a dataplane token or path to a file that has the token. When the user provides a path to the token, `kuma-dp` reads that token and adds it to the bootstrap request. To support rotation we are going to send both the token and path to the control plane in the bootstrap request. By default, we are going to use the current solution which uses `envoyGrpc` and `initial_metdata` for authentication. Token rotation needs to be enabled in the control-plane by setting `dpServer.auth.useTokenPath`. When the property is enabled, `kuma-cp` adds a path to the token to the Envoy configuration. In case there is no path, the `Envoy` will receive a configuration that doesn't support reloading. Envoy configuration: ```json "ads_config": { "api_type": "GRPC", "grpc_services": [ { "google_grpc": { "targeturi": "<kumacp_hostname>:5678", "channel_credentials": { "ssl_credentials": { "root_certs": { ... } } }, "call_credentials": [ { "from_plugin": { "name": "envoy.grpccredentials.filebased_metadata", "typed_config": { "@type": "type.googleapis.com/envoy.config.grpc_credential.v3.FileBasedMetadataConfig", "secret_data": { "filename": "<tokenfilepath>" } } } } ], ... } } ], ... } ``` Does not require implementation in Envoy Safe to switch from one to another model, in case of an error we can disable The flag adds some complexity (we can eventually deprecate and remove the flag) We would have to implement support for reading file values in EnvoyGRPC, which is already supported by GoogleGRPC. Already supported by GoogleGRPC https://github.com/kumahq/kuma/issues/4121 https://github.com/envoyproxy/envoy/issues/15380
Project: BFE (Orchestration & Management / Service Proxy)
File: mod_errors.md

mod_errors replaces error responses based on specified rules. conf/moderrors/moderrors.conf | Config Item | Description | | | - | | Basic.DataPath | String<br>Path for rule configuration | | Log.OpenDebug | Boolean<br>Whether enable debug logs<br>Default False | ```ini [Basic] DataPath = moderrors/errorsrule.data ``` | Config Item | Description | | -- | - | | Version | String<br>Version of config file | | Config | Object<br>Error rules for each product | | Config{k} | String<br>Product name | | Config{v} | Object<br> A list of error rules | | Config{v}[] | Object<br>A error rule | | Config{v} | | Config{v}[].Actions | Object<br>Action | | Config{v}[].Actions.Cmd | String<br>Name of Action | | Config{v}[].Actions.Params | Object<br>Parameters of Action | | Config{v}[].Actions.Params[] | String<br>A Parameter | | Action | Description | | -- | - | | RETURN | Return response generated from specified static html | | REDIRECT | Redirect to specified location | ```json { "Version": "20190101000000", "Config": { "example_product": [ { "Cond": "rescodein(\"404\")", "Actions": [ { "Cmd": "RETURN", "Params": [ "200", "text/html", "../conf/mod_errors/404.html" ] } ] }, { "Cond": "rescodein(\"500\")", "Actions": [ { "Cmd": "REDIRECT", "Params": [ "http://example.org/error.html" ] } ] } ] } } ```
Project: wasmCloud (Orchestration & Management / Scheduling & Orchestration)
File: 0011-split-rpc-events.md

Allow all cloud events to remain published on `wasmbus.evt.{lattice}` but move the publication of invocation success and failure to a separate topic, `wasmbus.rpcevt.{lattice}`. There are two problems with the status quo, one can causes crashes while the other is one of organization/optimization. The first issue is a fairly niche case. If an actor is using the `wasmcloud:messaging` capability provider to subscribe to a `wasmbus.evt.` topic, then each time the capability provider delivers (or fails to) a message to the actor, the host emits an _invocation succeeded/failed_ event, also on `wasmbus.evt.`. This will then trigger that same actor to receive the invocation event, which will in turn emit another invocation event, and so on until infinity or the host crashes. To avoid this death spiral, we need to not emit invocation events on the same topic as the other events. The second issue is just one of organization and traffic optimization. In a steady state lattice, invocation events will happen exponentially more than any of the other ones (followed closely by heartbeats). If we're not interested in the high-traffic events, our code/actor shouldn't have to manually filter them out and can instead bypass them with a subscription wildcard. Despite the limited nature of the main use case, this will cause a crash Optimization and efficiency of consumers of lattice events Split event publication so RPC (invocation) events are published on `wasmbus.rpcevt.` while all other events stay on `wasmbus.evt.` No crashes More optimized consumer flow
Project: Avro (Orchestration & Management / Remote Procedure Call)
File: avro-1.7.4-released.md

title: "Avro 1.7.4" linkTitle: "Avro 1.7.4" date: 2013-02-26 <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> Apache Avro 1.7.4 has been released!
Project: Volcano (Orchestration & Management / Scheduling & Orchestration)
File: how_to_use_pytorch_plugin.md

Pytorch plugin is designed to optimize the user experience when running pytorch jobs, it not only allows users to write less yaml, but also ensures the normal operation of Pytorch jobs. The Pytorch Plugin will do three things: Open ports used by Pytorch for all containers of the job Force open `svc` plugins Add some envs such like `MASTERADDR`, `MASTERPORT`, `WORLD_SIZE`, `RANK` which pytorch distributed training needed to containers automatically | ID | Name | Type | Default Value | Required | Description | Example | | - | | | - | -- | - | | | 1 | master | string | master | No | Name of Pytorch master | --master=master | | 2 | worker | string | worker | No | Name of Pytorch worker | --worker=worker | | 3 | port | string | 23456 | No | The port to open for the container | --port=23456 | ```yaml apiVersion: batch.volcano.sh/v1alpha1 kind: Job metadata: name: pytorch-job spec: minAvailable: 1 schedulerName: volcano plugins: pytorch: ["--master=master","--worker=worker","--port=23456"] # Pytorch plugin register tasks: replicas: 1 name: master policies: event: TaskCompleted action: CompleteJob template: spec: containers: image: gcr.io/kubeflow-ci/pytorch-dist-sendrecv-test:1.0 imagePullPolicy: IfNotPresent name: master restartPolicy: OnFailure replicas: 2 name: worker template: spec: containers: image: gcr.io/kubeflow-ci/pytorch-dist-sendrecv-test:1.0 imagePullPolicy: IfNotPresent name: worker workingDir: /home restartPolicy: OnFailure ```
Project: kcp (Orchestration & Management / Scheduling & Orchestration)
File: pull_request_template.md

<!-- Thanks for creating a pull request! If this is your first time, please make sure to review CONTRIBUTING.MD. Please copy the appropriate `:text:` or icon to the beginning of your PR title: :sparkles: feature :bug: bug fix :book: docs :memo: proposal :warning: breaking change :seedling: other/misc :question: requires manual review/categorization --> Fixes # <!-- Please add a release note in the block below. Leave NONE only if no user-facing changes are in this PR. --> ```release-note NONE ```
Project: hami (Orchestration & Management / Scheduling & Orchestration)
File: benchmark.md

Three instances from ai-benchmark have been used to evaluate vGPU-device-plugin performance as follows | Test Environment | description | | - | :: | | Kubernetes version | v1.12.9 | | Docker version | 18.09.1 | | GPU Type | Tesla V100 | | GPU Num | 2 | | Test instance | description | | - | :: | | nvidia-device-plugin | k8s + nvidia k8s-device-plugin | | vGPU-device-plugin | k8s + VGPU k8s-device-pluginwithout virtual device memory | | vGPU-device-plugin(virtual device memory) | k8s + VGPU k8s-device-pluginwith virtual device memory | Test Cases: | test id | case | type | params | | - | :--: | :-: | :: | | 1.1 | Resnet-V2-50 | inference | batch=50,size=346*346 | | 1.2 | Resnet-V2-50 | training | batch=20,size=346*346 | | 2.1 | Resnet-V2-152 | inference | batch=10,size=256*256 | | 2.2 | Resnet-V2-152 | training | batch=10,size=256*256 | | 3.1 | VGG-16 | inference | batch=20,size=224*224 | | 3.2 | VGG-16 | training | batch=2,size=224*224 | | 4.1 | DeepLab | inference | batch=2,size=512*512 | | 4.2 | DeepLab | training | batch=1,size=384*384 | | 5.1 | LSTM | inference | batch=100,size=1024*300 | | 5.2 | LSTM | training | batch=10,size=1024*300 | Test Result: To reproduce: install k8s-vGPU-schedulerand configure properly run benchmark job ``` $ kubectl apply -f benchmarks/ai-benchmark/ai-benchmark.yml ``` View the result by using kubctl logs ``` $ kubectl logs [pod id]
Project: Gloo (Orchestration & Management / API Gateway)
File: development.md

title: Development weight: 60 Developers can work on Gloo Edge in a number of different ways. You can contribute to Gloo Edge open-source, help with documentation, or extend Gloo Edge's functionality via the addition of plugins. You can contribute to the Gloo Edge open-source project by logging issues, generating PRs, or helping with documentation. More information about contributing to Gloo Edge can be found in our section. Gloo Edge invites developers to extend Gloo Edge's functionality and adapt to new use cases via the addition of plugins. Gloo Edge's plugin based architecture makes it easy to extend functionality in a variety of areas: : extensible through the use of along with : automatically discover service endpoints from catalogs such as and : annotate services with information discovered by polling services directly (such as OpenAPI endpoints and gRPC methods). : customize what happens to requests when they match a route or virtual host : customize what happens to requests when they are routed to a service Operators for Configuration: Gloo Edge exposes its intermediate language for proxy configuration via the Custom Resource, allowing operators to leverage Gloo Edge for multiple use cases. With the optional , you can set up API gateway and GraphQL server functionality for your apps, without running in the same process (or even the same container) as Gloo Edge. To get started with development around Gloo Edge, we recommend taking a look at our .
Project: Karmada (Orchestration & Management / Scheduling & Orchestration)
File: dependenciesdistributor_test.md

| Test Case | E2E Describe Text | Comments | |||--| | Test the configmap will propagate automatically | configmap automatically propagation testing | | | Test the secret will propagate automatically | secret automatically propagation testing | | | Test the PVC will propagate automatically | persistentVolumeClaim automatically propagation testing | | | Test the serviceAccount will propagate automatically | serviceAccount automatically propagation testing | | | Test the configmap will propagate automatically | configmap propagate automatically testing | |
Project: Contour (Orchestration & Management / Service Proxy)
File: CHANGELOG-v1.18.1.md

We are delighted to present version 1.18.1 of Contour, our layer 7 HTTP reverse proxy for Kubernetes clusters. As an additional mitigation for , has been backported to this release. Previously an ExternalName service with an address of `localhost` and port matching the Envoy admin interface listener port could expose admin endpoints that would allow an attacker to shut down Envoy remotely. Contour's bootstrap command now configures Envoy to listen on a Unix domain socket to ensure an ExternalName service cannot expose the "writable" endpoints of the admin interface. For backwards compatibility Contour now configures an additional Envoy listener to expose endpoints useful for debugging. This listener does not expose any endpoints that can be used to modify data, set Envoy healthcheck status, or shut down Envoy. See for some more detail. Note: While this fix does mitigate some aspects of CVE-2021-32783, ExternalName service usage is still discouraged as they can still be used to expose services across namespaces. If you have been using the Contour example YAMLs to deploy Contour and Envoy, no changes are required except to deploy the updated YAMLs for release 1.18.1. If you are managing your Envoy DaemonSet using another method, be sure to inspect the . An is required and it must be mounted in the and containers to ensure both have access to the Unix domain socket Envoy is now configured to listen on. Upgrades the default Envoy version to 1.19.1 for security and bug fixes. See the for more details.
Project: CoreDNS (Orchestration & Management / Coordination & Service Discovery)
File: coredns-1.0.5.md

+++ title = "CoreDNS-1.0.5 Release" description = "CoreDNS-1.0.5 Release Notes." tags = ["Release", "1.0.5", "Notes"] draft = false release = "1.0.5" date = "2018-01-25T11:10:29+00:00" author = "coredns" enabled = "default" +++ We are pleased to announce the of CoreDNS-1.0.5! This release has bug fixes, documentation fixes, polish and new plugins. Add ability to really compile out the default plugins. A new plugin route53* was added that enables serving zone data from AWS route53, see the . A new plugin on* was added. This is an external Caddy , that is now also available (by default) for CoreDNS; it allows you to run commands when an event is generated. cache doesn't apply a 5s minimum TTL anymore. It fixes prefetching and* correctly sets the metrics for cache hits and misses. dnssec fixes handing out expired* signatures after 8 days and properly filters out the qtype in the NSEC bitmap for NXDOMAIN responses. log* adds message ID `{>id}` to the default logging. health has gotten a lameduck option that will nack health, but will keep the server running for a configurable duration when CoreDNS is being shut down. If metrics are enabled health* exports a metric that curls the local endpoint and exports the duration. Useful for getting a sense of overloadedness of the process. rewrite* can now rewrite answers for `name regex` matches. This prevents DNS clients from ignoring the answers due to a mismatch with the original question. secondary* saw a bunch of fixes. The following people helped with getting this release done: Christian Nilsson, cricketliu, Francois Tur, Ilya Galimyanov, Miek Gieben, Paul Greenberg, Ruslan Drozhdzh, Tobias Schmidt, Yong Tang, Yue Ko. If you want to help, please check out one of the and start coding! For documentation and help, see our .
Project: Contour (Orchestration & Management / Service Proxy)
File: CHANGELOG-v1.17.2.md

We are delighted to present version 1.17.2 of Contour, our layer 7 HTTP reverse proxy for Kubernetes clusters. As an additional mitigation for , has been backported to this release. Previously an ExternalName service with an address of `localhost` and port matching the Envoy admin interface listener port could expose admin endpoints that would allow an attacker to shut down Envoy remotely. Contour's bootstrap command now configures Envoy to listen on a Unix domain socket to ensure an ExternalName service cannot expose the "writable" endpoints of the admin interface. For backwards compatibility Contour now configures an additional Envoy listener to expose endpoints useful for debugging. This listener does not expose any endpoints that can be used to modify data, set Envoy healthcheck status, or shut down Envoy. See for some more detail. Note: While this fix does mitigate some aspects of CVE-2021-32783, ExternalName service usage is still discouraged as they can still be used to expose services across namespaces. If you have been using the Contour example YAMLs to deploy Contour and Envoy, no changes are required except to deploy the updated YAMLs for release 1.17.2. If you are managing your Envoy DaemonSet using another method, be sure to inspect the . An is required and it must be mounted in the and containers to ensure both have access to the Unix domain socket Envoy is now configured to listen on. Upgrades the default Envoy version to 1.18.4 for security and bug fixes. See the for more details.
Project: Skipper (Orchestration & Management / Service Proxy)
File: SECURITY.md

We acknowledge that every line of code that we write may potentially contain security issues. We are trying to deal with it responsibly and provide patches as quickly as possible. We host our bug bounty program on HackerOne, it is currently private, therefore if you would like to report a vulnerability and get rewarded for it, please ask to join our program by filling this form: https://zeng.page.link/open-source-report-vulnerability You can also send you report via this form if your do not want to join our bug bounty program and just want to report a vulnerability or security issue. In the past there were issues with the process mentioned above. If you want to report a security vulnerability and face an issue, please reach out to the team of via e-mail.
Project: Gloo (Orchestration & Management / API Gateway)
File: code-generation.md

To run the full codegen process, use `make -B install-go-tools generated-code`. In most cases it is not necessary to run all sub-targets within `generated-code`. Here is a description of each sub-target and its purpose: | Target | Description | Use when... | Approximate runtime | |-|:-|:-|--| | `install-go-tools` | Invokes `mod-download` and `go install`s a number of tools used by `generated-code` | The `_output` dir is not present or you are otherwise unsure deps are installed | 3-8s (if mod download is up-to-date) | | `mod-download` | Calls `go mod download all` | Deps are not all present locally | 1s (if deps are already downloaded) | | `generated-code` | Generates all required code, cleaning and formatting as well; this target is executed in CI | You need to run all codegen steps without human error (ie prior to PR merge) | 4-5m | | `check-go-version` | Validates that local Go version matches go.mod | You need a sanity check | 1s | | `clean-solo-kit-gen` | Removes files generated by codegen other than docs and tests | An existing generated file may no longer be generated by codegen (rare) | 3s | | `go-generate-all` | Invokes all generate directives in the repo, most notably in `generate.go` which runs solo-kit codegen, and mockgen directives | There is an any API change | 3-5m | | `go-generate-apis` | Invokes the generate directive in `generate.go` which runs solo-kit codegen | There is a proto API change (prefer to use `generated-code-apis` which also formats) | 1m | | `go-generate-mocks` | Invokes all mockgen generate directives in the repo | There is an interface API change | 2:30-3m | | `generate-cli-docs` | Removes existing CLI docs and re-generates them | CLI code has been modified | 10-15s | | `generated-code-apis` | Generates and formats code from protos | There is a proto API change | 1:30s | | `generated-code-cleanup` | Executes cleanup and formatting targets | Preparing to open a PR without API changes | 30-40s | | `getter-check` | Ensures that accesses for fields which have "getter" functions are exclusively done via said "getter" functions | Any code has been written (just in case) | ~7s | | `mod-tidy` | Calls `go mod tidy` | Dependencies have been added, updated, or removed | 1-2s | | | `verify-enterprise-protos` | Validates that protos used in EE but not OSS are valid | Unsure whether protos used in EE are valid | 3-10s | | `generate-helm-files` | Generates helm chart and value files | The helm chart or values template has been modified | 1-3s | | `update-licenses` | Generates docs files containing attribution for all dependencies which require it | There is a new dependency or a depency bump | 7-12s | | `fmt` | Runs which updates imports and formats code | Code has been modified (any change, in case it's not properly formatted) | 9-13s |