cwielenb/CNCF-Dataset huggingface dataset:
Project: Fluid (Orchestration & Management / Scheduling & Orchestration)
File: ROADMAP.md

Support temporality through Kueue Once data migration is completed, run data preheating, triggering the running of machine learning tasks (such as tfjob, mpiJob, pytorchJob, sparkJob) After computation is completed, data migration and cache cleaning can be carried out Choose data access methods based on the scheduling results of the Kubernetes scheduler (default scheduler, Volcano, YuniKorn) If scheduled to ordinary nodes with shared operating system kernels, adaptively use csi plugin mode If scheduled to Kata container nodes with independent operating system kernels, you can use the sidecar mode adaptively and support scalable modifications by cloud vendors Support basic data operation Combine with Hugging face and Pytorch to support transparent data acceleration through pre-reading and multi-stream reading Support defining automated data flow operations Integrate with Kubeflow Pipelines to accelerate datasets in the pipeline Integrate with Fairing for model development and deployment in the notebook environment Integrate with KServe to facilitate model deployment Minimum container permission (remove the privileged permission of FUSE Pod) Minimum rbac permission Minimal container image installation Continuously provide best practice documentation Simplify deployment Merge Dataset/Runtime controllers into one binary package Simplify usage Support Runtimeless, Dataset as the single API entry for users to use Fluid Improve code quality Reduce repetitive code Improve test coverage Security hardening Minimize the permissions of controller's RBAC Regularly review and update the permissions when new runtime is introduced Enhance observability Provide monitoring and alerts for Datasets Enhance the quality of documentation Organize the documentation so users can navigate it easily and find the information Provide more practical examples and tutorials can significantly improve the user's comprehension and learning process. Maintain consistency in language, style, and formatting throughout the documentation
Project: Gloo (Orchestration & Management / API Gateway)
File: fips.md

title: FIPS Compliant Data Plane weight: 80 description: Installing Gloo Edge Enterprise with FIPS-compliant crypto libraries Gloo Edge Enterprise binaries have images available that were built with FIPS-compliant crypto libraries. These can be installed by setting `global.image.fips=true` via Helm. Add the following to your `value-overrides.yaml` file ```yaml global: image: fips: true ``` and use it to override the default values in the Gloo Edge chart with Helm 3 ```bash helm install gloo glooe/gloo-ee --namespace gloo-system \ -f value-overrides.yaml --create-namespace --set-string licensekey=YOURLICENSE_KEY ``` If you are building your own ExtAuth plugins, you will need to build those plugins with `goboring` as well. Follow the guide and use the value of `FIPSGOBUILD_IMAGE` in your docker builds. FIPS-compliant cryptography modules have been certified by the National Institute of Standards and Technology and meet the security standards required for use in government settings. Using FIPS-compliant cryptography libraries is a requirement for getting FIPS certification for your application. The FIPS-compliant binaries are built with `goboring`, which uses `CGO` to call out to FIPS-compliant crypto libraries. This adds overhead to cryptography operations and can complicate cross-compilation. If your project does not require FIPS-compliant cryptography, installing FIPS-compliant Gloo Edge is not recommended. During the build and release process, the FIPS-compliant images are validated to ensure they are built with FIPS-compliant crypto libraries. Below are the steps that you can take to validate the images yourself: Download the FIPS-compliant image. ```sh docker pull quay.io/solo-io/gloo-ee-fips:1.16.0-beta1 ``` Create a container with the image that you downloaded. ```sh docker create --name gloo-ee quay.io/solo-io/gloo-ee-fips:1.16.0-beta1 ``` Copy the Gloo binary from the container to your local machine. ```sh docker cp gloo-ee:/usr/local/bin/gloo . ``` Download the lastest . ```sh go install github.com/rsc/goversion@latest ``` Use the `goversion` CLI to print the cryto libraries that are linked in the Gloo binary. ```sh goversion -crypto gloo ``` Example output for standard Gloo Edge images: ``` gloo go1.20.9 (standard crypto) ``` Example output for FIPS-compliant Gloo Edge images: ``` gloo go1.20.9 X:boringcrypto (boring crypto) ```
Project: Skipper (Orchestration & Management / Service Proxy)
File: shadow-traffic.md

This tutorial will show how to setup routing for shadow traffic, where one backend (main) will receive the full traffic, while a shadowing backend (test) will receive only a certain percentage of the same traffic. Before the shadow traffic, we are sending all traffic to the main backend. ```sh main: * -> "https://main.example.org"; ``` Before generating the shadow traffic, we create an identical clone of the main route that will handle only 10% of the traffic, while the rest stays being handled by the main route. ```sh main: * -> "https://main.example.org"; split: Traffic(.1) -> "https://main.example.org"; ``` The route introduced next won't handle directly any incoming requests, because they won't be matched by the predicate, but it is prepared to send tee requests to the alternative, 'shadow' backend. ```sh main: * -> "https://main.example.org"; split: Traffic(.1) -> "https://main.example.org"; shadow: Tee("shadow-test-1") && True() -> "https://shadow.example.org"; ``` Now we can apply the filter to the 'split' route, using the same label as we did in the predicate. ```sh main: * -> "https://main.example.org"; split: Traffic(.1) -> teeLoopback("shadow-test-1") -> "https://main.example.org"; shadow: Tee("shadow-test-1") && True() -> "https://shadow.example.org"; ``` *Note that as of now, we need to increase the weight of the 'shadow' route by adding the `True()` predicate in order to avoid that the 'split' route would match the cloned request again.* After this, the 'split' route will still send all the handled requests, 10% of the total traffic, to the main backend, while the rest of the traffic is routed there by the 'main' route. However, the filter will also clone the traffic of the 'split' route, 10% of the total, and reapply the routing on it, during which these requests will be matched by the predicate, and sent to the shadow backend.
Project: Kusk Gateway (Orchestration & Management / API Gateway)
File: when-can-you-use-kusk.md

Kusk fits with both new and existing APIs: If your current project already uses OpenAPI or you want to start using it, Kusk fits well in both scenarios and can replace your existing Kubernetes Ingress Controller. Kusk will provide your current APIs with great documentation portal for the consumers of your APIs, request validation for instant protection and response mocking out of the box instantly. Kusk is ready to handle any request size as it uses Envoy under-the-hood to handle the traffic of your APIs. If you are just starting a new API project in Kubernetes, Kusk lets you mock your APIs so you can start developing your APIs without implementing the services. Kusk also provides your API with out-of-the-box security features like rate-limiting, authentication, CORS, and more.
Project: Gloo (Orchestration & Management / API Gateway)
File: envoy.glooe.solo.io.project.sk.md

title: "envoy.glooe.solo.io.project" weight: 5 <!-- Code generated by solo-kit. DO NOT EDIT. --> API Version: `envoy.glooe.solo.io.v1` <!-- Start of HubSpot Embed Code --> <script type="text/javascript" id="hs-script-loader" async defer src="//js.hs-scripts.com/5130874.js"></script> <!-- End of HubSpot Embed Code -->
Project: Volcano (Orchestration & Management / Scheduling & Orchestration)
File: elastic-scheduler.md

This feature allows Volcano to schedule workloads based on the `[min,max]` config to improve resource utilization rate and shorten the execution time of training job. For example, K8s cluster has 10 GPUs, and I want to use Volcano to schedule training jobs(tfjob/pytorchjob/vcjob) in two queues: queue1 and queue2 ||weight| reclaimable| deserved GPUs| ||||| |queue1| 1| false| 5| |queue2| 1| false| 5| If there is a job1-1 running in queue1, we set pod6 to pod10 as elastic pods/resources which can be preempted when queue1's resource is shortage. The elastic pods have the lowest priority. Specifically,these pods will be created last and be preempted first. elastic pods can be created only when there are free resources. elastic pods will be preempted if there are not enough resources for running minAvailable pods. ```yaml apiVersion: batch.volcano.sh/v1alpha1 kind: Job metadata: name: job1-1 spec: minAvailable: 5 #min queue: queue1 tasks: replicas: 10 #max name: job1-1 template: metadata: name: job1-1 spec: containers: image: train_script name: xx resources: limits: cpu: 1 nvidia.com/gpu: 1 ``` In detail, there are some principles for elastic schedule if job1-1 and job1-2 are submited at the same time, `job1.minAvailable` pods and `job2.minAvailable` pods will be created first. And then `job1/job2.elastic` pods will be created if there are extra resource. if submit job1-1 and then submit job1-2 in queue1, elastic pods in job1-1 will be preempted if submit job1-1 and then submit job2-1 in queue2, elastic pods in job1-1 will be preempted Enqueue action Modify the logic of job enqueue process. For elastic pods can be preempted at any time, elastic resources are free resources in a queue. So we will fix `jobEnqueueableFns` in `overcommit` and `proportion` plugin. it should be noticed that if total elastic resources can not meet new-job's minRequest and the new-job should also be pending. Allocate action(already implemented) All pods will be created initially(by controller/operator), but minAvailable pods will be scheduled first and then schedule elastic pods if there are free resources. Preempt action in queue scope Preempt elastic pods if there are starving job in the same queue(already implemented). It is not necessary to preempt elastic pods if total elastic resources can not meet the starving job's minRequest. Reclaim action in cluster scope If a queue is overused, reclaim its elastic resources whether this queue's `reclaimable` filed is true or false.
Project: CoreDNS (Orchestration & Management / Coordination & Service Discovery)
File: coredns-1.2.1.md

+++ title = "CoreDNS-1.2.1 Release" description = "CoreDNS-1.2.1 Release Notes." tags = ["Release", "1.2.1", "Notes"] release = "1.2.1" date = "2018-08-28T07:10:29+01:00" author = "coredns" +++ We are pleased to announce the of CoreDNS-1.2.1! This release features bugfixes (mostly in the plugin), documentation improvements and one new plugin: . A new plugin called was added. When starting up it detects resolver loops and stops the process if one is detected. The following people helped with getting this release done. Good to see a whole bunch of new names, as well as the usual suspects: Bingshen Wang, Chris O'Haver, Eugen Kleiner, Francois Tur, Jiacheng Xu, Karsten Weiss, Lorenzo Fontana, Miek Gieben, Nitish Tiwari, Stanislav Zapolsky, varyoo, Yong Tang, Zach Eddy. For documentation see our (in progress!) . For help and other resources, see our .
Project: Kusk Gateway (Orchestration & Management / API Gateway)
File: traffic_splitting.md

Traffic splitting in Kusk allows for traffic to be routed to 2 or more targets for the same API. The traffic to the API or API path will be split between services running different version of the same service. Setting property `x-kusk.upstreams` allows users to configure several endpoints for the API to which traffic will be split according to the value of `service.weight`. ```yaml openapi: 3.0.0 info: title: simple-api version: 0.1.0 x-kusk: upstreams: service: name: simple-api-servicev1 namespace: default port: 80 weight: 50 service: name: simple-api-servicev2 namespace: test-namespace port: 80 weight: 50 .. ``` The sum of the weights must be equal to 100. In this example traffic will be split equaly between services. Any number of services can be used as long the total weight equals to 100. To debug easily Kusk will set response header `x-kusk-weighted-cluster:[service_name]` for each weighted service. The property `x-kusk.upstreams` is mutually exclusive with `x-kusk.upstream` as well with `x-kusk.validation` will not accept multiple upstreams. Upstreams can be combination `service` and `host` where each must have weight value assinged. ```yaml openapi: 3.0.0 info: title: simple-api version: 0.1.0 x-kusk: upstreams: service: name: simple-api-servicev1 namespace: default port: 80 weight: 50 host: hostname: simple-api-servicev2 port: 80 weight: 50 .. ```
Project: Traefik (Orchestration & Management / API Gateway)
File: service-by-label.md

In general when configuring a Traefik provider, a service assigned to one (or several) router(s) must be defined as well for the routing to be functional. There are, however, exceptions when using label-based configurations: If a label defines a router (e.g. through a router Rule) and a label defines a service (e.g. implicitly through a loadbalancer server port value), but the router does not specify any service, then that service is automatically assigned to the router. If a label defines a router (e.g. through a router Rule) but no service is defined, then a service is automatically created and assigned to the router. !!! info "" As one would expect, in either of these cases, if in addition a service is specified for the router, then that service is the one assigned, regardless of whether it actually is defined or whatever else other services are defined.
Project: Docker Swarm (Orchestration & Management / Scheduling & Orchestration)
File: CONDUCT.md

Cobra will follow a steady release cadence. Non breaking changes will be released as minor versions quarterly. Patch bug releases are at the discretion of the maintainers. Users can expect security patch fixes to be released within relatively short order of a CVE becoming known. For more information on security patch fixes see the CVE section below. Releases will follow . Users tracking the Master branch should expect unpredictable breaking changes as the project continues to move forward. For stability, it is highly recommended to use a release. We will maintain two major releases in a moving window. The N-1 release will only receive bug fixes and security updates and will be dropped once N+1 is released. Deprecation of Go versions or dependent packages will only occur in major releases. To reduce the change of this taking users by surprise, any large deprecation will be preceded by an announcement in the and an Issue on Github. Maintainers will make every effort to release security patches in the case of a medium to high severity CVE directly impacting the library. The speed in which these patches reach a release is up to the discretion of the maintainers. A low severity CVE may be a lower priority than a high severity one. Cobra maintainers will use GitHub issues and the as the primary means of communication with the community. This is to foster open communication with all users and contributors. Breaking changes are generally allowed in the master branch, as this is the branch used to develop the next release of Cobra. There may be times, however, when master is closed for breaking changes. This is likely to happen as we near the release of a new version. Breaking changes are not allowed in release branches, as these represent minor versions that have already been released. These version have consumers who expect the APIs, behaviors, etc, to remain stable during the lifetime of the patch stream for the minor release. Examples of breaking changes include: Removing or renaming exported constant, variable, type, or function. Updating the version of critical libraries such as `spf13/pflag`, `spf13/viper` etc... Some version updates may be acceptable for picking up bug fixes, but maintainers must exercise caution when reviewing. There may, at times, need to be exceptions where breaking changes are allowed in release branches. These are at the discretion of the project's maintainers, and must be carefully considered before merging. Maintainers will ensure the Cobra test suite utilizes the current supported versions of Golang. Changes to this document and the contents therein are at the discretion of the maintainers. None of the contents of this document are legally binding in any way to the maintainers or the users.
Project: Gloo (Orchestration & Management / API Gateway)
File: percent.proto.sk.md

title: "percent.proto" weight: 5 <!-- Code generated by solo-kit. DO NOT EDIT. --> Identifies a percentage, in the range [0.0, 100.0]. ```yaml "value": float ``` | Field | Type | Description | | -- | - | -- | | `value` | `float` | | A fractional percentage is used in cases in which for performance reasons performing floating point to integer conversions during randomness calculations is undesirable. The message includes both a numerator and denominator that together determine the final fractional value. Example*: 1/100 = 1%. Example*: 3/10000 = 0.03%. ```yaml "numerator": int "denominator": .solo.io.envoy.type.FractionalPercent.DenominatorType ``` | Field | Type | Description | | -- | - | -- | | `numerator` | `int` | Specifies the numerator. Defaults to 0. | | `denominator` | | Specifies the denominator. If the denominator specified is less than the numerator, the final fractional percentage is capped at 1 (100%). | Fraction percentages support several fixed denominator values. | Name | Description | | -- | -- | | `HUNDRED` | 100. Example: 1/100 = 1%. | | `TEN_THOUSAND` | 10,000. Example: 1/10000 = 0.01%. | | `MILLION` | 1,000,000. Example: 1/1000000 = 0.0001%. | <!-- Start of HubSpot Embed Code --> <script type="text/javascript" id="hs-script-loader" async defer src="//js.hs-scripts.com/5130874.js"></script> <!-- End of HubSpot Embed Code -->
Project: Avro (Orchestration & Management / Remote Procedure Call)
File: avro-1.5.4-released.md

title: "Avro 1.5.4" linkTitle: "Avro 1.5.4" date: 2011-09-12 <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> Apache Avro 1.5.4 has been released!
Project: DolphinScheduler (Orchestration & Management / Scheduling & Orchestration)
File: hana.md

| Datasource | Description | |-|| | Datasource | Select HANA. | | Datasource name | Enter the name of the DataSource. | | Description | Enter a description of the DataSource. | | IP/Host Name | Enter the HANA service IP. | | Port | Enter the HANA service port. | | Username | Set the username for HANA connection. | | Password | Set the password for HANA connection. | | Database name | Enter the database name of the HANA connection. | | Jdbc connection parameters | Parameter settings for HANA connection, in JSON format. | No, read section example in `DataSource Center` section to activate this datasource.
Project: BFE (Orchestration & Management / Service Proxy)
File: mod_rewrite.md

mod_rewrite modifies the URI of HTTP request based on defined rules. conf/modrewrite/modrewrite.conf | Config Item | Description | | -- | | | Basic.DataPath | String<br>Path of rule configuration | ```ini [Basic] DataPath = mod_rewrite/rewrite.data ``` conf/mod_rewrite/rewrite.data | Config Item | Description | | -- | | | Version | String<br>Version of config file | | Config | Struct<br>Rewrite rules for each product | | Config{k} | String<br>Product name | | Config{v} | Object<br>A ordered list of rewrite rules | | Config{v}[] | Object<br>A rewrite rule | | Config{v} | | Config{v}[].Actions | Object<br>A ordered list of rewrite actions | | Config{v}[].Actions[] | Object<br>A rewrite action | | Config{v}[].Actions[].Cmd | Object<br>Name of rewrite action | | Config{v}[].Actions[].Params | Object<br>Parameters of rewrite action | | Config{v}[].Last | Integer<br>If true, stop to check the remaining rules | | Action | Description | | - | - | | HOST_SET | Set host to specified value | | HOSTSETFROMPATHPREFIX | Set host to specified path prefix | | HOSTSUFFIXREPLACE | Replace suffix of host | | PATH_SET | Set path to specified value | | PATHPREFIXADD | Add prefix to original path | | PATHPREFIXTRIM | Trim prefix from original path | | QUERY_ADD | Add query | | QUERY_DEL | Delete query | | QUERYDELALL_EXCEPT | Del all queries except specified queries | | QUERY_RENAME | Rename query | ```json { "Version": "20190101000000", "Config": { "example_product": [ { "Cond": "reqpathprefix_in(\"/rewrite\", false)", "Actions": [ { "Cmd": "PATHPREFIXADD", "Params": [ "/bfe/" ] } ], "Last": true } ] } } ```
Project: DolphinScheduler (Orchestration & Management / Scheduling & Orchestration)
File: datasource.md

The data source center supports POSTGRESQL, HIVE/IMPALA, SPARK, CLICKHOUSE, SQLSERVER data sources by default. If you are using MySQL or ORACLE data source, you need to place the corresponding driver package in the lib directory org.apache.dolphinscheduler.spi.datasource.DataSourceChannel org.apache.dolphinscheduler.spi.datasource.DataSourceChannelFactory org.apache.dolphinscheduler.spi.datasource.client.DataSourceClient In the first step, the data source plug-in can implement the above interfaces and inherit the general client. For details, refer to the implementation of data source plug-ins such as sqlserver and mysql. The addition methods of all RDBMS plug-ins are the same. Add the driver configuration in the data source plug-in pom.xml We provide APIs for external access of all data sources in the dolphin scheduler data source API module In additional, the `DataSourceChannelFactory` extends from `PrioritySPI`, this means you can set the plugin priority, when you have two plugin has the same name, you can customize the priority by override the `getIdentify` method. The high priority plugin will be load, but if you have two plugin with the same name and same priority, the server will throw `IllegalArgumentException` when load the plugin. Support data sources such as kafka, http, files, sparkSQL, FlinkSQL, etc.
Project: Avro (Orchestration & Management / Remote Procedure Call)
File: api-csharp.md

title: "C# API" linkTitle: "C# API" weight: 103 manualLink: /docs/++version++/api/csharp/html/ <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> The C# API documentation can be found <a href="/docs/++version++/api/csharp/html/">here</a>.