cwielenb/CNCF-Dataset huggingface dataset:
Project: Traefik (Orchestration & Management / API Gateway)
File: documentation.md

title: "Traefik Contribution Documentation" description: "Found something unclear in the Traefik Proxy documentation and want to give a try at explaining it better? Read the guide to building documentation." Features Are Better When You Know How to Use Them {: .subtitle } You've found something unclear in the documentation and want to give a try at explaining it better? Let's see how. This is built with . Please make sure you have the following requirements installed: You can build the documentation and test it locally (with live reloading), using the `docs-serve` target: ```bash $ make docs-serve docker build -t traefik-docs -f docs.Dockerfile . docker run --rm -v /home/user/go/github/traefik/traefik:/mkdocs -p 8000:8000 traefik-docs mkdocs serve [I 170828 20:47:48 server:283] Serving on http://0.0.0.0:8000 [I 170828 20:47:48 handlers:60] Start watching changes [I 170828 20:47:48 handlers:62] Start detecting changes ``` !!! tip "Default URL" Your local documentation server will run by default on <http://127.0.0.1:8000>. If you only want to build the documentation without serving it locally, you can use the following command: ```bash $ make docs-build ... ``` Please make sure you have the following requirements installed: ```bash $ python --version Python 2.7.2 $ pip --version pip 1.5.2 ``` Then, install MkDocs with `pip`. ```bash pip install --user -r requirements.txt ``` To build the documentation locally and serve it locally, run `mkdocs serve` from the root directory. This will start a local server. ```bash $ mkdocs serve INFO - Building documentation... INFO - Cleaning site directory [I 160505 22:31:24 server:281] Serving on http://127.0.0.1:8000 [I 160505 22:31:24 handlers:59] Start watching changes [I 160505 22:31:24 handlers:61] Start detecting changes ``` To check that the documentation meets standard expectations (no dead links, html markup validity, ...), use the `docs-verify` target. ```bash $ make docs-verify docker build -t traefik-docs-verify ./script/docs-verify-docker-image ## Build Validator image ... docker run --rm -v /home/travis/build/traefik/traefik:/app traefik-docs-verify ## Check for dead links and w3c compliance === Checking HTML content... Running ["HtmlCheck", "ImageCheck", "ScriptCheck", "LinkCheck"] on /app/site/basics/index.html on *.html... ``` !!! note "Clean & Verify" If you've made changes to the documentation, it's safer to clean it before verifying it. ```bash $ make docs ... ``` Will perform all necessary steps for you. !!! note "Disabling Documentation Verification" Verification can be disabled by setting the environment variable `DOCSVERIFYSKIP` to `true`: ```shell DOCSVERIFYSKIP=true make docs-verify ... DOCSLINTSKIP is true: no linting done. ```
Project: Volcano (Orchestration & Management / Scheduling & Orchestration)
File: proportional.md

Volcano scheduler handles jobs requiring different types of resources, such as GPU, CPU, memory. Under particular circumstances, we may specify a 'primary' resource(e.g., GPU in deep learning), and preserve the amount of associated 'secondary' resources by a pre-set proportion. This plugin works in the phase of predicates, dedicates to ensure the node's idle resource is enough for the proportion after jobs requiring secondary resources are scheduled. Considering we have a node with 74CPUs, 8GPUs, 128G memory. As no job is submitted, resource NodeAllocatable is equal to NodeIdle. Node | NodeAllocatable | NodeIdle || nodeC0-0 | cpu 74, memory 128G, nvidia.com/gpu 8 | cpu 74, memory 128G, nvidia.com/gpu 8 | Then two jobs requiring 8CPUs, 8G memory are submitted, and scheduled to the node; the resource status is as below: Job | Pod | Resource | Node | NodeAllocatable | NodeIdle ||||| default/single-1000-0 | single-1000-0 | cpu 8, memory 8G, nvidia.com/gpu 0 | nodeC0-0 | cpu 74, memory 128G, nvidia.com/gpu 8 | cpu 66, memory 120G, nvidia.com/gpu 8 | default/single-1000-1 | single-1000-1 | cpu 8, memory 8G, nvidia.com/gpu 0 | nodeC0-0 | cpu 66, memory 120G, nvidia.com/gpu 8 | cpu 58, memory 112G, nvidia.com/gpu 8 | If we take GPU as primary resource and want to use 1GPU 'binded' with 8CPUs, the left 58 CPUs are insufficent for 8 GPUs; the proportion plugin is designed to solve this problem. Firstly set the proportion binding in volcano-scheduler.conf: ```yaml actions: "enqueue, allocate, backfill" tiers: plugins: name: predicates arguments: predicate.ProportionalEnable: true predicate.resources: nvidia.com/gpu,nvidia.com/v100-sxm2-16gb predicate.resources.nvidia.com/gpu.cpu: 8 predicate.resources.nvidia.com/gpu.memory: 8 predicate.resources.nvidia.com/v100-sxm2-16gb.cpu: 16 predicate.resources.nvidia.com/v100-sxm2-16gb.memory: 16 ``` The proportion is GPU:CPU:MEMORY=1:8:8, and let the test scenario just as above: Node | NodeAllocatable | NodeIdle || nodeC0-0 | cpu 74, memory 128G, nvidia.com/gpu 8 | cpu 74, memory 128G, nvidia.com/gpu 8 | Job | Pod | Resource | Node | NodeAllocatable | NodeIdle ||||| default/single-1000-0 | single-1000-0 | cpu 8, memory 8G, nvidia.com/gpu 0 | nodeC0-0 | cpu 74, memory 128G, nvidia.com/gpu 8 | cpu 66, memory 120G, nvidia.com/gpu 8 | default/single-1000-1 | single-1000-1 | cpu 8, memory 8G, nvidia.com/gpu 0 | - | - | - | After job single-1000-0 is scheduled, the Idel resouce is 8GPUs, 66CPUs, 120G memory. During the predicate phase, this plugin caculates the resource left if job single-1000-1 is scheduled`(node.Idel.CPU - task.Resreq.CPU < node.Idel.GPU * cpuRatio || node.Idel.Memory - task.Resreq.Memory < node.Idel.GPU * memoryRatio)`; the result is 8GPUs, 58CPUs, 112G memory, that unsatisfies the 1:8:8 proportion. Therefore nodeC0-0 is removed from the predicateNodes, and NodeIdle remains 8GPUs, 66CPUs, 120G memory.
Project: Karmada (Orchestration & Management / Scheduling & Orchestration)
File: resource_test.md

| Test Case | E2E Describe Text | Comments | |--||-| | DeploymentStatus collection testing: Test the status collection of Deployment resources | DeploymentStatus collection | | | ServiceStatus collection testing: Test the status collection of Service resources | ServiceStatus collection | | | NodePort Service collection testing: Test the status collection of NodePort type Service resources | NodePort Service collection | | | IngressStatus collection testing: Test the status collection of Ingress resources | IngressStatus collection | | | JobStatus collection testing: Test the status collection of Job resources | JobStatus collection | | | DaemonSetStatus collection testing: Test the status collection of DaemonSet resources | DaemonSetStatus collection | | | StatefulSetStatus collection testing: Test the status collection of StatefulSet resources | StatefulSetStatus collection | | | PodDisruptionBudget collection testing: Test the status collection of PodDisruptionBudget resources | PodDisruptionBudget collection | | | Workload status synchronization testing: Test the synchronization of workload status when a cluster fails and recovers quickly | Workload status synchronization | |
Project: Easegress (Orchestration & Management / API Gateway)
File: bug_report.md

name: Bug report about: Create a report to help us improve title: '[Bug]: ' labels: [bug] assignees: '' Describe the bug A clear and concise description of what the bug is. To Reproduce Steps to reproduce the behavior: Execute '...' Send a request to '....' See error Expected behavior A clear and concise description of what you expected to happen. Version The version number of Easegress. Configuration Easegress Configuration ```yaml ``` HTTP server configuration ```yaml ``` Pipeline Configuration ```yaml ``` Logs ``` Easegress logs, if applicable. ``` OS and Hardware OS: [e.g. Ubuntu 20.04] CPU:[e.g. Intel(R) Core(TM) i5-8265U] Memory: [e.g. 16GB] Additional context Add any other context about the problem here. Thanks for contributing !
Project: Gloo (Orchestration & Management / API Gateway)
File: ip.proto.sk.md

title: "ip.proto" weight: 5 <!-- Code generated by solo-kit. DO NOT EDIT. --> Matches a specific IP address against a set of possibly overlapping subnets using a trie. ```yaml "rangeMatchers": []xds.type.matcher.v3.IPMatcher.IPRangeMatcher ``` | Field | Type | Description | | -- | - | -- | | `rangeMatchers` | | Match IP address by CIDR ranges. | Specifies a list of IP address ranges and a match action. ```yaml "ranges": []xds.core.v3.CidrRange "onMatch": .xds.type.matcher.v3.Matcher.OnMatch "exclusive": bool ``` | Field | Type | Description | | -- | - | -- | | `ranges` | | A non-empty set of CIDR ranges. | | `onMatch` | | Match action to apply when the IP address is within one of the CIDR ranges. | | `exclusive` | `bool` | Indicates whether this match option should be considered if there is a more specific matcher. Exclusive matchers are not selected whenever a more specific matcher exists (e.g. matcher with a longer prefix) even when the more specific matcher fails its nested match condition. Non-exclusive matchers are considered if the more specific matcher exists but its nested match condition does not entirely match. Non-exclusive matchers are selected in the order of their specificity first (longest prefix first), then the order of declaration next. For example, consider two range matchers: an exclusive matcher X on ``0.0.0.0/0`` and a matcher Y on ``192.0.0.0/2`` with a nested match condition Z. For the input IP ``192.168.0.1`` matcher Y is the most specific. If its nested match condition Z does not accept the input, then the less specific matcher X does not apply either despite the input being within the range, because matcher X is exclusive. The opposite is true if matcher X is not marked as exclusive. In that case matcher X always matches whenever matcher "Y rejects the input. | <!-- Start of HubSpot Embed Code --> <script type="text/javascript" id="hs-script-loader" async defer src="//js.hs-scripts.com/5130874.js"></script> <!-- End of HubSpot Embed Code -->
Project: Apache Zookeeper (Orchestration & Management / Coordination & Service Discovery)
File: zookeeperSnapshotAndRestore.md

<!-- Copyright 2002-2004 The Apache Software Foundation Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. //--> Zookeeper is designed to withstand machine failures. A Zookeeper cluster can automatically recover from temporary failures such as machine reboot. It can also tolerate up to (N-1)/2 permanent failures for a cluster of N members due to hardware failures or disk corruption, etc. When a member permanently fails, it loses access to the cluster. If the cluster permanently loses more than (N-1)/2 members, it disastrously fails and loses quorum. Once the quorum is lost, the cluster cannot reach consensus and therefore cannot continue to accept updates. To recover from such disastrous failures, Zookeeper provides snapshot and restore functionalities to restore a cluster from a snapshot. Snapshot and restore operate on the connected server via Admin Server APIs Snapshot and restore are rate limited to protect the server from being overloaded Snapshot and restore require authentication and authorization on the root path with ALL permission. The supported auth schemas are digest, x509 and IP. <a name="zookeeper_snapshot"></a> Recovering a cluster needs a snapshot from a ZooKeeper cluster. Users can periodically take snapshots from a live server which has the highest zxid and stream out data to a local or external storage/file system (e.g., S3). ```bash curl -H 'Authorization: digest root:root_passwd' http://hostname:adminPort/commands/snapshot?streaming=true --output snapshotFileName ``` <a name="zookeeper_restore"></a> Restoring a cluster needs a single snapshot as input stream. Restore can be used for recovering a cluster for quorum lost or building a brand-new cluster with seed data. All members should restore using the same snapshot. The following are the recommended steps: Blocking traffic on the client port or client secure port before restore starts Take a snapshot of the latest database state using the snapshot admin server command if applicable For each server Moving the files in dataDir and dataLogDir to different location to prevent the restored database from being overwritten when server restarts after restore Restore the server using restore admin server command Unblocking traffic on the client port or client secure port after restore completes ```bash curl -H 'Content-Type:application/octet-stream' -H 'Authorization: digest root:root_passwd' -POST http://hostname:adminPort/commands/restore --data-binary "@snapshotFileName" ```
Project: Capsule (Orchestration & Management / Scheduling & Orchestration)
File: charmed.md

is a Kubernetes distribution coming with out-of-the-box tools that support deployments and operational management and make microservice development easier. Combined with Capsule, Charmed Kubernetes allows users to further reduce the operational overhead of Kubernetes setup and management. The Charm package for Capsule is available to Charmed Kubernetes users via .
Project: Contour (Orchestration & Management / Service Proxy)
File: external-service-routing.md

HTTPProxy supports routing traffic to `ExternalName` service types, but this is disabled by default, as it can lead to inadvertent exposure of the Envoy Admin UI, allowing remote shutdown and restart of Envoy. Please see for all the details. It can also be used to expose services in namespaces a user does not have access to, using an ExternalName of `service.namespace.svc.cluster.local`. Please see for more details. We do not recommend enabling ExternalName Services without a strong use case, and understanding of the security implications. However, To enable ExternalName processing, you must set the `enableExternalNameService` configuration file setting to `true`. This will allow the following configuration to be valid. Contour looks at the `spec.externalName` field of the service and configures the route to use that DNS name instead of utilizing EDS. Note that hostnames of `localhost` or some other synonyms will be rejected (because of the aforementioned security issues). There's nothing specific in the HTTPProxy object that needs to be configured other than referencing a service of type `ExternalName`. HTTPProxy supports the `requestHeadersPolicy` field to rewrite the `Host` header after first handling a request and before proxying to an upstream service. This field can be used to ensure that the forwarded HTTP request contains the hostname that the external resource is expecting. **Note:** The ports are required to be specified. ```yaml apiVersion: v1 kind: Service metadata: labels: run: externaldns name: externaldns namespace: default spec: externalName: foo-basic.bar.com ports: name: http port: 80 protocol: TCP targetPort: 80 type: ExternalName ``` To proxy to another resource outside the cluster (e.g. A hosted object store bucket for example), configure that external resource in a service type `externalName`. Then define a `requestHeadersPolicy` which replaces the `Host` header with the value of the external name service defined previously. Finally, if the upstream service is served over TLS, set the `protocol` field on the service to `tls` or annotate the external name service with: `projectcontour.io/upstream-protocol.tls: 443,https`, assuming your service had a port 443 and name `https`.
Project: Fluid (Orchestration & Management / Scheduling & Orchestration)
File: automatic_clean_up_data_operation.md

Fluid's universal data operations describe operations such as data prefetch, data migration, elastic scaling, cache cleaning, metadata backup, and recovery. Similar to the Kubernetes Job's automatic cleaning mechanism, Fluid also provides automatic cleanup data operation, leveraging the Time-to-Live (TTL) mechanism to limit the lifecycle of data operations that have finished execution. This document will briefly demonstrate the utilization of these features. Before we start, please refer to to install Fluid on your Kubernetes Cluster. Make sure that all the components required by Fluid are set up correctly like this: ```shell $ kubectl get pod -n fluid-system alluxioruntime-controller-5b64fdbbb-84pc6 1/1 Running 0 8h csi-nodeplugin-fluid-fwgjh 2/2 Running 0 8h csi-nodeplugin-fluid-ll8bq 2/2 Running 0 8h dataset-controller-5b7848dbbb-n44dj 1/1 Running 0 8h ``` Check the Dataset and AlluxioRuntime objects to be created ```shell $ cat<<EOF >dataset.yaml apiVersion: data.fluid.io/v1alpha1 kind: Dataset metadata: name: hbase spec: mounts: mountPoint: https://mirrors.tuna.tsinghua.edu.cn/apache/hbase/stable/ name: hbase apiVersion: data.fluid.io/v1alpha1 kind: AlluxioRuntime metadata: name: hbase spec: replicas: 1 tieredstore: levels: mediumtype: MEM path: /dev/shm quota: 2Gi high: "0.95" low: "0.7" EOF ``` Create the Dataset and AlluxioRuntime ```shell $ kubectl create -f dataset.yaml ``` Wait for the Dataset and AlluxioRuntime to be ready You can check their status by running: ```shell $ kubectl get datasets hbase ``` Dataset and Runtime are all ready if you see something like this: ```shell NAME UFS TOTAL SIZE CACHED CACHE CAPACITY CACHED PERCENTAGE PHASE AGE hbase 1.21GiB 0.00B 2.00GiB 0.0% Bound 75s ``` We use Dataload to show the automatic cleanup of data operations here. Check the Dataload objects to be created ```shell $ cat <<EOF > dataload.yaml apiVersion: data.fluid.io/v1alpha1 kind: DataLoad metadata: name: hbase-dataload spec: dataset: name: hbase namespace: default ttlSecondsAfterFinished: 300 EOF ``` Here, we use the `spec.ttlSecondsAfterFinished` field to indicate how many seconds the data operation will be cleaned up after the job is Complete or Failed, in seconds. Create the Dataload ```shell $ kubectl apply -f dataload.yaml ``` Watch the Dataload status ```shell $ kubectl get dataload -w NAME DATASET PHASE AGE DURATION hbase-dataload hbase Executing 7s Unfinished hbase-dataload hbase Complete 29s 7s hbase-dataload hbase Complete 5m29s 7s $ kubectl get dataload hbase-dataload Error from server (NotFound): dataloads.data.fluid.io "hbase-dataload" not found ``` It can be seen that 300s after the execution of `hbase-dataload` is completed, the dataload will be automatically cleaned. Because the TTL-after-finished controller (Fluid dataset-controller) uses timestamps stored in the Data Operation to determine whether the TTL has expired or not, this feature is sensitive to time skew in your cluster, which might lead to premature or delayed cleanup of Job objects. Be cautious when setting a non-zero TTL.
Project: DolphinScheduler (Orchestration & Management / Scheduling & Orchestration)
File: snowflake.md

| Datasource | Description | |-|--| | Datasource | Select SNOWFLAKE. | | Datasource name | Enter the name of the DataSource. | | Description | Enter a description of the DataSource. | | IP/Host Name | Enter the SNOWFLAKE service IP. | | Port | Enter the SNOWFLAKE service port. | | Username | Set the username for SNOWFLAKE connection. | | Password | Set the password for SNOWFLAKE connection. | | Database name | Enter the database name of the SNOWFLAKE connection. | | Datawarehouse name | Enter the datawarehouse name of the SNOWFLAKE connection. | | Jdbc connection parameters | Parameter settings for SNOWFLAKE connection, in JSON format. |
Project: Aeraki Mesh (Orchestration & Management / Service Mesh)
File: MAINTAINERS.md

<!-- --> Aeraki Mesh Maintainers are given in the list below, in no particular order. | | Name | Company | Repos | |:-:|::|:-:|::| | <img width="30px" src="https://avatars.githubusercontent.com/u/1321762"> | | Tetrate.io | aeraki, meta-protocol-proxy | | <img width="30px" src="https://avatars.githubusercontent.com/u/15240576"> | | Alauda | meta-protocol-proxy | | <img width="30px" src="https://avatars.githubusercontent.com/u/3876735"> | | Baidu | meta-protocol-proxy | | <img width="30px" src="https://avatars.githubusercontent.com/u/12084604"> | | Tencent Music | aeraki | | <img width="30px" src="https://avatars.githubusercontent.com/u/28644213"> | | BOSS Zhipin | aeraki | | <img width="30px" src="https://avatars.githubusercontent.com/u/48784001"> | | Tencent | aeraki | | <img width="30px" src="https://avatars.githubusercontent.com/u/19339970"> | | Bdidu | aeraki |
Project: MetalLB (Orchestration & Management / Service Proxy)
File: k3s.md

title: Issues with K3s weight: 20 K3S come with its own service load balancer named Klipper. You need to disable it in order to run MetalLB. To disable Klipper, run the server with the `--disable servicelb` option, as described in [K3s documentation](https://rancher.com/docs/k3s/latest/en/networking/ )
Project: etcd (Orchestration & Management / Coordination & Service Discovery)
File: mirror_maker.md

Mirror maker mirrors a prefix in the key-value space of an etcd cluster into another prefix in another cluster. Mirroring is designed for copying configuration to various clusters distributed around the world. Mirroring usually has very low latency once it completes synchronizing with the initial state. Mirror maker utilizes the etcd watcher facility to immediately inform the mirror of any key modifications. Based on our experiments, the network latency between the mirror maker and the two clusters accounts for most of the latency. If the network is healthy, copying configuration held in etcd to the mirror should take under one second even for a world-wide deployment. If the mirror maker fails to connect to one of the clusters, the mirroring will pause. Mirroring can be resumed automatically once connectivity is reestablished. The mirroring mechanism is unidirectional. Changing the value on the mirrored cluster won't reflect the value back to the origin cluster. The mirror maker only mirrors key-value pairs; metadata, such as version number or modification revision, is discarded. However, mirror maker still attempts to preserve update ordering during normal operation, but there is no ordering guarantee during initial sync nor during failure recovery following network interruption. As a rule of thumb, the ordering of the updates on the mirror should not be considered reliable. ``` +-+ | | | source | +--+ | cluster +-> | mirror | | | | maker | +-+ ++-+ | v +-+ | | | mirror | | cluster | | | +-+ ``` Mirror-maker is a built-in feature of .