cwielenb/CNCF-Dataset huggingface dataset:
Project: Fluid (Orchestration & Management / Scheduling & Orchestration)
File: gcs_configuration.md

If `Google Cloud Storage (GCS)` is selected as the underlying storage system of Alluxio, additional configuration is required for Alluxio to properly access the mounted GCS storage system. This document shows how to do the special configuration required for Alluxio in Fluid in a declarative manner. For more information, see . The that has permission to access the bucket Please refer to the to complete the installation. In this example, we will use `GCS UFS version 2` which uses `Google Application Credentials` to access GCS. It is the recommended way to access GCS. Accessing GCS in Alluxio is a bit different from other object storage systems like S3. You have to mount the `Google Application Credentials` which is a `json` file into your Alluxio master and workers, and provide the path to the json file in the Alluxio properties. Create Secret Run the following command to create a secret called `gcscreds` with key `gcloud-application-credentials.json` and value of the json file. ```sh kubectl create secret generic gcscreds \ --from-file=gcloud-application-credentials.json=<name of your google application credentials>.json ``` Create AlluxioRuntime Resource Object Here you create your Alluxio runtime and mount the above secret into master and workers, the path in both master and worker must be the same. ```yaml $ cat << EOF > runtime.yaml apiVersion: data.fluid.io/v1alpha1 kind: AlluxioRuntime metadata: name: gcs-data spec: ... volumes: name: secret-volume secret: secretName: gcscreds # secret name you created master: volumeMounts: name: secret-volume readOnly: true mountPath: "/var/secrets" worker: volumeMounts: name: secret-volume readOnly: true mountPath: "/var/secrets" EOF ``` Create the runtime ``` $ kubectl create -f runtime.yaml ``` Create Dataset Resource Object Here you specify your GCS bucket and directory in the `mountPoint`, and provide the path to your `google application credentials` (the path you mounted in the runtime previously) in the `options`. ```yaml $ cat << EOF > dataset.yaml apiVersion: data.fluid.io/v1alpha1 kind: Dataset metadata: name: gcs-data spec: mounts: mountPoint: gs://<bucket-name>/<path-to-data> name: gcs options: fs.gcs.credential.path: /var/secrets/gcloud-application-credentials.json accessModes: ReadOnlyMany EOF ``` Create your dataset ``` $ kubectl create -f dataset.yaml ``` The bucket specified in 'dataset.yaml' will be mounted to the '/gcs' directory in Alluxio.
Project: DolphinScheduler (Orchestration & Management / Scheduling & Orchestration)
File: datasync.md

is an online data transfer service that simplifies, automates, and accelerates moving data between on-premises storage systems and AWS Storage services, as well as between AWS Storage services. DataSync can copy data to and from: Network File System (NFS) file servers Server Message Block (SMB) file servers Hadoop Distributed File System (HDFS) Object storage systems Amazon Simple Storage Service (Amazon S3) buckets Amazon EFS file systems Amazon FSx for Windows File Server file systems Amazon FSx for Lustre file systems Amazon FSx for OpenZFS file systems Amazon FSx for NetApp ONTAP file systems AWS Snowcone devices The follow shows the DolphinScheduler DataSync task plugin features: Create an AWS DataSync task and execute, continuously get the execution status until the task completes. Click `Project -> Management-Project -> Name-Workflow Definition`, and click the "Create Workflow" button to enter the DAG editing page. Drag from the toolbar <img src="../../../../img/tasks/icons/datasync.png" width="15"/> task node to canvas. Please refer to `Default Task Parameters` section for default parameters. Here are some specific parameters for the DataSync plugin: name: Task name destinationLocationArn: The Amazon Resource Name (ARN) of an AWS storage resource's location. Visit sourceLocationArn: The Amazon Resource Name (ARN) of the source location for the task. Visit cloudWatchLogGroupArn: The Amazon Resource Name (ARN) of the Amazon CloudWatch log group that is used to monitor and log events in the task. Visit or json: JSON parameters to construct datasync task, and supports parameters like options, etc. Visit: [AWS CreateTask API] Request Syntax (https://docs.aws.amazon.com/datasync/latest/userguide/API_CreateTask.html) The following shows the task plugin example: The task requires AWS configurations, modify the value in file `common.properties` ```yaml resource.aws.access.key.id=<YOUR AWS ACCESS KEY> resource.aws.secret.access.key=<YOUR AWS SECRET KEY> resource.aws.region=<AWS REGION> ```
Project: Gloo (Orchestration & Management / API Gateway)
File: glooctl_istio.md

title: "glooctl istio" weight: 5 Commands for interacting with Istio in Gloo ``` glooctl istio [flags] ``` ``` -h, --help help for istio --name string name of the resource to read or write -n, --namespace string namespace for reading or writing resources (default "gloo-system") ``` ``` -c, --config string set the path to the glooctl config file (default "<home_directory>/.gloo/glooctl-config.yaml") --consul-address string address of the Consul server. Use with --use-consul (default "127.0.0.1:8500") --consul-allow-stale-reads Allows reading using Consul's stale consistency mode. --consul-datacenter string Datacenter to use. If not provided, the default agent datacenter is used. Use with --use-consul --consul-root-key string key prefix for for Consul key-value storage. (default "gloo") --consul-scheme string URI scheme for the Consul server. Use with --use-consul (default "http") --consul-token string Token is used to provide a per-request ACL token which overrides the agent's default token. Use with --use-consul -i, --interactive use interactive mode --kube-context string kube context to use when interacting with kubernetes --kubeconfig string kubeconfig to use, if not standard one --use-consul use Consul Key-Value storage as the backend for reading and writing config (VirtualServices, Upstreams, and Proxies) ``` - CLI for Gloo - Disables Istio mTLS for a given upstream - Enables Istio mTLS for a given upstream - Enable SDS & istio-proxy sidecars in gateway-proxy pod - Remove SDS & istio-proxy sidecars from gateway-proxy pod
Project: DolphinScheduler (Orchestration & Management / Scheduling & Orchestration)
File: DS-License.md

As we know that DolphinScheduler is an open-source undergoing project at The Apache Software Foundation (ASF), which means that you have to follow the Apache way to become the DolphinScheduler contributor. Furthermore, Apache has extremely strict rules according to the License. This passage will explain the ASF license and how to avoid License risks at the early stage when you participate in DolphinScheduler. Note: This article only applies to the Apache projects. You have to pay attention to the following open-source software protocols which Apache projects support when you intend to add a new feature to the DolphinScheduler (or other Apache projects), which functions refers to other open-source software references. If the 3rd party software is not present at the above policy, we are sorry that your code can not pass the audit and we suggest searching for other substitute plans. Besides, when you demand new dependencies in the project, please email us about the reason and the outcome of the influence to dev@dolphinscheduler.apache.org to discuss. Besides, you need at least 3 positive votes from the PPMC to finish the whole step. Moreover, when we intend to refer a new software ( not limited to 3rd party jar, text, CSS, js, pics, icons, audios etc and modifications based on 3rd party files) to our project, we need to use them legally in addition to the permission of ASF. Refer to the following article: For example, we should contain the NOTICE file (every open-source project has NOTICE file, generally under root directory) of ZooKeeper in our project when we are using ZooKeeper. As the Apache explains, "Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work. We are not going to dive into every 3rd party open-source license policy, you may look up them if interested. In general, we would have our License-check scripts to our project. DolphinScheduler-License is provided by which differ a bit from other open-source projects. All in all, we are trying to make sure avoiding the license issues at the first time. We need to follow the following steps when we need to add new jars or external resources: Add the name and the version of the jar file in the known-dependencies.txt Add relevant maven repository address under 'dolphinscheduler-dist/release-docs/LICENSE' directory Append relevant NOTICE files under 'dolphinscheduler-dist/release-docs/NOTICE' directory and make sure they are no different to the original repository Add relevant source code protocols under 'dolphinscheduler-dist/release-docs/license/' directory and the file name should be named as license+filename.txt. Eg: license-zk.txt
Project: OpenFunction (Orchestration & Management / Scheduling & Orchestration)
File: GOVERNANCE.md

This document defines the project governance for OpenFunction. Users:* Members that engage with the OpenFunction community via any medium (Slack, WeChat, GitHub, mailing lists, etc.). Contributors:* Regular contributions to projects (documentation, code reviews, responding to issues, participation in proposal discussions, contributing code, etc.). Maintainers*: The OpenFunction project leaders. They are responsible for the overall health and direction of the project; final reviewers of PRs and responsible for releases. Some Maintainers are responsible for one or more components within a project, acting as technical leads for that component. Maintainers are expected to contribute code and documentation, review PRs including ensuring the quality of code, triage issues, proactively fix bugs, and perform maintenance tasks for these components. All decisions made by maintainers have to be a . are responsible for activities around maintaining and updating the OpenFunction. Final decisions on the features reside with the project maintainers. Maintainers MUST remain active. If they are unresponsive for >3 months, they will be automatically removed unless a of the other project maintainers agrees to extend the period to be greater than 3 months. New maintainers can be added to the project by a vote of the existing maintainers. A potential maintainer may be nominated by an existing maintainer. A vote is conducted in private between the current maintainers over the course of a one-week voting period. At the end of the week, votes are counted and a pull request is made on the repo adding the new maintainer to the file. Individuals interested in becoming maintainers may submit an stating their interest. Existing maintainers can choose if they would like to nominate these individuals to be a maintainer following the process above. A maintainer may step down by submitting an stating their intent. Changes to this governance document require a pull request with approval from a of the current maintainers. New projects can be added to the organization after a successful vote of the existing maintainers. When OpenFunction wants to archive an existing project, maintainers have to vote where a of the current maintainers agrees to archive the project with a clear indication of why it is being archived. We must not delete projects to avoid customer confusion.
Project: Gravitee.io (Orchestration & Management / API Gateway)
File: management-api-proxy-response-template.md

Response template must be defined according to the `Accept` request header. You can define as many content-type as you need to manage different response body, for example for `application/json` or `text/xml`. If none of the defined template is corresponding to the incoming `Accept` request header, you can define a fallback response using the type `/`. It no fallback is defined, the normal response from the policy will be sent to the API consumer. For each type, you must have to configure the response with: A status code HTTP headers A body The response body template is supporting the Expression Language so you can access many elements to provide a clean and clear response message. For the API-KEY: ```json { "message": "{#error.message}", "api-key": "{#parameters['api-key']}" } ``` In this example, `#error.message` is corresponding to the normal message send by the policy.
Project: Pipy (Orchestration & Management / Service Proxy)
File: README-QUIC.md

Using OpenSSL with QUIC ======================= From OpenSSL 3.2, OpenSSL features support for making QUIC connections as a client. Users interested in using the new QUIC functionality are encouraged to look at some of the following resources: The new [OpenSSL Guide], which provides introductory guides on the use of TLS, QUIC, and other OpenSSL functionality. The [OpenSSL Guide] incorporates various code samples. The complete source for these can be . The [openssl-quic(7) manual page], which provides a basic reference overview of QUIC functionality and how use of QUIC differs from use of TLS with regard to our API. The demos, which demonstrate the use of QUIC using simple examples. These can be [found in the source tree under `doc/designs/ddd`]. The [demo found in `demos/http3`], which provides an HTTP/3 client example using the nghttp3 HTTP/3 library. FAQ QUIC is a state-of-the-art secure transport protocol carried over UDP. It can serve many of the use cases of SSL/TLS as well as those of DTLS. QUIC delivers a number of advantages such as support for multiple streams of communication; it is the basis for HTTP/3 [RFC 9114]; fast connection initiation; and connection migration (enabling a connection to survive IP address changes). For a more complete description of what QUIC is and its advantages see the [QUIC Introduction] in the [OpenSSL Guide]. For a comprehensive overview of OpenSSL's QUIC implementation, see the [openssl-quic(7) manual page]. There are many HTTP/3 implementations in C available. The use of one such HTTP/3 library with OpenSSL QUIC is demonstrated via the [demo found in `demos/http3`]. The [OpenSSL Guide] provides introductory examples for how to make use of OpenSSL QUIC. The demos may also be helpful to illustrate the changes needed if you are trying to adapt an existing application. There is basic support for single-stream QUIC using `openssl s_client`: ```shell $ openssl s_client -quic -alpn myalpn -connect host:port ``` In the above example replace `host` with the hostname of the server (e.g. `www.example.com`) and `port` with the port for the server (e.g. `443`). Replace `myalpn` with the Application Layer Protocol to use (e.g.`h3` represents HTTP/3). IANA maintains a standard list of [ALPN ids] that can be used. This example connects to a QUIC server and opens a single bidirectional stream. Data can be passed via stdin/stdout as usual. This allows test usage of QUIC using simple TCP/TLS-like usage. Note that OpenSSL has no direct support for HTTP/3 so connecting to an HTTP/3 server should be possible but sending an HTTP/3 request or receiving any response data is not.
Project: Volcano (Orchestration & Management / Scheduling & Orchestration)
File: CHANGELOG_1.0.md

Fix job's scheduler name does not take effect (, ) Fix podgroup status and event (, ) Support job scale down to zero (, ) Fix queue capability validation failed when some running jobs finished or deleted (, ) GPU Sharing Volcano now supports gpu sharing between different pods (, , ). Preempt abd reclaim enhancement Volcano is now able to support preempt for batch job (, ). Dynamic scale up and down Volcano job now supports dynamically scale up and down (, ). Support integrate with flink operator Users are now able to run flink job with volcano. Follow the to make use of the feature. ). Support DAG job with argo Users are now able to run DAG job with volcano. Follow the to make use of the feature. ). Update go version to 1.14 (, ) Bump to k8s 1.18 to keep up with kubernetes (, ) Add mindspore example (, ) Add golangCi-lint check (, , , , , , , , , . ) Allow specifying admission webhook port (, ) Add support for most requested priority (, ) Set pod DNSPolicy to ClusterFirstWithHostNet when hostnetwork set (, ) Add bestNodeFn for plugins to select best node of its own (, ) Add e2e cases for drf (, ) Add e2e cases for reclaim ( , ) Add e2e cases for preempt (, ) Add e2e for queue (, ) Add test code for queue controller (, ) Fix panic in controller (, ) Fix panic in allocate (, ) Fix job phase transition time set (, ) Fix crd to support job patch ops (, )
Project: Gloo (Orchestration & Management / API Gateway)
File: metrics.proto.sk.md

title: "metrics.proto" weight: 5 <!-- Code generated by solo-kit. DO NOT EDIT. --> Copyright 2013 Prometheus Team Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. - ```yaml "name": string "value": string ``` | Field | Type | Description | | -- | - | -- | | `name` | `string` | | | `value` | `string` | | ```yaml "value": float ``` | Field | Type | Description | | -- | - | -- | | `value` | `float` | | ```yaml "value": float ``` | Field | Type | Description | | -- | - | -- | | `value` | `float` | | ```yaml "quantile": float "value": float ``` | Field | Type | Description | | -- | - | -- | | `quantile` | `float` | | | `value` | `float` | | ```yaml "sampleCount": int "sampleSum": float "quantile": []io.prometheus.client.Quantile ``` | Field | Type | Description | | -- | - | -- | | `sampleCount` | `int` | | | `sampleSum` | `float` | | | `quantile` | | | ```yaml "value": float ``` | Field | Type | Description | | -- | - | -- | | `value` | `float` | | ```yaml "sampleCount": int "sampleSum": float "bucket": []io.prometheus.client.Bucket ``` | Field | Type | Description | | -- | - | -- | | `sampleCount` | `int` | | | `sampleSum` | `float` | | | `bucket` | | | ```yaml "cumulativeCount": int "upperBound": float ``` | Field | Type | Description | | -- | - | -- | | `cumulativeCount` | `int` | | | `upperBound` | `float` | | ```yaml "label": []io.prometheus.client.LabelPair "gauge": .io.prometheus.client.Gauge "counter": .io.prometheus.client.Counter "summary": .io.prometheus.client.Summary "untyped": .io.prometheus.client.Untyped "histogram": .io.prometheus.client.Histogram "timestampMs": int ``` | Field | Type | Description | | -- | - | -- | | `label` | | | | `gauge` | | | | `counter` | | | | `summary` | | | | `untyped` | | | | `histogram` | | | | `timestampMs` | `int` | | ```yaml "name": string "help": string "type": .io.prometheus.client.MetricType "metric": []io.prometheus.client.Metric ``` | Field | Type | Description | | -- | - | -- | | `name` | `string` | | | `help` | `string` | | | `type` | | | | `metric` | | | Description: | Name | Description | | -- | -- | | COUNTER | | | GAUGE | | | SUMMARY | | | UNTYPED | | | HISTOGRAM | | <!-- Start of HubSpot Embed Code --> <script type="text/javascript" id="hs-script-loader" async defer src="//js.hs-scripts.com/5130874.js"></script> <!-- End of HubSpot Embed Code -->
Project: Gloo (Orchestration & Management / API Gateway)
File: advanced_http.proto.sk.md

title: "advanced_http.proto" weight: 5 <!-- Code generated by solo-kit. DO NOT EDIT. --> - ```yaml "responseMatchers": []advancedhttp.options.gloo.solo.io.ResponseMatcher "noMatchHealth": .advancedhttp.options.gloo.solo.io.HealthCheckResult ``` | Field | Type | Description | | -- | - | -- | | `responseMatchers` | | A bunch of match rules, the first match wins out and short-circuits. | | `noMatchHealth` | | The default health response if none of the response health checks were matches. If omitted, defaults to unhealthy. | Defines a transformation template. ```yaml "responseMatch": .advancedhttp.options.gloo.solo.io.ResponseMatch "matchHealth": .advancedhttp.options.gloo.solo.io.HealthCheckResult ``` | Field | Type | Description | | -- | - | -- | | `responseMatch` | | Defines the parameters to determine a single match. | | `matchHealth` | | The health response if this response_match is a match. If omitted, defaults to healthy. | ResponseMatches can be used to extract information from the request/response. ```yaml "jsonKey": .advancedhttp.options.gloo.solo.io.JsonKey "ignoreErrorOnParse": bool "header": string "body": .google.protobuf.Empty "regex": string ``` | Field | Type | Description | | -- | - | -- | | `jsonKey` | | Configuration to get the json key. Treats the body as raw text if omitted. | | `ignoreErrorOnParse` | `bool` | If set to true, Envoy will not throw an exception in case the json body parsing fails. | | `header` | `string` | Extract information from headers. Only one of `header` or `body` can be set. | | `body` | | Extract information from the request/response body. Only one of `body` or `header` can be set. | | `regex` | `string` | Only strings matching this regular expression will be considered a match. The most simple value for this field is '.*', which matches the whole source. The field is required. | ```yaml "path": []advancedhttp.options.gloo.solo.io.JsonKey.PathSegment ``` | Field | Type | Description | | -- | - | -- | | `path` | | The path to retrieve the Value. | Specifies the segment in a path to retrieve value. ```yaml "key": string ``` | Field | Type | Description | | -- | - | -- | | `key` | `string` | If specified, use the key to retrieve the value. If the key is not found, the value defaults to empty string. | Description: | Name | Description | | -- | -- | | default | | | healthy | | | degraded | | | unhealthy | | <!-- Start of HubSpot Embed Code --> <script type="text/javascript" id="hs-script-loader" async defer src="//js.hs-scripts.com/5130874.js"></script> <!-- End of HubSpot Embed Code -->
Project: Contour (Orchestration & Management / Service Proxy)
File: CHANGELOG-v1.24.1.md

We are delighted to present version v1.24.1 of Contour, our layer 7 HTTP reverse proxy for Kubernetes clusters. Update to Envoy v1.25.1. See the for more information about the content of the release. For a fresh install of Contour, consult the . To upgrade an existing Contour installation, please consult the . Contour v1.24.1 is tested against Kubernetes 1.24 through 1.26. If you're using Contour and want to add your organization to our adopters list, please visit this . If you prefer to keep your organization name anonymous but still give us feedback into your usage and scenarios for Contour, please post on this .