cwielenb/CNCF-Dataset huggingface dataset:
Project: Envoy (Orchestration & Management / Service Proxy)
File: upstream_filters.md

Before coverting a filter to be an upstream HTTP filter you should do some basic functionality analysis. Make sure The filter does not use any downstream-only functionality, accessed via downstreamCallbacks() such as clearing cached routes, or performing internal redirects. Either the filter does not sendLocalReply, or you test and document how sendLocalReply code paths will play with hedging / retries (cut off the hedge attempt, and local-reply failures won't trigger retries) Either the filter does not access streamInfo in a non-const way, or you test and document how the filter interacts with hedging and retries. Note that for hedging, a single downstream StreamInfo is accessible in parallel to both instances of the upstream HTTP filter instance, so it must be resiliant to parallel access. Any code accessing the downstream connection checks to make sure it is present. The downstream connection will not be available for mirrored/shadowed requests. Once you've done this, you're ready to convert. An example converted filter is the Envoy Assuming your filter inherits from FactoryBase: Change from inheriting from FactoryBase to inheriting from DualFactoryBase This changes the type of createFilterFactoryFromProto. Update accordingly. Please note if you use the init manager or a stats context you must get them from DualInfo rather than from the server factory context or xDS reloads will not work correctly and may leak memory. Add ``using UpstreamMyFilterFactory = MyFilterFactory;`` in your config.h file and ``REGISTER_FACTORY(UpstreamMyFilterFactory, Server::Configuration::UpstreamHttpFilterConfigFactory);`` to your config.cc file. If your filter is listed in ``source/extensions/extensions_metadata.yaml`` add ``envoy.filters.http.upstream`` to the filter category. Your filter should now be available as an upstream HTTP filter. An example PR for conversion is
Project: gRPC (Orchestration & Management / Remote Procedure Call)
File: bug_report_ruby.md

name: Report a gRPC Ruby bug about: Create a report to help us improve labels: kind/bug, priority/P2, lang/ruby assignees: apolcyn <!-- PLEASE DO NOT POST A QUESTION HERE. This form is for bug reports and feature requests ONLY! For general questions and troubleshooting, please ask/look for answers at StackOverflow, with "grpc" tag: https://stackoverflow.com/questions/tagged/grpc For questions that specifically need to be answered by gRPC team members, please ask/look for answers at grpc.io mailing list: https://groups.google.com/forum/#!forum/grpc-io Issues specific to grpc-java, grpc-go, grpc-node, grpc-dart, grpc-web should be created in the repository they belong to (e.g. https://github.com/grpc/grpc-LANGUAGE/issues/new) --> Please provide either 1) A unit test for reproducing the bug or 2) Specific steps for us to follow to reproduce the bug. If theres not enough information to debug the problem, gRPC team may close the issue at their discretion. Youre welcome to re-open the issue once you have a reproduction. Make sure you include information that can help us debug (full error message, exception listing, stack trace, logs). See for how to diagnose problems better.
Project: Docker Swarm (Orchestration & Management / Scheduling & Orchestration)
File: MAINTAINERS.md

This page lists all active maintainers of this repository. If you were a maintainer and would like to add your name to the Emeritus list, please send us a PR. See for governance guidelines and how to become a maintainer. See for general contribution guidelines. , Google LLC , Google LLC , Google LLC , Google LLC , Google LLC , Google LLC , Google LLC , Google LLC , Google LLC , Google LLC , Google LLC , Google LLC , Google LLC , Google LLC , Google LLC
Project: Apache Mesos (Orchestration & Management / Scheduling & Orchestration)
File: 2014-05-29-mesos-0-18-1-and-0-18-2-released.md

layout: post title: Mesos 0.18.1 and 0.18.2 Released permalink: /blog/mesos-mesos-0-18-1-and-0-18-2-released/ published: true post_author: display_name: Niklas Quarfot Nielsen twitter: quarfot tags: Release We recently tagged two new bug fixes including our latest release: version . A bug was introduced in 0.18.1, prompting the release of 0.18.2. If you're using 0.18.1, it is highly recommended that you upgrade. The latest version of Mesos is available on our page. With the release of 0.18.1, Maven is now used to compile and package Mesos' Java files. In addition, the Mesos Java artifacts shade the protocol buffer dependency, allowing frameworks to independently declare their protobuf dependency version. Furthermore, a new extract flag has been added to the URI protocol buffer which makes it possible to control whether the fetcher should extract a given artifact. Various bugs in the Mesos master, the containerizer and the JNI code have been fixed as well. (see ) 0.18.2 fixed a bug which caused the mesos-fetcher to not apply the executer bit. For additional details, please see: . If upgrading from 0.18.0 or 0.18.1, an upgrade can be applied seamlessly to running clusters. However, Maven is now a build dependency and must be installed in order to build Mesos with Java support. If upgrading from earlier versions than 0.18.0, please refer to the document. We encourage you to try out this release, and let us know what you think on the . You can also get in touch with us via or via .
Project: Kuma (Orchestration & Management / Service Mesh)
File: 036-internal-listeners.md

status: accepted At the moment we don't clearly distinguish between listeners used for service to service communication and internal listeners, like Prometheus listeners. We will add prefix `_` to internal listeners. At the moments internal listeners names start with `kuma` prefix, problem is that service can also have this prefix. To be able to easily distinguish between service listeners and internal listeners, we can add `_` prefix to listener name. I've looked into usage of these names in metrics, dashboards and other places and didn't find anything we should worry about. Also this convention is already used in MeshMetric and we didn't find any issues. When migrating resources names remember to add info on how to migrate in UPGRADE.md.
Project: Gloo (Orchestration & Management / API Gateway)
File: run-tests.md

- - - - - Gloo Edge testing leverages the test framework. As outlined in the linked documentation, Ginkgo pairs with the matcher library to provide a BDD-style testing framework. For more details about how to write tests, check out our . There are a few common make targets that can be used to run tests The `test` target provides a wrapper around invoking `ginkgo` with a set of useful flags. This is the base target that is used by all other test targets. Run tests with coverage reporting Run unit tests (ie non e2e) Run Run Run Shared environment variables that can be used to control the behavior of the tests are defined in . Below are a few that are commonly used: The `GINKGOUSERFLAGS` environment variable can be used to pass flags to Ginkgo. For example, to run the tests with very verbose output, you can run: ```bash GINKGOUSERFLAGS="-vv" make test ```` For the full set of available Ginkgo flags, check out the The `TEST_PKG` environment variable can be used to run a specific test suite. For example, to run the `test` test suite, you can run: ```bash TEST_PKG=test make test ``` If you would like to run multiple test suites, you can separate them with a comma: ```bash TEST_PKG=package1,package2 make test ``` If you would like to recursively run tests in a directory, you can use the `...` syntax: ```bash TEST_PKG=test/... make test ``` The `WAITONFAIL` environment variable can be used to prevent Ginkgo from cleaning up the Gloo Edge installation in case of failure. This is useful to inspect resources created by the test. A command to resume the test run (and thus clean up resources) will be logged to the output. See for more details about when it can be used. The `INVALIDTESTREQS` environment variable can be used to control the behavior of tests which depend on environment conditions that aren't satisfied. Options are `skip`, `run`, `fail`. The default is `fail`. For example, to skip tests that contain requirements which your machine doesn't support, run: ```bash INVALIDTESTREQS=skip make test ```
Project: Nomad (Orchestration & Management / Scheduling & Orchestration)
File: checklist-rpc-endpoint.md

Prefer adding a new message to changing any existing RPC messages. [ ] `Request` struct and `RequestType` constant in `nomad/structs/structs.go`. Append the constant, old constant values must remain unchanged. Just add the request type to this file, all other resource definitions must be on their own separate file. [ ] In `nomad/fsm.go`, add a dispatch case to the switch statement in `(n nomadFSM) Apply` `nomadFSM` method to decode the request and call the state method [ ] State method for modifying objects in a `Txn` in the `state` package, located in `nomad/state/`. Every new resource should have its own file and test file, named using the convention `nomad/state/statestore[resource].go` and `nomad/state/statestore[resource]_test.go` [ ] Handler for the request in `nomad/foo_endpoint.go` RPCs are resolved by matching the method name for bound structs Register any new RPC structs in `nomad/server.go` Authentication: For RPCs that support HTTP APIs, call `Authenticate` before forwarding. Return any error after frowarding, and call `ResolveACL` to get an ACL to check. For RPCs that support client-to-server RPCs only, use `AuthenticateClientOnly` before forwarding. Check the `AllowClientOp` ACL after forwarding. For RPCs that support server-to-server RPCs only, use `AuthenticateServerOnly` before forwarding. Check the `AllowServerOp` ACL before forwarding. Authorization: Use `ResolveACL` to turn the authenticated request into an ACL to check. For Update/Get/Delete RPCs, check ACLs before hitting the state store. For List RPCs, use ACLs as a filter on the query. Never check that the ACL object is `nil` to bypass authorization. The authorization methods in `acl/acl.go` should already handle `nil` ACL objects correctly (by rejecting them). [ ] Wrapper for the HTTP request in `command/agent/foo_endpoint.go` Backwards compatibility requires a new endpoint, an upgraded client or server may be forwarding this request to an old server, without support for the new RPC RPCs triggered by an internal process may not need support Check ACLs as an optimization [ ] `nomad/core_sched.go` sends many RPCs `ServersMeetMinimumVersion` asserts that the server cluster is upgraded, so use this to guard sending the new RPC, else send the old RPC Version must match the actual release version! [ ] If implementing a Client RPC... Use `QueryOptions` instead of `WriteRequest` in the Request struct as `WriteRequest` is only for Raft writes. Set `QueryOptions.AllowStale = true` in the Server* RPC forwarder to avoid an infinite loop between leaders and followers when a Client RPC is forwarded through a follower. See https://github.com/hashicorp/nomad/issues/16517 [ ] Changelog for RPCs with an HTTP endpoint, include ACLs, params, and example response body.
Project: GoFr (Orchestration & Management / Remote Procedure Call)
File: pull_request_template.md

Description: Provide a concise explanation of the changes made. Mention the issue number(s) this PR addresses (if applicable). Highlight the motivation behind the changes and the expected benefits. Breaking Changes (if applicable): List any breaking changes introduced by this PR. Explain the rationale behind these changes and how they will impact users. Additional Information: Mention any relevant dependencies or external libraries used. Include screenshots or code snippets (if necessary) to clarify the changes. Checklist: [ ] I have formatted my code using `goimport` and `golangci-lint`. [ ] All new code is covered by unit tests. [ ] This PR does not decrease the overall code coverage. [ ] I have reviewed the code comments and documentation for clarity. Thank you for your contribution!
Project: Fluid (Orchestration & Management / Scheduling & Orchestration)
File: concepts.md

A dataset is a collection of logically related data that is used by computation engines, such as Spark for big data and TensorFlow for AI. Intelligent applications of these datasets create core value for industry. Dataset management has multiple dimensions, including security, version control, and data acceleration. We aim to provide support for dataset management with a focus on data acceleration. For example, we support aggregation of data from different storage sources, portability, and data features. Data Source: Supports multiple data sources with different protocols, including HDFS, S3, OSS, and the native Kubernetes Persistent Volume Claim protocol. Multiple data sources can also be mounted under different subdirectories in a unified namespace. Placement Policy: cached dataset on nodes of different types using the strong and weak affinity and toleration of the nodeAffinity in Kubernetes semantics. <div align="center"> <img src="../../../static/concepts/dataset.png" title="perspective" height="60%" width="60%" alt=""> </div> At the same time, Dataset provides observability, such as how much data is in the dataset, how much cache space is currently available, and what the cache hit rate is. Users can use this information to decide whether to scale up or down. <div align="center"> <img src="../../../static/concepts/dataset-status.png" title="perspective" height="60%" width="60%" alt=""> </div> Dataset is a unified abstract concept, and the actual data operations are implemented by specific runtimes. Due to the differences in storage, there are different runtime interfaces. The introduction of runtime is necessary for accessing the data. The API specification here can be defined relatively flexibly, but the lifecycle of the runtime is unifiedly defined by Fluid, and the implementer of the runtime needs to complete the specific implementation according to the common interface definition. In Fluid, the Runtime is divided into two main categories: CacheRuntime implements cache acceleration, including the open-source distributed cache Alluxio which mainly accelerates S3, HDFS, and JuiceFS, JindoFS which accelerates OSS and OSS+HDFS, and GooseFS which supports COS. ThinRuntime provides a unified access interface, such as supporting distributed storage systems like s3fs and nfs-fuse. Fluid's universal data operations describe operations such as data prefetch, data migration, elastic scaling, cache cleaning, metadata backup, and recovery. The directory to be prefetched and the preheating strategy can be one-time, scheduled, or event-triggered can be specified. Support manual scaling, elastic scaling, and scheduled scaling as various strategies for scaling. Supports both importing data from external storage into a dataset before using it, and using a dataset while importing data into it. Full concept <div align="center"> <img src="../../../static/concepts/concept.png" title="perspective" height="60%" width="60%" alt=""> </div>
Project: k8gb (Orchestration & Management / Coordination & Service Discovery)
File: provider_rfc2136.md

In order to enable the provider RFC2136 on ExternalDNS, the following should be changed in the values.yaml of the K8GB helm chart: One authentication method should be enabled on the values: Insecure This method doesn't use any authentication and anonymous updates to the DNS records can be executed TSIG This method uses TSIG authentication that relies on a token provided for the DNS records update. GSS-TSIG This method uses GSS-TSIG authentication, which is a variation of the TSIG method, but uses Kerberos for the generation of tokens for authentication and authorization Method used by Active Directory Windows DNS GSS-TSIG kerberos-username this key should have the value of a Active Directory user account that has permissions for DNS updates kerberos-password password of the user account that will be used. Be aware that this isn't encrypted and so far ExternalDNS doesn't support adding a Secret reference for this value, so it will be stored in clear text kerberos-realm domain that will be used for authentication of the user ```yaml rfc2136: enabled: true rfc2136Opts: host: yourAcDc.k8gb.local port: 53 rfc2136auth: insecure: enabled: false tsig: enabled: false tsigCreds: tsig-secret-alg: hmac-sha256 tsig-keyname: externaldns-key gssTsig: enabled: true gssTsigCreds: kerberos-username: someServiceAccount kerberos-password: insecurePlainTextPassword kerberos-realm: yourKerberosRealm.domain ```
Project: CoreDNS (Orchestration & Management / Coordination & Service Discovery)
File: coredns-008.md

+++ date = "2017-06-14T22:52:11Z" description = "CoreDNS-008 Release Notes." release = "008" tags = ["Release", "008", "Notes"] title = "CoreDNS-008 Release" author = "coredns" +++ CoreDNS-008 has been ! CoreDNS is a DNS server that chains plugins, where each plugin implements a DNS feature. Release v008 has a lot of content, with new plugin and major features added to existing plugin. Please note there is an incompatible change to the `log` directive - it now only logs to `stdout` and so only allows `stdout` as the file name (which of course may be omitted). `-log` flag was changed into a boolean as all logging will be written to standard output. hosts* allows CoreDNS to read a `/etc/hosts` styled file and generate responses from that. debug* can disable the `panic/recover` that is enabled by default. Mostly useful for testing/non-prod use cases to generate stack traces. chaos* now returns the correct `version.bind` `TXT` record. kubernetes* Now returns a proper NS record for the cluster domain Supports `ExternalName` services, which was an oversight in the 1.0.0 version of the Now supports federation records Has had some other bug fixes. file* Now supports DNAME Refuse to load a zone without a SOA record. file, auto* don't reload a zone when the SOA's serial hasn't changed. secondary* now behaves properly if queried before the zone has been transferred log, errors output everything to stdout and let `journald` or `docker` (or whatever) that care of further handling. This is backwards incompatible* change wrt to the Corefile: `log query.log` will return an error. cache* got a new cache implementation to be more scalable and a new `prefetch` option for fetching records before the TTL expires. proxy does not use `singleinflight` anymore, removing a potential bottleneck on the single mutex in that implementation; it now forwards all* queries it get to the upstream nameserver. The following people helped with getting this release done: Chris Aniszczyk, Chris O'Haver, cricketliu, Eric Yan, John Belamaric, Jonas stanbck, Manuel Alejandro de Brito Fontes, Miek Gieben, Pat Moroney, Yong Tang If you want to help, please check out one of the and start coding! For documentation and help, see our .
Project: Kubeflow (Orchestration & Management / Scheduling & Orchestration)
File: CONTRIBUTING.md

Welcome to the Kubeflow project! We'd love to accept your patches and contributions to this project. Please read the . The contributor's guide shows you where to find the Contributor License Agreement (CLA) that you need to sign, helps you get started with your first contribution to Kubeflow, and describes the pull request and review workflow in detail, including the OWNERS files and automated workflow tool.
Project: Gloo (Orchestration & Management / API Gateway)
File: glooctl_edit_route_ratelimit_client-config.md

title: "glooctl edit route ratelimit client-config" weight: 5 Add rate-limits (Enterprise) Configure rate-limits, which are composed of rate-limit actions that translate request characteristics to rate-limit descriptor tuples. For available actions and more information see: https://www.envoyproxy.io/docs/envoy/v1.9.0/api-v2/api/v2/route/route.proto#route-ratelimit-action This is a Gloo Enterprise feature. ``` glooctl edit route ratelimit client-config [flags] ``` ``` -h, --help help for client-config ``` ``` -c, --config string set the path to the glooctl config file (default "<home_directory>/.gloo/glooctl-config.yaml") --consul-address string address of the Consul server. Use with --use-consul (default "127.0.0.1:8500") --consul-allow-stale-reads Allows reading using Consul's stale consistency mode. --consul-datacenter string Datacenter to use. If not provided, the default agent datacenter is used. Use with --use-consul --consul-root-key string key prefix for for Consul key-value storage. (default "gloo") --consul-scheme string URI scheme for the Consul server. Use with --use-consul (default "http") --consul-token string Token is used to provide a per-request ACL token which overrides the agent's default token. Use with --use-consul -x, --index uint32 edit the route with this index in the virtual service route list -i, --interactive use interactive mode --kube-context string kube context to use when interacting with kubernetes --kubeconfig string kubeconfig to use, if not standard one --name string name of the resource to read or write -n, --namespace string namespace for reading or writing resources (default "gloo-system") -o, --output OutputType output format: (yaml, json, table, kube-yaml, wide) (default table) --resource-version string the resource version of the resource we are editing. if not empty, resource will only be changed if the resource version matches --use-consul use Consul Key-Value storage as the backend for reading and writing config (VirtualServices, Upstreams, and Proxies) ``` - Configure rate-limits (Enterprise)