cwielenb/CNCF-Dataset huggingface dataset:
Project: Traefik (Orchestration & Management / API Gateway)
File: statsd.md

title: "Traefik StatsD Documentation" description: "Traefik supports several metrics backends, including StatsD. Learn how to implement it for observability in Traefik Proxy. Read the technical documentation." To enable the Statsd: ```yaml tab="File (YAML)" metrics: statsD: {} ``` ```toml tab="File (TOML)" [metrics] [metrics.statsD] ``` ```bash tab="CLI" --metrics.statsd=true ``` Required, Default="localhost:8125" Address instructs exporter to send metrics to statsd at this address. ```yaml tab="File (YAML)" metrics: statsD: address: localhost:8125 ``` ```toml tab="File (TOML)" [metrics] [metrics.statsD] address = "localhost:8125" ``` ```bash tab="CLI" --metrics.statsd.address=localhost:8125 ``` Optional, Default=true Enable metrics on entry points. ```yaml tab="File (YAML)" metrics: statsD: addEntryPointsLabels: true ``` ```toml tab="File (TOML)" [metrics] [metrics.statsD] addEntryPointsLabels = true ``` ```bash tab="CLI" --metrics.statsd.addEntryPointsLabels=true ``` Optional, Default=false Enable metrics on routers. ```yaml tab="File (YAML)" metrics: statsD: addRoutersLabels: true ``` ```toml tab="File (TOML)" [metrics] [metrics.statsD] addRoutersLabels = true ``` ```bash tab="CLI" --metrics.statsd.addrouterslabels=true ``` Optional, Default=true Enable metrics on services. ```yaml tab="File (YAML)" metrics: statsD: addServicesLabels: true ``` ```toml tab="File (TOML)" [metrics] [metrics.statsD] addServicesLabels = true ``` ```bash tab="CLI" --metrics.statsd.addServicesLabels=true ``` Optional, Default=10s The interval used by the exporter to push metrics to statsD. ```yaml tab="File (YAML)" metrics: statsD: pushInterval: 10s ``` ```toml tab="File (TOML)" [metrics] [metrics.statsD] pushInterval = "10s" ``` ```bash tab="CLI" --metrics.statsd.pushInterval=10s ``` Optional, Default="traefik" The prefix to use for metrics collection. ```yaml tab="File (YAML)" metrics: statsD: prefix: traefik ``` ```toml tab="File (TOML)" [metrics] [metrics.statsD] prefix = "traefik" ``` ```bash tab="CLI" --metrics.statsd.prefix=traefik ```
Project: Kusk Gateway (Orchestration & Management / API Gateway)
File: jwt.md

Kusk allows you to protect your APIs in Kubernetes using JWT. In this guide you'll learn how to add JWT authentication to your API. For the purposes of this example, we're using Auth0's JWT authentication. Signup for an account at . Create an Auth0 API Application. Go to Applications > APIs Configure the following Auth0 Application fields: Name (e.g. `example-api`) Identifier (e.g. `https://example-api`) Signing algorithm: RS256 Take note of your Auth0 domain, which you can find here: ```sh kubectl create deployment hello-world --image=kubeshop/kusk-hello-world:v1.0.0 kubectl expose deployment hello-world --name hello-world-svc --port=8080 ``` ```sh title="Expected output" deployment.apps/hello-world created service/hello-world-svc exposed ``` A minimal example of the configuration for the JWT filter is: ```yaml title="api.yaml" openapi: 3.1.0 info: title: jwt-oauth0 description: jwt-oauth0 version: 0.1.0 x-kusk: upstream: service: name: hello-world-svc namespace: default port: 8080 auth: jwt: providers: name: kusk-gateway-jwt issuer: $ISSUER_URL audiences: $JWT_AUDIENCE jwks: $JWKS_URL paths: /hello: get: description: Returns a hello world greeting operationId: /hello responses: {} ``` You are required to change: `issuer` (for example: `https://$AUTH0_DOMAIN`) `audiences` (for example: `https://$AUTH0_IDENTIFIER`, you can get the Identifier field from the Settings tab) `jwks` (for example: `https://$AUTH0_DOMAIN/.well-known/jwks.json`) After that, deploy the API by running: ``` kusk deploy -i api.yaml ``` The best way to get a token to test authentication with from Auth0 is using the `curl` command they offer in the test tab: Run the `curl` command and take note of the token: ```sh curl --request POST \ --url https://aabedraba.eu.auth0.com/oauth/token \ --header 'content-type: application/json' \ --data '{"clientid":"$CLIENTID","clientsecret":"$CLIENTSECRET","audience":"https://example-api","granttype":"clientcredentials"}' ``` ```sh title="Expected output" {"accesstoken":"TOKENTOCOPY","expiresin":86400,"token_type":"Bearer"} ``` Get the IP address of Kusk by running: ```sh kusk ip 12.34.56.78 ``` ```sh curl -H 'Authorization: Bearer "YOUR_TOKEN"' 12.34.56.78/hello ``` ```sh title="Expected output" { "message": "Hello from an implemented service!" } ``` ```sh curl 12.34.56.78/hello Jwt is missing ``` ```sh curl -H 'Authorization: Bearer invalid-token' 12.34.56.78/hello Jwt header is an invalid JSONShould pass since Bearer token is present ``` With Kusk, authenticating your APIs using JWT tokens is as easy as adding an extension to your OpenAPI definition. Stay safe out there!
Project: DolphinScheduler (Orchestration & Management / Scheduling & Orchestration)
File: appendix.md

DolphinScheduler task plugins share some common default parameters. Each type of task contains all or some default parameters as follows: | Parameter | Description | |--|| | Node Name | The name of the task. Node names within the same workflow must be unique. | | Run Flag | Indicating whether to schedule the task. If you do not need to execute the task, you can turn on the `Prohibition execution` switch. | | Cache Execution | Indicating whether this node needs to be cached. If it is cached, the same identifier (same task version, same task definition, same parameter input) task is cached. When the task has been cached, it will not be executed again, and the result will be reused directly. | | Description | Describing the function of this node. | | Task Priority | When the number of the worker threads is insufficient, the worker executes task according to the priority. When two tasks have the same priority, the worker will execute them in `first come first served` fashion. | | Worker Group | Machines which execute the tasks. If you choose `default`, scheduler will send the task to a random worker. | | Task Group Name | Resource group of tasks. It will not take effect if not configured. | | Environment Name | Environment to execute the task. | | Number of Failed Retries | The number of task retries for failures. You could select it by drop-down menu or fill it manually. | | Failure Retry Interval | Interval of task retries for failures. You could select it by drop-down menu or fill it manually. | | CPU Quota | Assign the specified CPU time quota to the task executed. Takes a percentage value. Default -1 means unlimited. For example, the full CPU load of one core is 100%, and that of 16 cores is 1600%. You could configure it by . | | Max Memory | Assign the specified max memory to the task executed. Exceeding this limit will trigger oom to be killed and will not automatically retry. Takes an MB value. Default -1 means unlimited. You could configure it by . | | Timeout Alarm | Alarm for task timeout. When the task exceeds the "timeout threshold", an alarm email will send. | | Delayed Execution Time | The time that a task delays for execution in minutes. | | Resources | Resources which your task node uses. | | Predecessor Task | The upstream task of the current task node. |
Project: Apache Mesos (Orchestration & Management / Scheduling & Orchestration)
File: 2018-09-05-mesoscon-2018-cfp-is-now-closed.md

layout: post title: MesosCon 2018 CFP is now closed! permalink: /blog/mesoscon-2018-cfp-is-now-closed/ published: true post_author: display_name: Andrew Schwartzmeyer gravatar: a0f86f38a2770f3ec0bbb1ef07f2d31b twitter: andschwa tags: MesosCon, Announcement Thank you everyone for submitting proposals, our Call For Proposals is now closed. We would like to communicate an important change: MesosCon 2018 will instead be held in the San Francisco area on November 5th-7th, 2018. The dates remain unchanged, but it will not be held in New York City as previously announced. After much discussion, the majority of the community found that the Bay Area would be easier to attend. We apologize for this last minute decision and any inconvenience that it might cause you. We are trying to have a small satellite event in New York City, and are also trying to allow for remote participation where feasible. Thank you again for everyone who submitted talks. We hope to inform each submitter by the end of September, and to announce the final format and venue at this time. MesosCon is a community run conference and your support is needed. Sponsoring this event will provide a great opportunity for your organization to further engage with the Mesos community. to learn more. Help us make #MesosCon 2018 a big success by spreading the word! Hope to see you at this year's MesosCon!
Project: wasmCloud (Orchestration & Management / Scheduling & Orchestration)
File: DEVELOPMENT.md

This document serves as a guide and reference for people looking to develop `wash`. - - - - To contribute to `wash`, you just need installed. To run any `wash` tests, you need to install . With a Rust toolchain already installed, you can simply install this with: ```bash cargo install cargo-nextest --locked ``` The dependency check script will also install this for you, see that section below. To run the `wash build` integration tests that compile components using actual language toolchains, you must have those toolchains installed. Currently the requirements for this are: - The `wasm32-unknown-unknown` target must be installed. You can install this with: `rustup target add wasm32-unknown-unknown`. - TinyGo also requires to be installed. To make it easy to ensure you have all the right tools installed to run all of the `wash` tests, we've created a Python script at `tools/depscheck.py`. You can run this using `make deps-check` or `python3 ./tools/depscheck.py`. While developing `wash`, consider installing the following optional development tools: (`cargo install cargo-watch`) to enable the `*-watch` commands These will be automatically installed using the `deps_check.py` script as well. To build the project: ```console make build ``` To build continuously (thanks to ): ```console make build-watch ``` To test all unit tests: ```console make test ``` To test all unit tests continuously: ```console make test-watch ``` To test a specific target test(s) continuously: ```console TARGET=integrationnewhandlesdashednames make test-watch ``` For us to be able to merge in any commits, they need to be signed off. If you didn't do so, the PR bot will let you know how to fix it, but it's worth knowing how to do it in advance. There are a few options: use `git commit -s` in the CLI in `vscode`, go to settings and set the `git.alwaysSignOff` setting. Note that the dev container configuration in this repo sets this up by default. manually add "Signed-off-by: NAME <EMAIL>" at the end of each commit You may also be able to use GPG signing in lieu of a sign off.
Project: CoreDNS (Orchestration & Management / Coordination & Service Discovery)
File: coredns-1.5.0.md

+++ title = "CoreDNS-1.5.0 Release" description = "CoreDNS-1.5.0 Release Notes." tags = ["Release", "1.5.0", "Notes"] release = "1.5.0" date = 2019-04-06T08:24:07+00:00 author = "coredns" +++ A new : CoreDNS-1.5.0. Two new plugins in this release: , and . And some polish and simplifications in the core server code. The use of TIMEOUT and no_reload in and have been fully deprecated. As is the plugin. And a update on two important and active bugs: seems to hone in on Docker and/or the container environment being a contributing factor. is because of TLS session negotiating in the forward plugin. The plugin was added that signals a plugin is ready to receive queries. First user is the kubernetes* plugin. The plugin was added to implement forwarding gRPC. If you were relying on this in the you can migrate to this one. The plugin was added that adds a context.WithTimeout to each context (but not enabled - yet). The plugin adds dnstap support. The plugin now supports wildcards. The plugin adds a `block` option that enables the block profiling. The plugin adds a `corednspluginenabled` metric that shows which plugins are enabled. The plugin returns the maintainers when queried for "authors.bind TXT CH". The `resyncperiod` option in now defaults to zero seconds, which disables resyncing. The `resyncperiod` option in is deprecated and will be made a no-op in 1.6.0 and removed in 1.7.0. Aleks, Chris O'Haver, David, dilyevsky, Francois Tur, Iigo, Jiacheng Xu, John Belamaric, Matt Greenfield, MengZeLee, Miek Gieben, peiranliushop, Rajveer Malviya, Ruslan Drozhdzh, Stefan Budeanu, Xiao An, Yong Tang.
Project: Emissary-Ingress (Orchestration & Management / API Gateway)
File: HACKING.md

If you have a dependency on another `.mk` file includes, include it with `include $(dir $(lastword $(MAKEFILE_LIST)))common.mk`. `.PHONY` targets that you wish to be user-visible should have a `## Help text` usage comment. See `help.mk` for more information. Wrap your `.mk` files in ifeq ($(words $(filter $(abspath $(lastword $(MAKEFILELIST))),$(abspath $(MAKEFILELIST)))),1) endif include guards to make sure they are only included once; similar to how you would with a C header file. The Make `export` directive only affects recipes, and does not affect `$(shell )`. Because of this, you shoud not call `go` inside of `$(shell )`, as `$GO111MODULE` may not have the correct value. Make sure to pass `--fail` to `curl` when downloading things; otherwise it will silently save 404 HTML/XML pages. Don't depend on anything in ./build-aux/bin/ during clean and clobber rules. The `prelude.mk` cleanup might remove it before your cleanup bit runs. For Go programs, `cd` to the program's sourcedirectory, and use `go run .` to run it: cd $(dir $(_myfile.mk))bin-go/PROGRAM && GO111MODULE=on go run . ARGS... Only tolerate that grossness in your cleanup rules. `check` and `check-FOO` are `.PHONY` targets that run tests. `test-FOO` is an executable program that when run tests FOO. Perhaps the `check-FOO` Make target compiles and runs the `test-FOO` program. `test` is the POSIX `test(1)` command. Don't use it as a Makefile rule name. (That is, use "check" as a verb, and "test" as a noun.) Internal "private" variables should be named `_snippet-name.VAR`; for example, a variable internal to `k8s.mk` might be named `_k8s.push`. Everything should work with GNU Make 3.81 AND newer versions. Avoid Make features introduced in 3.82 or later. The 3.813.82 update changed precedence of pattern rules from "parse-order-based" to "stem-length-based". Be careful that your pattern rules work in BOTH systems. Requires `go` 1.11 or newer. Using `--` to separate positional arguments isn't POSIX, but is implemented in `getopt(3)` and `getopt_long(3)` in every major libc (including macOS). Therefore, `--` working is a reasonable base assumption. Known exceptions: macOS `chmod` Prefer `curl` to `wget`; macOS ships with `curl`, it doesn't ship with `wget`. See "Naming conventions".. See . Place `.PHONY:` immediately after the rule definition. Use pattern rules instead of "old-fashioned suffix rules" (as the GNU Make manual refers to them).
Project: Gloo (Orchestration & Management / API Gateway)
File: security-posture.md

title: "Security Posture" description: Expected release cadence and support of Gloo Edge weight: 33 Review the following information about security posture of Solo's Gloo Edge Envoy extensions. For more information, see the . The security posture includes extensions for both the Open Source and Enterprise versions of Gloo Edge. Each extension has the filter name and the the classification of the filter's security posture. The following table describes the security posture values. | Value | Description | | -- | -- | | dataplaneagnostic | Do not expose this extension to data plane attacks, for both untrusted downstreams and upstream services. | | requirestrusteddownstreamandupstream | Use this extension only when both the downstream and upstream services are trusted.| | robusttountrusted_downstream | You can use these hardened filters only with untrusted downstream services. Do not use with untrusted upstream services, as these filters assume that the upstream services are trusted.| | robusttountrusteddownstreamand_upstream | You can use these hardened filters with both untrusted downstream and upstream services.| | unknown | Use these filters with your own security procedures. These filters have an unknown security posture. | Review the following Open Source and Enterprise security postures for Gloo Edge Envoy extensions. You can also download this . ```yaml {{< readfile file="/reference/security-posture.yaml">}} ```
Project: Apache Mesos (Orchestration & Management / Scheduling & Orchestration)
File: clang-format.md

title: Apache Mesos - ClangFormat layout: documentation is an automatic source code formatting tool which helps us focus on the code rather than the formatting. The provided configurations try to honor the as much as possible, but there are some limitations which require manual attention. Even with these limitations however, ClangFormat will be extremely useful for your workflow! sudo apt-add-repository "deb http://llvm.org/apt/trusty/ llvm-toolchain-trusty main" sudo apt-get update sudo apt-get install clang-format-3.8 $ brew install clang-format By default, ClangFormat uses the configuration defined in a `.clang-format` or `_clang-format` file located in the nearest parent directory of the input file. The `./support/setup-dev.sh` script creates a `.clang-format` symlink at the top-level directory which points to `support/clang-format` for ClangFormat to find. Add the following to your `.vimrc`: Ubuntu: map <C-K> :pyf /usr/share/vim/addons/syntax/clang-format-3.8.py<cr> imap <C-K> <c-o>:pyf /usr/share/vim/addons/syntax/clang-format-3.8.py<cr> OS X: map <C-K> :pyf /usr/local/share/clang/clang-format.py<cr> imap <C-K> <c-o>:pyf /usr/local/share/clang/clang-format.py<cr> The first line enables clang-format for `NORMAL` and `VISUAL` mode, the second line adds support for `INSERT` mode. Change `C-K` to another binding if you need clang-format on a different key (`C-K` stands for `Ctrl+k`). With this integration you can press the bound key and clang-format will format the current line in `NORMAL` and `INSERT` mode or the selected region in `VISUAL` mode. The line or region is extended to the next bigger syntactic entity. It operates on the current, potentially unsaved buffer and does not create or save any files. To revert a formatting, just undo. Source: http://llvm.org/releases/3.8.0/tools/clang/docs/ClangFormat.html Add the following to your `.emacs`: Ubuntu: (load "/usr/share/emacs/site-lisp/clang-format-3.8/clang-format.el") (global-set-key [C-M-tab] 'clang-format-region) OS X: (load "/usr/local/share/clang/clang-format.el") (global-set-key [C-M-tab] 'clang-format-region) This binds the function `clang-format-region` to `C-M-tab`, which then formats the current line or selected region. Source: http://llvm.org/releases/3.8.0/tools/clang/docs/ClangFormat.html Parameters and arguments should be indented by 4 spaces rather than 2. Should not follow Google's style of wrapping on open parentheses, we should try to reduce "jaggedness" in the code.
Project: Envoy (Orchestration & Management / Service Proxy)
File: header_map.md

The Envoy header map implementation (`HeaderMapImpl`) has the following properties: Headers are stored in a linked list (`HeaderList`) in the order they are added, with pseudo headers kept at the front of the list. Once there at at least 3 years, the header map will also use a map (`HeaderLazyMap`), in addition to the linked list, for faster access to the headers. O(1) direct access is possible for common headers needed during data plane processing. This is provided by a table of pointers that reach directly into a linked list that is populated when headers are added or removed from the map. When O(1) headers are accessed by direct method (`DEFINEINLINEHEADER` and `CustomInlineHeaderBase`) they use direct pointer access to see whether a header is present, add it, modify it, etc. When headers are added by name a trie is used to lookup the pointer in the table (`StaticLookupTable`). Custom headers can be registered statically against a specific implementation (request headers, request trailers, response headers, and response trailers) via core code and extensions (`CustomInlineHeaderRegistry`). Each registered header increases the size of the table by the size of a single pointer. Operations that search, replace, etc. for a header by name will be O(1). O(1) registered headers are tracked during static initialization via the `CustomInlineHeaderBase` class. The first time a header map is constructed (in practice this is after bootstrap load and the Envoy header prefix is finalized when `getAllHeaderMapImplInfo` is called), the `StaticLookupTable` is finalized for each header map type. No further changes are possible after this point. The `StaticLookupTable` defines the amount of variable pointer table space that is require for each header map type. Each concrete header map type derives from `InlineStorage` with a variable length member at the end of the definition. Each concrete header map type uses a factory function and a provide constructor. The required size is determined via the `inlineHeadersSize` function.
Project: hami (Orchestration & Management / Scheduling & Orchestration)
File: design.md

<img src="../../imgs/arch.png" width = "800" /> The architect of HAMi is shown in the figure above, It is organized in the form of "chart". MutatingWebhook The MutatingWebhook checks the validity of each task, and set the "schedulerName" to "HAMi scheduler" if the resource requests have been recognized by HAMi If Not, the MutatingWebhook does nothing and pass this task to default-scheduler. Scheduler HAMi support default kube-scheduler and volcano-scheduler, it implements an extender and register 'Filter' and 'Score' methods to deal with sharable devices. When a pod with sharable device request arrives, 'Filter' searches the cluster and returns a list of 'available' nodes. 'Score' scores each node 'Filter' returned, and pick the highest one to host the pod. It patches the schedule decision on corresponding pod annotations, for the detailed protocol, see protocol.md DevicePlugin When the schedule decision is made, scheduler calls devicePlugin on that node to generate environment variables and mounts according to pod annotations. Please note that, the DP used here is a customized version, you need to install according to README document with that device. Most officaial DP will not fit in HAMi, and will result in unexpected behaviour InContainer Control The implementation of in-container hard limit is different for diffent devices. For example, HAMi-Core is responsible for NVIDIA devices. libvgpu-control.so is responsible for iluvatar devices, etc. HAMi needs to pass the correct environment variables in order for it to operate. <img src="./imgs/flowchart.jpeg" width = "600" /> In summary, The flowchart of pod is descirbed as the figure above.