cwielenb/CNCF-Dataset huggingface dataset:
Project: DolphinScheduler (Orchestration & Management / Scheduling & Orchestration)
File: load-balance.md

Load balancing refers to the reasonable allocation of server pressure through routing algorithms (usually in cluster environments) to achieve the maximum optimization of server performance. DolphinScheduler-Master allocates tasks to workers, and by default provides three algorithms: Weighted random (random) Smoothing polling (round-robin) Linear load (lower weight) The default configuration is the linear load. As the routing sets on the client side, the master service, you can change master.host.selector in master.properties to configure the algorithm. e.g. master.host.selector=random (case-insensitive) The configuration file is worker.properties All the load algorithms above are weighted based on weights, which affect the routing outcome. You can set different weights for different machines by modifying the `worker.weight` value. Consider JIT optimization, worker runs at low power for a period of time after startup, so that it can gradually reach its optimal state, a process we call preheating. If you are interested, you can read some articles about JIT. So the worker gradually reaches its maximum weight with time after starts up ( by default ten minutes, there is no configuration about the pre-heating duration, it's recommend to submit a PR if have needs to change the duration). This algorithm is relatively simple, select a worker by random (the weight affects its weighting). An obvious drawback of the weighted polling algorithm, which is under special weights circumstance, weighted polling scheduling generates an imbalanced sequence of instances, and this unsmooth load may cause some instances to experience transient high loads, leading to a risk of system crash. To address this scheduling flaw, we provide a smooth weighted polling algorithm. Each worker has two weights parameters, weight (which remains constant after warm-up is complete) and currentweight (which changes dynamically). For every route, calculate the currentweight + weight and is iterated over all the workers, the weight of all the workers sum up and count as totalweight, then the worker with the largest currentweight is selected as the worker for this task. By meantime, set worker's currentweight-totalweight. This algorithm reports its own load information to the registry at regular intervals. Make decision on two main pieces of information: load average (default is the number of CPU cores * 2) available physical memory (default is 0.3, in G) If either of these is lower than the configured item, then this worker will not participate in the load. (no traffic will be allocated) You can customise the configuration by changing the following properties in worker.properties worker.max.cpu.load.avg=-1 (worker max cpu load avg, only higher than the system cpu load average, worker server can be dispatched tasks. default value -1: the number of cpu cores * 2) worker.reserved.memory=0.3 (worker reserved memory, only lower than system available memory, worker server can be dispatched tasks. default value 0.3, the unit is percentage)
Project: kratos (Orchestration & Management / Remote Procedure Call)
File: feature-request.md

name: "\U0001F4A1 Feature Request" about: For ideas or feature requests, start a new discussion. title: "[Feature]" labels: feature assignees: '' Please see the FAQ in our main README.md before submitting your issue. <!-- In order to accurately distinguish that the needs put forward by users are the needs of most users and reasonable needs, solicit community opinions through the process, and the features adopted by the community will be realized as new functions. In order to make the proposal process as simple as possible, the process includes three stages: feature request - > proposal - > pull-request, where feature, proposal is issue and pull-request is the specific function implementation. In order to help the community correctly understand the requirements of the feature, the feature request issue needs to describe the functional requirements and relevant references or documents in detail. And the feature request issue can contain the basic description of the function, which can be used as a reference for the function implementation in the proposal. Proposal contains the basic implementation methods of functions, such as interface definition, general usage of functions, etc. After the function is realized, a merge request will be initiated to associate the proposal issue with the function issue. After the merger is completed, all questions will be closed and the process will end. When more than five maintainer members agree to implement the feature, a proposal issue will be created for detailed design. The status of the proposal is divided into: under discussion, finalized and abandoned. After reaching the final status, start specific implementation (PR can also be implemented synchronously during the discussion) If the maintainer team members have major differences on a requirement, the final decision is made by @Terry Mao. --> <!-- example: We hope to add event interface to Kratos framework to access middleware such as Kafka and rabbitmq --> <!-- example: The event interface should be added to Kratos. The interface should contain subscribers and publishers, and the message body should contain key value head --> <!-- example: - -->
Project: Traefik (Orchestration & Management / API Gateway)
File: datadog.md

title: "Traefik Datadog Metrics Documentation" description: "Traefik Proxy supports Datadog for backend metrics. Read the technical documentation to enable Datadog for observability." To enable the Datadog: ```yaml tab="File (YAML)" metrics: datadog: {} ``` ```toml tab="File (TOML)" [metrics] [metrics.datadog] ``` ```bash tab="CLI" --metrics.datadog=true ``` Required, Default="127.0.0.1:8125" Address instructs exporter to send metrics to datadog-agent at this address. This address can be a Unix Domain Socket (UDS) address with the following form: `unix:///path/to/datadog.socket`. ```yaml tab="File (YAML)" metrics: datadog: address: 127.0.0.1:8125 ``` ```toml tab="File (TOML)" [metrics] [metrics.datadog] address = "127.0.0.1:8125" ``` ```bash tab="CLI" --metrics.datadog.address=127.0.0.1:8125 ``` Optional, Default=true Enable metrics on entry points. ```yaml tab="File (YAML)" metrics: datadog: addEntryPointsLabels: true ``` ```toml tab="File (TOML)" [metrics] [metrics.datadog] addEntryPointsLabels = true ``` ```bash tab="CLI" --metrics.datadog.addEntryPointsLabels=true ``` Optional, Default=false Enable metrics on routers. ```toml tab="File (TOML)" [metrics] [metrics.datadog] addRoutersLabels = true ``` ```yaml tab="File (YAML)" metrics: datadog: addRoutersLabels: true ``` ```bash tab="CLI" --metrics.datadog.addrouterslabels=true ``` Optional, Default=true Enable metrics on services. ```yaml tab="File (YAML)" metrics: datadog: addServicesLabels: true ``` ```toml tab="File (TOML)" [metrics] [metrics.datadog] addServicesLabels = true ``` ```bash tab="CLI" --metrics.datadog.addServicesLabels=true ``` Optional, Default=10s The interval used by the exporter to push metrics to datadog-agent. ```yaml tab="File (YAML)" metrics: datadog: pushInterval: 10s ``` ```toml tab="File (TOML)" [metrics] [metrics.datadog] pushInterval = "10s" ``` ```bash tab="CLI" --metrics.datadog.pushInterval=10s ``` Optional, Default="traefik" The prefix to use for metrics collection. ```yaml tab="File (YAML)" metrics: datadog: prefix: traefik ``` ```toml tab="File (TOML)" [metrics] [metrics.datadog] prefix = "traefik" ``` ```bash tab="CLI" --metrics.datadog.prefix=traefik ```
Project: Consul (Orchestration & Management / Service Mesh)
File: bug_report.md

name: Bug Report about: You're experiencing an issue with Consul that is different than the documented behavior. <!-- When filing a bug, please include the following headings if possible. Any example text in this template can be deleted. --> <!-- Please provide a paragraph or two about the issue you're experiencing. --> <!-- Please provide steps to reproduce the bug, without any details it would be hard to troubleshoot: Steps to reproduce this issue, eg: Create a cluster with n client nodes n and n server nodes Run `curl ...` View error --> <! Please provide both `consul info` and agent HCL config for both client and servers to help us better diagnose the issue. Take careful steps to remove any sensitive information from config files that include secrets such as Gossip keys. > <details> <summary>Client info</summary> ``` Output from client 'consul info' command here ``` ``` Client agent HCL config ``` </details> <details> <summary>Server info</summary> ``` Output from server 'consul info' command here ``` ``` Server agent HCL config ``` </details> <!-- OS, Architecture, and any other information you can provide about the environment. --> <!-- Include appropriate Client or Server log fragments. If the log is longer than a few dozen lines, please include the URL to the of the log instead of posting it in the issue. Use `-log-level=TRACE` on the client and server to capture the maximum log detail. -->
Project: MOSN (Orchestration & Management / Service Proxy)
File: README_EN.md

This example project demonstrates how to configure MOSN with Wasm extension to deal with SofaRPC request. Protocol between MOSN is Bolt. For the convenience of demonstration, MOSN listens to one port. Once receiving a Bolt request, MOSN will response directly. ``` config.json // MOSN config file filter.go // Wasm source file makefile // makefile to compile wasm source file into wasm extension client.go // Mocked SofaRPC client ``` ``` make ``` This operation will generate filter.wasm ``` docker pull mosnio/mosn-wasm:v0.21.0 ``` ``` docker run -it --rm -p 2045:2045 -v $(pwd):/etc/wasm/ mosnio/mosn-wasm:v0.21.0 ``` ``` go run client.go ``` And we should observe wasm-related log printed in MOSN side.snio/mosn-wasm:v0.21.0 ``` ``` go run server.go ``` ``` curl -v http://127.0.0.1:2045/ -d "haha" ``` Now we should be able to observe wasm-related log printed in MOSN side. ``` [INFO] response header from http://127.0.0.1:2046/: Content-Length: 39 [INFO] response header from http://127.0.0.1:2046/: Content-Type: text/plain; charset=utf-8 [INFO] response header from http://127.0.0.1:2046/: From: external http server [INFO] response body from http://127.0.0.1:2046/: response body from external http server ```body from external http server ``` On the external HTTP server, we will observe a new http header "Foo: bar" ``` [UPSTREAM]receive request / Accept -> [/] X-Forwarded-Proto -> [http] X-Request-Id -> [5c59b8b3-08fb-4baa-90ce-319e88a8f0df] Foo -> [bar] X-Envoy-Expected-Rq-Timeout-Ms -> [15000] User-Agent -> [curl/7.29.0] ```
Project: CloudWeGo (Orchestration & Management / Remote Procedure Call)
File: PULL_REQUEST_TEMPLATE.md

<!-- Add one of the following kinds: build: Changes that affect the build system or external dependencies (example scopes: gulp, broccoli, npm) ci: Changes to our CI configuration files and scripts (example scopes: Travis, Circle, BrowserStack, SauceLabs) docs: Documentation only changes feat: A new feature optimize: A new optimization fix: A bug fix perf: A code change that improves performance refactor: A code change that neither fixes a bug nor adds a feature style: Changes that do not affect the meaning of the code (white space, formatting, missing semi-colons, etc) test: Adding missing tests or correcting existing tests chore: Changes to the build process or auxiliary tools and libraries such as documentation generation --> <!-- The description of the title will be attached in Release Notes, so please describe it from user-oriented, what this PR does / why we need it. Please check your PR title with the below requirements: --> [ ] This PR title match the format: \<type\>(optional scope): \<description\> [ ] The description of this PR title is user-oriented and clear enough for others to understand. <!-- Provide more detailed info for review(e.g., it's recommended to provide perf data if this is a perf type PR). --> en: zh(optional): <!-- Automatically closes linked issue when PR is merged. Eg: `Fixes #<issue number>`, or `Fixes (paste link of issue)`. --> <!-- If the current PR requires user awareness at the usage level, please submit a PR to update user docs. -->
Project: gRPC (Orchestration & Management / Remote Procedure Call)
File: siddharth_shukla.md

Project Overview ================ The project, titled 'GRPC Python compatibility support', involved collaborating with the GRPC team to improve the library compatibility for the GRPC Python library. Python is, originally, a specification for a programming language. This specification has been implemented differently in different implementations of the . A small, and by no means exhaustive, list of some major python implementations is: : The reference implementation : Python implemented in Java : CPython implementation that enables .NET library usage : Python implemented in .NET : Python implemented completely in Python : Replaces the dependency for the C call stack with it's own stack The development in this project revolved around introducing changes to the codebase that enable support for latest stable as well as development releases of the reference implementation (CPython) of the Python programming language namely `Python 3.4`, `Python 3.5`,and `Python 3.6` as well as the stable releases of the PyPy implementation. Special changes were required to enable PyPy support because PyPy has a non-deterministic garbage collector that does not rely on reference counting unlike the CPython garbage collector. The changes to the codebase involved changes to the library code as well as changes to the tests and scripts in the test infrastructure which resulted in both the library as well as the testing infrastructure being Python 3.x and PyPy compatible. The list of merged commits, as of 22.08.2016 23:59 CEST, is summarized here for the perusal of those interested: The list of unmerged pull requests is as follows: The list of tasks that have pending unsubmitted pull requests is as follows: Modify run_tests.py to enable testing of new languages without affecting old branches. Project Details =============== Title: GRPC Python compatibility support Student: Mentors: , Duration: May 23 - August 23 Hat tip: , ,
Project: BFE (Orchestration & Management / Service Proxy)
File: session_ticket_key.data.md

sessionticketkey.data records the session ticket key. | Config Item | Description | | - | | | Version | String<br>Version of config file | | SessionTicketKey | String<br>The session ticket key. length is 48 and contains only [a-z0-9] | ```json { "Version": "20190101000000", "SessionTicketKey": "08a0d852ef494143af613ef32d3c39314758885f7108e9ab021d55f422a454f7c9cd5a53978f48fa1063eadcdc06878f" } ```
Project: Kubeflow (Orchestration & Management / Scheduling & Orchestration)
File: tf_serving_iap.md

<!-- START doctoc generated TOC please keep comment here to allow auto update --> <!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --> Table of Contents generated with <!-- END doctoc generated TOC please keep comment here to allow auto update --> To send prediction request through ingress and IAP, read doc
Project: Higress (Orchestration & Management / API Gateway)
File: non--crash-security--bug.md

name: Non-{crash,security} bug about: Create a report to help us improve title: '' labels: '' assignees: '' If you are reporting any crash or any potential security issue, do not open an issue in this repo. Please report the issue via (Alibaba Security Response Center) where the issue will be triaged appropriately. of this repository and believe that this is not a duplicate. If there is an exception, please attach the exception trace: ``` Just paste your stack trace here! ``` xxx xxx xxx Higress version: OS : Others:
Project: Apache bRPC (Orchestration & Management / Remote Procedure Call)
File: iobuf.md

brpc uses as data structure for attachment in some protocols and HTTP body. It's a non-contiguous zero-copied buffer, proved in previous projects, and good at performance. The interface of `IOBuf` is similar to `std::string`, but not the same. If you've used the `BufHandle` in Kylin before, you should notice the convenience of `IOBuf`: the former one is badly encapsulated, leaving the internal structure directly in front of users, who must carefully handle the referential countings, very error prone and leading to bugs. Default constructor does not allocate memory. Copyable. Modifications to the copy doesn't affect the original one. Copy the managing structure of IOBuf only rather the payload. Append another IOBuf without copying payload. Can append string, by copying payload. Read from or write into file descriptors. Serialize to or parse from protobuf messages. constructible like a std::ostream using IOBufBuilder. Used as universal string-like structure in the program. Lifetime of IOBuf should be short, to prevent the referentially counted blocks(8K each) in IOBuf lock too many memory. Cut 16 bytes from front-side of sourcebuf and append to destbuf: ```c++ sourcebuf.cut(&destbuf, 16); // cut all bytes of source_buf when its length < 16 ``` Just pop 16 bytes from front-side of source_buf: ```c++ sourcebuf.popfront(16); // Empty source_buf when its length < 16 ``` Append another IOBuf to back-side ```c++ buf.append(another_buf); // no data copy ``` Append std::string to back-sie ```c++ buf.append(str); // copy data of str into buf ``` Parse a protobuf message from the IOBuf ```c++ IOBufAsZeroCopyInputStream wrapper(&iobuf); pb_message.ParseFromZeroCopyStream(&wrapper); ``` Parse IOBuf in user-defined formats ```c++ IOBufAsZeroCopyInputStream wrapper(&iobuf); CodedInputStream coded_stream(&wrapper); coded_stream.ReadLittleEndian32(&value); ... ``` Serialize a protobuf message into the IOBuf ```c++ IOBufAsZeroCopyOutputStream wrapper(&iobuf); pb_message.SerializeToZeroCopyStream(&wrapper); ``` Built IOBuf with printable data ```c++ IOBufBuilder os; os << "anything can be sent to std::ostream"; os.buf(); // IOBuf ``` Directly printable to std::ostream. Note that the iobuf in following example should only contain printable characters. ```c++ std::cout << iobuf << std::endl; // or std::string str = iobuf.to_string(); // note: allocating memory printf("%s\n", str.c_str()); ``` IOBuf is good at performance: | Action | Throughput | QPS | | - | -- | - | | Read from file -> Cut 12+16 bytes -> Copy -> Merge into another buffer ->Write to /dev/null | 240.423MB/s | 8586535 | | Read from file -> Cut 12+128 bytes -> Copy-> Merge into another buffer ->Write to /dev/null | 790.022MB/s | 5643014 | | Read from file -> Cut 12+1024 bytes -> Copy-> Merge into another buffer ->Write to /dev/null | 1519.99MB/s | 1467171 |
Project: Gloo (Orchestration & Management / API Gateway)
File: glooe_solo_io.project.sk.md

title: "glooesoloio.project" weight: 5 <!-- Code generated by solo-kit. DO NOT EDIT. --> API Version: `glooe.solo.io.enterprise` Gloo is a high-performance, plugin-extendable, platform-agnostic API Gateway built on top of Envoy. Gloo is designed for microservice, monolithic, and serverless applications. By employing function-level routing, Gloo can completely decouple client APIs from upstream APIs at the routing level. Gloo serves as an abstraction layer between clients and upstream services, allowing front-end teams to work independently of teams developing the microservices their apps connect to. <!-- Start of HubSpot Embed Code --> <script type="text/javascript" id="hs-script-loader" async defer src="//js.hs-scripts.com/5130874.js"></script> <!-- End of HubSpot Embed Code -->
Project: Gravitee.io (Orchestration & Management / API Gateway)
File: management-api-policy-studio-design.md

Policies let you customize, enhance API behavior and functionalities. Policies can enforce security, enhance performance, transform request/response and let you implement some logic via the Groovy Policy or dynamic routing policy. Policy can be chained, applied to root path (/) or for specific paths. You can drag and drop policy at the center of the screen to apply policy for a specific path, and the right part of the screen let you configure your policy. The (+) button let you create a new path to specify its behavior via the policies.
Project: Contour (Orchestration & Management / Service Proxy)
File: kind.md

title: Creating a Contour-compatible kind cluster This guide walks through creating a kind (Kubernetes in Docker) cluster on your local machine that can be used for developing and testing Contour. Download & install Docker and kind: Docker kind Create a kind configuration file locally. This file will instruct kind to create a cluster with one control plane node and one worker node, and to map ports 80 and 443 on your local machine to ports 80 and 443 on the worker node container. This will allow us to easily get traffic to Contour/Envoy running inside the kind cluster from our local machine. Copy the text below into the local yaml file `kind-config.yaml`: ```yaml kind: Cluster apiVersion: kind.x-k8s.io/v1alpha4 nodes: role: control-plane role: worker extraPortMappings: containerPort: 80 hostPort: 80 listenAddress: "0.0.0.0" containerPort: 443 hostPort: 443 listenAddress: "0.0.0.0" ``` Create a kind cluster using the config file from above: ```yaml $ kind create cluster --config kind-config.yaml ``` Verify the nodes are ready by running: ```yaml $ kubectl get nodes ``` You should see 2 nodes listed with status Ready: kind-control-plane kind-worker Congratulations, you have created your cluster environment. You're ready to install Contour. Note: When you are done with the cluster, you can delete it by running: ```yaml $ kind delete cluster ``` See https://projectcontour.io/getting-started/ for how to install Contour into your kind cluster.
Project: Avro (Orchestration & Management / Remote Procedure Call)
File: new-pmc-martin-grigorov.md

title: "New PMC member: Martin Grigorov" linkTitle: "New PMC member: Martin Grigorov" date: 2022-09-13 <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> The Project Management Committee (PMC) for Apache Avro is pleased to announce that Martin Grigorov has accepted our invitation to become a PMC member. He has has been active, reliable and responsive to the community and a solid contributor to various SDKs, bringing well-thought out reviews and comments to both old and new PRs and JIRA. He definitely stepped up for the website refactoring and preparing for the 1.11.1 release! Please join me in welcoming Martin to the Avro PMC!