cwielenb/CNCF-Dataset huggingface dataset:
Project: Karmada (Orchestration & Management / Scheduling & Orchestration)
File: USER_GUIDE.md

In the example below, we create a client, an index with non-default settings, insert a document to the index, search for the document, delete the document and finally delete the index. ```go package main import ( "os" "context" "crypto/tls" "fmt" opensearch "github.com/opensearch-project/opensearch-go" opensearchapi "github.com/opensearch-project/opensearch-go/opensearchapi" "net/http" "strings" ) const IndexName = "go-test-index1" func main() { // Initialize the client with SSL/TLS enabled. client, err := opensearch.NewClient(opensearch.Config{ Transport: &http.Transport{ TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, // For testing only. Use certificate for validation. }, Addresses: []string{"https://localhost:9200"}, Username: "admin", // For testing only. Don't store credentials in code. Password: "admin", }) if err != nil { fmt.Println("cannot initialize", err) os.Exit(1) } // Print OpenSearch version information on console. fmt.Println(client.Info()) // Define index mapping. mapping := strings.NewReader(`{ 'settings': { 'index': { 'numberofshards': 4 } } }`) // Create an index with non-default settings. res := opensearchapi.CreateRequest{ Index: IndexName, Body: mapping, } fmt.Println("creating index", res) // Add a document to the index. document := strings.NewReader(`{ "title": "Moneyball", "director": "Bennett Miller", "year": "2011" }`) docId := "1" req := opensearchapi.IndexRequest{ Index: IndexName, DocumentID: docId, Body: document, } insertResponse, err := req.Do(context.Background(), client) if err != nil { fmt.Println("failed to insert document ", err) os.Exit(1) } fmt.Println(insertResponse) // Search for the document. content := strings.NewReader(`{ "size": 5, "query": { "multi_match": { "query": "miller", "fields": ["title^2", "director"] } } }`) search := opensearchapi.SearchRequest{ Body: content, } searchResponse, err := search.Do(context.Background(), client) if err != nil { fmt.Println("failed to search document ", err) os.Exit(1) } fmt.Println(searchResponse) // Delete the document. delete := opensearchapi.DeleteRequest{ Index: IndexName, DocumentID: docId, } deleteResponse, err := delete.Do(context.Background(), client) if err != nil { fmt.Println("failed to delete document ", err) os.Exit(1) } fmt.Println("deleting document") fmt.Println(deleteResponse) // Delete previously created index. deleteIndex := opensearchapi.IndicesDeleteRequest{ Index: []string{IndexName}, } deleteIndexResponse, err := deleteIndex.Do(context.Background(), client) if err != nil { fmt.Println("failed to delete index ", err) os.Exit(1) } fmt.Println("deleting index", deleteIndexResponse) } ```
Project: Apache Thrift (Orchestration & Management / Remote Procedure Call)
File: README.apache.md

Thrift PHP/Apache Integration License ======= Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. Building PHP Thrift Services with Apache ======================================== Thrift can be embedded in the Apache webserver with PHP installed. Sample code is provided below. Note that to make requests to this type of server you must use a THttpClient transport. Sample Code =========== ```php <?php namespace MyNamespace; / Include path */ $THRIFT_ROOT = '/your/thrift/root/lib'; / Init Autloader */ requireonce $THRIFTROOT . '/Thrift/ClassLoader/ThriftClassLoader.php'; $loader = new ThriftClassLoader(); $loader->registerNamespace('Thrift', $THRIFT_ROOT); $loader->registerDefinition('Thrift', $THRIFT_ROOT . '/packages'); $loader->register(); use Thrift\Transport\TPhpStream; use Thrift\Protocol\TBinaryProtocol; / Example of how to build a Thrift server in Apache/PHP */ class ServiceHandler implements ServiceIf { // Implement your interface and methods here } header('Content-Type: application/x-thrift'); $handler = new ServiceHandler(); $processor = new ServiceProcessor($handler); // Use the TPhpStream transport to read/write directly from HTTP $transport = new TPhpStream(TPhpStream::MODER | TPhpStream::MODEW); $protocol = new TBinaryProtocol($transport); $transport->open(); $processor->process($protocol, $protocol); $transport->close(); ```
Project: Envoy (Orchestration & Management / Service Proxy)
File: summary.md

Summary of changes: Removed the Swift/C++ interop layer in Envoy Mobile. Addd retry policy to ext_proc. Added HTTP downstream remote reset response flag. Added support for the Fluentd access logger. Introduced `MemoryAllocatorManager` to configure heap memory release rate. Envoy Mobile added `CONNECT` Proxy support for iOS. Redis: support echo command. Envoy Mobile setting QUIC newtowrk idle timeout to 30 seconds. Sending server preferred address to non-QUICHE clients. Avoid concatenation of JWT duplicated headers. HTTP: Keep `Transfer-Encoding` header for `trailers`. Envoy Mobile setting the socket receive buffer to 1MB for QUIC. Added `FULL_SCAN` support to least-request load-balancing algorithm. awslambda and extproc filters can be used as an upstream filter. Hosts marked as draining in and EDS update are now excluded. Envoy Mobile supports log-levels. Added support for URI tempate matching for RBAC. Fixed load balancing initialization bug. Supporting `%UPSTREAMCONNECTIONID%` in access logs. Added request and response attributes support to ext_proc. Added support sending dynamic metadata to ext_proc. Re-enable the nghttp2 codec for HTTP/2 connections by default.
Project: Apache Mesos (Orchestration & Management / Scheduling & Orchestration)
File: user-groups.html.md

layout: community_section breadcrumb: Community Mesos User Groups (MUGs) are responsible for organizing local meetups, hackathons, and other events to foster the adoption of Mesos and support its community. It's easy. If you're interested in organizing a MUG in your city, share your interest on the user mailing list (user@mesos.apache.org) and we'll add you to our list. MUGs typically use Meetup.com or Twitter to coordinate. We encourage groups to be organized by at least two organizations, and share responsibility for hosting and organizing events.
Project: Apache Zookeeper (Orchestration & Management / Coordination & Service Discovery)
File: README.md

ZooKeeper Fatjar ================ This package contains build to create a fat zookeeper jar. Fatjar can be used to run: zookeeper server zookeeper client distributed load generator for testing (generateLoad) container that will instantiate classes as directed by an instance manager (ic) system test (systest) jmh micro benchmarks (jmh) Use following command to build fatjar ``` mvn clean install -P fatjar -DskipTests ``` To run the fatjar use: ``` java -jar zoookeeper-<version>-fatjar.jar ```
Project: Apache Mesos (Orchestration & Management / Scheduling & Orchestration)
File: 2013-11-12-mesos-0-14-2-released.md

layout: post title: Apache Mesos 0.14.2 Released permalink: /blog/mesos-0-14-2-released/ published: true post_author: display_name: Ben Mahler gravatar: fb43656d4d45f940160c3226c53309f5 twitter: bmahler tags: Release We recently released Mesos 0.14.2, a bugfix release with only a minor change related to cgroups isolation in 0.14.1. If you're using 0.14.1 with cgroups isolation, it is recommended to upgrade to avoid unnecessary out-of-memory (OOM) killing of jobs. The latest version of Mesos is available on our page. If upgrading from 0.14.x, this upgrade can be applied seamlessly to running clusters. However, if you're upgrading from 0.13.x on a running cluster, please refer to the document, which details how a seamless upgrade from 0.13.x to 0.14.x can be performed. With the release of 0.14.1, when using cgroups isolation, the OOM semantics were altered to enable the kernel OOM killer and to use the memory soft limit combined with memory threshold notifications to induce OOMs in user-space. This was done to attempt to capture memory statistics at the time of OOM for diagnostic purposes. However, this proved to trigger unintended OOMs as the memory purging that occurs when the hard limit is reached was no longer occurring. 0.14.2 no longer uses threshold notifications; the memory hard limit is now used instead to preserve the previous OOM semantics. For additional details, please see: , .
Project: Gloo (Orchestration & Management / API Gateway)
File: empty.proto.sk.md

title: "empty.proto" weight: 5 <!-- Code generated by solo-kit. DO NOT EDIT. --> Protocol Buffers - Google's data interchange format Copyright 2008 Google Inc. All rights reserved. https://developers.google.com/protocol-buffers/ Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. Neither the name of Google Inc. nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance: service Foo { rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); } The JSON representation for `Empty` is empty JSON object `{}`. ```yaml ``` | Field | Type | Description | | -- | - | -- | <!-- Start of HubSpot Embed Code --> <script type="text/javascript" id="hs-script-loader" async defer src="//js.hs-scripts.com/5130874.js"></script> <!-- End of HubSpot Embed Code -->
Project: Avro (Orchestration & Management / Remote Procedure Call)
File: avro-1.9.2-released.md

title: "Avro 1.9.2" linkTitle: "Avro 1.9.2" date: 2020-02-19 <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> Apache Avro 1.9.2 has been released!
Project: Apache bRPC (Orchestration & Management / Remote Procedure Call)
File: backup_request.md

Sometimes in order to ensure availability, we need to visit two services at the same time and get the result coming back first. There are several ways to achieve this in brpc: Channel opens backup request. Channel sends the request to one of the servers and when the response is not returned after ChannelOptions.backuprequestms ms, it sends to another server, taking the response that coming back first. After backuprequestms is set up properly, in most of times only one request should be sent, causing no extra pressure to back-end services. Read as example code. In this example, client sends backup request after 2ms and server sleeps for 20ms on purpose when the number of requests is even to trigger backup request. After running, the log in client and server is as following. "Index" is the number of request. After the server receives the first request, it will sleep for 20ms on purpose. Then the client sends the request with the same index. The final delay is not affected by the intentional sleep. /rpcz also shows that the client triggers backup request after 2ms and sends the second request. You can look the default cdf(Cumulative Distribution Function) graph of latency provided by brpc, or add it by your own. The y-axis of the cdf graph is a latency(us by default), and the x-axis is the proportion of requests whose latencies are less than the corresponding value in y-aixs. In the following graph, Choosing backuprequestms=2ms could approximately cover 95.5% of the requests, while choosing backuprequestms=10ms could cover 99.99% of the requests. The way of adding it by yourself: ```c++ ... bvar::LatencyRecorder myfunclatency("my_func"); ... butil::Timer tm; tm.start(); my_func(); tm.stop(); myfunclatency << tm.uelapsed(); // u represents for microsecond, and selapsed(), melapsed(), nelapsed() correspond to second, millisecond, nanosecond. // All work is done here. Myfuncqps, myfunclatency, myfunclatency_cdf and many other counters would be shown in /vars. ``` . More details please refer to the above program. . The problem of this method is that the program always sends two requests, doubling the pressure to back-end services. It is uneconomical in any sense and should be avoided as much as possible.
Project: Istio (Orchestration & Management / Service Mesh)
File: LICENSE.md

Copyright (C) 2021 Matt Layher Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. OFTWARE.
Project: Traefik (Orchestration & Management / API Gateway)
File: contenttype.md

title: "Traefik ContentType Documentation" description: "Traefik Proxy's HTTP middleware automatically sets the `Content-Type` header value when it is not set by the backend. Read the technical documentation." Handling Content-Type auto-detection {: .subtitle } The Content-Type middleware sets the `Content-Type` header value to the media type detected from the response content, when it is not set by the backend. !!! info The scope of the Content-Type middleware is the MIME type detection done by the core of Traefik (the server part). Therefore, it has no effect against any other `Content-Type` header modifications (e.g.: in another middleware such as compress). ```yaml tab="Docker & Swarm" labels: "traefik.http.middlewares.autodetect.contenttype=true" ``` ```yaml tab="Kubernetes" apiVersion: traefik.io/v1alpha1 kind: Middleware metadata: name: autodetect spec: contentType: {} ``` ```yaml tab="Consul Catalog" "traefik.http.middlewares.autodetect.contenttype=true" ``` ```yaml tab="File (YAML)" http: middlewares: autodetect: contentType: {} ``` ```toml tab="File (TOML)" [http.middlewares] [http.middlewares.autodetect.contentType] ``` !!! warning `autoDetect` option is deprecated and should not be used. Moreover, it is redundant with an empty ContentType middleware declaration. `autoDetect` specifies whether to let the `Content-Type` header, if it has not been set by the backend, be automatically set to a value derived from the contents of the response.
Project: DolphinScheduler (Orchestration & Management / Scheduling & Orchestration)
File: task.md

org.apache.dolphinscheduler.spi.task.TaskChannel The plug-in can implement the above interface. It mainly includes creating tasks (task initialization, task running, etc.) and task cancellation. If it is a yarn task, you need to implement org.apache.dolphinscheduler.plugin.task.api.AbstractYarnTask. We provide APIs for external access to all tasks in the dolphinscheduler-task-api module, while the dolphinscheduler-spi module is the spi general code library, which defines all the plug-in modules, such as the alarm module, the registry module, etc., you can read and view in detail . In additional, the `TaskChannelFactory` extends from `PrioritySPI`, this means you can set the plugin priority, when you have two plugin has the same name, you can customize the priority by override the `getIdentify` method. The high priority plugin will be load, but if you have two plugin with the same name and same priority, the server will throw `IllegalArgumentException` when load the plugin. NOTICE Since the task plug-in involves the front-end page, the front-end SPI has not yet been implemented, so you need to implement the front-end page corresponding to the plug-in separately. If there is a class conflict in the task plugin, you can use to solve this problem.
Project: KubeStellar (Orchestration & Management / Scheduling & Orchestration)
File: hosting-cluster.md

Status of this document: it is the barest of a start. Much more needs to be written. This requires a pre-existing Kind cluster that has an Ingress controller that is listening on host port 9443 and configured with TLS passthrough. The examples say to create a Kind cluster for hosting using the following command. ```shell kflex init --create-kind ``` To use a pre-existing Kind cluster instead, make sure that your current kubeconfig context is for accessing that cluster and issue the following command. ```shell kflex init ``` All the subsequent kubectl and helm commands that say to use the kubeconfig context named `kind-kubeflex` need to be modified to use the appropriate kubeconfig context for accessing the hosting cluster. This is similar to using an existing Kind cluster but requires an additional modification. Modify the `kflex` init command and subsequent kubeconfig context references as in the existing-kind-cluster scenario. Additionally, the recipe for registering a WEC with the ITS needs to be modified. In the `clusteradm` command, omit the `--force-internal-endpoint-lookup` flag. If following the example commands literally, this means to define `flags=""` rather than `flags="--force-internal-endpoint-lookup"`.
Project: Gloo (Orchestration & Management / API Gateway)
File: glooctl_get_upstreamgroup.md

title: "glooctl get upstreamgroup" weight: 5 read an upstream group or list upstream groups in a namespace usage: glooctl get upstreamgroup [NAME] [--namespace=namespace] [-o FORMAT] ``` glooctl get upstreamgroup [flags] ``` ``` -h, --help help for upstreamgroup ``` ``` -c, --config string set the path to the glooctl config file (default "<home_directory>/.gloo/glooctl-config.yaml") --consul-address string address of the Consul server. Use with --use-consul (default "127.0.0.1:8500") --consul-allow-stale-reads Allows reading using Consul's stale consistency mode. --consul-datacenter string Datacenter to use. If not provided, the default agent datacenter is used. Use with --use-consul --consul-root-key string key prefix for for Consul key-value storage. (default "gloo") --consul-scheme string URI scheme for the Consul server. Use with --use-consul (default "http") --consul-token string Token is used to provide a per-request ACL token which overrides the agent's default token. Use with --use-consul -i, --interactive use interactive mode --kube-context string kube context to use when interacting with kubernetes --kubeconfig string kubeconfig to use, if not standard one --name string name of the resource to read or write -n, --namespace string namespace for reading or writing resources (default "gloo-system") -o, --output OutputType output format: (yaml, json, table, kube-yaml, wide) (default table) --use-consul use Consul Key-Value storage as the backend for reading and writing config (VirtualServices, Upstreams, and Proxies) ``` - Display one or a list of Gloo resources
Project: Contour (Orchestration & Management / Service Proxy)
File: contour-debug-log.md

The `contour serve` subcommand has two command-line flags that can be helpful for debugging. The `--debug` flag enables general Contour debug logging, which logs more information about how Contour is processing API resources. The `--kubernetes-debug` flag enables verbose logging in the Kubernetes client API, which can help debug interactions between Contour and the Kubernetes API server. This flag requires an integer log level argument, where higher number indicates more detailed logging.
Project: BFE (Orchestration & Management / Service Proxy)
File: mod_trust_clientip.md

modtrustclientip checks the client IP of incoming request/connnection against trusted ip dictionary. If matched, the request/connection is marked as trusted. conf/modtrustclientip/modtrustclientip.conf | Config Item | Description | | -- | | | Basic.DataPath | String<br>Path of rule configuration | ```ini [Basic] DataPath = modtrustclientip/trustclientip.data ``` conf/modtrustclientip/trustclientip.data | Config Item | Type | Description | | -- | | | | Version | String | Version of config file | | Config | Object | Trusted ip config | | Config{k} | Struct | Label | Config{v} | String | A list of ip segments | | Config{v}[] | Object | A ip segment | | Config{v}[].Begin | String | Start ip address | | Config{v}[].End | String | End ip address | ```json { "Version": "20190101000000", "Config": { "inner-idc": [ { "Begin": "10.0.0.0", "End": "10.255.255.255" } ] } } ``` | Metric | Description | | - | -- | | CONNADDRINTERNAL | Counter for connection from internal | | CONNADDRINTERNALNOTTRUST | Counter for connection from internal and not trust | | CONN_TOTAL | Counter for all connnetion checked | | CONNTRUSTCLIENTIP | Counter for connection from trust address |