cwielenb/CNCF-Dataset huggingface dataset:
Project: KubeStellar (Orchestration & Management / Scheduling & Orchestration)
File: best-practices.md

Note: This section is under construction and includes partial information. As KubeStellar is built on top of Kubernetes all Kuberentes limitations and recommendations apply to KubeStellar as well. These recommendations can be found in . In addition, KubeStellar Transport Plugin is built on top of OCM, so KubeStellar also comply to some of OCM's limitations. Users should take into account the following restrictions: The ManifestWork shouldn't exceed 500KB
Project: MetalLB (Orchestration & Management / Service Proxy)
File: expand.en.md

+++ description = "Expandable/collapsible sections of text" title = "Expand" +++ The `expand` shortcode displays an expandable/collapsible section of text. {{% expand title="Expand me..." %}}Thank you! That's some text with a footnote[^1] That's some more text with a footnote.[^someid] Blue light glows blue. {{% /expand %}} {{% notice note %}} This only works in modern browsers flawlessly. While Internet Explorer 11 has issues in displaying it, the functionality still works. {{% /notice %}} While the examples are using shortcodes with named parameter you are free to use positional as well or also call this shortcode from your own partials. {{< tabs groupid="shortcode-parameter">}} {{% tab title="shortcode" %}} ````go {{%/ expand title="Expand me..." /%}}Thank you!{{%/ /expand /%}} ```` {{% /tab %}} {{% tab title="shortcode (positional)" %}} ````go {{%/ expand "Expand me..." /%}}Thank you!{{%/ /expand /%}} ```` {{% /tab %}} {{% tab title="partial" %}} ````go {{ partial "shortcodes/expand.html" (dict "page" . "title" "Expand me..." "content" "Thank you!" )}} ```` {{% /tab %}} {{< /tabs >}} | Name | Position | Default | Notes | |--|-||-| | title | 1 | `"Expand me..."` | Arbitrary text to appear next to the expand/collapse icon. | | open | 2 | `false` | When `true` the content text will be initially shown as expanded. | | **&lt;content&gt;** | | &lt;empty&gt; | Arbitrary text to be displayed on expand. | ````go {{%/ expand /%}}Yes, you did it!{{%/ /expand /%}} ```` {{% expand %}}Yes, you did it!{{% /expand %}} ````go {{%/ expand title="Expand me..." open="true" /%}}No need to press you!{{%/ /expand /%}} ```` {{% expand title="Expand me..." open="true" %}}No need to press you!{{% /expand %}} ````go {{%/ expand title="Show me almost endless possibilities" /%}} You can add standard markdown syntax: multiple paragraphs bullet point lists emphasized, bold and even _bold emphasized_ text etc. ```plaintext ...and even source code ``` the possibilities are endless (almost - including other shortcodes may or may not work) {{%/ /expand /%}} ```` {{% expand title="Show me almost endless possibilities" %}} You can add standard markdown syntax: multiple paragraphs bullet point lists emphasized, bold and even _bold emphasized_ text etc. ```plaintext ...and even source code ``` the possibilities are endless (almost - including other shortcodes may or may not work) {{% /expand %}}
Project: KubeBrain (Orchestration & Management / Coordination & Service Discovery)
File: quick_start.md

KubeBrain adapts to dgraph-io/badger to implement a stand-alone version for testing. ```shell make badger ./bin/kube-brain --key-prefix "/" ``` KubeBrain adapts to TiKV to implement kubernetes meta-information storage based on distributed key-value database. For TiKV cluster deployment, please refer to TiUP. For the local test environment, you can use tiup playground. ```shell tiup playground --mode tikv-slim --kv 1 --pd 1 ``` ```shell make tikv ./bin/kube-brain --key-prefix "/" --pd-addr=127.0.0.1:2379 --port=3379 --peer-port=3380 --compatible-with-etcd=true ``` When multiple KubeBrain share a tikv cluster `key-prefix` is consistent with the etcd-prefixparameter of the apiserver corresponding to the apiserver. `compatible-with-etcd` should set true to make all nodes provide all etcd APIs used in apiserver The community version of the APIServer storage only supports the API of etcd3, and KubeBrain is compatible with the API of etcd3 used in the community version of APIServer, but this method does not take full advantage of KubeBrain. ```shell ./kube-apiserver / --storage-backend=etcd3 \ --etcd-servers=http://127.0.0.1:3379 \ ``` The customized APIServer accesses the storage system by extending the storage interface of APIServer, and integrating functions such as cluster state checking, read-write separation of load, etc. in the SDK. The custom api-server will be released soon.
Project: Gloo (Orchestration & Management / API Gateway)
File: glooctl_install_gateway.md

title: "glooctl install gateway" weight: 5 install the Gloo Gateway on Kubernetes requires kubectl to be installed ``` glooctl install gateway [flags] ``` ``` --create-namespace Create the namespace to install gloo into (default true) -d, --dry-run Dump the raw installation yaml instead of applying it to kubernetes -f, --file string Install Gloo from this Helm chart archive file rather than from a release -h, --help help for gateway -n, --namespace string namespace to install gloo into (default "gloo-system") --release-name string helm release name (default "gloo") --values strings List of files with value overrides for the Gloo Helm chart, (e.g. --values file1,file2 or --values file1 --values file2) --version string version to install (e.g. 1.4.0, defaults to latest) ``` ``` -c, --config string set the path to the glooctl config file (default "<home_directory>/.gloo/glooctl-config.yaml") --consul-address string address of the Consul server. Use with --use-consul (default "127.0.0.1:8500") --consul-allow-stale-reads Allows reading using Consul's stale consistency mode. --consul-datacenter string Datacenter to use. If not provided, the default agent datacenter is used. Use with --use-consul --consul-root-key string key prefix for for Consul key-value storage. (default "gloo") --consul-scheme string URI scheme for the Consul server. Use with --use-consul (default "http") --consul-token string Token is used to provide a per-request ACL token which overrides the agent's default token. Use with --use-consul -i, --interactive use interactive mode --kube-context string kube context to use when interacting with kubernetes --kubeconfig string kubeconfig to use, if not standard one --use-consul use Consul Key-Value storage as the backend for reading and writing config (VirtualServices, Upstreams, and Proxies) -v, --verbose If true, output from kubectl commands will print to stdout/stderr ``` - install gloo on different platforms - install the Gloo Enterprise Gateway on Kubernetes
Project: ngrok (Orchestration & Management / API Gateway)
File: README.md

](https://pkg.go.dev/golang.ngrok.com/ngrok) ](https://github.com/ngrok/ngrok-go/actions/workflows/buildandtest.yml) ](https://github.com/ngrok/ngrok-rust/blob/main/LICENSE-MIT) is a simplified API-first ingress-as-a-service that adds connectivity, security, and observability to your apps. ngrok-go is an open source and idiomatic library for embedding ngrok networking directly into Go applications. If youve used ngrok before, you can think of ngrok-go as the ngrok agent packaged as a Go library. ngrok-go lets developers serve Go apps on the internet in a single line of code without setting up low-level network primitives like IPs, certificates, load balancers and even ports! Applications using ngrok-go listen on ngroks global ingress network but they receive the same interface any Go app would expect (net.Listener) as if it listened on a local port by calling net.Listen(). This makes it effortless to integrate ngrok-go into any application that uses Go's net or net/http packages. See for example usage, or the tests in . For working with the , check out the . The best way to install the ngrok agent SDK is through `go get`. ```sh go get golang.ngrok.com/ngrok ``` A full API reference is included in the . Check out the for more information about what you can do with ngrok. For additional information, be sure to also check out the ! For more examples of using ngrok-go, check out the folder. The following example uses ngrok to start an http endpoint with a random url that will route traffic to the handler. The ngrok URL provided when running this example is accessible by anyone with an internet connection. The ngrok authtoken is pulled from the `NGROK_AUTHTOKEN` environment variable. You can find your authtoken by logging into the . You can run this example with the following command: ```sh NGROKAUTHTOKEN=xxxxxxxx go run examples/http/main.go ``` ```go package main import ( "context" "fmt" "log" "net/http" "golang.ngrok.com/ngrok" "golang.ngrok.com/ngrok/config" ) func main() { if err := run(context.Background()); err != nil { log.Fatal(err) } } func run(ctx context.Context) error { ln, err := ngrok.Listen(ctx, config.HTTPEndpoint(), ngrok.WithAuthtokenFromEnv(), ) if err != nil { return err } log.Println("Ingress established at:", ln.URL()) return http.Serve(ln, http.HandlerFunc(handler)) } func handler(w http.ResponseWriter, r *http.Request) { fmt.Fprintln(w, "Hello from ngrok-go!") } ``` The best place to get support using ngrok-go is through the . If you find bugs or would like to contribute code, please follow the instructions in the . Changes to `ngrok-go` are tracked under . Check out Read about updates on Open an or Join our Follow us on Subscribe to our ngrok-go is licensed under the terms of the MIT license. See for details.
Project: GoFr (Orchestration & Management / Remote Procedure Call)
File: README.md

This GoFr example demonstrates a simple Subscriber that subscribes asynchronously to a given topic and commits based on the handler response. Run the docker image of kafka and ensure that your provided topics are created before subscribing. ```console docker run --name kafka-1 -p 9092:9092 \ -e KAFKAENABLEKRAFT=yes \ -e KAFKACFGPROCESS_ROLES=broker,controller \ -e KAFKACFGCONTROLLERLISTENERNAMES=CONTROLLER \ -e KAFKACFGLISTENERS=PLAINTEXT://:9092,CONTROLLER://:9093 \ -e KAFKACFGLISTENERSECURITYPROTOCOL_MAP=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT \ -e KAFKACFGADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092 \ -e KAFKACFGAUTOCREATETOPICS_ENABLE=true \ -e KAFKABROKERID=1 \ -e KAFKACFGCONTROLLERQUORUMVOTERS=1@127.0.0.1:9093 \ -e ALLOWPLAINTEXTLISTENER=yes \ -e KAFKACFGNODE_ID=1 \ -v kafka_data:/bitnami \ bitnami/kafka:3.4 ``` Now run the example using below command : ```console go run main.go ```
Project: Fluid (Orchestration & Management / Scheduling & Orchestration)
File: ADOPTERS.md

The following are the adopters of project Fluid. If you are using Fluid to simplify and accelerate your applications to access data in Kubernetes, please feel free to add yourself into the following list by a pull request. There're several phases as follow: Evaluation:* Known Fluid, that's interesting; evaluating the features/scopes of Fluid Testing:* Take Fluid as one of candidates, testing Kubernetes cluster with Fluid Staging:* Decide to use Fluid, testing it in pre-product environment Production:* Already put Fluid into product environment The companies/organizations are listed in alphabetical order grouped by the phase. | Organization | Contact | Phases | Description of Use | | | - | -- | | | | | Testing | 4Paradigm Sage EE | | | | Testing | ML & Kubernetes | | | | Staging | Alsenal AI Platform | | | | Staging | AI Platform on k8s | | | | Testing | HAOMO Auto Machine Learning Platform | | | | Production | PAI Deep Learning Containers | | | | Production | BigData platform | | | | Testing | Tencent Kubernetes Engine | | | | Staging | Atlas Deep Learning Platform | | | | Production | weibo deep learning platform | | | | Production | bilibili AI platform | | | | Production | LakeCache Containers | | | | Testing | Pxing AI training platform | | | | Testing | MetaSpore distributed multimodal AI platform | | | | Testing | AI Platform on k8s | | | | Testing | Recommendation | | | | Staging | Cloud Container Engine | | | | Testing | AI Platform on k8s | | | | Production | AI Platform on k8s | | | | Testing | AI Platform on k8s | | | | Staging | Cloudml AI Platform | | | | Testing | AI Platform on k8s | | | | Production | AI Platform on k8s | | | | Production | Data acceleration | | | | Production | AI Platform on k8s | | | | Production | Bigdata Platform and Autonomous Driving Simulation Platform | | | | Testing | AI platform training data cache and big emr job data process data cache | | | | Production | Data Platform on k8s | | | | Testing | AI training platform| | | | Production | AI Platform on k8s| | | | Testing | AI Platform on k8s| | | | Testing | Elastic Container Orchestration Service on k8s | | | | Staging | AI Platform(KubeAI) on k8s | | | | Production | AI Training/Big Data/Stream Media Platforms In Aicheng Technology At Large Scale | | | | Production | AIGC Platform on k8s | | | | Production | AIGC Platform on k8s | | | | Production | AI Platform on k8s | | | | Production | Accelerate Spark Service on BigData |
Project: CoreDNS (Orchestration & Management / Coordination & Service Discovery)
File: coredns-1.1.4.md

+++ title = "CoreDNS-1.1.4 Release" description = "CoreDNS-1.1.4 Release Notes." tags = ["Release", "1.1.4", "Notes"] release = "1.1.4" date = "2018-06-19T09:39:29+01:00" author = "coredns" +++ We are pleased to announce the of CoreDNS-1.1.4! This release has a few enhancements in the plugins, and a few (Docker) improvements. As said in the , we are making the `-log` command line flag a noop. This is also a heads up that in the next release - 1.2.0 - the current etcd plugin will be replaced by a new plugin that supports etcd3, see this [pull request](https://github.com/coredns/coredns/pull/1702). The Docker image is now built using a multistage build. This means the final image is based on `scratch` and all architectures now have certificates in the image (not just the amd64 one). We are also deprecating: the reverse* plugin has been removed, but we allow it still in the configuration. the `google_https` protocol has been a noop in the proxy* plugin. In the next release (1.2.0) this code will removed completely. Further more: file* now always queries local zones when trying to find a CNAME target. log* will now always log in seconds (not micro, or milliseconds). forward* erases expired connection after some time. The following people helped with getting this release done: Francois Tur, Malcolm Akinje, Mario Kleinsasser, Miek Gieben, Ruslan Drozhdzh, Yong Tang. For documentation see our (in progress!) . For help and other resources, see our .
Project: BFE (Orchestration & Management / Service Proxy)
File: mod_redirect.md

mod_redirect redirects HTTP requests based on defined rules. conf/modredirect/modredirect.conf | Config Item | Description | | -- | | | Basic.DataPath | String<br>Path of rule configuration | ```ini [Basic] DataPath = mod_redirect/redirect.data ``` conf/mod_redirect/redirect.data | Config Item | Description | | -- | | | Version | String<br>Version of config file | | Config | Struct<br>Redirect rules for each product. | | Config{k} | String<br>Product name | | Config{v} | Object<br>A ordered list of redirect rules | | Config{v}[] | Object<br>A redirect rule | | Config{v} | | Config{v}[].Actions | Object<br>A ordered list of redirect actions | | Config{v}[].Actions[] | Object<br>A redirect action | | Config{v}[].Actions[].Cmd | Object<br>Name of redirect action | | Config{v}[].Actions[].Params | Object<br>Parameters of redirect action | | Config{v}[].Status | Integer<br>Status code | | Action | Description | | -- | -- | | URL_SET | Redirect to specified URL | | URLFROMQUERY | Redirect to URL parsed from specified query in request | | URLPREFIXADD | Redirect to URL concatenated by specified prefix and the original URL | | SCHEME_SET | Redirect to the original URL but with scheme changed. supported scheme: http\|https | ```json { "Version": "20190101000000", "Config": { "example_product": [ { "Cond": "reqpathprefix_in(\"/redirect\", false)", "Actions": [ { "Cmd": "URL_SET", "Params": ["https://example.org"] } ], "Status": 301 } ] } } ```
Project: Contour (Orchestration & Management / Service Proxy)
File: proxy-proto.md

title: How to Configure PROXY v1/v2 Support If you deploy Contour as a Deployment or Daemonset, you will likely use a `type: LoadBalancer` Service to request an from your hosting provider. If you use the Elastic Load Balancer (ELB) service from Amazon's EC2, you need to perform a couple of additional steps to enable the protocol. Here's why: External load balancers typically operate in one of two modes: a layer 7 HTTP proxy, or a layer 4 TCP proxy. The former cannot be used to load balance TLS traffic, because your cloud provider attempts HTTP negotiation on port 443. So the latter must be used when Contour handles HTTP and HTTPS traffic. However this leads to a situation where the remote IP address of the client is reported as the inside address of your cloud provider's load balancer. To rectify the situation, you can add annotations to your service and flags to your Contour Deployment or DaemonSet to enable the protocol which forwards the original client IP details to Envoy. In GKE clusters a `type: LoadBalancer` Service is provisioned as a Network Load Balancer and will forward traffic to your Envoy instances with their client addresses intact. Your services should see the addresses in the `X-Forwarded-For` or `X-Envoy-External-Address` headers without having to enable a PROXY protocol. To instruct EC2 to place the ELB into `tcp`+`PROXY` mode, add the following annotations to the `contour` Service: ``` apiVersion: v1 kind: Service metadata: annotations: service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: '*' name: contour namespace: projectcontour spec: type: LoadBalancer ... ``` ``` ... spec: containers: image: ghcr.io/projectcontour/contour:<version> imagePullPolicy: Always name: contour command: ["contour"] args: ["serve", "--incluster", "--use-proxy-protocol"] ... ```
Project: DolphinScheduler (Orchestration & Management / Scheduling & Orchestration)
File: open-api.md

Generally, projects and processes are created through pages, but considering the integration with third-party systems requires API calls to manage projects and workflows. Log in to the scheduling system, click "Security", then click "Token manage" on the left, and click "Create token" to create a token. Select the "Expiration time" (Token validity time), select "User" (choose the specified user to perform the API operation), click "Generate token", copy the `Token` string, and click "Submit". Open the API documentation Addresshttp://{API server ip}:12345/dolphinscheduler/swagger-ui/index.html?language=en_US&lang=en select a test API, the API selected for this test is `queryAllProjectList` > projects/list Open `Postman`, fill in the API address, enter the `Token` in `Headers`, and then send the request to view the result: ``` token: The Token just generated ``` This demonstrates how to use the calling api to create the corresponding project. By consulting the api documentation, configure the KEY as Accept and VALUE as the parameter of application/json in the headers of Postman. And then configure the required projectName and description parameters in Body. Check the post request result. The returned `msg` information is "success", indicating that we have successfully created the project through API. If you are interested in the source code of creating a project, please continue to read the following
Project: Volcano (Orchestration & Management / Scheduling & Orchestration)
File: dynamic-conf.md

Volcano consists of three components: Scheduler, Webhook, and Controller. The Scheduler component schedules jobs by performing a series of actions and plugins to find the most suitable node for them. The Controller Manager manages the lifecycle of CRD (Custom Resource Definition) resources and is mainly composed of Queue ControllerManager, PodGroupControllerManager, and VCJob ControllerManager. The Admission component is responsible for validating CRD API resources. Currently, the startup configuration information for Volcano components is static and cannot be updated at runtime. When locate a problem of volcano, usually the existing log level is not enough and the problem cannot be located, however, after restart volcano-scheduler and increasing the log level, the problem context is lost, making it impossible to continue locating. Therefore, it is convenient to update the log level without restarting volcano to identify the problem. The startup configuration information for Volcano components is currently static, and we need to select appropriate configurations and make them dynamic. I will identify all the static parameters used when launching Volcano and choose the configurations that are necessary to be dynamically adjustable under specific scenarios. ``` -v, --v: The level of verbosity for log levels. ``` `-v `The inability to update the log level of Volcano's Scheduler at runtime poses a significant burden on administrators during operation and maintenance. It is crucial to add the capability of dynamic configuration hot updates to enhance the efficiency of problem identification in production environments. Only focus on the static parameters that start the Schedule component The vc-scheduler pod has an internal socket service to monitor changes in the klog.scok file. Because the socket service in the pod cannot be directly accessed in the k8s Node, the klog.sock file needs to be mounted from the pod to the cluster. Use The curl command modifies the klog.sock file. example curl command example ``` sudo curl --unix-socket /tmp/socks/klog.sock "http://localhost/setlevel?level=5&duration=60s" ``` 1Modifying configmap may affect other plugins.If Volcano deployed in a customer cluster, the configmap cannot be modified easily. 2It'll be confusing that both options and configmap has same parameters,but have different value. In deployment.yaml and configmap have different klog level. deployment.yaml ``` name: volcano-scheduler image: volcanosh/vc-scheduler:latest args: -v=3 ``` configmap ``` volcano-scheduler.conf: | actions: "enqueue, allocate, backfill" configurations: name: dynamicConf arguments: klogLevel: "5" ```