cwielenb/CNCF-Dataset huggingface dataset:
Project: StarRocks (App Definition and Development / Database)
File: DBeaver.md

displayed_sidebar: "English" DBeaver is a SQL client software application and a database administration tool, which offers a helpful assistant that walks you through the process of connecting to a database. Make sure that you have installed DBeaver. You can download the DBeaver Community edition at or the DBeaver PRO edition at . Follow these steps to connect to a database: Launch DBeaver. Click the plus sign (+) icon in the upper-left corner of the DBeaver window or choose Database > New Database Connection in the menu bar to access the assistant. Select the MySQL driver. In the Select your database step, you are presented with a list of available drivers. Click Analytical in the left-side pane to quickly locate the MySQL driver. Then, double-click the MySQL icon. Configure the connection to the database. In the Connection Settings step, go to the Main tab and configure the following essential connection settings: Server Host: the FE host IP address of your StarRocks cluster. Port: the FE query port of your StarRocks cluster, for example, `9030`. Database: the target database in your StarRocks cluster. Both internal and external databases are supported, but the functionality for external databases may be incomplete. Username: the username that is used to log in to your StarRocks cluster, for example, `admin`. Password: the password that is used to log in to your StarRocks cluster. You can also view and edit the properties of the MySQL driver on the Driver properties tab if necessary. To edit a specific property, click the row in the Value column for that property. Test the connection to the database. Click Test Connection to verify the accuracy of the connection settings. A dialog box displaying the MySQL driver's information appears. Click OK in the dialog box to confirm the information. After you have successfully configured the connection settings, click Finish to complete the process. Connect to the database. After the connection is established, you can view it in the left-side database connection tree and DBeaver can effectively connect to the database.
Project: Beam (App Definition and Development / Streaming & Messaging)
File: 37_ai_tfx_orchestration.md

Prompt: How can I orchestrate Apache Beam pipelines with TFX? Response: You can use Apache Beam for various tasks within your machine learning project, including data validation, preprocessing, model validation, and model deployment and inference. However, a comprehensive ML workflow also involves additional steps like data exploration, feature engineering, and model training. Additionally, ML workflows require metadata and artifact tracking for reproducibility and auditability. Popular tools for building and managing ML workflows include , , and . TFX is an end-to-end platform designed for deploying production ML pipelines. TFX pipelines consist of a sequence of components that together form an ML pipeline. Each TFX pipeline DAG requires its own orchestrator. The natively supported orchestrators include , , and Apache Beam itself. To integrate an Apache Beam pipeline into a TFX pipeline, you need to follow these steps: Create TFX components by specifying interfaces and containerizing the component implementation. Create the TFX pipeline by connecting these components and specifying how inputs and outputs are exchanged between them. Compile and execute the TFX pipeline in the target environment. TFX provides a rich set of for constructing ML pipelines. These components can be used as building blocks for creating custom components. TFX heavily relies on Apache Beam libraries for implementing data-parallel pipelines within these components. You must execute components created using these libraries with one of the supported Apache Beam runners. To launch the pipeline, you must provide two configurations: the orchestrator for the TFX pipeline and the pipeline options necessary for running Apache Beam pipelines. You can find the complete TFX example code in the .
Project: YugabyteDB (App Definition and Development / Database)
File: yba_universe.md

Manage YugabyteDB Anywhere universes Manage YugabyteDB Anywhere universes ``` yba universe [flags] ``` ``` -h, --help help for universe ``` ``` -a, --apiToken string YugabyteDB Anywhere api token. --config string Config file, defaults to $HOME/.yba-cli.yaml --debug Use debug mode, same as --logLevel debug. --disable-color Disable colors in output. (default false) -H, --host string YugabyteDB Anywhere Host (default "http://localhost:9000") -l, --logLevel string Select the desired log level format. Allowed values: debug, info, warn, error, fatal. (default "info") -o, --output string Select the desired output format. Allowed values: table, json, pretty. (default "table") --timeout duration Wait command timeout, example: 5m, 1h. (default 168h0m0s) --wait Wait until the task is completed, otherwise it will exit immediately. (default true) ``` - yba - Command line tools to manage your YugabyteDB Anywhere (Self-managed Database-as-a-Service) resources. - Create YugabyteDB Anywhere universe - Delete a YugabyteDB Anywhere universe - Describe a YugabyteDB Anywhere universe - List YugabyteDB Anywhere universes - Manage YugabyteDB Anywhere universe nodes - Restart a YugabyteDB Anywhere Universe - Upgrade a YugabyteDB Anywhere universe
Project: Redis (App Definition and Development / Database)
File: crash_report.md

name: Crash report about: Submit a crash report title: '[CRASH] <short description>' labels: '' assignees: '' Notice! If a Redis module was involved, please open an issue in the module's repo instead! If you're using docker on Apple M1, please make sure the image you're using was compiled for ARM! Crash report Paste the complete crash log between the quotes below. Please include a few lines from the log preceding the crash report to provide some context. ``` ``` Additional information OS distribution and version Steps to reproduce (if any)
Project: KICS (Provisioning / Security & Compliance)
File: dd690686-2bf9-4012-a821-f61912dd77be.md

title: Client Certificate Disabled hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: dd690686-2bf9-4012-a821-f61912dd77be Query name: Client Certificate Disabled Platform: GoogleDeploymentManager Severity: <span style="color:#bb2124">High</span> Category: Insecure Configurations URL: Kubernetes Clusters must be created with Client Certificate enabled, which means 'masterAuth' must have 'clientCertificateConfig' with the attribute 'issueClientCertificate' equal to true<br> ```yaml title="Positive test num. 1 - yaml file" hl_lines="4" resources: name: cluster type: container.v1.cluster properties: name: my-cluster ``` ```yaml title="Positive test num. 2 - yaml file" hl_lines="6" resources: name: cluster type: container.v1.cluster properties: name: my-cluster masterAuth: clientKey: key ``` ```yaml title="Positive test num. 3 - yaml file" hl_lines="8" resources: name: cluster type: container.v1.cluster properties: name: my-cluster masterAuth: clientCertificateConfig: issueClientCertificate: false ``` ```yaml title="Negative test num. 1 - yaml file" resources: name: cluster type: container.v1.cluster properties: name: my-cluster masterAuth: clientCertificateConfig: issueClientCertificate: true ```
Project: Apache Storm (App Definition and Development / Streaming & Messaging)
File: dynamic-worker-profiling.md

title: Dynamic Worker Profiling layout: documentation documentation: true In multi-tenant mode, storm launches long-running JVMs across cluster without sudo access to user. Self-serving of Java heap-dumps, jstacks and java profiling of these JVMs would improve users' ability to analyze and debug issues when monitoring it actively. The storm dynamic profiler lets you dynamically take heap-dumps, jprofile or jstack for a worker jvm running on stock cluster. It let user download these dumps from the browser and use your favorite tools to analyze it The UI component page provides list workers for the component and action buttons. The logviewer lets you download the dumps generated by these logs. Please see the screenshots for more information. Using the Storm UI In order to request for heap-dump, jstack, start/stop/dump jprofile or restart a worker, click on a running topology, then click on specific component, then you can select workers by checking the box of any of the worker's executors in the Executors table, and then click on Start","Heap", "Jstack" or "Restart Worker" in the "Profiling and Debugging" section. In the Executors table, click the checkbox in the Actions column next to any executor, and any other executors belonging to the same worker are automatically selected. When the action has completed, any output files created will available at the link in the Actions column. For start jprofile, provide a timeout in minutes (or 10 if not needed). Then click on Start. To stop the jprofile logging click on the Stop button. This dumps the jprofile stats and stops the profiling. Refresh the page for the line to disappear from the UI. Click on "My Dump Files" to go the logviewer UI for list of worker specific dump files. Configuration The "worker.profiler.command" can be configured to point to specific pluggable profiler, heapdump commands. The "worker.profiler.enabled" can be disabled if plugin is not available or jdk does not support Jprofile flight recording so that worker JVM options will not have "worker.profiler.childopts". To use different profiler plugin, you can change these configuration.
Project: StarRocks (App Definition and Development / Database)
File: deployment_prerequisites.md

displayed_sidebar: "English" This topic describes the hardware and software requirements that your servers must meet before deploying StarRocks. For recommended hardware specifications of your StarRocks cluster, see . StarRocks relies on AVX2 instruction sets to fully unleash its vectorization capability. Therefore, in a production environment, we highly recommend you deploy StarRocks on machines with x86 architecture CPUs. You can run the following command in your terminal to check if the CPUs on your machines support the AVX2 instruction sets: ```Bash cat /proc/cpuinfo | grep avx2 ``` NOTE ARM architecture does not support SIMD instruction sets, and therefore is less competitive than x86 architecture in some scenarios. Therefore, we only recommend deploying StarRocks on ARM architecture in a development environment. No specific requirement is imposed on memory kits used for StarRocks. See for the recommended memory size. StarRocks supports both HDD and SSD as storage medium. If your applications require real-time data analytics, intensive data scans, or random disk access, we strongly recommend you use SSD storage. If your applications involve with the persistent index, you must use SSD storage. We recommend that you use 10 Gigabit Ethernet networking to ensure stable data transmission across nodes within your StarRocks cluster. StarRocks supports deployments on CentOS Linux 7.9 or Ubuntu Linux 22.04. You must install JDK 8 on your servers to run StarRocks. For v2.5 and later versions, JDK 11 is recommended. CAUTION StarRocks does not support JRE. If you want to install StarRocks on Ubuntu 22.04, you must install JDK 11. Follow these steps to install JDK 8: Navigate to the path for the JDK installation. Download JDK by running the following command: ```Bash wget --no-check-certificate --no-cookies \ --header "Cookie: oraclelicense=accept-securebackup-cookie" \ http://download.oracle.com/otn-pub/java/jdk/8u131-b11/d54c1d3a095b4ff2b6607d096fa80163/jdk-8u131-linux-x64.tar.gz ```
Project: KubeBlocks by ApeCloud (App Definition and Development / Database)
File: kbcli_plugin_index_update.md

title: kbcli plugin index update update all configured indexes ``` kbcli plugin index update [flags] ``` ``` -h, --help help for update ``` ``` --as string Username to impersonate for the operation. User could be a regular user or a service account in a namespace. --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups. --as-uid string UID to impersonate for the operation. --cache-dir string Default cache directory (default "$HOME/.kube/cache") --certificate-authority string Path to a cert file for the certificate authority --client-certificate string Path to a client certificate file for TLS --client-key string Path to a client key file for TLS --cluster string The name of the kubeconfig cluster to use --context string The name of the kubeconfig context to use --disable-compression If true, opt-out of response compression for all requests to the server --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure --kubeconfig string Path to the kubeconfig file to use for CLI requests. --match-server-version Require server version to match client version -n, --namespace string If present, the namespace scope for this CLI request --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0") -s, --server string The address and port of the Kubernetes API server --tls-server-name string Server name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used --token string Bearer token for authentication to the API server --user string The name of the kubeconfig user to use ``` - Manage custom plugin indexes
Project: Operator Framework (App Definition and Development / Application Definition & Image Build)
File: operator-sdk_pkgman-to-bundle.md

title: "operator-sdk pkgman-to-bundle" Migrates packagemanifests to bundles 'pkgman-to-bundle' command helps in migrating OLM packagemanifests to bundles which is the preferred OLM packaging format. This command takes an input packagemanifest directory and generates bundles for each of the versions of manifests present in the input directory. Additionally, it also provides the flexibility to build bundle images for each of the generated bundles. The generated bundles are always written on disk. Location for the generated bundles can be specified using '--output-dir'. If not specified, the default location would be 'bundle/' directory. The base container image name for the bundles can be provided using '--image-tag-base' flag. This should be provided without the tag, since the tag for the images would be the bundle version, (ie) image names will be in the format &lt;baseimage&gt;:&lt;bundleversion&gt;. Specify the build command for building container images using '--build-cmd' flag. The default build command is 'docker build'. The command will need to be in the 'PATH' or fully qualified path name should be provided. ``` operator-sdk pkgman-to-bundle <packagemanifestdir> [flags] ``` ``` $ tree packagemanifests/ packagemanifests etcd 0.0.1 etcdcluster.crd.yaml etcdoperator.clusterserviceversion.yaml 0.0.2 etcdbackup.crd.yaml etcdcluster.crd.yaml etcdoperator.v0.0.2.clusterserviceversion.yaml etcdrestore.crd.yaml etcd.package.yaml $ operator-sdk pkgman-to-bundle packagemanifests --image-tag-base quay.io/example/etcd INFO[0000] Packagemanifests will be migrated to bundles in bundle directory INFO[0000] Creating bundle/bundle-0.0.1/bundle.Dockerfile INFO[0000] Creating bundle/bundle-0.0.1/metadata/annotations.yaml ... $ tree bundles/ bundles/ bundle-0.0.1 bundle manifests etcdcluster.crd.yaml etcdoperator.clusterserviceversion.yaml metadata annotations.yaml tests scorecard config.yaml bundle.Dockerfile bundle-0.0.2 bundle manifests etcdbackup.crd.yaml etcdcluster.crd.yaml etcdoperator.v0.0.2.clusterserviceversion.yaml etcdrestore.crd.yaml metadata annotations.yaml bundle.Dockerfile A custom command to build bundle images can also be specified using the '--build-cmd' flag. For example, $ operator-sdk pkgman-to-bundle packagemanifests --image-tag-base quay.io/example/etcd --build-cmd "podman build -f bundle.Dockerfile . -t" Images for the both the bundles will be built with the following names: quay.io/example/etcd:0.0.1 and quay.io/example/etcd:0.0.2. ``` ``` --build-cmd string Build command to be run for building images. By default 'docker build' is run. -h, --help help for pkgman-to-bundle --image-tag-base string Base container image name for bundle image tags, ex. my.reg/foo/bar-operator-bundle will become my.reg/foo/bar-operator-bundle:${package-dir-name} for each child directory name in the packagemanifests directory --output-dir string Directory to write bundle to. (default "bundles") ``` ``` --plugins strings plugin keys to be used for this subcommand execution --verbose Enable verbose logging ``` -
Project: Chef Habitat (App Definition and Development / Application Definition & Image Build)
File: plan_variables.md

+++ title = "Plan Variables" description = "Set package, service, and cache paths, compiler options, install location and context with plan variables" gh_repo = "habitat" [menu] [menu.habitat] title = "Plan Variables" identifier = "habitat/reference/plan-variables" parent = "habitat/reference" +++ The following variables can be used in your plans to help get binaries and libraries to build and install in the correct locations in your package. The values for the variables listed below are default values or read-only values. pkg_prefix : The absolute path for your package. pkg_dirname : The directory created when Habitat extracts an archive. If a .tar file extracts to a directory that's different from the filename, then you would need to override this value to match the directory name created during extraction. Default value: `${pkgname}-${pkgversion}` pkgsvcpath : Where the running service is located. Read-only value: `${HABROOTPATH}/svc/${pkg_name}` pkgsvcdata_path : Where the running service data is located. Read-only value: `${pkgsvcpath}/data` pkgsvcfiles_path : Where the gossiped configuration files are located. Read-only value: `${pkgsvcpath}/files` pkgsvcvar_path : Where the running service variable data is located. Read-only value: `${pkgsvcpath}/var` pkgsvcconfig_path : Where the running service configuration is located. Read-only value: `${pkgsvcpath}/config` pkgsvcstatic_path : Where the running service static data is located. Read-only value: `${pkgsvcpath}/static` CACHE_PATH : A temporary directory that will be clean on every build. Read-only value: `${HABCACHESRCPATH}/${pkgdirname}` HABCACHESRC_PATH : The default path where source archives are downloaded, extracted, and compiled. Read-only value: `${HABROOTPATH}/cache/src` HABCACHEARTIFACT_PATH : The default download root path for packages. Read-only value: `${HABROOTPATH}/cache/artifacts` HABPKGPATH : The root path containing all locally installed packages. Read-only value: `${HABROOTPATH}/pkgs` PLAN_CONTEXT : The location on your local dev machine for the files in your plan directory. CFLAGS : C compiler options. LDFLAGS : C linker options. PREFIX : Where to install the software; same as `pkg_prefix`. LDRUNPATH : Where to find the binaries at run time.
Project: SeaTunnel (App Definition and Development / Streaming & Messaging)
File: connector-check.md

```shell bin/seatunnel-connector.sh ``` ```text Usage: seatunnel-connector.sh [options] Options: -h, --help Show the usage message -l, --list List all supported plugins(sources, sinks, transforms) (default: false) -o, --option-rule Get option rule of the plugin by the plugin identifier(connector name or transform name) -pt, --plugin-type SeaTunnel plugin type, support [source, sink, transform] ``` ```shell bin/seatunnel-connector.sh -l bin/seatunnel-connector.sh -l -pt sink bin/seatunnel-connector.sh -o Paimon bin/seatunnel-connector.sh -o Paimon -pt sink ```
Project: Kui (App Definition and Development / Application Definition & Image Build)
File: two-choices.md

This guidebook will sequence the user through two choices. === "Sounds great, let's go!" Note that choices will be discarded if they do not contain a code block. ```shell echo "Ok, got it" ``` Description for first choice. === "Option A" Description for ${choice}. ```shell echo You chose ${choice} ``` === "Option B" Description for ${choice}. ```shell echo You chose ${choice} ``` Description for second choice. === "Option X" Description for ${choice}. ```shell echo You then chose ${choice} ``` === "Option Y" Description for ${choice}. ```shell echo You then chose ${choice} ```
Project: Dapr (App Definition and Development / Application Definition & Image Build)
File: v1.12.4.md

This update includes bug fixes: When restarting a significant deployment on Kubernetes (> 20 pods), many pods will be removed from placement table. When the restarts happens concurrently with leadership changes in placement service, the placement table can get into a corrupt state that requires the restart of placement service Impacts users running Dapr 1.12.0-1.12.3 that uses actors and deployments with many sidecar instances. Race condition between placement leadership changes and placement table updates due to pods being terminated. Mitigate by reducing the chances of leadership changes in placement service and fix a bug that can cause terminated pods to remain in placement table forever. When performing service invocation to another app using Dapr (using either HTTP or gRPC), the caller sidecar establishes a gRPC connection with the target sidecar. When the target app is deployed with multiple replicas (scaled horizontally), a connection is established with each replica and requests are load-balanced across all replicas. In Dapr 1.12.3 and lower running on Kubernetes, due to the way this logic was implemented, if the connection with one of the replicas became idle, it would have caused the connection between the caller sidecar and all replicas to be severed, possibly interrupting other in-flight service invocation calls. This issue impacts users running Dapr 1.12.3 and lower, running on Kubernetes, that use service invocation to invoke apps that are scaled horizontally. This issue does not impact users that are running outside of Kubernetes or who are using other Dapr nameresolution components (including mDNS or Consul). On Kubernetes, the gRPC connection was established with the DNS name of the target Service, and we allowed the gRPC-Go library to perform name resolution with the Kubernetes DNS server and establish a connection with each replica. When the connection with one of the replicas becomes idle, the caller app receives a "GOAWAY" message which is interpreted as a signal to terminate all connections with all replicas of the app. We have changed the internals of service invocation, so we always perform DNS resolution in the caller Dapr sidecar, also on Kubernetes. The gRPC-Go library receives an individual IP address to connect to, so "GOAWAY" messages only impact the connection with an individual replica.
Project: StarRocks (App Definition and Development / Database)
File: SET_CATALOG.md

displayed_sidebar: "English" Switches to a specified catalog in the current session. This command is supported from v3.0 onwards. NOTE For a newly deployed StarRocks v3.1 cluster, you must have the USAGE privilege on the destination external catalog if you want to run SET CATALOG to switch to that catalog. You can use to grant the required privileges. For a v3.1 cluster upgraded from an earlier version, you can run SET CATALOG with the inherited privilege. ```SQL SET CATALOG <catalog_name> ``` `catalog_name`: the name of the catalog to use in the current session. You can switch to an internal or external catalog. If the catalog that you specify does not exist, an exception is thrown. Run the following command to switch to a Hive catalog named `hive_metastore` in the current session: ```SQL SET CATALOG hive_metastore; ``` Run the following command to switch to the internal catalog `default_catalog` in the current session: ```SQL SET CATALOG default_catalog; ```