cwielenb/CNCF-Dataset huggingface dataset:
Project: StarRocks (App Definition and Development / Database)
File: SHOW_ROUTINE_LOAD_TASK.md

displayed_sidebar: "English" import RoutineLoadPrivNote from '../../../assets/commonMarkdown/RoutineLoadPrivNote.md' Shows the execution information of load tasks within a Routine Load job. :::note <RoutineLoadPrivNote /> For the relationship between a Routine Load job and the load tasks in it, see ::: ```SQL SHOW ROUTINE LOAD TASK [ FROM <db_name>] [ WHERE JobName = <job_name> ] ``` :::note You can add the `\G` option to the statement (such as `SHOW ROUTINE LOAD TASK WHERE JobName = <job_name>\G`) to vertically display the return result rather than in the usual horizontal table format. ::: | Parameter | Required | Description | | - | | -- | | db_name | No | The name of the database to which the Routine Load job belongs. | | JobName | No | The name of the Routine Load job. | | Parameter | Description | | -- | | | TaskId | The globally unique ID of the load task, automatically generated by StarRocks. | | TxnId | The ID of the transaction to which the load task belongs. | | TxnStatus | The status of the transaction to which the load task belongs. `UNKNOWN` indicates that the transaction is not started yet, possibly because the load task is not dispatched or executed. | | JobId | The ID of the load job. | | CreateTime | The date and time when the load task was created. | | LastScheduledTime | The date and time when the load task was last scheduled. | | ExecuteStartTime | The date and time when the load task was executed. | | Timeout | The timeout period for the load task, controlled by the FE parameter and the `tasktimeoutsecond` parameter in for the Routine Load job. | | BeId | The ID of the BE that executes the load task. | | DataSourceProperties | The load task's progress (measured in the offset) of consuming messages in partitions of the topic. | | Message | The information returned for the load task, including task error information. | View all load tasks in the Routine Load job `exampletblordertest`. ```SQL MySQL [exampledb]> SHOW ROUTINE LOAD TASK WHERE JobName = "exampletbl_ordertest"; +--+-+--+-+++++++--+ | TaskId | TxnId | TxnStatus | JobId | CreateTime | LastScheduledTime | ExecuteStartTime | Timeout | BeId | DataSourceProperties | Message | +--+-+--+-+++++++--+ | abde6998-c19a-43d6-b48c-6ca7e14144a3 | -1 | UNKNOWN | 10208 | 2023-12-22 12:46:10 | 2023-12-22 12:47:00 | NULL | 60 | -1 | Progress:{"0":6},LatestOffset:null | there is no new data in kafka/pulsar, wait for 10 seconds to schedule again | +--+-+--+-+++++++--+ 1 row in set (0.00 sec) ```
Project: ClickHouse (App Definition and Development / Database)
File: v22.5.3.21-stable.md

sidebar_position: 1 sidebar_label: 2022 Backported in : Fix possible crash in `Distributed` async insert in case of removing a replica from config. (). Backported in : Any allocations inside OvercommitTracker may lead to deadlock. Logging was not very informative so it's easier just to remove logging. Fixes . (). Backported in : Fix bug in filesystem cache that could happen in some corner case which coincided with cache capacity hitting the limit. Closes . (). Backported in : Fix error `Block structure mismatch` which could happen for INSERT into table with attached MATERIALIZED VIEW and enabled setting `extremes = 1`. Closes and . (). Backported in : Fixed error `Not found column Type in block` in selects with `PREWHERE` and read-in-order optimizations. (). Backported in : Declare RabbitMQ queue without default arguments `x-max-length` and `x-overflow`. (). Backported in : Fix incorrect fetch postgresql tables query fro PostgreSQL database engine. Closes . (). Retry docker buildx commands with progressive sleep in between (). Add docker_server.py running to backport and release CIs ().
Project: SeaTunnel (App Definition and Development / Streaming & Messaging)
File: Email.md

Email sink connector Send the data as a file to email. The tested email version is 1.5.6. | name | type | required | default value | |--|--|-|| | emailfromaddress | string | yes | - | | emailtoaddress | string | yes | - | | email_host | string | yes | - | | emailtransportprotocol | string | yes | - | | emailsmtpauth | string | yes | - | | emailauthorizationcode | string | yes | - | | emailmessageheadline | string | yes | - | | emailmessagecontent | string | yes | - | | common-options | | no | - | Sender Email Address . Address to receive mail. SMTP server to connect to. The protocol to load the session . Whether to authenticate the customer. authorization code,You can obtain the authorization code from the mailbox Settings. The subject line of the entire message. The body of the entire message. Sink plugin common parameters, please refer to for details. ```bash EmailSink { emailfromaddress = "xxxxxx@qq.com" emailtoaddress = "xxxxxx@163.com" email_host="smtp.qq.com" emailtransportprotocol="smtp" emailsmtpauth="true" emailauthorizationcode="" emailmessageheadline="" emailmessagecontent="" } ``` Add Email Sink Connector
Project: YugabyteDB (App Definition and Development / Database)
File: sysbench-ysql.md

title: Benchmark YSQL performance using sysbench headerTitle: sysbench linkTitle: sysbench description: Benchmark YSQL performance using sysbench. headcontent: Benchmark YSQL performance using sysbench menu: v2.18: identifier: sysbench-ysql parent: benchmark weight: 5 type: docs sysbench is a popular tool for benchmarking databases like PostgreSQL and MySQL, as well as system capabilities like CPU, memory, and I/O. The is forked from the with a few modifications to better reflect YugabyteDB's distributed nature. To ensure the recommended hardware requirements are met and the database is correctly configured before benchmarking, review the . Install sysbench using the following steps: ```sh $ cd $HOME $ git clone https://github.com/yugabyte/sysbench.git $ cd sysbench $ ./autogen.sh && ./configure --with-pgsql && make -j && sudo make install ``` This installs the sysbench utility in `/usr/local/bin`. Make sure you have the `ysqlsh` exported to the `PATH` variable. ```sh $ export PATH=$PATH:/path/to/ysqlsh ``` Start your YugabyteDB cluster by following the steps in . {{< tip title="Tip" >}} You will need the IP addresses of the nodes in the cluster for the next step. {{< /tip>}} Run the `run_sysbench.sh` shell script to load the data and run the various workloads: ```sh ./run_sysbench.sh --ip <ip> ``` This script runs all 8 workloads using 64 threads with the number of tables as 10 and the table size as 100k. If you want to run the benchmark with a different count of tables and tablesize, do the following: ```sh ./run_sysbench.sh --ip <ip> --numtables <number of tables> --tablesize <number of rows in each table> ``` You can choose to run the following workloads individually: oltp_insert oltppointselect oltpwriteonly oltpreadonly oltpreadwrite oltpupdateindex oltpupdatenon_index oltp_delete Before starting the workload, load the data as follows: ```sh $ sysbench <workload> \ --tables=10 \ --table-size=100000 \ --rangekeypartitioning=true \ --db-driver=pgsql \ --pgsql-host=127.0.0.1 \ --pgsql-port=5433 \ --pgsql-user=yugabyte \ --pgsql-db=yugabyte \ prepare ``` Run a workload as follows: ```sh $ sysbench <workload> \ --tables=10 \ --table-size=100000 \ --rangekeypartitioning=true \ --db-driver=pgsql \ --pgsql-host=127.0.0.1 \ --pgsql-port=5433 \ --pgsql-user=yugabyte \ --pgsql-db=yugabyte \ --threads=64 \ --time=120 \ --warmup-time=120 \ run ``` The following results are for a 3-node cluster with each node on a c5.4xlarge AWS instance (16 cores, 32 GB of RAM, and 2 EBS volumes), all in the same AZ with the client VM running in the same AZ. | Workload | Throughput (txns/sec) | Latency (ms) | | :-- | :-- | :-- | | OLTPREADONLY | 3276 | 39 | | OLTPREADWRITE | 487 | 265 | | OLTPWRITEONLY | 1818 | 70 | | OLTPPOINTSELECT | 95695 | 1.3 | | OLTP_INSERT | 6348 | 20.1 | | OLTPUPDATEINDEX | 4052 | 31 | | OLTPUPDATENON_INDEX | 11496 | 11 | | OLTP_DELETE | 67499 | 1.9 |
Project: KOTS (App Definition and Development / Application Definition & Image Build)
File: apko_melange_build.md

This doc describes a non-production-ready process for building a minimal `kots` image using `melange` and `apko`: is a tool for reproducibly building APK packages from source is a tool for reproducibly building container images from APK packages Building with `melange` and `apko` produces smaller, more reproducible images, which can be easier to operate and easier to keep free of vulnerabilities. First, build the package from source, using `melange`. To start, if there isn't already a signing key for the package, we need to generate one: ```sh melange keygen ``` We only need to build for x86_64, which is faster than building for arm64 since it doesn't require qemu. ```sh melange build melange.yaml --arch=x86_64 ``` Only building for your local platform makes builds faster, since it doesn't have to emulate with qemu. If you're on an arm64 machine (e.g., Apple Silicon), use `--arch=aarch64` here and below. Then, build the image from the newly built `kotsadm` package, and the other packages needed by the image, using `apko`: ```sh apko publish apko.yaml ttl.sh/kotsadm --arch=x86_64 ``` This will print the image to stdout, so you can run it: ```sh docker run $(apko publish ...) ``` The above steps are automated in as a presubmit check for PRs. The image this workflow produces is only meant for validation, and not meant for production use cases at this time. https://edu.chainguard.dev/open-source/melange/overview/ https://edu.chainguard.dev/open-source/apko/overview/
Project: ClickHouse (App Definition and Development / Database)
File: v21.3.20.1-lts.md

sidebar_position: 1 sidebar_label: 2022 Backported in : Quota limit was not reached, but the limit was exceeded. This PR fixes . (). Backported in : Fix null pointer dereference in low cardinality data when deserializing LowCardinality data in the Native format. (). Backported in : fix crash when used fuzzBits with multiply same FixedString, Close . (). Fix data race in ProtobufSchemas (). Fix possible Pipeline stuck in case of StrictResize processor. (). Fix arraySlice with null args. (). Fix queries with hasColumnInTable constant condition and non existing column (). Merge (). Merge (). Cherry pick to 21.3: Merge ().
Project: KubeVela (App Definition and Development / Application Definition & Image Build)
File: apply-terraform-config.eg.md

```yaml apiVersion: core.oam.dev/v1beta1 kind: Application metadata: name: apply-terraform-resource namespace: default spec: components: [] workflow: steps: name: provider type: apply-terraform-provider properties: type: alibaba name: my-alibaba-provider accessKey: <accessKey> secretKey: <secretKey> region: cn-hangzhou name: configuration type: apply-terraform-config properties: source: path: alibaba/cs/dedicated-kubernetes remote: https://github.com/FogDong/terraform-modules providerRef: name: my-alibaba-provider writeConnectionSecretToRef: name: my-terraform-secret namespace: vela-system variable: name: regular-check-ack newnatgateway: true vpc_name: "tf-k8s-vpc-regular-check" vpc_cidr: "10.0.0.0/8" vswitchnameprefix: "tf-k8s-vsw-regualr-check" vswitch_cidrs: [ "10.1.0.0/16", "10.2.0.0/16", "10.3.0.0/16" ] k8snameprefix: "tf-k8s-regular-check" k8s_version: 1.24.6-aliyun.1 k8spodcidr: "192.168.5.0/24" k8sservicecidr: "192.168.2.0/24" k8sworkernumber: 2 cpucorecount: 4 memory_size: 8 tags: created_by: "Terraform-of-KubeVela" created_from: "module-tf-alicloud-ecs-instance" ```
Project: RBAC Manager (Provisioning / Security & Compliance)
File: README.md

<div align="center" class="no-border"> <img src="/img/rbac-manager-logo.svg" height="200" alt="RBAC Manager" style="padding-bottom: 20px"> <br> <a href="https://join.slack.com/t/fairwindscommunity/shared_invite/zt-e3c6vj4l-3lIH6dvKqzWII5fSSFDi1g"> <img src="https://img.shields.io/static/v1?label=Slack&message=Join+our+Community&color=4a154b&logo=slack"> </a> <a href="https://github.com/FairwindsOps/rbac-manager/releases"> <img src="https://img.shields.io/github/v/release/FairwindsOps/rbac-manager"> </a> <a href="https://goreportcard.com/report/github.com/FairwindsOps/rbac-manager"> <img src="https://goreportcard.com/badge/github.com/FairwindsOps/rbac-manager"> </a> <a href="https://insights.fairwinds.com/gh/FairwindsOps/rbac-manager"> <img src="https://insights.fairwinds.com/v0/gh/FairwindsOps/polaris/badge.svg"> </a> </div> RBAC Manager is designed to simplify authorization in Kubernetes. This is an operator that supports declarative configuration for RBAC with new custom resources. Instead of managing role bindings or service accounts directly, you can specify a desired state and RBAC Manager will make the necessary changes to achieve that state. Check out the <!-- Begin boilerplate --> The goal of the Fairwinds Community is to exchange ideas, influence the open source roadmap, and network with fellow Kubernetes users. or to get involved! <a href="https://insights.fairwinds.com/auth/register/"> <img src="https://www.fairwinds.com/hubfs/DocBanners/FairwindsOSSUserGroup740x125v6.png" alt="Love Fairwinds Open Source? Automate Fairwinds Open Source for free with Fairwinds Insights. Click to learn more" /> </a> Enjoying rbac-manager? Check out some of our other projects: - Audit, enforce, and build policies for Kubernetes resources, including over 20 built-in checks for best practices - Right-size your Kubernetes Deployments by compare your memory and CPU settings against actual usage - Detect Kubernetes resources that have been deprecated or removed in future versions - Check to see if any of your Helm charts have updates available Or
Project: Devfile (App Definition and Development / Application Definition & Image Build)
File: terminology.md

The following terms may be used throughout design proposals for devfile registries The container image containing the devfile stacks and index.json used to bootstrap the OCI registry with devfile stacks. It also runs the webserver that functions as a proxy to the OCI registry, and hosts the index.json. The GitHub repository that hosts the devfile stacks for consumption within an OCI registry. For example, . The GitHub repository that hosts toolings for OCI devfile registries. This includes the , and the . It also contains the and Dockerfile for creating the Devfile index image. The process that takes the devfile registry repository, generates the index.json and builds it up into the devfile index image. The process that pushes the devfile stacks on the devfile index container to the OCI registry.
Project: Trivy (Provisioning / Security & Compliance)
File: principles.md

This document outlines the guiding principles and governance framework for the Trivy project. Trivy is a security scanner focused on static analysis and designed with simplicity and security at its core. All new proposals to the project must adhere to the following principles. Trivy operates without requiring container or VM image startups, eliminating the need for Docker or similar runtimes, except for scanning images stored within a container runtime. This approach enhances security and efficiency by minimizing dependencies. Operating as a single binary, Trivy is independent of external environments and avoids executing external OS commands or processes. If specific functionality, like Maven's, is needed, Trivy opts for internal reimplementations or processing outputs of the tool without direct execution of external tools. This approach obviously requires more effort but significantly reduces security risks associated with executing OS commands and dependency errors due to external environment versions. Simplifying the scanner's use by making it operational immediately upon binary download facilitates easier initiation of scans. Trivy must be ready to use immediately after installation. It's unacceptable for Trivy not to function without setting up a database or writing configuration files by default. Such setups should only be necessary for users requiring specific customizations. Security often isn't a top priority for many organizations and can be easily deferred. Trivy aims to lower the barrier to entry by simplifying the setup process, making it easier for users to start securing their projects. Trivy prioritizes the identification of security issues, excluding features unrelated to security, such as performance metrics or content listings of container images. It can, however, produce and output intermediate representations like SBOMs for comprehensive security assessments. Trivy serves as a tool with opinions on security, used to warn users about potential issues. Trivy is designed to detect unintended vulnerable states in projects, such as the use of vulnerable versions of dependencies or misconfigurations in Infrastructure as Code (IaC) that may unintentionally expose servers to the internet. The focus is on identifying developer mistakes or undesirable states, not on detecting intentional attacks, such as malicious images and malware. Aqua Security offers a premium version with several features not available in the open-source Trivy project. While detailed information can be found , it's beneficial to highlight specific functionalities frequently inquired about: As mentioned in , Trivy is a static analysis security scanner, making runtime security outside its scope. Runtime security needs are addressed by . As mentioned in , detection of intentional attacks, such as malware or malicious container images, is not covered by Trivy and is supported in . Trivy primarily operates via CLI for displaying results, with a richer UI available in .
Project: CyberArk Conjur (Provisioning / Key Management)
File: CHANGELOG.md

This Gem has been moved into Conjur. All Conjur Rack Changelog entries should appear in the main Changelog. Support Ruby 3. Bump `slosilo` to v3.0 with ruby 3. Remove pinned `bundler` version, use default system bundler. Bump `slosilo` to v2.2 in order to be FIPS compliant Bump `rack` to v2, `bundler` to v1.16 in gemspec Add Jenkinsfile to project Ignore headers such as Conjur-Privilege or Conjur-Audit if they're not supported by the API (instead of erroring out). Support for JWT Slosilo tokens. Initial support for Conjur 5. Add TRUSTED_PROXIES support resolve 'own' token to CONJUR_ACCOUNT env var add #optional paths to Conjur::Rack authenticator Add handling for `Conjur-Audit-Roles` and `Conjur-Audit-Resources` Change `globalsudo?` to `globalelevate?` Add `validatedglobalprivilege` helper function to get the global privilege, if any, which has been submitted with the request and verified by the Conjur server. Add handling for `X-Forwarded-For` and `X-Conjur-Privilege`
Project: YugabyteDB (App Definition and Development / Database)
File: helm-chart.md

title: Deploy single zone on EKS using Helm Chart linkTitle: Amazon Elastic Kubernetes Service (EKS) description: Use Helm Chart to deploy a single-zone Kubernetes cluster on Amazon Elastic Kubernetes Service (EKS). menu: v2.18: parent: deploy-kubernetes-sz name: Amazon EKS identifier: k8s-eks-1 weight: 622 type: docs <ul class="nav nav-tabs-alt nav-tabs-yb"> <li > <a href="../helm-chart/" class="nav-link active"> <i class="fa-regular fa-dharmachakra" aria-hidden="true"></i> Helm chart </a> </li> </ul> Amazon EKS runs multi-zone Kubernetes clusters by default and has no support for single-zone deployments. As described in , the managed control plane runs in multiple availability zones by default to protect cluster administration against zone failures. Similarly, the worker nodes are automatically placed in multiple availability zones of the chosen region to protect the cluster itself from zone failures. Refer to the instructions for getting started with YugabyteDB on Amazon EKS.
Project: Beam (App Definition and Development / Streaming & Messaging)
File: 25_io_csv.md

Prompt: Is CSV format supported in Apache Beam? Response: is a data format for storing and exchanging data. Apache Beam supports reading data from and writing data to CSV files using the `ReadFromText` and `WriteToText` transforms in the `TextIO` module. For more information, see the TextIO connector documentation: To get started with CSV and Apache Beam, refer to the . Here's an example of Apache Beam pipeline code for reading data from a CSV file: ```python class CsvOptions(PipelineOptions): @classmethod def addargparse_args(cls, parser): parser.add_argument( '--file_path', default="gs://your-bucket/your-file.csv", help='Csv file path' ) options = CsvOptions() with beam.Pipeline(options=options) as p: output = (p | "Read from Csv file" >> ReadFromCsv( path=options.file_path ) | "Log Data" >> Map(logging.info)) ```
Project: ShardingSphere (App Definition and Development / Database)
File: create-shadow-rule.en.md

+++ title = "CREATE SHADOW RULE" weight = 1 +++ The `CREATE SHADOW RULE` syntax is used to create a shadow rule. {{< tabs >}} {{% tab name="Grammar" %}} ```sql CreateShadowRule ::= 'CREATE' 'SHADOW' 'RULE' ifNotExists? shadowRuleDefinition (',' shadowRuleDefinition)* ifNotExists ::= 'IF' 'NOT' 'EXISTS' shadowRuleDefinition ::= ruleName '(' storageUnitMapping shadowTableRule (',' shadowTableRule)* ')' storageUnitMapping ::= 'SOURCE' '=' storageUnitName ',' 'SHADOW' '=' storageUnitName shadowTableRule ::= tableName '(' shadowAlgorithm ')' shadowAlgorithm ::= 'TYPE' '(' 'NAME' '=' algorithmType ',' propertiesDefinition ')' ruleName ::= identifier storageUnitName ::= identifier tableName ::= identifier algorithmName ::= identifier algorithmType ::= string propertiesDefinition ::= 'PROPERTIES' '(' key '=' value (',' key '=' value)* ')' key ::= string value ::= literal ``` {{% /tab %}} {{% tab name="Railroad diagram" %}} <iframe frameborder="0" name="diagram" id="diagram" width="100%" height="100%"></iframe> {{% /tab %}} {{< /tabs >}} Duplicate `ruleName` cannot be created; `storageUnitMapping` specifies the mapping relationship between the `source` database and the shadow library. You need to use the storage unit managed by RDL, please refer to ; `shadowAlgorithm` can act on multiple `shadowTableRule` at the same time; If `algorithmName` is not specified, it will be automatically generated according to `ruleName`, `tableName` and `algorithmType`; `algorithmType` currently supports `VALUEMATCH`, `REGEXMATCH` and `SQL_HINT`; `ifNotExists` caluse is used for avoid `Duplicate shadow rule` error. Create a shadow rule ```sql CREATE SHADOW RULE shadow_rule( SOURCE=demo_ds, SHADOW=demodsshadow, torder(TYPE(NAME="SQLHINT")), torderitem(TYPE(NAME="VALUEMATCH", PROPERTIES("operation"="insert","column"="userid", "value"='1'))) ); ``` Create a shadow rule with `ifNotExists` clause ```sql CREATE SHADOW RULE IF NOT EXISTS shadow_rule( SOURCE=demo_ds, SHADOW=demodsshadow, torder(TYPE(NAME="SQLHINT")), torderitem(TYPE(NAME="VALUEMATCH", PROPERTIES("operation"="insert","column"="userid", "value"='1'))) ); ``` `CREATE`, `SHADOW`, `RULE`, `SOURCE`, `SHADOW`, `TYPE`, `NAME`, `PROPERTIES`