cwielenb/CNCF-Dataset huggingface dataset:
Project: CockroachDB (App Definition and Development / Database)
File: 20220607_kvconnector_refactor.md

Feature Name: Multitenancy code movement Status: draft Start Date: 2022-05-25 Authors: knz ajs ben RFC PR: #82516 Cockroach Issue: Jira CRDB-16053, CRDB-16091 (non-public) Note: this RFC document is a placeholder that describes the high-level changes incurred by an actual RFC proposal that is kept internal to CRL employees. We would like all clusters including self-hosted and dedicated to run multitenant, albeit with just 2 tenants; this is one of the goals of the "arch unification project". To achieve this, we bring under BSL the capability to automatically define and run 2 tenants in-process. Bring the API definition (go interface) of KV Connector (`pkg/ccl/kvccl/kvtenantccl/connector.go`) under the BSL. However: Keep the current implementation of the API, which supports network connections, under CCL. Define a new implementation of the API suitable for in-process tenant servers, in the BSL tree. Update TestServer and the related tenant server creation code in unit tests (and cockroach demo) to use the in-memory KV Connector implementation. Move all the unit test code that is only dependent on the ability to start multitenant servers under the BSL directories where it belongs. This proposal makes it possible to use a BSL-only build of CockroachDB with multitenancy. The full RFC proposal is here: https://docs.google.com/document/d/1s0PVIKFywX3G9lZ3ds8Hi51cAIHph-pp2PCcKt7iwaw/edit# (internal only)
Project: Checkov (Provisioning / Security & Compliance)
File: gitlab_ci.md

layout: default title: gitlab_ci resource scans nav_order: 1 | | Id | Type | Entity | Policy | IaC | Resource Link | |-|-|--|--|-|--|--| | 0 | CKVGITLABCI1 | jobs | *.script | | 1 | CKVGITLABCI2 | jobs | *.rules | Avoid creating rules that generate double pipelines | gitlab_ci | | | 2 | CKVGITLABCI3 | jobs | *.image | | 3 | CKVGITLABCI3 | jobs | *.services |
Project: KICS (Provisioning / Security & Compliance)
File: 086031e1-9d4a-4249-acb3-5bfe4c363db2.md

title: Cloud Storage Anonymous or Publicly Accessible hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 086031e1-9d4a-4249-acb3-5bfe4c363db2 Query name: Cloud Storage Anonymous or Publicly Accessible Platform: Ansible Severity: <span style="color:#ff0000">Critical</span> Category: Access Control URL: Cloud Storage Buckets must not be anonymously or publicly accessible, which means the attribute 'entity' must not be 'allUsers' or 'allAuthenticatedUsers'<br> ```yaml title="Positive test num. 1 - yaml file" hl_lines="11 28 22" name: create a bucket1 google.cloud.gcpstoragebucket: name: ansible-storage-module1 project: test_project auth_kind: serviceaccount serviceaccountfile: "/tmp/auth.pem" state: present defaultobjectacl: bucket: bucketName1 entity: allUsers role: READER name: create a bucket2 google.cloud.gcpstoragebucket: name: ansible-storage-module2 project: test_project auth_kind: serviceaccount serviceaccountfile: "/tmp/auth.pem" state: present acl: bucket: bucketName2 entity: allAuthenticatedUsers defaultobjectacl: bucket: bucketName2 entity: allUsers role: READER name: create a bucket3 google.cloud.gcpstoragebucket: name: ansible-storage-module3 project: test_project auth_kind: serviceaccount serviceaccountfile: "/tmp/auth.pem" state: present ``` ```yaml title="Negative test num. 1 - yaml file" name: create a bucket google.cloud.gcpstoragebucket: name: ansible-storage-module project: test_project auth_kind: serviceaccount serviceaccountfile: /tmp/auth.pem state: present acl: bucket: bucketName entity: group-example@googlegroups.com ```
Project: Numaflow (App Definition and Development / Streaming & Messaging)
File: edges-buffers-buckets.md

This document describes the concepts of `Edge`, `Buffer` and `Bucket` in a pipeline. `Edge` is the connection between the vertices, specifically, `edge` is defined in the pipeline spec under `.spec.edges`. No matter if the `to` vertex is a Map, or a Reduce with multiple partitions, it is considered as one edge. In the following pipeline, there are 3 edges defined (`in` - `aoti`, `aoti` - `compute-sum`, `compute-sum` - `out`). ```yaml apiVersion: numaflow.numaproj.io/v1alpha1 kind: Pipeline metadata: name: even-odd-sum spec: vertices: name: in source: http: {} name: atoi scale: min: 1 udf: container: image: quay.io/numaio/numaflow-go/map-even-odd:v0.5.0 name: compute-sum partitions: 2 udf: container: image: quay.io/numaio/numaflow-go/reduce-sum:v0.5.0 groupBy: window: fixed: length: 60s keyed: true name: out scale: min: 1 sink: log: {} edges: from: in to: atoi from: atoi to: compute-sum from: compute-sum to: out ``` Each `edge` could have a name for internal usage, the naming convention is `{pipeline-name}-{from-vertex-name}-{to-vertex-name}`. `Buffer` is `InterStepBuffer`. Each buffer has an owner, which is the vertex who reads from it. Each `udf` and `sink` vertex in a pipeline owns a group of partitioned buffers. Each buffer has a name with the naming convention `{pipeline-name}-{vertex-name}-{index}`, where the `index` is the partition index, starting from 0. This naming convention applies to the buffers of both map and reduce udf vertices. When multiple vertices connecting to the same vertex, if the `to` vertex is a Map, the data from all the from vertices will be forwarded to the group of partitoned buffers round-robinly. If the `to` vertex is a Reduce, the data from all the from vertices will be forwarded to the group of partitioned buffers based on the partitioning key. A Source vertex does not have any owned buffers. But a pipeline may have multiple Source vertices, followed by one vertex. Same as above, if the following vertex is a map, the data from all the Source vertices will be forwarded to the group of partitioned buffers round-robinly. If it is a reduce, the data from all the Source vertices will be forwarded to the group of partitioned buffers based on the partitioning key. `Bucket` is a K/V store (or a pair of stores) used for watermark propagation. There are 3 types of buckets in a pipeline: `Edge Bucket`: Each edge has a bucket, used for edge watermark propagation, no matter if the vertex that the edge leads to is a Map or a Reduce. The naming convention of an edge bucket is `{pipeline-name}-{from-vertex-name}-{to-vertex-name}`. `Source Bucket`: Each Source vertex has a source bucket, used for source watermark propagation. The naming convention of a source bucket is `{pipeline-name}-{vertex-name}-SOURCE`. `Sink Bucket`: Sitting on the right side of a Sink vertex, used for sink watermark. The naming convention of a sink bucket is `{pipeline-name}-{vertex-name}-SINK`. Map Reduce
Project: YDB (App Definition and Development / Database)
File: staticmap.md

Transforms a structure or tuple by applying a lambda function to each item. Arguments: Structure or tuple. Lambda for processing items. Result: a structure or tuple with the same number and naming of items as in the first argument, and with item data types determined by lambda results. Examples: ```yql SELECT * FROM ( SELECT StaticMap(TableRow(), ($item) -> { return CAST($item AS String); }) FROM my_table ) FLATTEN COLUMNS; -- converting all columns to rows ```
Project: KICS (Provisioning / Security & Compliance)
File: 1a4bc881-9f69-4d44-8c9a-d37d08f54c50.md

title: S3 Bucket Allows Public Policy hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 1a4bc881-9f69-4d44-8c9a-d37d08f54c50 Query name: S3 Bucket Allows Public Policy Platform: Terraform Severity: <span style="color:#bb2124">High</span> Category: Access Control URL: S3 bucket allows public policy<br> ```tf title="Positive test num. 1 - tf file" hl_lines="9 18" resource "awss3bucket" "positive1" { bucket = "example" } resource "awss3bucketpublicaccess_block" "positive2" { bucket = awss3bucket.example.id blockpublicacls = true blockpublicpolicy = false ignorepublicacls = false } // comment // comment // comment // comment // comment resource "awss3bucketpublicaccess_block" "positive3" { bucket = awss3bucket.example.id blockpublicacls = true ignorepublicacls = false } ``` ```tf title="Positive test num. 2 - tf file" hl_lines="9" module "s3_bucket" { source = "terraform-aws-modules/s3-bucket/aws" version = "3.7.0" bucket = "my-s3-bucket" acl = "private" restrictpublicbuckets = true blockpublicacls = true blockpublicpolicy = false versioning = { enabled = true } policy = <<POLICY { "Version": "2012-10-17", "Id": "MYBUCKETPOLICY", "Statement": [ { "Sid": "IPAllow", "Effect": "Deny", "Action": "s3:*", "Resource": "arn:aws:s3:::mytftest_bucket/*", "Condition": { "IpAddress": {"aws:SourceIp": "8.8.8.8/32"} } } ] } POLICY } ``` ```tf title="Positive test num. 3 - tf file" hl_lines="1" module "s3_bucket" { source = "terraform-aws-modules/s3-bucket/aws" version = "3.7.0" bucket = "my-s3-bucket" acl = "private" restrictpublicbuckets = true blockpublicacls = true versioning = { enabled = true } policy = <<POLICY { "Version": "2012-10-17", "Id": "MYBUCKETPOLICY", "Statement": [ { "Sid": "IPAllow", "Effect": "Deny", "Action": "s3:*", "Resource": "arn:aws:s3:::mytftest_bucket/*", "Condition": { "IpAddress": {"aws:SourceIp": "8.8.8.8/32"} } } ] } POLICY } ``` ```tf title="Negative test num. 1 - tf file" resource "awss3bucket" "negative1" { bucket = "example" } resource "awss3bucketpublicaccess_block" "negative2" { bucket = awss3bucket.example.id blockpublicacls = true blockpublicpolicy = true ignorepublicacls = false } ``` ```tf title="Negative test num. 2 - tf file" module "s3_bucket" { source = "terraform-aws-modules/s3-bucket/aws" version = "3.7.0" bucket = "my-s3-bucket" acl = "private" restrictpublicbuckets = true blockpublicacls = true blockpublicpolicy = true versioning = { enabled = true } policy = <<POLICY { "Version": "2012-10-17", "Id": "MYBUCKETPOLICY", "Statement": [ { "Sid": "IPAllow", "Effect": "Deny", "Action": "s3:*", "Resource": "arn:aws:s3:::mytftest_bucket/*", "Condition": { "IpAddress": {"aws:SourceIp": "8.8.8.8/32"} } } ] } POLICY } ```
Project: YugabyteDB (App Definition and Development / Database)
File: handling-zone-failures.md

title: Handling zone failures headerTitle: Handling zone failures linkTitle: Zone failures description: YugabyteDB can handle in-region failure of zones headcontent: YugabyteDB can handle in-region failure of zones menu: stable: identifier: handling-zone-failures parent: fault-tolerance weight: 30 type: docs YugabyteDB is resilient to a single-domain failure in a deployment with a replication factor (RF) of 3. To survive zone failures, you deploy across multiple zones. Let's see how YugabyteDB survives a zone failure. Consider a setup where YugabyteDB is deployed across three zones in a single region (us-east-1). Say it is an RF 3 cluster with leaders and followers distributed across the 3 zones with 3 tablets (A, B, and C). Suppose one of your zones, us-east-1b, fails. In this case, the connections established by your application to the nodes in us-east-1b start timing out (typical timeout is 15s). If new connections are attempted, they will immediately fail, and some tablets will be leaderless. In the following illustration, tablet B has lost its leader. All the nodes in the cluster constantly ping each other for a liveness check. When a node goes offline, it is identified within 3s and a leader election is triggered. This results in the promotion of one of the followers of the offline tablets to leaders. Leader election is very fast and there is no data loss. In the illustration, you can see that one of the followers of the tablet B leader in zone-a has been elected as the new leader. Once new leaders have been elected, there are no leader-less tablets and the cluster becomes fully functional. There is no data loss as the follower that was elected as the leader has the latest data (guaranteed by RAFT replication). The recovery time is about 3s. But note that the cluster is now under-replicated because some of the followers are currently offline. From the point a zone outage occurs, it takes about 3s for all requests to succeed as it takes about 3s for the cluster to realize that nodes are offline and complete a leader election. Because of default TCP timeouts, connections already established by applications will take about 15s to fail and on reconnect, they will reconnect to other active nodes. At this point, the cluster is under-replicated because of the loss of followers. If the failed nodes don't come back online within 15 minutes, the followers are considered failed and new followers will be created to guarantee the replication factor (3). This in essence happens only when the failure of nodes is considered to be a long-term failure.
Project: KubeBlocks by ApeCloud (App Definition and Development / Database)
File: kbcli_cluster_list-ops.md

title: kbcli cluster list-ops List all opsRequests. ``` kbcli cluster list-ops [flags] ``` ``` kbcli cluster list-ops kbcli cluster list-ops mycluster ``` ``` -A, --all-namespaces If present, list the requested object(s) across all namespaces. Namespace in current context is ignored even if specified with --namespace. -h, --help help for list-ops --name string The OpsRequest name to get the details. -o, --output format prints the output in the specified format. Allowed values: table, json, yaml, wide (default table) -l, --selector string Selector (label query) to filter on, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2). Matching objects must satisfy all of the specified label constraints. --show-labels When printing, show all labels as the last column (default hide labels column) --status strings Options include all, pending, creating, running, canceling, failed. by default, outputs the pending/creating/running/canceling/failed OpsRequest. (default [pending,creating,running,canceling,failed]) --type strings The OpsRequest type ``` ``` --as string Username to impersonate for the operation. User could be a regular user or a service account in a namespace. --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups. --as-uid string UID to impersonate for the operation. --cache-dir string Default cache directory (default "$HOME/.kube/cache") --certificate-authority string Path to a cert file for the certificate authority --client-certificate string Path to a client certificate file for TLS --client-key string Path to a client key file for TLS --cluster string The name of the kubeconfig cluster to use --context string The name of the kubeconfig context to use --disable-compression If true, opt-out of response compression for all requests to the server --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure --kubeconfig string Path to the kubeconfig file to use for CLI requests. --match-server-version Require server version to match client version -n, --namespace string If present, the namespace scope for this CLI request --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0") -s, --server string The address and port of the Kubernetes API server --tls-server-name string Server name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used --token string Bearer token for authentication to the API server --user string The name of the kubeconfig user to use ``` - Cluster command.
Project: YugabyteDB (App Definition and Development / Database)
File: analyze-schema.md

title: analyze schema reference headcontent: yb-voyager analyze schema linkTitle: analyze schema description: YugabyteDB Voyager analyze schema reference menu: preview_yugabyte-voyager: identifier: voyager-analyze-schema parent: schema-migration weight: 20 type: docs dumped in the export schema step. ```sh yb-voyager analyze-schema [ <arguments> ... ] ``` The valid arguments for analyze schema are described in the following table: | Argument | Description/valid options | | :- | : | | -e, --export-dir | Path to the export directory. This directory is a workspace used to store exported schema DDL files, export data files, migration state, and a log file.| | -h, --help | Command line help. | | --output-format | Format in which the report file is generated. One of `html`, `txt`, `json`, or `xml`. | | &#8209;&#8209;send&#8209;diagnostics | Enable or disable sending information to Yugabyte. <br>Default: true<br> Accepted parameters: true, false, yes, no, 0, 1 | | -y, --yes | Answer yes to all prompts during the export schema operation. <br>Default: false<br> Accepted parameters: true, false, yes, no, 0, 1 | ```sh yb-voyager analyze-schema --export-dir /dir/export-dir --output-format txt ```
Project: KubeVela (App Definition and Development / Application Definition & Image Build)
File: k8s-objects.eg.md

```yaml apiVersion: core.oam.dev/v1beta1 kind: Application metadata: name: app-raw spec: components: name: myjob type: k8s-objects properties: objects: apiVersion: batch/v1 kind: Job metadata: name: pi spec: template: spec: containers: name: pi image: perl command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"] restartPolicy: Never backoffLimit: 4 ```
Project: ShardingSphere (App Definition and Development / Database)
File: show-migration-check-algorithm.en.md

+++ title = "SHOW MIGRATION CHECK ALGORITHM" weight = 9 +++ The `SHOW MIGRATION RULE` syntax is used to query migration check algorithm. {{< tabs >}} {{% tab name="Grammar" %}} ```sql ShowMigrationCheckAlgorithm ::= 'SHOW' 'MIGRATION' 'CHECK' 'ALGORITHMS' ``` {{% /tab %}} {{% tab name="Railroad diagram" %}} <iframe frameborder="0" name="diagram" id="diagram" width="100%" height="100%"></iframe> {{% /tab %}} {{< /tabs >}} | Column | Description | |--|--| | type | migration check algorithm type | | supporteddatabasetypes | supported database type | | description | Description of migration check algorithm | Query migration check algorithm ```sql SHOW MIGRATION CHECK ALGORITHMS; ``` ```sql mysql> SHOW MIGRATION CHECK ALGORITHMS; +-+--+-+ | type | supporteddatabasetypes | description | +-+--+-+ | CRC32_MATCH | MySQL | Match CRC32 of records. | | DATA_MATCH | SQL92,MySQL,MariaDB,PostgreSQL,openGauss,Oracle,SQLServer,H2 | Match raw data of records. | +-+--+-+ 2 rows in set (0.03 sec) ``` `SHOW`, `MIGRATION`, `CHECK`, `ALGORITHMS`
Project: YugabyteDB (App Definition and Development / Database)
File: concatenation-operator.md

title: "|| (concatenation operator) [JSON]" headerTitle: "|| (concatenation)" linkTitle: "|| (concatenation)" description: Concatenate two jsonb values using the JSON concatenation operator ("||"). menu: v2.18: identifier: concatenation-operator parent: json-functions-operators weight: 14 type: docs Purpose: Concatenate two `jsonb` values. The rule for deriving the output value depends upon the JSON data types of the operands. Signature: ``` input values: jsonb || jsonb return value: jsonb ``` Notes: There is no `json` overload. If both sides of the operator are primitive JSON values, then the result is an array of these values: ```plpgsql do $body$ declare j_left constant jsonb := '17'; j_right constant jsonb := '"x"'; j_expected constant jsonb := '[17, "x"]'; begin assert jleft || jright = j_expected, 'unexpected'; end; $body$; ``` If one side is a primitive JSON value and the other is an array , then the result is an array: ```plpgsql do $body$ declare j_left constant jsonb := '17'; j_right constant jsonb := '["x", true]'; j_expected constant jsonb := '[17, "x", true]'; begin assert jleft || jright = j_expected, 'unexpected'; end; $body$; ``` If each side is an object, and no key-value pair in the RHS object has the same key as any key-value pair in the LHS object then the result is an object with all of the key-value pairs present: ```plpgsql do $body$ declare j_left constant jsonb := '{"a": 1, "b": 2}'; j_right constant jsonb := '{"p":17, "q": 19}'; j_expected constant jsonb := '{"a": 1, "b": 2, "p": 17, "q": 19}'; begin assert jleft || jright = j_expected, 'unexpected'; end; $body$; ``` If the key of any key-value pair in the RHS object collides with a key of a key-value pair in the LHS object, then the key-value pair from the RHS object wins, just as when the keys of such pairs collide in a single object: ```plpgsql do $body$ declare j_left constant jsonb := '{"a": 1, "b": 2}'; j_right constant jsonb := '{"p":17, "a": 19}'; j_expected constant jsonb := '{"a": 19, "b": 2, "p": 17}'; begin assert jleft || jright = j_expected, 'unexpected'; end; $body$; ``` If one side is an object and the other is an array, then the object is absorbed as a value in the array: ```plpgsql do $body$ declare j_left constant jsonb := '{"a": 1, "b": 2}'; j_right constant jsonb := '[false, 42, null]'; j_expected constant jsonb := '[{"a": 1, "b": 2}, false, 42, null]'; begin assert jleft || jright = j_expected, 'unexpected'; end; $body$; ```
Project: StarRocks (App Definition and Development / Database)
File: pipe_files.md

displayed_sidebar: "English" `pipe_files` provides the status of the data files to be loaded via a specified pipe. This view is supported from StarRocks v3.2 onwards. The following fields are provided in `pipe_files`: | Field | Description | | - | | | DATABASE_NAME | The name of the database in which the pipe is stored. | | PIPE_ID | The unique ID of the pipe. | | PIPE_NAME | The name of the pipe. | | FILE_NAME | The name of the data file. | | FILE_VERSION | The digest of the data file. | | FILE_SIZE | The size of the data file. Unit: bytes. | | LAST_MODIFIED | The last time when the data file was modified. Format: `yyyy-MM-dd HH:mm:ss`. Example: `2023-07-24 14:58:58`. | | LOAD_STATE | The load status of the data file. Valid values: `UNLOADED`, `LOADING`, `FINISHED`, and `ERROR`. | | STAGED_TIME | The date and time when the data file was first recorded by the pipe. Format: `yyyy-MM-dd HH:mm:ss`. Example: `2023-07-24 14:58:58`. | | STARTLOADTIME | The date and time when the loading of the data file began. Format: `yyyy-MM-dd HH:mm:ss`. Example: `2023-07-24 14:58:58`. | | FINISHLOADTIME | The date and time when the loading of the data file finished. Format: `yyyy-MM-dd HH:mm:ss`. Example: `2023-07-24 14:58:58`. | | ERROR_MSG | The details about the load error for the data file. |
Project: ArangoDB (App Definition and Development / Database)
File: is_basic_result.md

+++ title = "`isbasicresult<T>`" description = "An integral constant type true for `basic_result<T, E, NoValuePolicy>` types." +++ An integral constant type true for {{% api "basicresult<T, E, NoValuePolicy>" %}} types. This does not match anything not exactly a `basicresult`. If you want to match types like `basicresult` but not equal to it, consider {{% api "basicresult<T>" %}} or {{% api "valueorerror<T>" %}}. Overridable: Not overridable. Namespace: `BOOSTOUTCOMEV2_NAMESPACE` Header: `<boost/outcome/basic_result.hpp>` Variable alias: `isbasicresult_v<T>`