cwielenb/CNCF-Dataset huggingface dataset:
Project: Beam (App Definition and Development / Streaming & Messaging)
File: spark-runner.md

title: "Spark Runner Roadmap" <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> Support for Spark 3 in Beam's Spark runner is ongoing. For info on the various tasks please refer to the JIRA ticket. JIRA: Contact:
Project: openGemini (App Definition and Development / Database)
File: ROADMAP.md

v0.1.0 (released) [x] standalone and cluster version,local storage. [x] Supports certificate verification and user authentication. [x] Supports data compression. [x] Supports InfluxDB v1 APIs and Prometheus Remote Read/Write APIs. [x] Provided monitoring tool ts-monitor. v0.2.0 (2022.10) [x] Supports KubeEdge. [x] Real-time analysis framework. v1.0.0 (2023.02) [x] Full join [x] Stream computing [x] mult-level down-sampling [x] Time series data prediction. [x] Time series data anomaly detection. v1.1.0 (2023.09) [x] Full-text indexing [x] High series cardinality engine (kernel) [x] Support windows platform [x] Support data subscription [x] Support query manager [x] Support continuous query [x] Support show configs command v1.2.0 (2024.02) [x] Query performance optimization for IoT scenarios [x] LSM compaction, unorder data merge optimization [x] Enhangce log storage v1.2.1 (2024.04) [ ] Bug fix v1.2.2(2024.06) [ ] Bug fix v.1.3.0 (2024.12) [ ] Data replication [ ] Table level TTL [ ] OBS store [ ] Aggregation optimization v0.1.0 (2024.01) [x] Basic API for reading and writing (Line Protocol) v0.2.0 (2024.02) [ ] Support Arrow flight
Project: YugabyteDB (App Definition and Development / Database)
File: ddl_drop_keyspace.md

title: DROP KEYSPACE statement [YCQL] headerTitle: DROP KEYSPACE linkTitle: DROP KEYSPACE description: Use the DROP KEYSPACE statement to remove a keyspace from the system. menu: v2.18: parent: api-cassandra weight: 1260 type: docs Use the `DROP KEYSPACE` statement to remove a keyspace from the system. <svg class="rrdiagram" version="1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg" width="477" height="65" viewbox="0 0 477 65"><path class="connector" d="M0 22h5m53 0h30m82 0h20m-117 0q5 0 5 5v20q0 5 5 5h5m71 0h16q5 0 5-5v-20q0-5 5-5m5 0h30m32 0h10m64 0h20m-141 0q5 0 5 5v8q0 5 5 5h116q5 0 5-5v-8q0-5 5-5m5 0h10m116 0h5"/><rect class="literal" x="5" y="5" width="53" height="25" rx="7"/><text class="text" x="15" y="22">DROP</text><rect class="literal" x="88" y="5" width="82" height="25" rx="7"/><text class="text" x="98" y="22">KEYSPACE</text><rect class="literal" x="88" y="35" width="71" height="25" rx="7"/><text class="text" x="98" y="52">SCHEMA</text><rect class="literal" x="220" y="5" width="32" height="25" rx="7"/><text class="text" x="230" y="22">IF</text><rect class="literal" x="262" y="5" width="64" height="25" rx="7"/><text class="text" x="272" y="22">EXISTS</text><a xlink:href="../grammardiagrams#keyspace-name"><rect class="rule" x="356" y="5" width="116" height="25"/><text class="text" x="366" y="22">keyspacename</text></a></svg> ```ebnf dropkeyspace ::= DROP { KEYSPACE | SCHEMA } [ IF EXISTS ] keyspacename; ``` Where `keyspace_name` is an identifier. An error is raised if the specified `keyspace_name` does not exist unless `IF EXISTS` option is present. An error is raised if the specified keyspace is non-empty (contains tables or types). You can do this as shown below. ```sql ycqlsh> CREATE KEYSPACE example; ``` ```sql ycqlsh> DROP KEYSPACE example; ``` ```sql ycqlsh> DROP KEYSPACE IF EXISTS example; ```
Project: Beam (App Definition and Development / Streaming & Messaging)
File: enrichment-vertexai.md

title: "Enrichment with Vertex AI Feature Store" <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> {{< localstorage language language-py >}} <table> <tr> <td> <a> {{< button-pydoc path="apachebeam.transforms.enrichmenthandlers.vertexaifeature_store" class="VertexAIFeatureStoreEnrichmentHandler" >}} </a> </td> </tr> </table> In Apache Beam 2.55.0 and later versions, the enrichment transform includes a built-in enrichment handler for . The following example demonstrates how to create a pipeline that use the enrichment transform with the handler and the handler. The precomputed feature values stored in Vertex AI Feature Store uses the following format: {{< table >}} | user_id | age | gender | state | country | |:--:|:-:|:-:|:--:|:-:| | 21422 | 12 | 0 | 0 | 0 | | 2963 | 12 | 1 | 1 | 1 | | 20592 | 12 | 1 | 2 | 2 | | 76538 | 12 | 1 | 3 | 0 | {{< /table >}} {{< highlight language="py" >}} {{< codesample "sdks/python/apachebeam/examples/snippets/transforms/elementwise/enrichment.py" enrichmentwithvertex_ai >}} {{</ highlight >}} {{< paragraph class="notebook-skip" >}} Output: {{< /paragraph >}} {{< highlight class="notebook-skip" >}} {{< codesample "sdks/python/apachebeam/examples/snippets/transforms/elementwise/enrichmenttest.py" enrichmentwithvertexai >}} {{< /highlight >}} The precomputed feature values stored in Vertex AI Feature Store (Legacy) use the following format: {{< table >}} | entity_id | title | genres | |:-|:-|:--| | movie_01 | The Shawshank Redemption | Drama | | movie_02 | The Shining | Horror | | movie_04 | The Dark Knight | Action | {{< /table >}} {{< highlight language="py" >}} {{< codesample "sdks/python/apachebeam/examples/snippets/transforms/elementwise/enrichment.py" enrichmentwithvertexailegacy >}} {{</ highlight >}} {{< paragraph class="notebook-skip" >}} Output: {{< /paragraph >}} {{< highlight class="notebook-skip" >}} {{< codesample "sdks/python/apachebeam/examples/snippets/transforms/elementwise/enrichmenttest.py" enrichmentwithvertexai_legacy >}} {{< /highlight >}} Not applicable. {{< button-pydoc path="apachebeam.transforms.enrichmenthandlers.vertexaifeature_store" class="VertexAIFeatureStoreEnrichmentHandler" >}}
Project: Trivy-Operator (Provisioning / Security & Compliance)
File: policy-reporter.md

adds observability and monitoring possibilities to cluster security based on the , developed by the Kubernetes Policy Working Group. Key features are: New results can be send to like Grafana Loki, Slack, Discord or MS Teams A webbased dashboard to get a graphical overview of all results with optional filters E-Mail Reports on a regular basis about the current status of your cluster security Provide metrics to observe violations in well known monitoring solutions like Grafana Can also be integrated with other tools such as Tracee, Kyverno or Falco With it is possible to create generalized (Cluster)PolicyReport CRD from the various Trivy Operator CRDs, which allows integration with Policy Reporter.
Project: Apache Hadoop (App Definition and Development / Database)
File: RELEASENOTES.2.6.6.md

<! --> These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements. | Blocker | Upgrade Tomcat to 6.0.48* Tomcat 6.0.46 starts to filter weak ciphers. Some old SSL clients may be affected. It is recommended to upgrade the SSL client. Run the SSL client against https://www.howsmyssl.com/a/check to find out its TLS version and cipher suites.
Project: Podman (App Definition and Development / Application Definition & Image Build)
File: podman-network-prune.1.md

% podman-network-prune 1 podman\-network\-prune - Remove all unused networks podman network prune [options] Remove all unused networks. An unused network is defined by a network which has no containers connected or configured to connect to it. It does not remove the so-called default network which goes by the name of podman. Provide filter values. The filters argument format is of `key=value`. If there is more than one filter, then pass multiple OPTIONS: --filter foo=bar --filter bif=baz. Supported filters: | Filter | Description | |::|-| | label | Only remove networks, with (or without, in the case of label!=[...] is used) the specified labels. | | until | Only remove networks created before given timestamp. | The `label` filter accepts two formats. One is the `label`=key or `label`=key=value, which removes networks with the specified labels. The other format is the `label!`=key or `label!`=key=value, which removes networks without the specified labels. The `until` filter can be Unix timestamps, date formatted timestamps, or Go duration strings (e.g. 10m, 1h30m) computed relative to the machines time. Do not prompt for confirmation Prune networks: ``` podman network prune ``` Prune all networks created not created in the last two hours: ``` podman network prune --filter until=2h ``` , , February 2021, Originally compiled by Brent Baude <bbaude@redhat.com>
Project: Kui (App Definition and Development / Application Definition & Image Build)
File: advanced.md

In your markdown source, add this to the frontmatter, at the top. If you already have frontmatter, add only the `title:` line. ```yaml title: My title ``` In your markdown source, use section breaks `` to mark the boundaries between splits. Then, define a `layout` structure in your frontmatter, which maps section indices (starting from 1) to the desired split location. For example: ```yaml layout: 1: left 2: bottom ```
Project: Tremor (App Definition and Development / Streaming & Messaging)
File: expr.md

The `Expr` rule aliases the `SimpleExpr` rule. The alias allows higher levels of the DSL such as the rules in the deployment or query language to avoid some of the internal complexity in the scripting language. Within the scripting DSLs grammar the different forms and variations of expression are significant. However, in the higher level we limit exposure to a subset of these forms. This is done for convenience, and for consistency of usage, and ease of learning the language.
Project: Stolon (App Definition and Development / Database)
File: custom_pg_hba_entries.md

Stolon manages the pg_hba.conf file entries. The first rules are generated by stolon to permit local keeper connections and remote replication connections since these are needed to ensure the correct operation of the cluster. Users can specify custom pghba.conf entries setting the `pgHBA` option. It must be a list of string containing additional pghba.conf entries. They will be added to the pg_hba.conf generated by stolon. Since clients connection will pass through the stolon-proxy the host part of the entries should match at least the stolon-proxies source addresses. For the same reason it's not possible to directly filter by client. If you have clients that requires different accesses you should use different set of stolon proxies for every kind of access. NOTE: these lines aren't validated so if some of them are wrong postgres will refuse to start or, on reload, will log a warning and ignore the updated pg_hba.conf file. Stolon will just check that the string doesn't contain newlines characters. By default, if no custom pg_hba entries are defined (clusterpsec pgHBA option is null, not an empty list), to keep backward compatibility, stolon will add two rules to permit tcp (both ipv4 and ipv6) connections from every host to all dbs and usernames with md5 password authentication: ``` host all all 0.0.0.0/0 md5 host all all ::0/0 md5 ```
Project: YugabyteDB (App Definition and Development / Database)
File: key-value-workload-ycql.md

title: Benchmark YCQL performance with key-value workloads headerTitle: Key-value workload linkTitle: Key-value workload description: Benchmark YCQL performance with key-value workloads. headcontent: Benchmark YCQL performance with key-value workloads menu: v2.18: identifier: key-value-workload-1-ycql parent: benchmark weight: 6 type: docs <ul class="nav nav-tabs-alt nav-tabs-yb"> <li > <a href="../key-value-workload-ysql/" class="nav-link"> <i class="icon-postgres" aria-hidden="true"></i> YSQL </a> </li> <li > <a href="../key-value-workload-ycql/" class="nav-link active"> <i class="icon-cassandra" aria-hidden="true"></i> YCQL </a> </li> </ul> Use this benchmark to test the performance of YugabyteDB using a key-value workload. For this benchmark, you will set up a three-node YugabyteDB cluster with a replication factor of `3`. Amazon Web Services (AWS) Instance type: i3.4xlarge Storage: 2 x 1.9 TB NVMe SSD (comes preconfigured with the instance) Google Cloud Platform (GCP) Instance type: n1-standard-16 Storage: 2 x 375 GB SSD on-premises data center Instance: 16 CPU cores Storage: 1 x 200 GB SSD (minimum) RAM size: 30 GB (minimum) We will use the to perform this benchmark. Download the YugabyteDB workload generator JAR file (`yb-sample-apps.jar`) as follows: {{% yb-sample-apps-path %}} To run the workload generator tool, you must have: Java runtime or JDK installed. Set the environment variable $ENDPOINTS to the IP addresses (including hosts and ports) for the nodes of the cluster. ```output ENDPOINTS="X.X.X.X:9042,X.X.X.X:9042,X.X.X.X:9042" ``` Run the key-value workload with higher number of write threads (representing write-heavy workload). Load 1B keys of 256 bytes each across 256 writer threads ```sh $ java -jar ./yb-sample-apps.jar \ --workload CassandraKeyValue \ --nodes $ENDPOINTS \ --nouuid \ --value_size 256 \ --numthreadsread 0 \ --numthreadswrite 256 \ --numuniquekeys 1000000000 ``` Name | Observation --| Write Ops/sec | ~90k Write Latency | ~2.5-3.0 ms/op CPU (User + Sys) | 60% Run the key-value workload with higher number of read threads (representing read-heavy workload). Load 1M keys of 256 bytes and access them with 256 reader threads. ```sh $ java -jar ./yb-sample-apps.jar \ --workload CassandraKeyValue \ --nodes $ENDPOINTS \ --nouuid \ --value_size 256 \ --numthreadsread 256 \ --numthreadswrite 0 \ --numuniquekeys 1000000 ``` | Name | Observation | | : | :- | | (Read) Ops/sec | ~150k | | (Read) Latency | ~1.66 ms/op | | CPU (User + Sys) | 60% | Run the key-value workload in batch mode and higher number of write threads (representing batched, write-heavy workload). Load 1B keys of 256 bytes each across 64 writer threads in batches of 25 each. ```sh $ java -jar ./yb-sample-apps.jar \ --workload CassandraBatchKeyValue \ --nodes $ENDPOINTS \ --nouuid \ --batch_size 25 \ --value_size 256 \ --numthreadsread 0 \ --numthreadswrite 64 \ --numuniquekeys 1000000000 ``` | Name | Observation | | : | :- | | (Batch Write) Ops/sec | ~140k | | (Batch Write) Latency | ~9.0 ms/op | | CPU (User + Sys) | 80% |
Project: KICS (Provisioning / Security & Compliance)
File: b7d0181d-0a9b-4611-9d1c-1ad4f0b620ff.md

title: Etcd Peer Client Certificate Authentication Set To False hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: b7d0181d-0a9b-4611-9d1c-1ad4f0b620ff Query name: Etcd Peer Client Certificate Authentication Set To False Platform: Kubernetes Severity: <span style="color:#ff7213">Medium</span> Category: Secret Management URL: When using etcd commands, the '--peer-client-cert-auth' flag should be set to true<br> ```yaml title="Positive test num. 1 - yaml file" hl_lines="21" apiVersion: apps/v1 kind: Deployment metadata: name: app-etcd-deployment spec: selector: matchLabels: app: app replicas: 1 template: metadata: labels: app: app version: v1 spec: serviceAccountName: database containers: name: database image: gcr.io/google_containers/etcd:v3.2.18 imagePullPolicy: IfNotPresent command: ["etcd"] args: ["--peer-client-cert-auth=false"] nodeSelector: kubernetes.io/hostname: worker02 restartPolicy: OnFailure ``` ```yaml title="Positive test num. 2 - yaml file" hl_lines="21" apiVersion: apps/v1 kind: Deployment metadata: name: app-etcd-deployment spec: selector: matchLabels: app: app replicas: 1 template: metadata: labels: app: app version: v1 spec: serviceAccountName: database containers: name: database image: gcr.io/google_containers/etcd:v3.2.18 imagePullPolicy: IfNotPresent command: ["etcd"] args: [] nodeSelector: kubernetes.io/hostname: worker02 restartPolicy: OnFailure ``` ```yaml title="Negative test num. 1 - yaml file" apiVersion: apps/v1 kind: Deployment metadata: name: app-etcd-deployment spec: selector: matchLabels: app: app replicas: 1 template: metadata: labels: app: app version: v1 spec: serviceAccountName: database containers: name: database image: gcr.io/google_containers/etcd:v3.2.18 imagePullPolicy: IfNotPresent command: ["etcd"] args: ["--peer-client-cert-auth=true"] nodeSelector: kubernetes.io/hostname: worker02 restartPolicy: OnFailure ```
Project: StarRocks (App Definition and Development / Database)
File: base64_to_bitmap.md

displayed_sidebar: "English" Before you import bitmap data into StarRocks, you need to serialize the data and encode the data as a Base64 string. When you import the Base64 string into StarRocks, you need to convert the string into bitmap data. This function is used to convert Base64 strings into bitmap data. This function is supported from v2.3. ```Haskell BITMAP base64tobitmap(VARCHAR bitmap) ``` `bitmap`: The supported data type is VARCHAR. Before you load Bitmap data into StarRocks, you can use Java or C++ to , add an element, serialize the data, and encode the data as a Base64 string. Then, pass the Base64 string as an input parameter into this function. Returns a value of the BITMAP type. Create a database named `bitmapdb` and a table named `bitmap`. Use Stream Load to import JSON data into `bitmaptable`. During this process, use base64to_bitmap to convert the Base64 string in the JSON file into bitmap data. Create a database and a table in StarRocks. In this example, a Primary Key table is created. ```SQL CREATE database bitmapdb; USE bitmapdb; CREATE TABLE `bitmap_table` ( `tagname` varchar(65533) NOT NULL COMMENT "Tag name", `tagvalue` varchar(65533) NOT NULL COMMENT "Tag value", `userid` bitmap NOT NULL COMMENT "User ID" ) ENGINE=OLAP PRIMARY KEY(`tagname`, `tagvalue`) COMMENT "OLAP" DISTRIBUTED BY HASH(`tagname`) PROPERTIES ( "replication_num" = "3", "storage_format" = "DEFAULT" ); ``` Use to import JSON data into `bitmap_table`. Suppose there is a JSON file named simpledata. This file has the following content and `userid` is a Base64-encoded string. ```JSON { "tagname": "Product", "tagvalue": "Insurance", "userid":"AjowAAABAAAAAAACABAAAAABAAIAAwA=" } ``` Use base64tobitmap to convert `userid` into a bitmap value. ```Plain curl --location-trusted -u <username>:<password>\ -H "columns: c1,c2,c3,tagname=c1,tagvalue=c2,userid=base64tobitmap(c3)"\ -H "label:bitmap123"\ -H "format: json"\ -H "jsonpaths: [\"$.tagname\",\"$.tagvalue\",\"$.userid\"]"\ -T simpleData http://host:port/api/bitmapdb/bitmaptable/stream_load ``` Query data from `bitmap_table`. ```Plaintext mysql> select tagname,tagvalue,bitmaptostring(userid) from bitmap_table; +--+-+-+ | tagname | tagvalue | bitmaptostring(`userid`) | +--+-+-+ | Product | Insurance | 1,2,3 | +--+-+-+ 1 rows in set (0.01 sec) ```
Project: ClickHouse (App Definition and Development / Database)
File: v20.6.6.7-stable.md

Backported in : Added Redis requirepass authorization. (). Backported in : Stop query execution if exception happened in `PipelineExecutor` itself. This could prevent rare possible query hung. (). Backported in : Stop query execution if exception happened in `PipelineExecutor` itself. This could prevent rare possible query hung. Continuation of . (). Backported in : Fix bug which leads to wrong merges assignment if table has partitions with a single part. (). Backported in : Check for array size overflow in `topK` aggregate function. Without this check the user may send a query with carefully crafter parameters that will lead to server crash. This closes . (). Backported in : Fix rare segfaults in functions with combinator -Resample, which could appear in result of overflow with very large parameters. (). Backported in : Fix wrong Decimal multiplication result caused wrong decimal scale of result column. ().
Project: KICS (Provisioning / Security & Compliance)
File: 9b877bd8-94b4-4c10-a060-8e0436cc09fa.md

title: User With Privilege Escalation By Actions 'glue:UpdateDevEndpoint' hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 9b877bd8-94b4-4c10-a060-8e0436cc09fa Query name: User With Privilege Escalation By Actions 'glue:UpdateDevEndpoint' Platform: Terraform Severity: <span style="color:#ff7213">Medium</span> Category: Access Control URL: User with privilege escalation by actions 'glue:UpdateDevEndpoint' and Resource set to '*'. For more information see https://rhinosecuritylabs.com/aws/aws-privilege-escalation-methods-mitigation/.<br> ```tf title="Positive test num. 1 - tf file" hl_lines="1" resource "awsiamuser" "cosmic" { name = "cosmic" } resource "awsiamuserpolicy" "testinline_policy" { name = "testinlinepolicy" user = awsiamuser.cosmic.name policy = jsonencode({ Version = "2012-10-17" Statement = [ { Action = [ "glue:UpdateDevEndpoint", ] Effect = "Allow" Resource = "*" }, ] }) } resource "awsiampolicy_attachment" "test-attach" { name = "test-attachment" users = [awsiamuser.cosmic.name] roles = [awsiamrole.role.name] groups = [awsiamgroup.group.name] } ``` ```tf title="Negative test num. 1 - tf file" resource "awsiamuser" "cosmic2" { name = "cosmic2" } resource "awsiamuserpolicy" "inlinepolicyruninstances2" { name = "inlinepolicyrun_instances" user = awsiamuser.cosmic2.name policy = jsonencode({ Version = "2012-10-17" Statement = [ { Action = [ "ec2:Describe*", ] Effect = "Allow" Resource = "*" }, ] }) } ```