cwielenb/CNCF-Dataset huggingface dataset:
Project: Chef Infra (Provisioning / Automation & Configuration)
File: resource_property_validation_messaging.md

Custom resources provide multiple property validators, allowing authors to control property input beyond just simple data types. Authors can expect strings to match predefined strings, match a regex, or return true from a callback method. This gives the author great control over the input data, but doesn't provide the consumer with much information when the validator fails. This RFC provides the author with the ability to control the error text when the validator fails. As an author of custom resources, I want to control property inputs while providing useful error messaging on failure, so that users can easily understand why input data is not acceptable As a consumer of custom resources, I want detailed errors when I pass incorrect data to a resource, so that I quickly resolve failed chef-client runs. This design specifies a `validation_message` option for properties, which accepts a string. This message will be shown on failure in place of a generic failure message. in resources/example.rb ```ruby property :version, kind_of: String, default: '8.0.35' regex: [/^(?:(\d+)\.)?(?:(\d+)\.)?(\*|\d+)$/] validation_message: 'Version must be a X.Y.Z format String type' ```
Project: secureCodeBox (Provisioning / Security & Compliance)
File: values.yaml.md

title: values.yaml sidebar_position: 2 The `values.yaml` is also created by `helm create new-scanner`. Most of these generated fields are not necessary for the secureCodeBox. In the following we will describe the important fields. The final `values.yaml` will look something like this: ```yaml parser: image: repository: docker.io/securecodebox/parser-nmap tag: null ttlSecondsAfterFinished: null backoffLimit: 3 env: [] scopeLimiterAliases: {} scanner: image: repository: docker.io/securecodebox/parser-nmap tag: null ttlSecondsAfterFinished: null backoffLimit: 3 resources: {} env: [] extraVolumes: [] extraVolumeMounts: [] extraContainers: [] securityContext: runAsNonRoot: true readOnlyRootFilesystem: true allowPrivilegeEscalation: false privileged: false capabilities: drop: all ``` The two top-level fields `scanner` and `parser` define the containers and settings for the scanner and parser, respectively. All fields below are common for both scanner and parser. The `image` field contains the container image and tag used for the scanner or parser. For the scanner, this could be the official image of the scanner or a custom image, if one is needed. Usually the `tag` of the image is `null` and will default to the charts `appVersion` (for the scanner) or `version` (for the parser). See below how to use a local docker image. For WPScan the official image can be used so the `image` fields for scanner and parser may look like this: ```yaml scanner: image: repository: wpscanteam/wpscan tag: null parser: image: repository: docker.io/securecodebox/parser-wpscan tag: null ``` Defines how long the scanner job after finishing will be available (see: ). The `resources` field can limit or request resources for the scan / parse job (see: ). A basic example could be the following: ```yaml resources: requests: memory: "256Mi" cpu: "250m" limits: memory: "512Mi" cpu: "500m" ``` Optional environment variables mapped into the job (see: ). Optional Volumes mapped into the job (see: ). Optional VolumeMounts mapped into the job (see: ). Optional additional Containers started with the job (see: ). Optional securityContext set on the container (see: ). Optional scopeLimiterAliases set on the parse definition (see ) Optional affinity settings that control how the job is scheduled (see: ) Optional tolerations settings that control how the job is scheduled (see: )
Project: GreptimeDB (App Definition and Development / Database)
File: how-to-implement-sql-statement.md

This document introduces how to implement SQL statements in GreptimeDB. The execution entry point for SQL statements locates at Frontend Instance. You can see it has implemented `SqlQueryHandler`: ```rust impl SqlQueryHandler for Instance { type Error = Error; async fn doquery(&self, query: &str, queryctx: QueryContextRef) -> Vec<Result<Output>> { // ... } } ``` Normally, when a SQL query arrives at GreptimeDB, the `do_query` method will be called. After some parsing work, the SQL will be feed into `StatementExecutor`: ```rust // in Frontend Instance: self.statementexecutor.executesql(stmt, query_ctx).await ``` That's where we handle our SQL statements. You can just create a new match arm for your statement there, then the statement is implemented for both GreptimeDB Standalone and Cluster. You can see how `DESCRIBE TABLE` is implemented as an example. Now, what if the statements should be handled differently for GreptimeDB Standalone and Cluster? You can see there's a `SqlStatementExecutor` field in `StatementExecutor`. Each GreptimeDB Standalone and Cluster has its own implementation of `SqlStatementExecutor`. If you are going to implement the statements differently in the two mode ( like `CREATE TABLE`), you have to implement them in their own `SqlStatementExecutor`s. Summarize as the diagram below: ```text SQL query | v ++ | SqlQueryHandler::do_query | ++ | | SQL parsing v +--+ | StatementExecutor::execute_sql | +--+ | | SQL execution v +-+ | commonly handled statements like | | "plan_exec" for selection or | +-+ | | For Standalone | | For Cluster v v ++ ++ | SqlStatementExecutor impl | | SqlStatementExecutor impl | | in Datanode Instance | | in Frontend DistInstance | ++ ++ ``` Note that some SQL statements can be executed in our QueryEngine, in the form of `LogicalPlan`. You can follow the invocation path down to the `QueryEngine` implementation from `StatementExecutor::plan_exec`. For now, there's only one `DatafusionQueryEngine` for both GreptimeDB Standalone and Cluster. That lone query engine works for both modes is because GreptimeDB read/write data through `Table` trait, and each mode has its own `Table` implementation. We don't have any bias towards whether statements should be handled in query engine or `StatementExecutor`. You can implement one kind of statement in both places. For example, `Insert` with selection is handled in query engine, because we can easily do the query part there. However, `Insert` without selection is not, for the cost of parsing statement to `LogicalPlan` is not neglectable. So generally if the SQL query is simple enough, you can handle it in `StatementExecutor`; otherwise if it is complex or has some part of selection, it should be parsed to `LogicalPlan` and handled in query engine.
Project: KICS (Provisioning / Security & Compliance)
File: 2b3c671f-1b76-4741-8789-ed1fe0785dc4.md

title: PostgreSQL Server Without Connection Throttling hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 2b3c671f-1b76-4741-8789-ed1fe0785dc4 Query name: PostgreSQL Server Without Connection Throttling Platform: Terraform Severity: <span style="color:#ff7213">Medium</span> Category: Observability URL: Ensure that Connection Throttling is set for the PostgreSQL server<br> ```tf title="Positive test num. 1 - tf file" hl_lines="19 12 5" resource "azurermpostgresqlconfiguration" "positive1" { name = "connection_throttling" resourcegroupname = data.azurermresourcegroup.example.name servername = azurermpostgresql_server.example.name value = "off" } resource "azurermpostgresqlconfiguration" "positive2" { name = "connection_throttling" resourcegroupname = data.azurermresourcegroup.example.name servername = azurermpostgresql_server.example.name value = "Off" } resource "azurermpostgresqlconfiguration" "positive3" { name = "connection_throttling" resourcegroupname = data.azurermresourcegroup.example.name servername = azurermpostgresql_server.example.name value = "OFF" } ``` ```tf title="Negative test num. 1 - tf file" resource "azurermpostgresqlconfiguration" "negative1" { name = "connection_throttling" resourcegroupname = data.azurermresourcegroup.example.name servername = azurermpostgresql_server.example.name value = "on" } resource "azurermpostgresqlconfiguration" "negative2" { name = "connection_throttling" resourcegroupname = data.azurermresourcegroup.example.name servername = azurermpostgresql_server.example.name value = "On" } resource "azurermpostgresqlconfiguration" "negative3" { name = "connection_throttling" resourcegroupname = data.azurermresourcegroup.example.name servername = azurermpostgresql_server.example.name value = "ON" } ```
Project: Flink (App Definition and Development / Streaming & Messaging)
File: lateral-view.md

title: "Lateral View Clause" weight: 6 type: docs <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> Lateral view clause is used in conjunction with user-defined table generating functions()) such as `explode()`. A UDTF generates zero or more output rows for each input row. A lateral view first applies the UDTF to each row of base and then joins results output rows to the input rows to form a virtual table having the supplied table alias. ```sql lateralView: LATERAL VIEW [ OUTER ] udtf( expression ) tableAlias AS columnAlias [, ... ] fromClause: FROM baseTable lateralView [, ... ] ``` The column alias can be omitted. In this case, aliases are inherited from fields name of StructObjectInspector which is returned from UDTF. Lateral View Outer User can specify the optional `OUTER` keyword to generate rows even when a `LATERAL VIEW` usually would not generate a row. This happens when the UDTF used does not generate any rows which happens easily with when the column to explode is empty. In this case, the source row would never appear in the results. `OUTER` can be used to prevent that and rows will be generated with `NULL` values in the columns coming from UDTF. Multiple Lateral Views A FROM clause can have multiple LATERAL VIEW clauses. Subsequent LATERAL VIEWS can reference columns from any of the tables appearing to the left of the LATERAL VIEW. Assuming you have one table: ```sql CREATE TABLE pageAds(pageid string, addid_list array<int>); ``` And the table contains two rows: ```sql front_page, [1, 2, 3]; contact_page, [3, 4, 5]; ``` Now, you can use `LATERAL VIEW` to convert the column `addid_list` into separate rows: ```sql SELECT pageid, adid FROM pageAds LATERAL VIEW explode(adid_list) adTable AS adid; -- result front_page, 1 front_page, 2 front_page, 3 contact_page, 3 contact_page, 4 contact_page, 5 ``` Also, if you have one table: ```sql CREATE TABLE t1(c1 array<int>, c2 array<int>); ``` You can use multiple lateral view clauses to convert the column `c1` and `c2` into separate rows: ```sql SELECT myc1, myc2 FROM t1 LATERAL VIEW explode(c1) myTable1 AS myc1 LATERAL VIEW explode(c2) myTable2 AS myc2; ``` When the UDTF doesn't produce rows, then `LATERAL VIEW` won't produce rows. You can use `LATERAL VIEW OUTER` to still produce rows, with `NULL` filling the corresponding column. ```sql SELECT * FROM t1 LATERAL VIEW OUTER explode(array()) C AS a; ```
Project: StarRocks (App Definition and Development / Database)
File: views.md

displayed_sidebar: "English" `views` provides information about all user-defined views. The following fields are provided in `views`: | Field | Description | | -- | | | TABLE_CATALOG | The name of the catalog to which the view belongs. This value is always `def`. | | TABLE_SCHEMA | The name of the database to which the view belongs. | | TABLE_NAME | The name of the view. | | VIEW_DEFINITION | The `SELECT` statement that provides the definition of the view. | | CHECKOPTION | The value of the `CHECKOPTION` attribute. The value is one of `NONE`, `CASCADE`, or `LOCAL`. | | IS_UPDATABLE | Whether the view is updatable. The flag is set to `YES` (true) if `UPDATE` and `DELETE` (and similar operations) are legal for the view. Otherwise, the flag is set to `NO` (false). If a view is not updatable, statements such `UPDATE`, `DELETE`, and `INSERT` are illegal and are rejected. | | DEFINER | The user of the user who created the view. | | SECURITY_TYPE | The view `SQL SECURITY` characteristic. The value is one of `DEFINER` or `INVOKER`. | | CHARACTERSETCLIENT | | | COLLATION_CONNECTION | |
Project: Checkov (Provisioning / Security & Compliance)
File: gitlab_configuration.md

layout: default title: gitlab_configuration resource scans nav_order: 1 | | Id | Type | Entity | Policy | IaC | Resource Link | |-|--|-|-||-|--| | 0 | CKVGITLAB1 | gitlabconfiguration | * | Merge requests should require at least 2 approvals | gitlabconfiguration | | | 1 | CKVGITLAB2 | gitlabconfiguration | * | Ensure all Gitlab groups require two factor authentication | gitlabconfiguration | |
Project: Apache Hadoop (App Definition and Development / Database)
File: RELEASENOTES.0.13.0.md

<! --> These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements. | Major | MiniDFSCluster exists a race condition that lead to data node resources are not properly released* Resolved race condition in shutting down MiniDFSCluster data node that prevented resources from being deallocated properly.
Project: Beam (App Definition and Development / Streaming & Messaging)
File: when-in-processing-time.md

type: runners title: "When in processing time?" layout: "capability-template" <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> <!-- Summary table --> {{< documentation/capability-matrix-big cap-data="capability-matrix" cap-view="full" cap-index=4 >}}
Project: KICS (Provisioning / Security & Compliance)
File: 727c4fd4-d604-4df6-a179-7713d3c85e20.md

title: EFS Not Encrypted hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 727c4fd4-d604-4df6-a179-7713d3c85e20 Query name: EFS Not Encrypted Platform: Ansible Severity: <span style="color:#bb2124">High</span> Category: Encryption URL: Elastic File System (EFS) must be encrypted<br> ```yaml title="Positive test num. 1 - yaml file" hl_lines="17 6 25" name: foo community.aws.efs: state: present name: myTestEFS encrypt: no tags: Name: myTestNameTag purpose: file-storage targets: subnet_id: subnet-748c5d03 security_groups: ["sg-1a2b3c4d"] name: foo2 community.aws.efs: state: present name: myTestEFS encrypt: false tags: Name: myTestNameTag purpose: file-storage targets: subnet_id: subnet-748c5d03 security_groups: ["sg-1a2b3c4d"] name: foo3 community.aws.efs: state: present name: myTestEFS tags: Name: myTestNameTag purpose: file-storage targets: subnet_id: subnet-748c5d03 security_groups: ["sg-1a2b3c4d"] ``` ```yaml title="Negative test num. 1 - yaml file" name: foo community.aws.efs: state: present name: myTestEFS encrypt: yes tags: Name: myTestNameTag purpose: file-storage targets: subnet_id: subnet-748c5d03 security_groups: [sg-1a2b3c4d] name: foo2 community.aws.efs: state: present name: myTestEFS encrypt: true tags: Name: myTestNameTag purpose: file-storage targets: subnet_id: subnet-748c5d03 security_groups: [sg-1a2b3c4d] ```
Project: Chef Habitat (App Definition and Development / Application Definition & Image Build)
File: README.md

This is a tool designed to simulate a supervisor network using docker-compose. It takes much inspiration from . I wanted a tool that I could use to get a network of Habitat supervisors up and running quickly. More than that though, I specifically wanted to be able to test supervisors that were under active development. I wanted to run `cargo build` on the `sup` crate and then immediately be able to test the changes I had just made with a minimum of hassle and fuss. That's why I made this. To start the network, run `make up`. This will build `hab`, `hab-launch`, and `hab-sup`, write out a `CTL_SECRET` file, and start the network. By default it will start 1 bastion node and 3 peer nodes. The number of peer nodes that are started can be controlled via `HABSUPTESTNETWORKSIZE`. ``` make up HABSUPTESTNETWORKSIZE=5 ``` To stop the network, run `make down`. This not only stops the network and all the containers, but also removes them, as well as removes the `CTL_SECRET` file. If you want to just stop containers, without removing them, you can run `make stop`. `make start` will start them back up from a stopped state. Nodes are started up in detached mode. Presumably, you'd like to look at the logs for a service at some point. You can do that via `make logs`. This shows the logs for the `rando` service by default but you can display the logs of the bastion service by setting the `HABSUPTESTNETWORKLOG_SERVICE` variable to `bastion`. ``` make logs HABSUPTESTNETWORKLOG_SERVICE=bastion ``` If you would like to get a shell prompt into one of the containers, you can run `make console`. This will get you into the `rando_1` container by default. You can specify a different container using the `HABSUPTESTNETWORKPEER_NAME` variable. ``` make console HABSUPTESTNETWORKPEERNAME=rando2 ``` If you wish to simulate a departure, you can kill a container using `make kill`. Which container is killed works the same as `make console`. Sometimes you may want the IP address of one of the containers, perhaps to do a service load on a remote supervisor, or perhaps to satisfy your own curiosity. `make ip` will get you that. Specifying the container works the same as `make console`.
Project: ClickHouse (App Definition and Development / Database)
File: json.md

slug: /en/sql-reference/data-types/json sidebar_position: 26 sidebar_label: JSON :::note This feature is experimental and is not production-ready. If you need to work with JSON documents, consider using instead. ::: Stores JavaScript Object Notation (JSON) documents in a single column. `JSON` is an alias for `Object('json')`. :::note The JSON data type is an obsolete feature. Do not use it. If you want to use it, set `allowexperimentalobject_type = 1`. ::: Example 1 Creating a table with a `JSON` column and inserting data into it: ```sql CREATE TABLE json ( o JSON ) ENGINE = Memory ``` ```sql INSERT INTO json VALUES ('{"a": 1, "b": { "c": 2, "d": [1, 2, 3] }}') ``` ```sql SELECT o.a, o.b.c, o.b.d[3] FROM json ``` ```text o.ao.b.carrayElement(o.b.d, 3) 1 2 3 ``` Example 2 To be able to create an ordered `MergeTree` family table the sorting key has to be extracted into its column. For example, to insert a file of compressed HTTP access logs in JSON format: ```sql CREATE TABLE logs ( timestamp DateTime, message JSON ) ENGINE = MergeTree ORDER BY timestamp ``` ```sql INSERT INTO logs SELECT parseDateTimeBestEffort(JSONExtractString(json, 'timestamp')), json FROM file('access.json.gz', JSONAsString) ``` When displaying a `JSON` column ClickHouse only shows the field values by default (because internally, it is represented as a tuple). You can display the field names as well by setting `outputformatjsonnamedtuplesasobjects = 1`: ```sql SET outputformatjsonnamedtuplesasobjects = 1 SELECT * FROM json FORMAT JSONEachRow ``` ```text {"o":{"a":1,"b":{"c":2,"d":[1,2,3]}}} ```
Project: Kapitan (Provisioning / Automation & Configuration)
File: remove.md

This input type simply removes files or directories. This can be helpful if you can't control particular files generated during other compile inputs. For example, to remove a file named `copytarget`, specify an entry to `inputpaths`, `compiled/${kapitan:vars:target}/copy_target`. ```yaml parameters: target_name: removal kapitan: vars: target: ${target_name} compile: input_type: copy input_paths: copy_target output_path: . input_type: remove input_paths: compiled/${kapitan:vars:target}/copy_target output_path: . ``` As a reminder, each input block within the compile array is run sequentially for a target in Kapitan. If we reversed the order of the inputs above like so: ```yaml parameters: target_name: removal kapitan: vars: target: ${target_name} compile: input_type: remove input_paths: compiled/${kapitan:vars:target}/copy_target output_path: . input_type: copy input_paths: copy_target output_path: . ``` The first input block would throw an error because the copy input command hasn't run yet to produce the file being removed by the remove input block. Supported output types: N/A (no need to specify `output_type`)
Project: ClickHouse (App Definition and Development / Database)
File: v23.3.9.55-lts.md

sidebar_position: 1 sidebar_label: 2023 Backported in : Do not store blocks in `ANY` hash join if nothing is inserted. (). Backported in : Fix incorrect projection analysis which invalidates primary keys. This issue only exists when `queryplanoptimizeprimarykey = 1, queryplanoptimize_projection = 1` . This fixes . This fixes . (). Backported in : Packing inline cache into docker images sometimes causes strange special effects. Since we don't use it at all, it's good to go. (). Backported in : The compiler's profile data (`-ftime-trace`) is uploaded to ClickHouse Cloud., the second attempt after . (). Backported in : Preserve environment parameters in `clickhouse start` command. Fixes . (). Fix optimization to move functions before sorting. (). Fix Block structure mismatch in Pipe::unitePipes for FINAL (). Fix binary arithmetic for Nullable(IPv4) (). Support IPv4 and IPv6 as dictionary attributes (). Fix ORDER BY tuple of WINDOW functions (). Disable expression templates for time intervals (). Fix `countSubstrings()` hang with empty needle and a column haystack (). Fixed inserting into Buffer engine (). The implementation of AnyHash was non-conformant. (). init and destroy ares channel on demand.. (). Fix crash in function `tuple` with one sparse column argument (). clickhouse-keeper: fix implementation of server with poll() (). Fix password leak in show create mysql table (). Fix incorrect normal projection AST format (). Fix loading lazy database during system.table select query (). Fix wrong columns order for queries with parallel FINAL. (). Fix: interpolate expression takes source column instead of same name aliased from select expression. (). Fix crash in comparison functions due to incorrect query analysis (). Fix deadlocks in StorageTableFunctionProxy (). Disable testreversedns_query/test.py (). Disable testhostregexpmultipleptr_records/test.py ().