cwielenb/CNCF-Dataset huggingface dataset:
Project: Checkov (Provisioning / Security & Compliance)
File: Contribute Python-Based Policies.md

layout: default published: true title: Contribute Python-Based Policies nav_order: 2 After identifying a Custom Policy's IaC type and provider, place the file with its code in `checkov/<scanner>/checks/<type>/<provider>`, where type is the Custom Policy's type and provider is the Custom Policy's provider. A Custom Policy is a class implementing an abstract base class that corresponds to some provider and type. For example, all Custom Policies of resource type and aws provider implement the resource base class found at `checkov/terraform/checks/resource/basecheck.py`. The resource check needs to implement its base abstract method named `scanresource_conf`, which accepts as an input a dictionary of all the key-valued resource attributes, and outputs a CheckResult. Define a policy as described . `checkov/terraform/checks/resource/aws/APIGatewayCacheEnable.py` ```python from checkov.terraform.checks.resource.baseresourcevalue_check import BaseResourceValueCheck from checkov.common.models.enums import CheckCategories class APIGatewayCacheEnable(BaseResourceValueCheck): def init(self): name = "Ensure API Gateway caching is enabled" id = "CKVAWS120" supportedresources = ['awsapigatewaystage'] categories = [CheckCategories.BACKUPANDRECOVERY] super().init(name=name, id=id, categories=categories, supportedresources=supportedresources) def getinspectedkey(self): return "cacheclusterenabled" check = APIGatewayCacheEnable() ``` Assuming the implemented checks class is file is found in checkov/terraform/checks/<type>/<provider> directory, named <ClassName>.py, create an appropriate unit test file in tests/terraform/checks/<type>/<provider> directory, named test_<ClassName>.py. The test suite should cover different check results; Test if the check outputs PASSED on a compliant configuration, and test if it output FAILED on a non-compliant configuration. You are also encouraged to test more specific components of the check, according to their complexity. `tests/terraform/checks/resource/aws/test_APIGatewayCacheEnable.py` ```python import unittest import hcl2 from checkov.common.models.enums import CheckResult from checkov.terraform.checks.resource.aws.APIGatewayCacheEnable import check class TestAPIGatewayCacheEnable(unittest.TestCase): def test_failure(self): hcl_res = hcl2.loads(""" resource "awsapigatewayrestapi" "example" { name = "example" } """) resourceconf = hclres scanresult = check.scanresourceconf(conf=resourceconf) self.assertEqual(CheckResult.FAILED, scan_result) def test_success(self): hcl_res = hcl2.loads(""" resource "awsapigatewayrestapi" "example" { name = "example" cacheclusterenabled = true } """) resourceconf = hclres scanresult = check.scanresourceconf(conf=resourceconf) self.assertEqual(CheckResult.PASSED, scan_result) if name == 'main': unittest.main() ```
Project: YugabyteDB (App Definition and Development / Database)
File: archive-changes.md

title: archive changes reference headcontent: yb-voyager archive changes linkTitle: archive changes description: YugabyteDB Voyager archive changes reference menu: preview_yugabyte-voyager: identifier: voyager-archive-changes parent: cutover-archive weight: 150 type: docs The archive changes command limits the disk space used by the locally queued CDC events. After the changes from the local queue are applied on the target YugabyteDB database (and source-replica database), they are eligible for deletion. The command gives an option to archive the changes before deleting by moving them to another directory. Note that even if some changes are applied to the target databases, locally queued CDC events are deleted only after the disk use exceeds 70%. ```text Usage: yb-voyager archive changes [ <arguments> ... ] ``` The valid arguments for archive changes are described in the following table: | Argument | Description/valid options | | :- | : | | &#8209;&#8209;delete&#8209;changes&#8209;without&#8209;archiving | Delete the imported changes without archiving them. Note that the changes are deleted from the export directory only after disk utilisation exceeds 70%. <br>Default: false <br> Accepted parameters: true, false, yes, no, 0, 1 | | -e, --export-dir | Path to the export directory. This directory is a workspace used to store exported schema DDL files, export data files, migration state, and a log file.| | --fs-utilization-threshold | Disk utilization threshold in percentage. <br>Default: 70 | | -h, --help | Command line help for archive changes. | | --move-to | Path to the directory where the imported change events are to be moved to. Note that the changes are deleted from the export directory only after the disk use exceeds 70%. | | --send-diagnostics | Enable or disable sending information to Yugabyte. <br>Default: true<br> Accepted parameters: true, false, yes, no, 0, 1 | | -y, --yes | Answer yes to all prompts during migration. <br>Default: false | Example to delete changes without archiving them to another destination is as follows: ```sh yb-voyager archive changes --export-dir /dir/export-dir --delete-changes-without-archiving true ``` Example to archive changes from the export directory to another destination is as follows: ```sh yb-voyager archive changes --export-dir /dir/export-dir --move-to /dir/archived-changes-dir ```
Project: Meshery (Provisioning / Automation & Configuration)
File: v0.6.128.md

layout: release date: 2023-08-18 tag: v0.6.128 Update `errorutil` in CI to looks for the right `server` directory @nebula-aac (#8517) Update `errorutil` in CI to looks for the right `server` directory @nebula-aac (#8517) Thank you to our contributors for making this release possible: @MUzairS15, @leecalcote and @nebula-aac
Project: ShardingSphere (App Definition and Development / Database)
File: show-mask-algorithm-plugins.en.md

+++ title = "SHOW MASK ALGORITHM PLUGINS" weight = 5 +++ The `SHOW MASK ALGORITHM PLUGINS` syntax is used to query all the implementations of the interface `org.apache.shardingsphere.mask.spi.MaskAlgorithm`. {{< tabs >}} {{% tab name="Grammar" %}} ```sql showMaskAlgorithmPlugins ::= 'SHOW' 'MASK' 'ALGORITHM' 'PLUGINS' ``` {{% /tab %}} {{% tab name="Railroad diagram" %}} <iframe frameborder="0" name="diagram" id="diagram" width="100%" height="100%"></iframe> {{% /tab %}} {{< /tabs >}} | Columns | Description | |--|--| | type | type | | type_aliases | type aliases | | description | description | Query all the implementations for `org.apache.shardingsphere.mask.spi.MaskAlgorithm` interface ```sql SHOW MASK ALGORITHM PLUGINS ``` ```sql SHOW MASK ALGORITHM PLUGINS; ++--+-+ | type | type_aliases | description | ++--+-+ | MD5 | | | | KEEPFIRSTNLASTM | | | | KEEPFROMXTOY | | | | MASKAFTERSPECIAL_CHARS | | | | MASKBEFORESPECIAL_CHARS | | | | MASKFIRSTNLASTM | | | | MASKFROMXTOY | | | | GENERICTABLERANDOM_REPLACE | | | ++--+-+ 8 rows in set (0.13 sec) ``` `SHOW`, `MASK`, `ALGORITHM`, `PLUGINS`
Project: StarRocks (App Definition and Development / Database)
File: table_constraints.md

displayed_sidebar: "English" `table_constraints` describes which tables have constraints. The following fields are provided in `table_constraints`: | Field | Description | | | | | CONSTRAINT_CATALOG | The name of the catalog to which the constraint belongs. This value is always `def`. | | CONSTRAINT_SCHEMA | The name of the database to which the constraint belongs. | | CONSTRAINT_NAME | The name of the constraint. | | TABLE_SCHEMA | The name of the database to which the table belongs. | | TABLE_NAME | The name of the table. | | CONSTRAINT_TYPE | The type of constraint. |
Project: YDB (App Definition and Development / Database)
File: percentile_median.md

Calculating percentiles using the amortized version of the algorithm. `MEDIAN`: An alias for `PERCENTILE(N, 0.5)`. {% note info "Restriction" %} The first argument (N) must be a table column name. If you need to bypass this restriction, use a subquery. The restriction is introduced to simplify calculations, since the implementation merges the calls with the same first argument (N) into a single pass. {% endnote %} ```yql SELECT MEDIAN(numeric_column), PERCENTILE(numeric_column, 0.99) FROM my_table; ```
Project: ClickHouse (App Definition and Development / Database)
File: v23.12.3.40-stable.md

sidebar_position: 1 sidebar_label: 2024 Backported in : When executing some queries, which require a lot of streams for reading data, the error `"Paste JOIN requires sorted tables only"` was previously thrown. Now the numbers of streams resize to 1 in that case. (). Backported in : Add `SYSTEM JEMALLOC PURGE` for purging unused jemalloc pages, `SYSTEM JEMALLOC (). Backported in : Allow to ignore schema evolution in Iceberg table engine and read all data using schema specified by the user on table creation or latest schema parsed from metadata on table creation. This is done under a setting `icebergengineignoreschemaevolution` that is disabled by default. Note that enabling this setting can lead to incorrect result as in case of evolved schema all data files will be read using the same schema. (). Delay reading from StorageKafka to allow multiple reads in materialized views (). Fix a stupid case of intersecting parts (). Disable maxjoinedblock_rows in ConcurrentHashJoin (). Fix stream partitioning in parallel window functions (). Fix double destroy call on exception throw in addBatchLookupTable8 (). Fix JSONExtract function for LowCardinality(Nullable) columns (). Multiple read file log storage in mv (). Fix: LIMIT BY and LIMIT in distributed query (). Fix not-ready set for system.tables (). Fix translate() with FixedString input (). Follow-up to (). Fix possible race in ManyAggregatedData dtor. (). Change log level for super imporant message in Keeper (). Decrease log level for one log message (). Fix fasttest by pinning pip dependencies (). No debug symbols in Rust ().
Project: StarRocks (App Definition and Development / Database)
File: CREATE_DATABASE.md

displayed_sidebar: "English" This statement is used to create databases. :::tip This operation requires the CREATE DATABASE privilege on the target catalog. You can follow the instructions in to grant this privilege. ::: ```sql CREATE DATABASE [IF NOT EXISTS] <db_name> ``` `db_name`: the name of the database to create. For the naming conventions, see . Create database `db_test`. ```sql CREATE DATABASE db_test; ```
Project: ClickHouse (App Definition and Development / Database)
File: build-cross-osx.md

slug: /en/development/build-cross-osx sidebar_position: 66 title: How to Build ClickHouse on Linux for macOS sidebar_label: Build on Linux for macOS This is for the case when you have a Linux machine and want to use it to build `clickhouse` binary that will run on OS X. This is intended for continuous integration checks that run on Linux servers. If you want to build ClickHouse directly on macOS, then proceed with . The cross-build for macOS is based on the , follow them first. The following sections provide a walk-through for building ClickHouse for `x8664` macOS. If youre targeting ARM architecture, simply substitute all occurrences of `x8664` with `aarch64`. For example, replace `x86_64-apple-darwin` with `aarch64-apple-darwin` throughout the steps. Follow the instructions from https://apt.llvm.org/ for your Ubuntu or Debian setup. For example the commands for Bionic are like: ``` bash sudo echo "deb [trusted=yes] http://apt.llvm.org/bionic/ llvm-toolchain-bionic-17 main" >> /etc/apt/sources.list sudo apt-get install clang-18 ``` Lets remember the path where we install `cctools` as ${CCTOOLS} ``` bash mkdir ~/cctools export CCTOOLS=$(cd ~/cctools && pwd) cd ${CCTOOLS} git clone https://github.com/tpoechtrager/apple-libtapi.git cd apple-libtapi git checkout 15dfc2a8c9a2a89d06ff227560a69f5265b692f9 INSTALLPREFIX=${CCTOOLS} ./build.sh ./install.sh cd .. git clone https://github.com/tpoechtrager/cctools-port.git cd cctools-port/cctools git checkout 2a3e1c2a6ff54a30f898b70cfb9ba1692a55fad7 ./configure --prefix=$(readlink -f ${CCTOOLS}) --with-libtapi=$(readlink -f ${CCTOOLS}) --target=x86_64-apple-darwin make install ``` Also, we need to download macOS X SDK into the working tree. ``` bash cd ClickHouse/cmake/toolchain/darwin-x86_64 curl -L 'https://github.com/phracker/MacOSX-SDKs/releases/download/11.3/MacOSX11.0.sdk.tar.xz' | tar xJ --strip-components=1 ``` ``` bash cd ClickHouse mkdir build-darwin cd build-darwin CC=clang-18 CXX=clang++-18 cmake -DCMAKEAR:FILEPATH=${CCTOOLS}/bin/x8664-apple-darwin-ar -DCMAKEINSTALLNAMETOOL=${CCTOOLS}/bin/x8664-apple-darwin-installnametool -DCMAKERANLIB:FILEPATH=${CCTOOLS}/bin/x8664-apple-darwin-ranlib -DLINKERNAME=${CCTOOLS}/bin/x8664-apple-darwin-ld -DCMAKETOOLCHAINFILE=cmake/darwin/toolchain-x86_64.cmake .. ninja ``` The resulting binary will have a Mach-O executable format and cant be run on Linux.
Project: ShardingSphere (App Definition and Development / Database)
File: button.en.md

title: Button description : "Nice buttons on your page." A button is a just a clickable button with optional icon. ``` {{%/ button href="https://getgrav.org/" /%}}Get Grav{{%/ /button /%}} {{%/ button href="https://getgrav.org/" icon="fas fa-download" /%}}Get Grav with icon{{%/ /button /%}} {{%/ button href="https://getgrav.org/" icon="fas fa-download" icon-position="right" /%}}Get Grav with icon right{{%/ /button /%}} ``` {{% button href="https://getgrav.org/" %}}Get Grav{{% /button %}} {{% button href="https://getgrav.org/" icon="fas fa-download" %}}Get Grav with icon{{% /button %}} {{% button href="https://getgrav.org/" icon="fas fa-download" icon-position="right" %}}Get Grav with icon right{{% /button %}}
Project: Meshery (Provisioning / Automation & Configuration)
File: v0.5.19.md

layout: release date: 2021-06-06 tag: v0.5.19 Bug: system start does not change platform if config file is already created @navendu-pottekkat (#3140) Generate and upload coverage report for unit tests @ramrodo (#3135) updated helm action @kumarabd (#3138) [WIP] Eliminate unnecessary warning when changing endpoint @metonymic-smokey (#3132) Bump github.com/layer5io/meshery-operator from 0.2.12 to 0.2.17 @dependabot (#3110) Updated Docs about the installation of Meshery with Windows w/signoff @debjani1413 (#3120)
Project: KICS (Provisioning / Security & Compliance)
File: 25d251f3-f348-4f95-845c-1090e41a615c.md

title: EFS Without KMS hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 25d251f3-f348-4f95-845c-1090e41a615c Query name: EFS Without KMS Platform: Terraform Severity: <span style="color:#edd57e">Low</span> Category: Encryption URL: Amazon Elastic Filesystem should have filesystem encryption enabled using KMS CMK customer-managed keys instead of AWS managed-keys<br> ```tf title="Positive test num. 1 - tf file" hl_lines="1" resource "awsefsfile_system" "positive1" { creation_token = "my-product" encrypted = true tags = { Name = "MyProduct" } } ``` ```tf title="Negative test num. 1 - tf file" resource "awsefsfile_system" "negative1" { creation_token = "my-product" encrypted = true kmskeyid = "1234abcd-12ab-34cd-56ef-1234567890ab" tags = { Name = "MyProduct" } } ```
Project: Beam (App Definition and Development / Streaming & Messaging)
File: yahoo.md

title: "Yahoo" icon: /images/logos/powered-by/yahoo.png hasNav: true hasLink: "https://www.yahoo.com/" <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->
Project: SPIRE (Provisioning / Key Management)
File: plugin_agent_nodeattestor_k8s_sat.md

This plugin has been deprecated in favor of the plugin and will be removed in a future release. Must be used in conjunction with the server-side k8s_sat plugin The `k8s_sat` plugin attests nodes running in inside of Kubernetes. The agent reads and provides the signed service account token to the server. *Note: If your cluster supports you should instead consider using the `k8s_psat` attestor due to the below.* The server-side `k8s_sat` plugin generates a one-time UUID and generates a SPIFFE ID with the form: ```xml spiffe://<trustdomain>/spire/agent/k8ssat/<cluster>/<UUID> ``` The main configuration accepts the following values: | Configuration | Description | Default | |||-| | `cluster` | Name of the cluster. It must correspond to a cluster configured in the server plugin. | | `token_path` | Path to the service account token on disk | "/var/run/secrets/kubernetes.io/serviceaccount/token" | The token path defaults to the default location Kubernetes uses to place the token and should not need to be overridden in most cases. A sample configuration with the default token path: ```hcl NodeAttestor "k8s_sat" { plugin_data { cluster = "MyCluster" } } ``` At this time, the service account token does not contain claims that could be used to strongly identify the node/daemonset/pod running the agent. This means that any container running in an allowed service account can masquerade as an agent, giving it access to any identity the agent is capable of issuing. It is STRONGLY recommended that agents run under a dedicated service account. It should be noted that due to the fact that SPIRE can't positively identify a node using this method, it is not possible to directly authorize identities for a distinct node or sets of nodes. Instead, this must be accomplished indirectly using a service account and deployment that leverages node affinity or node selectors.
Project: YDB (App Definition and Development / Database)
File: current_utc.md

`CurrentUtcDate()`, `CurrentUtcDatetime()` and `CurrentUtcTimestamp()`: Getting the current date and/or time in UTC. The result data type is specified at the end of the function name. The arguments are optional and work same as . Examples ```yql SELECT CurrentUtcDate(); ``` ```yql SELECT CurrentUtcTimestamp(TableRow()) FROM my_table; ```
Project: KusionStack (Provisioning / Automation & Configuration)
File: kusion_build.md

Build Kusion modules in a Stack to the Intent Build Kusion modules in a Stack to the Intent The command must be executed in a Stack or by specifying a Stack directory with the -w flag. You can provide a list of arguments to replace the placeholders defined in KCL, and use the --output flag to output the built results to a file ``` kusion build [flags] ``` ``` kusion build -D name=test -D age=18 kusion build -w appops/demo/dev kusion build -o output.yaml kusion build -Y settings.yaml kusion build --no-style=true ``` ``` -D, --argument stringToString Specify the top-level argument (default []) -h, --help help for build --no-style Disable the output style and color -o, --output string Specify the output file -Y, --setting strings Specify the command line setting files -w, --workdir string Specify the work directory ``` ``` --profile string Name of profile to capture. One of (none|cpu|heap|goroutine|threadcreate|block|mutex) (default "none") --profile-output string Name of the file to write the profile to (default "profile.pprof") ``` - Kusion is the Platform Orchestrator of Internal Developer Platform
Project: Airship (Provisioning / Automation & Configuration)
File: bug_report.md

name: Bug report about: Report a bug or unexpected treasuremap behavior title: '' labels: bug, triage assignees: '' Describe the bug A clear and concise description of what the bug is. If known add a label for the type of issue, i.e affecting the Documents vs its a CI/CD problem. Steps To Reproduce Minimal steps to reproduce the issue. Expected behavior A clear and concise description of what you expected to happen. Environment Treasuremap version: Treasuremp site type: Airshipctl version: Operating system: Kernel version: Kubernetes version: Go version: Hypervisor level 0 (if applicable): Hardware specs (e.g. 4 vCPUs, 16GB RAM, bare metal vs VM):
Project: Chef InSpec (Provisioning / Security & Compliance)
File: google_logging_folder_log_sink.md

+++ title = "googleloggingfolderlogsink resource" draft = false gh_repo = "inspec" platform = "gcp" [menu] [menu.inspec] title = "googleloggingfolderlogsink" identifier = "inspec/resources/gcp/googleloggingfolderlogsink.md googleloggingfolderlogsink resource" parent = "inspec/resources/gcp" +++ A `googleloggingfolderlogsink` is used to test a Google FolderLogSink resource ```ruby describe.one do googleresourcemanagerfolders(parent: 'organizations/12345').names.each do |folder_name| describe googleloggingfolderlogsink(folder: folder_name.split('/')[1], name: 'inspec-gcp-folder-sink') do it { should exist } its('filter') { should cmp 'resource.type = gce_instance AND severity >= ERROR' } end end end ``` Properties that can be accessed from the `googleloggingfolderlogsink` resource: `folder` : Id of the folder that this sink belongs to `name` : Name of the log sink. `filter` : An advanced logs filter. The only exported log entries are those that are in the resource owning the sink and that match the filter. `destination` : The export destination. `writer_identity` : An IAM identitya service account or groupunder which Logging writes the exported log entries to the sink's destination. This field is set by sinks.create and sinks.update based on the value of uniqueWriterIdentity in those methods. `include_children` : If the field is false, the default, only the logs owned by the sink's parent resource are available for export. If the field is true, then logs from all the projects, folders, and billing accounts contained in the sink's parent resource are also available for export. Whether a particular log entry from the children is exported depends on the sink's filter expression. Ensure the is enabled for the current project.