cwielenb/CNCF-Dataset huggingface dataset:
Project: Apache NiFi (App Definition and Development / Streaming & Messaging)
File: PULL_REQUEST_TEMPLATE.md

<!-- Licensed to the Apache Software Foundation (ASF) under one or more --> <!-- contributor license agreements. See the NOTICE file distributed with --> <!-- this work for additional information regarding copyright ownership. --> <!-- The ASF licenses this file to You under the Apache License, Version 2.0 --> <!-- (the "License"); you may not use this file except in compliance with --> <!-- the License. You may obtain a copy of the License at --> <!-- http://www.apache.org/licenses/LICENSE-2.0 --> <!-- Unless required by applicable law or agreed to in writing, software --> <!-- distributed under the License is distributed on an "AS IS" BASIS, --> <!-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. --> <!-- See the License for the specific language governing permissions and --> <!-- limitations under the License. --> Please complete the following tracking steps prior to pull request creation. issue created [ ] Pull Request title starts with Apache NiFi Jira issue number, such as `NIFI-00000` [ ] Pull Request commit message starts with Apache NiFi Jira issue number, as such `NIFI-00000` [ ] Pull Request based on current revision of the `main` branch [ ] Pull Request refers to a feature branch with one commit containing changes Please indicate the verification steps performed prior to pull request creation. [ ] Build completed using `mvn clean install -P contrib-check` [ ] JDK 21 also need to be implemented in the . according to the [ ] New dependencies are documented in applicable `LICENSE` and `NOTICE` files [ ] Documentation formatting appears as expected in rendered files
Project: Pulumi (Provisioning / Automation & Configuration)
File: index.md

Welcome to the Pulumi developer documentation! This documentation provides details on Pulumi internals, including but not limited to: How to build and test Pulumi Pulumi architectural details Specifications Use the navigation bar to the left to browse the docs. ```{toctree} :hidden: :caption: System Architecture architecture/overview architecture/resource-registration architecture/deployment-schema architecture/type-system architecture/import ``` ```{toctree} :hidden: :caption: Resource Providers providers/implementers-guide providers/metaschema ``` ```{toctree} :hidden: :caption: Meta build ```
Project: KICS (Provisioning / Security & Compliance)
File: 34b921bd-90a0-402e-a0a5-dc73371fd963.md

title: SES Policy With Allowed IAM Actions hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 34b921bd-90a0-402e-a0a5-dc73371fd963 Query name: SES Policy With Allowed IAM Actions Platform: Terraform Severity: <span style="color:#bb2124">High</span> Category: Access Control URL: SES policy should not allow IAM actions to all principals<br> ```tf title="Positive test num. 1 - tf file" hl_lines="4" resource "awssesidentity_policy" "positive1" { identity = awssesdomain_identity.example.arn name = "example" policy = <<EOF { "Version": "2012-10-17", "Statement": [ { "Action": "*", "Principal": { "AWS": "*" }, "Effect": "Allow", "Resource": "*", "Sid": "" } ] } EOF } ``` ```tf title="Negative test num. 1 - tf file" resource "awssesidentity_policy" "negative1" { identity = awssesdomain_identity.example.arn name = "example" policy = <<EOF { "Version": "2012-10-17", "Statement": [ { "Action": "*", "Principal": { "AWS": "arn:aws:iam::987654321145:root" }, "Effect": "Allow", "Resource": "*", "Sid": "" } ] } EOF } ```
Project: GraphScope (App Definition and Development / Database)
File: how_to_debug.md

This document shows how to debugging GraphScope under various conditions. Most of the logs will be streamed through the stdout of client, you could control the log level by ```python import graphscope graphscope.setoption(showlog=True) graphscope.setoption(loglevel='DEBUG') # could also be INFO, ERROR ``` As you may know, GraphScope is composed of three engines, where the detailed log location of each engine is Analytical Engine: `/tmp/grape_engine.INFO` Interactive Engine: Inside `/var/log/graphscope/` or `$HOME/log/graphscope` if GraphScope doesn't have permission of `/var/log`. You may find several folders named with a long number, which is the object id of the graph. There is also a `current` folder links to the log folder of latest created interactive instance. Learning Engine: `graphlearn.INFO` in the current directory. In kubernetes environment, besides most of the logs still output to console, you could find detailed logs in each pod's stdout, or files inside each pods. Note: You could use `kubectl logs <pod>` to inspect the stdout of the pod. Use `kubectl logs <pod> -c <container>` to inspect a specific container inside the pod. Coordinator: The stdout of coordinator pod. Analytical Engine: The stdout engine container in the engine pod. Interactive Engine: The stdout of executor container in the engine pod for the executor log. And the stdout of interactive-frontend pod for the frontend. The log files resides in the `/var/log/graphscope` of each container, respectively. Here is list with commands usually used for checking the status of the GraphScope deployment on K8s. Check the status of the pods in a specific namespace `kubectl get pods -n <namespace>` Get detailed information about a pod: `kubectl describe pod <pod-name> -n <namespace>` Check the logs of a container in a pod: `kubectl logs <pod-name> <container-name> -n <namespace>` Check the events related to a pod: `kubectl get events --field-selector involvedObject.name=<pod-name> -n <namespace>` Check the configuration of a resource: `kubectl get <resource-type> <resource-name> -o yaml -n <namespace>` Verify that the Docker image used in the deployment is correct and can be pulled by Kubernetes. Check the logs of the container in the pod to see if there are any error messages or warnings that could indicate the source of the problem. Increase the log level of the GraphScope components. This will produce more verbose output that may help pinpoint the source of the problem. ```python import graphscope graphscope.setoption(showlog=True) graphscope.setoption(loglevel='DEBUG') ``` Use kubectl exec to access a running container in a pod and run diagnostic commands inside it. Use kubectl port-forward to forward a port from a pod to your local machine and access the service directly from your browser or command line. Check the Kubernetes events to see if there are any events related to the pod that could explain the problem.
Project: Beam (App Definition and Development / Streaming & Messaging)
File: _index.md

title: "Beam Security" aliases: /security/CVE-2020-1929/ <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> Apache Beam uses the standard process outlined by the [Apache Security Team](https://www.apache.org/security/) for reporting vulnerabilities. Note that vulnerabilities should not be publicly disclosed until the project has responded. To report a possible security vulnerability, please email `security@apache.org` and `pmc@beam.apache.org`. This is a non-public list that will reach the Beam PMC. [CVE-2020-1929] Apache Beam MongoDB IO connector disables certificate trust verification Severity: Major Vendor: The Apache Software Foundation Versions Affected: Apache Beam 2.10.0 to 2.16.0 Description: The Apache Beam MongoDB connector in versions 2.10.0 to 2.16.0 has an option to disable SSL trust verification. However this configuration is not respected and the certificate verification disables trust verification in every case. This exclusion also gets registered globally which disables trust checking for any code running in the same JVM. Mitigation: Users of the affected versions should apply one of the following mitigations: Upgrade to Apache Beam 2.17.0 or later Acknowledgements: This issue was reported (and fixed) by Colm higeartaigh.
Project: KICS (Provisioning / Security & Compliance)
File: e200a6f3-c589-49ec-9143-7421d4a2c845.md

title: ELB With Security Group Without Inbound Rules hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: e200a6f3-c589-49ec-9143-7421d4a2c845 Query name: ELB With Security Group Without Inbound Rules Platform: CloudFormation Severity: <span style="color:#ff7213">Medium</span> Category: Networking and Firewall URL: An AWS Elastic Load Balancer (ELB) shouldn't have security groups without outbound rules<br> ```yaml title="Positive test num. 1 - yaml file" hl_lines="5" AWSTemplateFormatVersion: 2010-09-09 Resources: sgwithoutingress: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Limits security group egress traffic MyLoadBalancer: Type: AWS::ElasticLoadBalancing::LoadBalancer Properties: SecurityGroups: sgwithoutingress ``` ```json title="Positive test num. 2 - json file" hl_lines="6" { "AWSTemplateFormatVersion": "2010-09-09T00:00:00Z", "Resources": { "sgwithoutingress": { "Type": "AWS::EC2::SecurityGroup", "Properties": { "GroupDescription": "Limits security group egress traffic" } }, "MyLoadBalancer": { "Type": "AWS::ElasticLoadBalancing::LoadBalancer", "Properties": { "SecurityGroups": [ "sgwithoutingress" ] } } } } ``` ```yaml title="Negative test num. 1 - yaml file" AWSTemplateFormatVersion: 2010-09-09 Resources: sgwithingress: Type: AWS::EC2::SecurityGroup Properties: GroupDescription: Limits security group egress traffic SecurityGroupIngress: IpProtocol: tcp FromPort: 80 ToPort: 80 CidrIp: 0.0.0.0/0 MyLoadBalancer: Type: AWS::ElasticLoadBalancing::LoadBalancer Properties: SecurityGroups: sgwithingress ``` ```json title="Negative test num. 2 - json file" { "AWSTemplateFormatVersion": "2010-09-09T00:00:00Z", "Resources": { "sgwithingress": { "Type": "AWS::EC2::SecurityGroup", "Properties": { "GroupDescription": "Limits security group egress traffic", "SecurityGroupIngress": [ { "ToPort": 80, "CidrIp": "0.0.0.0/0", "IpProtocol": "tcp", "FromPort": 80 } ] } }, "MyLoadBalancer": { "Type": "AWS::ElasticLoadBalancing::LoadBalancer", "Properties": { "SecurityGroups": [ "sgwithingress" ] } } } } ```
Project: ClickHouse (App Definition and Development / Database)
File: v21.7.5.29-stable.md

sidebar_position: 1 sidebar_label: 2022 Backported in : Improve latency of short queries, that require reading from tables with large number of columns. (). Backported in : Fix zstd decompression in case there are escape sequences at the end of internal buffer. Closes . (). Backported in : Fix broken name resolution after rewriting column aliases. This fixes . (). Backported in : Fix issues with `CREATE DICTIONARY` query if dictionary name or database name was quoted. Closes . (). Backported in : Fix crash in rabbitmq shutdown in case rabbitmq setup was not started. Closes . (). Backported in : Update `chown` cmd check in clickhouse-server docker entrypoint. It fixes the bug that cluster pod restart failed (or timeout) on kubernetes. (). Backported in : Fix incorrect function names of groupBitmapAnd/Or/Xor. This fixes. (). Backported in : Fix potential nullptr dereference in window functions. This fixes . (). Backported in : Sometimes SET ROLE could work incorrectly, this PR fixes that. (). Backported in : Fix library-bridge ids load. (). Remove MySQLWireContext (). Update link to dpkg-deb in dockerfiles (). Try fix rabbitmq tests (). One more library bridge fix ().
Project: ClickHouse (App Definition and Development / Database)
File: v21.3.6.55-lts.md

sidebar_position: 1 sidebar_label: 2022 Backported in : If PODArray was instantiated with element size that is neither a fraction or a multiple of 16, buffer overflow was possible. No bugs in current releases exist. (). Backported in : Add profile event HedgedRequestsChangeReplica, change read data timeout from sec to ms. (). Backported in : Correctly check structure of async distributed blocks. (). Backported in : Fix very rare bug when quorum insert with `quorum_parallel=1` is not really "quorum" because of deduplication. (). Backported in : Check if table function view is used as a column. This complements https://github.com/ClickHouse/ClickHouse/pull/20350. (). Backported in : Follow-up fix for . Also fixes . (). Backported in : Fix usage of function `map` in distributed queries. (). Backported in : Fix wait for mutations on several replicas for ReplicatedMergeTree table engines. Previously, mutation/alter query may finish before mutation actually executed on other replicas. (). Backported in : Fix possible hangs in zk requests in case of OOM exception. Fixes . (). Backported in : Fix approx total rows accounting for reverse reading from MergeTree. (). Backported in : Fix pushdown of `HAVING` in case, when filter column is used in aggregation. (). Backported in : LIVE VIEW (experimental feature). Fix possible hanging in concurrent DROP/CREATE of TEMPORARY LIVE VIEW in `TemporaryLiveViewCleaner`, see https://gist.github.com/vzakaznikov/0c03195960fc86b56bfe2bc73a90019e. (). Backported in : Fixed a crash when using `mannWhitneyUTest` and `rankCorr` with window functions. This fixes . (). FileDictionarySource fix absolute file path ().
Project: SeaTunnel (App Definition and Development / Streaming & Messaging)
File: filter-rowkind.md

FilterRowKind transform plugin Filter the data by RowKind | name | type | required | default value | ||-|-|| | include_kinds | array | yes | | | exclude_kinds | array | yes | | The row kinds to include The row kinds to exclude. You can only config one of `includekinds` and `excludekinds`. Transform plugin common parameters, please refer to for details The RowKink of the data generate by FakeSource is `INSERT`, If we use `FilterRowKink` transform and exclude the `INSERT` data, we will write zero rows into sink. ```yaml env { job.mode = "BATCH" } source { FakeSource { resulttablename = "fake" row.num = 100 schema = { fields { id = "int" name = "string" age = "int" } } } } transform { FilterRowKind { sourcetablename = "fake" resulttablename = "fake1" exclude_kinds = ["INSERT"] } } sink { Console { sourcetablename = "fake1" } } ```
Project: KubeVirt (App Definition and Development / Application Definition & Image Build)
File: logging.md

For logging use the `Log` struct provided in the `log` package (see `pkg/log`). That struct offers several functions for providing logs with different log levels, verbosity and content. The Log struct offers several functions for logging with a decent log level: `Info(msg)`, `Warning(msg)`, `Error(msg)`, `Critical(msg)`. These functions also exist in a second flavor, where you can use a formatted string with additional arguments. Additionally you can set the verbosity with `V(verbosity)`, which takes effect on info level messages. Please check the for the meaning of the different log levels and verbosity. During runtime you will see: all info logs with a verbosity equal or lower to verbosity set by the `-v` command line flag all warning, error and critical logs If you don't set a verbosity for your info log statements, they will use the default value of `2`. Also, if you don't provide a `-v` command line flag, it will use a default of `2`. You can enhance the log statements with some helper functions: `Object(o)`: `o` has to be a Kubernetes resource, this will log the name, namespace, kind and uuid of the resource `With(...keyvals)`: logs the given key / value pairs `Reason(err)`: short for `With("reason", err)` `Key(name, kind)`: short for `With("name", name, "kind", kind)`, where given name can be in format `namespace/name`
Project: Dapr (App Definition and Development / Application Definition & Image Build)
File: v1.1.1.md

Add POST HTTP API for shutdown (https://github.com/dapr/dapr/pull/3020) If you are not using the HTTP shutdown API as part of the Kubernetes job support that was added in v1.1, you do not need to upgrade to 1.1.1. If you are using this feature, please read on. As a general rule, state mutating APIs should not be GET requests. The newly added shutdown API endpoint allows changing the state of the Dapr sidecar, by asking it to quit. We are adding a POST API that is functionally equivalent to the GET API. The GET API will be deprecated according to our . We strongly recommend that anyone looking to adopt Kubernetes job support via the HTTP shutdown API integrate with the POST API. Please see the for an example integration with the HTTP POST shutdown API.
Project: Chef InSpec (Provisioning / Security & Compliance)
File: ipnat.md

+++ title = "ipnat resource" draft = false gh_repo = "inspec" platform = "bsd" [menu] [menu.inspec] title = "ipnat" identifier = "inspec/resources/os/ipnat.md ipnat resource" parent = "inspec/resources/os" +++ Use the `ipnat` Chef InSpec audit resource to test rules that are defined for `IP NAT`. The purpose of the ipnat utility is to add or remove set of IP NAT rules. Rules are added to the end of the internal lists, matching the order in which they appear when given to ipnat. `ipnat -l` helps to view the current NAT table entry mappings. The rule match is done against the output rules of `ipnat -l`. {{< readfile file="content/inspec/reusable/md/inspec_installation.md" >}} An `ipnat` resource block declares tests for rules set for IP NAT: describe ipnat do it { should have_rule("RULE") } end where `have_rule('RULE')` tests the active rule for IP NAT. This must match the entire line taken from `ipnat -l`. The following examples show how to use this Chef InSpec audit resource. describe ipnat do it { should have_rule("map en0 192.0.0.0/8 -> 10.9.0.1/24") } end describe ipnat do it { should have_rule("map en0 192.0.0.0/8 -> 10.9.0.1/32 proxy port ftp ftp/tcp") } end {{< note >}} The rule specification must match the output of `ipnat -l` that depends on how you have built your rules. {{< /note >}} {{< readfile file="content/inspec/reusable/md/inspecmatcherslink.md" >}} This resource has the following special matchers. The `have_rule` matcher tests the named rule against the information in the output rule of `ipnat -l`: it { should have_rule("RULE") }
Project: Druid (App Definition and Development / Database)
File: thrift.md

id: thrift title: "Thrift" <!-- ~ Licensed to the Apache Software Foundation (ASF) under one ~ or more contributor license agreements. See the NOTICE file ~ distributed with this work for additional information ~ regarding copyright ownership. The ASF licenses this file ~ to you under the Apache License, Version 2.0 (the ~ "License"); you may not use this file except in compliance ~ with the License. You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, ~ software distributed under the License is distributed on an ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY ~ KIND, either express or implied. See the License for the ~ specific language governing permissions and limitations ~ under the License. --> To use this Apache Druid extension, `druid-thrift-extensions` in the extensions load list. This extension enables Druid to ingest thrift compact data online (`ByteBuffer`) and offline (SequenceFile of type `<Writable, BytesWritable>` or LzoThriftBlock File). You may want to use another version of thrift, change the dependency in pom and compile yourself. If you plan to read LZO-compressed Thrift files, you will need to download version 0.4.19 of the and place it in your `extensions/druid-thrift-extensions` directory. | Field | Type | Description | Required | | -- | -- | - | -- | | type | String | This should say `thrift` | yes | | parseSpec | JSON Object | Specifies the timestamp and dimensions of the data. Should be a JSON parseSpec. | yes | | thriftJar | String | path of thrift jar, if not provided, it will try to find the thrift class in classpath. Thrift jar in batch ingestion should be uploaded to HDFS first and configure `jobProperties` with `"tmpjars":"/path/to/your/thrift.jar"` | no | | thriftClass | String | classname of thrift | yes | Batch Ingestion example - `inputFormat` and `tmpjars` should be set. This is for batch ingestion using the HadoopDruidIndexer. The inputFormat of inputSpec in ioConfig could be one of `"org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat"` and `com.twitter.elephantbird.mapreduce.input.LzoThriftBlockInputFormat`. Be careful, when `LzoThriftBlockInputFormat` is used, thrift class must be provided twice. ```json { "type": "index_hadoop", "spec": { "dataSchema": { "dataSource": "book", "parser": { "type": "thrift", "jarPath": "book.jar", "thriftClass": "org.apache.druid.data.input.thrift.Book", "protocol": "compact", "parseSpec": { "format": "json", ... } }, "metricsSpec": [], "granularitySpec": {} }, "ioConfig": { "type": "hadoop", "inputSpec": { "type": "static", "inputFormat": "org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat", // "inputFormat": "com.twitter.elephantbird.mapreduce.input.LzoThriftBlockInputFormat", "paths": "/user/to/some/book.seq" } }, "tuningConfig": { "type": "hadoop", "jobProperties": { "tmpjars":"/user/huserprofile/du00/druid/test/book.jar", // "elephantbird.class.for.MultiInputFormat" : "${YOURTHRIFTCLASS_NAME}" } } } } ```
Project: Keycloak (Provisioning / Security & Compliance)
File: updating-database-schema.md

Updating Database Schema ======================== Keycloak supports automatically migrating the database to a new version. This is done by applying one or more change-sets to the existing database. This means if you need to do any changes to database schemas you need to create a change-set that can transform the schema as well as any existing data. This includes changes to: Realm entities User entities User session entities Event entities Creating a change-set We use Liquibase to support updating the database. The change-sets are located in . There's a separate file for each release that requires database changes. To manually create a change-set add a new file in the above location with the name `jpa-changelog-<version>.xml`. This file should contain a single `change-set` with `id` equal to the next version to be released and `author` set to the value `keycloak` . Then look at Liquibase documentation on how to write this file. Other option can be to look at existing changesets for the inspiration and create your changeset based on them. Add a reference to this file in . The file should have a single change-set and the id of the change-set should be the next version to be released. Testing database migration -- The first step may be to re-build Keycloak and check if server is started fully. This means that server can be started against empty DB with your changes. You can either use testing server based on embedded undertow or build fully distribution. See and for more details. Also see for testing on the more production non-H2 database. It is good idea to run DB migration afterwards. See for more details.