cwielenb/CNCF-Dataset huggingface dataset:
Project: Harbor (Provisioning / Container Registry)
File: 5-06-OVA-Configuration.md

Test 5-06 - OVA Configuration ======= To verify that the settings of an OVA version of Harbor can be configured and re-configured after power-on/off. User guide, Installation guide of Harbor OVA version. This test requires an OVA binary of Harbor. A vCenter, at least an ESX host, and a network that supports DHCP. A linux host with Docker CLI installed (Docker client). To-Do: verify all settings work at initial import verify all settings work after a hard power off followed by a power on Verify all settings can be changed as needed. (except networks, root password, Harbor password , mysql password) TBD None
Project: KICS (Provisioning / Security & Compliance)
File: e17fa86a-6222-4584-a914-56e8f6c87e06.md

title: Tiller Deployment Is Accessible From Within The Cluster hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: e17fa86a-6222-4584-a914-56e8f6c87e06 Query name: Tiller Deployment Is Accessible From Within The Cluster Platform: Kubernetes Severity: <span style="color:#bb2124">High</span> Category: Networking and Firewall URL: Check if any Tiller Deployment container allows access from within the cluster.<br> ```yaml title="Positive test num. 1 - yaml file" hl_lines="53 21" apiVersion: apps/v1 kind: Deployment metadata: labels: app: helm name: tiller name: tiller-bad-args spec: selector: matchLabels: name: tiller template: metadata: labels: app: helm name: tiller spec: containers: args: "--listen=10.7.2.8:44134" image: tiller-image name: tiller-v2 ports: containerPort: 44134 name: tiller protocol: TCP containerPort: 44135 name: http protocol: TCP serviceAccountName: tiller apiVersion: apps/v1 kind: Deployment metadata: labels: app: helm name: tiller name: tiller-deploy-no-args spec: selector: matchLabels: name: tiller template: metadata: labels: app: helm name: tiller spec: containers: name: tiller-v2 image: tiller-image serviceAccountName: tiller ``` ```yaml title="Negative test num. 1 - yaml file" apiVersion: apps/v1 kind: Deployment metadata: name: tiller-deploy labels: app: helm name: tiller spec: selector: matchLabels: app: helm name: tiller template: metadata: labels: app: helm name: tiller spec: serviceAccountName: tiller containers: name: tiller image: "tiller-image" args: ["--listen=127.0.0.1:44134"] ports: containerPort: 44134 name: tiller protocol: TCP containerPort: 44135 name: http protocol: TCP ```
Project: Grafeas (Provisioning / Security & Compliance)
File: design_principles.md

This page outlines the core principles that guide Grafeas's design: Universal coverage: Grafeas stores structured metadata against the software components unique identifier (e.g., container image digest), so you dont have to co-locate it with the components registry, and so it can store metadata about components from many different repositories. Hybrid cloud-friendly: Just as you can use JFrog Artifactory as the central, universal component repository across hybrid cloud deployments, you can use the Grafeas API as a central, universal metadata store. Pluggable: Grafeas makes it easy to add new metadata producers and consumers (for example, if you decide to add or change security scanners, add new build systems, etc.) Structured: Structured metadata schemas for common metadata types (e.g., vulnerability, build, attestation, and package index metadata) let you add new metadata types and providers, and the tools that depend on Grafeas can immediately understand those new sources. Strong access controls: Grafeas allows you to carefully control access for multiple metadata producers and consumers. Rich query-ability: With Grafeas, you can easily query all metadata across all of your components so you dont have to parse monolithic reports on each component.
Project: KICS (Provisioning / Security & Compliance)
File: integrations_bamboo.md

You can easily integrate KICS into Bamboo. The example below demonstrates how you can run KICS in a build plan linked to a GitHub repository. After you create a project with a build plan linked to a Github repository, you need to create a task to run KICS. The script body should contain `docker run -v ${PWD}:/path checkmarx/kics:latest scan -p /path -o /path/results --ci --ignore-on-exit results`. For example, to turn on this task every time a commit is merged in the Github repository, you need to create a plan branch that enables that. The build will run every time a commit is merged in the Github repository.
Project: Backstage (App Definition and Development / Application Definition & Image Build)
File: locations.md

id: locations title: GitLab Locations sidebar_label: Locations description: Integrating source code stored in GitLab into the Backstage catalog The GitLab integration supports loading catalog entities from gitlab.com or a self-hosted GitLab. Entities can be added to , or registered with the plugin. ```yaml integrations: gitlab: host: gitlab.com token: ${GITLAB_TOKEN} ``` :::note Note A public GitLab provider is added automatically at startup for convenience, so you only need to list it if you want to supply a . ::: Directly under the `gitlab` key is a list of provider configurations, where you can list the GitLab providers you want to fetch data from. Each entry is a structure with up to four elements: `host`: The host of the GitLab instance, e.g. `gitlab.company.com`. `token` (optional): An authentication token as expected by GitLab. The token need at least `api`, `readrepository` and `writerepository` scopes. If this is not supplied, anonymous access will be used. `apiBaseUrl` (optional): The URL of the GitLab API. For self-hosted installations, it is commonly at `https://<host>/api/v4`. For gitlab.com, this configuration is not needed as it can be inferred. `baseUrl` (optional): The base URL for this provider, e.g. `https://gitlab.com`. If this is not provided, it is assumed to be `https://{host}`.
Project: YugabyteDB (App Definition and Development / Database)
File: upgrade-software-prepare.md

title: Prepare to upgrade universes with a new version of YugabyteDB headerTitle: Prepare to upgrade a universe linkTitle: Prepare to upgrade description: Use YugabyteDB Anywhere to upgrade the YugabyteDB software on universes. headcontent: Review changes that may affect your automation menu: stable_yugabyte-platform: identifier: upgrade-software-prepare parent: upgrade-software weight: 10 type: docs If your universe is running on a , you need to update your OS before you can upgrade to the next major release of YugabyteDB. Refer to . Before starting the upgrade, review the following major changes in previous YugabyteDB releases. Depending on the upgrade you are planning, you may need to make changes to your automation. The YB Controller (YBC) service was introduced in v2.16.0 for all universes (except Kubernetes), and is required for YBA 2.16.0 and later. YBC is used to manage backup and restore, providing faster full backups, and introduces support for incremental backups. Impacts Firewall ports - update your firewall rules to allow incoming TCP traffic on port 18018, which is used by YBC, for all nodes in a universe. On-premises provider - if you use on-premises providers with manually-provisioned nodes, update your current procedures for manually provisioning instances to accommodate YBC. This includes the following: Set systemd-specific database service unit files (if used). Refer to . After upgrading nodes, manually install YBC on the nodes. Refer to . OS patching procedure - for universes created using an on-premises provider with manually-provisioned nodes, if your OS patching procedures involve re-installing YugabyteDB software on a node, you will need to update those procedures to accommodate YBC. YBC was introduced for Kubernetes clusters in v2.18.0. Refer to . The Node Agent was introduced for all universes in v2.18.2. Node agent is an RPC service running on a YugabyteDB node, and is used to manage communication between YBA and the nodes in universes. Except for Day 0 tasks during initial installation, YBA no longer uses SSH and SCP to manage nodes; instead, YBA connects to the Node agent process listening on port 9070, and performs all its management via this secure connection. For more information, refer to the . Impacts Firewall ports - update your firewall rules to allow incoming TCP traffic on port 9070 for all nodes in a universe. YugabyteDB Anywhere listens to node agents on port 443. On-premises provider - if you use on-premises providers with manually-provisioned nodes, you will need to update your current procedures for manually provisioning instances to include installing node agent. Refer to . procedure - for universes created using an on-premises provider with manually-provisioned nodes, if your OS patching procedures involve re-installing YugabyteDB software on a node, you will need to update those procedures to accommodate node agent.
Project: Beam (App Definition and Development / Streaming & Messaging)
File: 42_ai_custom_inference.md

Prompt: How can I use custom remote inference calls in my Apache Beam pipeline? Response: The optimal approach for conducting inference within an Apache Beam pipeline is by leveraging the provided by the Apache Beam Python SDK. This feature allows you to seamlessly incorporate ML models into your pipeline or execute remote inference calls. When developing custom inference calls, consider the following factors: API quotas*: heavy loads may lead to reaching API quota limits. You can use to specify the maximum number of parallel API calls. Use `directnumworkers` for the or `maxnumworkers` for the . Refer to the for information about other runners. Error handling*: it is essential to handle errors in case of API call failures. Consider implementing exponential backoff for retries or utilizing dead-letter queues for failed API calls. Monitoring*: incorporate monitoring and performance metrics to track the performance of your inference calls and the health of your pipeline. Batching*: batching can be used to send multiple inputs in a single API call for improved efficiency. To execute external API calls with the `RunInference` transform, you will need to create a `beam.DoFn` in the form of a custom model handler: ```python class CustomModelHandler(ModelHandler): """DoFn that accepts a batch of inputs and sends that batch to the remote API for inference""" def initialize_client(self): """Initiate the Custom remote API client.""" client = ... # Initialize the client return client def run_inference(self, batch, model, inference): client = self.initialize_client() inputs = ... # Process inputs from the batch input_requests = ... # Prepare input requests for the model batch_request = ... # Prepare batch request for the model responses = client.request(request=batch_request).responses return responses ``` Integrate this custom model handler into your pipeline as demonstrated in the following example: ```python with beam.Pipeline() as pipeline: _ = (pipeline | "Create inputs" >> beam.Create(<read input data>) | "Inference" >> RunInference(model_handler=CustomModelHandler()) | "Process outputs" >> beam.Map(<write output data>) ) ``` For a comprehensive example of using the RunInference API for remote inference calls, refer to the .
Project: KICS (Provisioning / Security & Compliance)
File: c2eae442-d3ba-4cb1-84ca-1db4f80eae3d.md

title: Lambda Function Without Dead Letter Queue hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: c2eae442-d3ba-4cb1-84ca-1db4f80eae3d Query name: Lambda Function Without Dead Letter Queue Platform: CloudFormation Severity: <span style="color:#edd57e">Low</span> Category: Insecure Configurations URL: AWS Lambda Function should be configured for a Dead Letter Queue(DLQ)<br> ```yaml title="Positive test num. 1 - yaml file" hl_lines="6" AWSTemplateFormatVersion: '2010-09-09' Description: VPC function. Resources: Function: Type: AWS::Lambda::Function Properties: Handler: index.handler Role: arn:aws:iam::123456789012:role/lambda-role Code: S3Bucket: my-bucket S3Key: function.zip Runtime: nodejs12.x Timeout: 5 TracingConfig: Mode: Active VpcConfig: SecurityGroupIds: sg-085912345678492fb SubnetIds: subnet-071f712345678e7c8 subnet-07fd123456788a036 Tags: Key: Description Value: VPC Function Key: Type Value: AWS Lambda Function ``` ```yaml title="Positive test num. 2 - yaml file" hl_lines="27 6" AWSTemplateFormatVersion: '2010-09-09' Description: VPC function. Resources: Function2: Type: AWS::Lambda::Function Properties: Handler: index.handler Role: arn:aws:iam::123456789012:role/lambda-role Code: S3Bucket: my-bucket S3Key: function.zip Runtime: nodejs12.x Timeout: 5 TracingConfig: Mode: Active VpcConfig: SecurityGroupIds: sg-085912345678492fb SubnetIds: subnet-071f712345678e7c8 subnet-07fd123456788a036 Tags: Key: Description Value: VPC Function Key: Type Value: AWS Lambda Function DeadLetterConfig: ``` ```yaml title="Negative test num. 1 - yaml file" AWSTemplateFormatVersion: '2010-09-09' Description: VPC function. Resources: Function3: Type: AWS::Lambda::Function Properties: Handler: index.handler Role: arn:aws:iam::123456789012:role/lambda-role Code: S3Bucket: my-bucket S3Key: function.zip Runtime: nodejs12.x Timeout: 5 TracingConfig: Mode: Active VpcConfig: SecurityGroupIds: sg-085912345678492fb SubnetIds: subnet-071f712345678e7c8 subnet-07fd123456788a036 Tags: Key: Description Value: VPC Function Key: Type Value: AWS Lambda Function DeadLetterConfig: TargetArn: arn:aws:sqs:us-east-1:2324243535:aaa ```
Project: Open Policy Agent (OPA) (Provisioning / Security & Compliance)
File: opa-wasm-dotnet.md

title: OPA Wasm .NET package software: csharp inventors: christophwille labels: category: wasm type: poweredbyopa tutorials: https://github.com/christophwille/dotnet-opa-wasm code: https://github.com/christophwille/dotnet-opa-wasm https://www.nuget.org/packages/Opa.Wasm/ docs_features: wasm-integration: note: 'This project implements the OPA Wasm module interface in C#. ' Call Rego policies in Wasm from C# .NET Core
Project: YDB (App Definition and Development / Database)
File: paging.md

This section provides recommendations for organizing paginated data output. To organize paginated output, we recommend selecting data sorted by primary key sequentially, limiting the number of rows with the LIMIT keyword. The query in listing 1 demonstrates the recommended way to organize paginated output. {% note info %} `$lastCity, $lastNumber`: Primary key values obtained from the previous query. {% endnote %} <small>Listing 1: Query for organizing paginated output</small> ```sql -- Table `schools`: -- -- | Name | Type | Key | -- -- | city | Utf8? | K0 | -- | number | Uint32? | K1 | -- | address | Utf8? | | -- DECLARE $limit AS Uint64; DECLARE $lastCity AS Utf8; DECLARE $lastNumber AS Uint32; SELECT * FROM schools WHERE (city, number) > ($lastCity, $lastNumber) ORDER BY city, number LIMIT $limit; ``` In the query example shown above, the `WHERE` clause uses a tuple comparison to select the next set of rows. Tuples are compared element by element from left to right, so the order of the fields in the tuple must match the order of the fields in the primary key to avoid table full scan. {% note warning "NULL value in key column" %} In {{ ydb-short-name }}, all columns, including key ones, may have a NULL value. Despite this, using NULL as key column values is highly discouraged, since the SQL standard doesn't allow NULL to be compared. As a result, concise SQL statements with simple comparison operators won't work correctly. Instead, you'll have to use cumbersome statements with IS NULL/IS NOT NULL expressions. {% endnote %} {% if oss %} {% endif %}
Project: KICS (Provisioning / Security & Compliance)
File: 826abb30-3cd5-4e0b-a93b-67729b4f7e63.md

title: RBAC Roles with Read Secrets Permissions hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 826abb30-3cd5-4e0b-a93b-67729b4f7e63 Query name: RBAC Roles with Read Secrets Permissions Platform: Terraform Severity: <span style="color:#ff7213">Medium</span> Category: Access Control URL: Roles and ClusterRoles with get/watch/list RBAC permissions on Kubernetes secrets are dangerous and should be avoided. In case of compromise, attackers could abuse these roles to access sensitive data, such as passwords, tokens and keys<br> ```tf title="Positive test num. 1 - tf file" hl_lines="48 9 27 62" resource "kubernetes_role" "example1" { metadata { name = "terraform-example1" labels = { test = "MyRole" } } rule { api_groups = [""] resources = ["secrets", "namespaces"] resource_names = ["foo"] verbs = ["get", "list", "watch"] } rule { api_groups = ["apps"] resources = ["deployments"] verbs = ["get", "list"] } } resource "kubernetesclusterrole" "example2" { metadata { name = "terraform-example2" } rule { api_groups = [""] resources = ["namespaces", "secrets"] verbs = ["get", "list", "watch"] } rule { api_groups = ["apps"] resources = ["deployments"] verbs = ["get", "list"] } } resource "kubernetes_role" "example3" { metadata { name = "terraform-example3" labels = { test = "MyRole" } } rule { api_groups = [""] resources = ["secrets", "namespaces"] resource_names = ["foo"] verbs = ["get", "list", "watch"] } } resource "kubernetesclusterrole" "example4" { metadata { name = "terraform-example4" } rule { api_groups = [""] resources = ["namespaces", "secrets"] verbs = ["get", "list", "watch"] } } ``` ```tf title="Negative test num. 1 - tf file" resource "kubernetes_role" "example1" { metadata { name = "terraform-example1" labels = { test = "MyRole" } } rule { api_groups = [""] resources = ["pods"] resource_names = ["foo"] verbs = ["get", "list", "watch"] } rule { api_groups = ["apps"] resources = ["deployments"] verbs = ["get", "list"] } } resource "kubernetesclusterrole" "example2" { metadata { name = "terraform-example2" } rule { api_groups = [""] resources = ["namespaces", "pods"] verbs = ["get", "list", "watch"] } } ```
Project: KubeBlocks by ApeCloud (App Definition and Development / Database)
File: bug_report.md

name: Bug report about: Create a report to help us improve title: "[BUG]" labels: kind/bug assignees: nayutah Describe the bug A clear and concise description of what the bug is. To Reproduce Steps to reproduce the behavior: Go to '...' Click on '....' Scroll down to '....' See error Expected behavior A clear and concise description of what you expected to happen. Screenshots If applicable, add screenshots to help explain your problem. Desktop (please complete the following information): OS: [e.g. iOS] Browser [e.g. chrome, safari] Version [e.g. 22] Additional context Add any other context about the problem here.
Project: KICS (Provisioning / Security & Compliance)
File: bc75ce52-a60a-4660-b533-bce837a5019b.md

title: BOM - GCP Redis hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: bc75ce52-a60a-4660-b533-bce837a5019b Query name: BOM - GCP Redis Platform: Terraform Severity: <span style="color:#CCCCCC">Trace</span> Category: Bill Of Materials URL: A list of Redis Instance resources found. Memorystore for Redis is a fully managed Redis service for Google Cloud. Applications running on Google Cloud can achieve extreme performance by leveraging the highly scalable, available, secure Redis service without the burden of managing complex Redis deployments.<br> ```tf title="Positive test num. 1 - tf file" hl_lines="1 20" resource "googleredisinstance" "cache" { name = "memory-cache" memorysizegb = 1 } resource "googlecomputeglobaladdress" "servicerange" { name = "address" purpose = "VPC_PEERING" address_type = "INTERNAL" prefix_length = 16 network = data.googlecomputenetwork.redis-network.id } resource "googleservicenetworkingconnection" "privateservice_connection" { network = data.googlecomputenetwork.redis-network.id service = "servicenetworking.googleapis.com" reservedpeeringranges = [googlecomputeglobaladdress.servicerange.name] } resource "googleredisinstance" "cache2" { name = "private-cache" tier = "STANDARD_HA" memorysizegb = 1 location_id = "us-central1-a" alternativelocationid = "us-central1-f" authorizednetwork = data.googlecompute_network.redis-network.id connectmode = "PRIVATESERVICE_ACCESS" redisversion = "REDIS4_0" display_name = "Terraform Test Instance" dependson = [googleservicenetworkingconnection.privateserviceconnection] } resource "googlecomputefirewall" "positive1" { name = "test-firewall" network = googlecomputenetwork.redis-network.name direction = "INGRESS" source_ranges = ["0.0.0.0/0"] allow { protocol = "icmp" } allow { protocol = "tcp" ports = ["22", "80", "3389", "8080", "1000-2000"] } source_tags = ["web"] } ``` ```tf title="Negative test num. 1 - tf file" resource "googlebigquerydataset" "negative1" { datasetid = "exampledataset" friendly_name = "test" description = "This is a test description" location = "EU" defaulttableexpiration_ms = 3600000 labels = { env = "default" } access { role = "OWNER" userbyemail = googleserviceaccount.bqowner.email } access { role = "READER" domain = "hashicorp.com" } } ```
Project: Monokle (App Definition and Development / Application Definition & Image Build)
File: how-to-browse-clusters.md

In this Monokle tutorial, we will illustrate the steps to connect to a cluster and navigate/update contained resources. Lets get started! Launch Monokle and ensure that the setting Show Cluster Selector box is checked. If this is the first time to configure a cluster in Monokle, you will see No Cluster Configured and the Configure button at the top of the screen: Clicking the Configure button opens the widget to specify the cluster to load by inserting its location: After clusters are configured, select the cluster you wish to work with from the drop down list at the top of the screen and click Load. Monokle will switch to the Cluster Mode, and the Navigator will reflect all the resources retrieved from the configured cluster. Select a resource to view its source code in the Editor. Editing resources in cluster mode allows quick changes to be made and deployed back to the cluster. Click the Diff button to analyze and compare the selected resource against the currently configured cluster. The comparison table will look like this: Click on the Deploy button to update the resource to the currently configured cluster. Then select the namespace for deployment. Select an existing namespace from the cluster, create a new namespace, or don't specify a namespace at all. By selecting None, Deploy will use the namespaces declared in the resources. The resource deployed dialog will be shown in the top-right corner of the console. Click on the Exit button on the top-right corner of the console to restore the resources of the currently selected folder. Please feel free to join our open source community on Discord with this and start your discussion.