cwielenb/CNCF-Dataset huggingface dataset:
Project: KICS (Provisioning / Security & Compliance)
File: 675e8eaa-2754-42b7-bf33-bfa295d1601d.md

title: Nifcloud LB Use Insecure TLS Policy Name hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 675e8eaa-2754-42b7-bf33-bfa295d1601d Query name: Nifcloud LB Use Insecure TLS Policy Name Platform: Terraform Severity: <span style="color:#bb2124">High</span> Category: Insecure Configurations URL: The lb use insecure tls policy<br> ```tf title="Positive test num. 1 - tf file" hl_lines="1" resource "nifcloudloadbalancer" "positive" { loadbalancername = "example" instance_port = 443 loadbalancerport = 443 } ``` ```tf title="Positive test num. 2 - tf file" hl_lines="1" resource "nifcloudloadbalancer" "positive" { loadbalancername = "example" instance_port = 443 loadbalancerport = 443 sslpolicyname = "Standard Ciphers A ver1" } ``` ```tf title="Negative test num. 1 - tf file" resource "nifcloudloadbalancer" "negative" { loadbalancername = "example" instance_port = 443 loadbalancerport = 443 sslpolicyname = "Standard Ciphers D ver1" } ```
Project: KICS (Provisioning / Security & Compliance)
File: 9127f0d9-2310-42e7-866f-5fd9d20dcbad.md

title: Cluster Allows Unsafe Sysctls hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 9127f0d9-2310-42e7-866f-5fd9d20dcbad Query name: Cluster Allows Unsafe Sysctls Platform: Kubernetes Severity: <span style="color:#bb2124">High</span> Category: Insecure Configurations URL: A Kubernetes Cluster must not allow unsafe sysctls, to prevent a pod from having any influence on any other pod on the node, harming the node's health or gaining CPU or memory resources outside of the resource limits of a pod. This means 'spec.securityContext.sysctls' must not specify unsafe sysctls and the attribute 'allowedUnsafeSysctls' must be undefined.<br> ```yaml title="Positive test num. 1 - yaml file" hl_lines="24 11 13" apiVersion: v1 kind: Pod metadata: name: sysctl-example spec: securityContext: sysctls: name: kernel.shmrmidforced value: "0" name: net.core.somaxconn value: "1024" name: kernel.msgmax value: "65536" containers: name: test1 image: nginx apiVersion: policy/v1beta1 kind: PodSecurityPolicy metadata: name: sysctl-psp spec: allowedUnsafeSysctls: kernel.msg* forbiddenSysctls: kernel.shmrmidforced seLinux: rule: RunAsAny supplementalGroups: rule: RunAsAny runAsUser: rule: RunAsAny fsGroup: rule: RunAsAny ``` ```yaml title="Positive test num. 2 - yaml file" hl_lines="18" apiVersion: apps/v1 kind: Deployment metadata: name: test-app labels: app: test-app spec: selector: matchLabels: app: test-app template: metadata: labels: app: test-app spec: securityContext: sysctls: name: kernel.sem value: "128 32768 128 4096" containers: name: test-ubuntu image: ubuntu ``` ```yaml title="Negative test num. 1 - yaml file" apiVersion: v1 kind: Pod metadata: name: sysctl-example spec: securityContext: sysctls: name: kernel.shmrmidforced value: "0" name: net.ipv4.iplocalport_range value: "0" containers: name: test1 image: nginx apiVersion: policy/v1beta1 kind: PodSecurityPolicy metadata: name: sysctl-psp spec: forbiddenSysctls: kernel.shmrmidforced seLinux: rule: RunAsAny supplementalGroups: rule: RunAsAny runAsUser: rule: RunAsAny fsGroup: rule: RunAsAny ``` ```yaml title="Negative test num. 2 - yaml file" apiVersion: apps/v1 kind: Deployment metadata: name: test-app-neg labels: app: test-app-neg spec: selector: matchLabels: app: test-app-neg template: metadata: labels: app: test-app-neg spec: securityContext: sysctls: name: kernel.shmrmidforced value: "0" name: net/ipv4/tcp_syncookies value: "1" containers: name: test-ubuntu image: ubuntu ```
Project: KICS (Provisioning / Security & Compliance)
File: e7dada38-af20-4899-8955-dabea84ab1f0.md

title: Nifcloud Router Undefined Security Group To Router hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: e7dada38-af20-4899-8955-dabea84ab1f0 Query name: Nifcloud Router Undefined Security Group To Router Platform: Terraform Severity: <span style="color:#bb2124">High</span> Category: Networking and Firewall URL: Missing security group for router<br> ```tf title="Positive test num. 1 - tf file" hl_lines="1" resource "nifcloud_router" "positive" { network_interface { networkid = "net-COMMONGLOBAL" } } ``` ```tf title="Negative test num. 1 - tf file" resource "nifcloud_router" "negative" { securitygroup = nifcloudsecuritygroup.router.groupname network_interface { networkid = "net-COMMONGLOBAL" } } ```
Project: Flink (App Definition and Development / Streaming & Messaging)
File: kubernetes_ha.md

title: Kubernetes HA Services weight: 3 type: docs aliases: /deployment/ha/kubernetes_ha.html <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> Flink's Kubernetes HA services use for high availability services. Kubernetes high availability services can only be used when deploying to Kubernetes. Consequently, they can be configured when using or the In order to use Flink's Kubernetes HA services you must fulfill the following prerequisites: Kubernetes >= 1.9. Service account with permissions to create, edit, delete ConfigMaps. Take a look at how to configure a service account for and for more information. In order to start an HA-cluster you have to configure the following configuration keys: (required): The `high-availability.type` option has to be set to `kubernetes`. ```yaml high-availability.type: kubernetes ``` (required): JobManager metadata is persisted in the file system `high-availability.storageDir` and only a pointer to this state is stored in Kubernetes. ```yaml high-availability.storageDir: s3://flink/recovery ``` The `storageDir` stores all metadata needed to recover a JobManager failure. (required): In order to identify the Flink cluster, you have to specify a `kubernetes.cluster-id`. ```yaml kubernetes.cluster-id: cluster1337 ``` Configure high availability mode in : ```yaml kubernetes.cluster-id: <cluster-id> high-availability.type: kubernetes high-availability.storageDir: hdfs:///flink/recovery ``` {{< top >}} To keep HA data while restarting the Flink cluster, simply delete the deployment (via `kubectl delete deployment <cluster-id>`). All the Flink cluster related resources will be deleted (e.g. JobManager Deployment, TaskManager pods, services, Flink conf ConfigMap). HA related ConfigMaps will be retained because they do not set the owner reference. When restarting the cluster, all previously running jobs will be recovered and restarted from the latest successful checkpoint. {{< top >}}
Project: YDB (App Definition and Development / Database)
File: README_simple.md

<img alt="ContourPy" src="https://raw.githubusercontent.com/contourpy/contourpy/main/docs/static/contourpylogo_horiz.svg" height="90"> ContourPy is a Python library for calculating contours of 2D quadrilateral grids. It is written in C++11 and wrapped using pybind11. It contains the 2005 and 2014 algorithms used in Matplotlib as well as a newer algorithm that includes more features and is available in both serial and multithreaded versions. It provides an easy way for Python libraries to use contouring algorithms without having to include Matplotlib as a dependency. Documentation*: https://contourpy.readthedocs.io Source code*: https://github.com/contourpy/contourpy | | | | | | | Latest release | ](https://pypi.python.org/pypi/contourpy) ](https://anaconda.org/conda-forge/contourpy) ](https://anaconda.org/anaconda/contourpy) | | Downloads | ](https://pepy.tech/project/contourpy) ](https://anaconda.org/conda-forge/contourpy) ](https://anaconda.org/anaconda/contourpy) | | Python version | ](https://pypi.org/project/contourpy/) | | Coverage | ](https://app.codecov.io/gh/contourpy/contourpy) |
Project: secureCodeBox (Provisioning / Security & Compliance)
File: documentation_issue.md

name: " Documentation Issue" about: "Did you come across parts of our documentation that should be fixed?" labels: documentation <!-- Thank you for reporting an issue in our documentation Before opening a new issue, please make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead. --> <!-- Be as specific as possible by naming the document, page, and ideally paragraph. --> <!-- Please let us know what exactly is the issue with that part of the documentation --> <!-- If you have ideas on how to fix this issue, please note them here, or consider creating a Pull Request -->
Project: EMQ Technologies (App Definition and Development / Streaming & Messaging)
File: breaking-12576.en.md

Starting from 5.6, the "Configuration Manual" document will no longer include the `bridges` config root. A `bridge` is now either `action` + `connector` for egress data integration, or `source` + `connector` for ingress data integration. Please note that the `bridges` config (in `cluster.hocon`) and the REST API path `api/v5/bridges` still works, but considered deprecated.
Project: ClickHouse (App Definition and Development / Database)
File: v22.7.7.24-stable.md

sidebar_position: 1 sidebar_label: 2022 Backported in : - Choose correct aggregation method for LowCardinality with BigInt. (). Backported in : Update cctz to the latest master, update tzdb to 2020e. (). Backported in : Update tzdata to 2022e to support the new timezone changes. Palestine transitions are now Saturdays at 02:00. Simplify three Ukraine zones into one. Jordan and Syria switch from +02/+03 with DST to year-round +03. (https://data.iana.org/time-zones/tzdb/NEWS). This closes . (). Backported in : Fix reusing of files > 4GB from base backup. (). Backported in : Fix a bug with projections and the `aggregatefunctionsnullforempty` setting. This bug is very rare and appears only if you enable the `aggregatefunctionsnullforempty` setting in the server's config. This closes . (). Backported in : `ALTER UPDATE` of attached part (with columns different from table schema) could create an invalid `columns.txt` metadata on disk. Reading from such part could fail with errors or return invalid data. Fixes . (). Backported in : Setting `additionaltablefilters` were not applied to `Distributed` storage. Fixes . (). Backported in : This closes . (). Add a warning message to release.py script, require release type (). Revert ().
Project: KubeBlocks by ApeCloud (App Definition and Development / Database)
File: kbcli_cluster_list-events.md

title: kbcli cluster list-events List cluster events. ``` kbcli cluster list-events [flags] ``` ``` kbcli cluster list-events kbcli cluster list-events mycluster ``` ``` -A, --all-namespaces If present, list the requested object(s) across all namespaces. Namespace in current context is ignored even if specified with --namespace. -h, --help help for list-events -l, --selector string Selector (label query) to filter on, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2). Matching objects must satisfy all of the specified label constraints. ``` ``` --as string Username to impersonate for the operation. User could be a regular user or a service account in a namespace. --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups. --as-uid string UID to impersonate for the operation. --cache-dir string Default cache directory (default "$HOME/.kube/cache") --certificate-authority string Path to a cert file for the certificate authority --client-certificate string Path to a client certificate file for TLS --client-key string Path to a client key file for TLS --cluster string The name of the kubeconfig cluster to use --context string The name of the kubeconfig context to use --disable-compression If true, opt-out of response compression for all requests to the server --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure --kubeconfig string Path to the kubeconfig file to use for CLI requests. --match-server-version Require server version to match client version -n, --namespace string If present, the namespace scope for this CLI request --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0") -s, --server string The address and port of the Kubernetes API server --tls-server-name string Server name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used --token string Bearer token for authentication to the API server --user string The name of the kubeconfig user to use ``` - Cluster command.
Project: GraphScope (App Definition and Development / Database)
File: tutorial_networkx_algorithms.md

In the , we have introduced how to manipulate graph data with NetworkX APIs. In this tutorial, we will show how to use GraphScope to perform graph analysis like Networkx. Usually, the graph analysis process of NetworkX starts with the construction of a graph. In the following example, we create an empty graph first, and then expand the data through the graph manipulation interfaces of NetworkX. ```python import networkx G = networkx.Graph() G.addedgesfrom([(1, 2), (1, 3)]) G.add_node(4) ``` Then we can query the graph information. ```python G.numberofnodes() G.numberofedges() sorted(d for n, d in G.degree()) ``` Finally, we can call the builtin algorithms of NetworkX to analyze the graph `G`. ```python list(networkx.connected_components(G)) networkx.clustering(G) ``` To use NetworkX APIs from GraphScope, we just need to replace `import networkx as nx` with `import graphscope.nx as nx`. According to the , we use create a graph `nx.Graph()` first. ```python import graphscope graphscope.setoption(showlog=True) import graphscope.nx as nx G = nx.Graph() G.add_node(1) G.addnodesfrom([2, 3]) G.addnodesfrom([(4, {"color": "red"}), (5, {"color": "green"})]) G.add_edge(1, 2) e = (2, 3) G.add_edge(*e) G.addedgesfrom([(1, 2), (1, 3)]) G.addedgesfrom([(1, 2), (2, 3, {'weight': 3.1415})]) ``` The interface of graph analysis module in GraphScope is also compatible with NetworkX. With the above created graph in place, we use `connectedcomponents` to analyze the connected components of the graph, use `clustering` to get the clustering coefficient of each vertex, and `allpairsshortestpath` to compute the shortest path between any two vertices. ```python list(nx.connected_components(G)) nx.clustering(G) sp = dict(nx.allpairsshortest_path(G)) sp[3] ``` Like NetworkX, you can draw a graph by `draw` interface, which relies on the drawing function of `Matplotlib`. You should install `matplotlib` first, ```bash pip3 install matplotlib ``` Then you can draw a graph with ```python nx.draw(G, withlabels=True, fontweight='bold') ``` Let's see how much performance can be improved by GraphScope over NetworkX, compared with NetworkX, by running the `clustering` algorithm on . Download dataset if it is not in environment: ```bash wget https://raw.githubusercontent.com/GraphScope/gstest/master/twitter.e -P /tmp ``` Then load dataset both in GraphScope and NetworkX. ```python import os import graphscope.nx as gs_nx import networkx as nx g1 = nx.read_edgelist( os.path.expandvars('/tmp/twitter.e'), nodetype=int, data=False, create_using=nx.Graph ) type(g1) g2 = gsnx.readedgelist( os.path.expandvars('/tmp/twitter.e'), nodetype=int, data=False, createusing=gsnx.Graph ) type(g2) ``` Run algorithm and display time both in GraphScope and NetworkX. ```python %%time retgs = gsnx.clustering(g2) %%time ret_nx = nx.clustering(g1) retgs == retnx ```
Project: Distribution (Provisioning / Container Registry)
File: migrationguide.md

This guide is intended to assist in the migration from the `azure-storage-blob-go` module, or previous betas of `azblob`, to the latest releases of the `azblob` module. The redesign of the `azblob` module separates clients into various sub-packages. In previous versions, the public surface area was "flat", so all clients and supporting types were in the `azblob` package. This made it difficult to navigate the public surface area. In `azure-storage-blob-go` a client constructor always requires a `url.URL` and `Pipeline` parameters. In `azblob` a client constructor always requires a `string` URL, any specified credential type, and a `*ClientOptions` for optional values. You pass `nil` to accept default options. ```go // new code client, err := azblob.NewClient("<my storage account URL>", cred, nil) ``` In `azure-storage-blob-go` you created a `Pipeline` with the required credential type. This pipeline was then passed to the client constructor. In `azblob`, you pass the required credential directly to the client constructor. ```go // new code. cred is an AAD token credential created from the azidentity module client, err := azblob.NewClient("<my storage account URL>", cred, nil) ``` The `azure-storage-blob-go` module provided limited support for OAuth token authentication via `NewTokenCredential`. This been replaced by using Azure Identity credentials from . Authentication with a shared key via `NewSharedKeyCredential` remains unchanged. In `azure-storage-blob-go` you created a `Pipeline` with `NewAnonymousCredential` to support anonymous or SAS authentication. In `azblob` you use the construtor `NewClientWithNoCredential()` instead. ```go // new code client, err := azblob.NewClientWithNoCredential("<public blob or blob with SAS URL>", nil) ``` In `azure-storage-blob-go` you explicitly created a `Marker` type that was used to page over results (). In `azblob`, operations that return paginated values return a `*runtime.Pager[T]`. ```go // new code pager := client.NewListBlobsFlatPager("my-container", nil) for pager.More() { page, err := pager.NextPage(context.TODO()) // process results } ``` In `azure-storage-blob-go` you explicitly created a HTTP pipeline with configuration before creating a client. This pipeline instance was then passed as an argument to the client constructor (). In `azblob` a HTTP pipeline is created during client construction. The pipeline is configured through the `azcore.ClientOptions` type. ```go // new code client, err := azblob.NewClient(account, cred, &azblob.ClientOptions{ ClientOptions: azcore.ClientOptions{ // configure HTTP pipeline options here }, }) ```
Project: KICS (Provisioning / Security & Compliance)
File: 133fee21-37ef-45df-a563-4d07edc169f4.md

title: CMK Is Unusable hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 133fee21-37ef-45df-a563-4d07edc169f4 Query name: CMK Is Unusable Platform: Ansible Severity: <span style="color:#ff7213">Medium</span> Category: Availability URL: AWS Key Management Service (KMS) must only possess usable Customer Master Keys (CMK), which means the CMKs must have the attribute 'enabled' set to true and the attribute 'pending_window' must be undefined.<br> ```yaml title="Positive test num. 1 - yaml file" hl_lines="6" name: Update IAM policy on an existing KMS key1 community.aws.aws_kms: alias: my-kms-key policy: '{"Version": "2012-10-17", "Id": "my-kms-key-permissions", "Statement": [ { <SOME STATEMENT> } ]}' state: present enabled: false ``` ```yaml title="Positive test num. 2 - yaml file" hl_lines="6" name: Update IAM policy on an existing KMS key2 community.aws.aws_kms: alias: my-kms-key policy: '{"Version": "2012-10-17", "Id": "my-kms-key-permissions", "Statement": [ { <SOME STATEMENT> } ]}' state: present pending_window: 8 ``` ```yaml title="Negative test num. 1 - yaml file" name: Update IAM policy on an existing KMS key community.aws.aws_kms: alias: my-kms-key policy: '{"Version": "2012-10-17", "Id": "my-kms-key-permissions", "Statement": [ { <SOME STATEMENT> } ]}' state: present enabled: true ```
Project: Apache Hadoop (App Definition and Development / Database)
File: s3_select.md

<! Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> S3 Select is a feature for Amazon S3 introduced in April 2018. It allows for SQL-like SELECT expressions to be applied to files in some structured formats, including CSV and JSON. It is no longer supported in Hadoop releases. Any Hadoop release built on the will reject calls to open files using the select APIs. If a build of Hadoop with S3 Select is desired, the relevant classes can be found in hadoop trunk commit `8bf72346a59c`. The path capabilities probe `fs.s3a.capability.select.sql` returns "false" for any and all `s3a://` paths. Any `openFile()` call where a SQL query is passed in as a `must()` clause SHALL raise `UnsupportedOperationException`: ```java // fails openFile("s3a://bucket/path") .must("fs.s3a.select.sql", "SELECT ...") .get(); ``` Any `openFile()` call to an S3A Path where a SQL query is passed in as a `may()` clause SHALL be logged at WARN level the first time it is invoked, then ignored. ```java // ignores the option after printing a warning. openFile("s3a://bucket/path") .may("fs.s3a.select.sql", "SELECT ...") .get(); ``` The `hadoop s3guard select` command is no longer supported. Previously, the command would either generate an S3 select or a error (with exit code 42 being the one for not enough arguments): ``` hadoop s3guard select select [OPTIONS] [-limit rows] [-header (use|none|ignore)] [-out path] [-expected rows] [-compression (gzip|bzip2|none)] [-inputformat csv] [-outputformat csv] <PATH> <SELECT QUERY> make an S3 Select call [main] INFO util.ExitUtil (ExitUtil.java:terminate(241)) - Exiting with status 42: 42: Too few arguments ``` Now it will fail with exit code 55 always: ``` hadoop s3guard select [main] INFO util.ExitUtil (ExitUtil.java:terminate(241)) - Exiting with status 55: 55: S3 Select is no longer supported ```