cwielenb/CNCF-Dataset huggingface dataset:
Project: StarRocks (App Definition and Development / Database)
File: append_trailing_char_if_absent.md

displayed_sidebar: "English" If the str string is not empty and does not contain trailingchar character in the end, it appends trailingchar character to the end. trailing_char can only contain one character. If it contains multiple characters, this function will return NULL. ```Haskell VARCHAR appendtrailingcharifabsent(VARCHAR str, VARCHAR trailing_char) ``` ```Plain Text MySQL [test]> select appendtrailingcharifabsent('a','c'); ++ |appendtrailingcharifabsent('a', 'c') | ++ | ac | ++ 1 row in set (0.02 sec) MySQL [test]> select appendtrailingcharifabsent('ac','c'); +-+ |appendtrailingcharifabsent('ac', 'c') | +-+ | ac | +-+ 1 row in set (0.00 sec) ``` APPENDTRAILINGCHARIFABSENT
Project: Nocalhost (App Definition and Development / Application Definition & Image Build)
File: CHANGELOG-6.0.md

See . 6.0 has removed support for the `_all` field. Only use `true` or `false` for boolean values, not `0` or `1` or `on` or `off`. Notice that 6.0 and future versions will default to single type indices, i.e. you may not use multiple types when e.g. adding an index with a mapping. See .
Project: Athenz (Provisioning / Key Management)
File: local_athenz_on_docker_ec2.md

In this introduction to Athenz, you will be able to run Athenz on your EC2 instance in AWS. AWS account with access to make changes to policies, IAM roles etc. Create a new policy with following JSON ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "sts:AssumeRole" ], "Resource": [ "arn:aws:iam:::role/" ] }, { "Effect": "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents", "logs:DescribeLogStreams", "logs:DescribeLogGroups" ], "Resource": [ "arn:aws:logs:::*" ] }, { "Effect": "Allow", "Action": [ "ec2:DescribeTags" ], "Resource": [ "*" ] } ] } ``` Create a new IAM role and associate the above created policy with it. !!! Note Make sure your VPC is set up with an Internet Gateway and necessary NACL, Route Table rules to allow EC2 instances in that VPC to be able to reach internet. Create a new EC2 instance in your VPC with the above created role as instance profile role. We will be using Amazon Linux 2 AMI and t2.2xlarge instance type since we will be running 5 docker containers for the purpose of this demo. SSH to the EC2 instance and run following commands to install git and docker - ```shell sudo yum update -y sudo yum install git sudo yum install docker -y sudo usermod -a -G docker ec2-user sudo service docker start ``` !!! Note You might have to log out and log back in for ec2-user's docker group membership to come into effect. Checkout Athenz from Github ```shell git clone https://github.com/AthenZ/athenz.git ``` `cd` to checked out directory and run following command: ```shell cd athenz && ./start-local-athenz.sh ``` This script will - download Athenz components docker images from DockerHub generate self-signed certificates to be used by Athenz components configure Athenz with meaningful defaults suitable for local environment ( for production set up, please refer to "Production set up" section of docs.) start local containers corresponding to Athenz components (ZMS, ZMS DB, ZTS, ZTS DB, UI)
Project: Beam (App Definition and Development / Streaming & Messaging)
File: 08_basic_windowing.md

Prompt: What is windowing in Apache Beam? Response: In stream processing, windowing allows you to divide data streams into logical units for efficient and correct parallel processing. With an unbounded data set, collecting all of the elements is impossible, because new elements are constantly being added. In the Beam model, any `PCollection` (including unbounded `PCollection` objects) can be subdivided into . Grouping transforms then consider the elements of each `PCollection` on a per-window basis. Because Apache Beam's default windowing strategy is to assign each element to a single, global window, you must explicitly specify a for your pipeline. The following code snippet shows how to divide a `PCollection` into 60-second windows: ```python from apache_beam import beam from apache_beam import window fixedwindoweditems = ( items | 'window' >> beam.WindowInto(window.FixedWindows(60))) ``` Beam provides a number of that you can use to subdivide your data into different types of windows: Fixed Time Windows Sliding Time Windows Per-Session Windows Single Global Window Calendar-based Windows (not supported by the Beam SDK for Python or Go) You can also create your own custom windowing function . You need to specify a to determine when to emit the results of your pipelines windowed computations. You can adjust the windowing strategy to allow for , which is data that arrives after the watermark has passed the end of the window. You can also specify how to handle late data, such as discarding or adding it to the next window.
Project: Apache Hadoop (App Definition and Development / Database)
File: RELEASENOTES.3.2.3.md

<! --> These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements. | Major | Install yarnpkg and upgrade nodejs in Dockerfile* In the Dockerfile, nodejs is upgraded to 8.17.0 and yarn 1.12.1 is installed. | Major | Update Dockerfile to use Bionic* The build image has been upgraded to Bionic. | Critical | [Hadoop 3] Both NameNodes can crash simultaneously due to the short JN socket timeout* The default value of the configuration hadoop.http.idle\_timeout.ms (how long does Jetty disconnect an idle connection) is changed from 10000 to 60000. This property is inlined during compile time, so an application that references this property must be recompiled in order for it to take effect. | Major | Migrate to Python 3 and upgrade Yetus to 0.13.0* <!-- markdown --> Upgraded Yetus to 0.13.0. Removed determine-flaky-tests-hadoop.py. Temporarily disabled shelldocs check in the Jenkins jobs due to YETUS-1099. | Major | Use spotbugs-maven-plugin instead of findbugs-maven-plugin* Removed findbugs from the hadoop build images and added spotbugs instead. Upgraded SpotBugs to 4.2.2 and spotbugs-maven-plugin to 4.2.0. | Major | Increase Quota initialization threads* The default quota initialization thread count during the NameNode startup process (dfs.namenode.quota.init-threads) is increased from 4 to 12.
Project: KICS (Provisioning / Security & Compliance)
File: 249328b8-5f0f-409f-b1dd-029f07882e11.md

title: Cluster Admin Rolebinding With Superuser Permissions hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 249328b8-5f0f-409f-b1dd-029f07882e11 Query name: Cluster Admin Rolebinding With Superuser Permissions Platform: Kubernetes Severity: <span style="color:#edd57e">Low</span> Category: Access Control URL: Ensure that the cluster-admin role is only used where required (RBAC)<br> ```yaml title="Positive test num. 1 - yaml file" hl_lines="11" kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: tiller-clusterrolebinding subjects: kind: ServiceAccount name: tiller namespace: kube-system roleRef: kind: ClusterRole name: cluster-admin apiGroup: "" ``` ```yaml title="Negative test num. 1 - yaml file" kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: tiller-clusterrolebinding subjects: kind: ServiceAccount name: tiller namespace: kube-system roleRef: kind: ClusterRole name: view apiGroup: "" ```
Project: secureCodeBox (Provisioning / Security & Compliance)
File: adr_0008.md

title: "ADR-0008: Drop the Former Decision To Provide Versioned Documentation" sidebar_label: "ADR-0008" | <!-- --> | <!-- --> | |-|-| | Status: | ACCEPTED (will SUPERSEDES ) | | Date: | 2021-07-15 | | Author(s): | Sven Strittmatter | In we decided to provide and maintain versions according to the major and minor versions of the secureCodeBox. Until now we never respected this decision on only provided a HEAD revision of the documentation. Since we have the plan to to a major breaking update to the secureCodeBox we have the urge to decide if we want to version our documentation or not because we must make breaking changes to the documentation, so that it will not match older versions of secureCodeBox. We do not provide any versions of the documentation! But we tag the documentation repository for major version increments, so that one looking for older documentation can checkout an older version. We do not want to invest the work to maintain more than the current version of the documentation. We do not want to encourage our customers to not upgrading secureCodeBox. We do not want to confuse our customers with old documentation maybe linked somewhere or preferred in search results. Someone using an old major version will have incompatible documentation. We strongly advise them to update always to the latest version. Someone using an old minor version will have features documented not available. We strongly advise them to update always to the latest version. We need to introduce a development branch to ensure that breaking changes are not visible on the site as long as it is not merged in the dev repository
Project: ko (App Definition and Development / Application Definition & Image Build)
File: ko_apply.md

Apply the input files with image references resolved to built/pushed image digests. This sub-command finds import path references within the provided files, builds them into Go binaries, containerizes them, publishes them, and then feeds the resulting yaml into "kubectl apply". ``` ko apply -f FILENAME [flags] ``` ``` ko apply -f config/ ko apply --preserve-import-paths -f config/ ko apply --local -f config/ cat config.yaml | ko apply -f - ko apply -f config -- --namespace=foo --kubeconfig=cfg.yaml ``` ``` --bare Whether to just use KODOCKERREPO without additional context (may not work properly with --tags). -B, --base-import-paths Whether to use the base path without MD5 hash after KODOCKERREPO (may not work properly with --tags). --disable-optimizations Disable optimizations when building Go code. Useful when you want to interactively debug the created container. -f, --filename strings Filename, directory, or URL to files to use to create the resource -h, --help help for apply --image-label strings Which labels (key=value) to add to the image. --image-refs string Path to file where a list of the published image references will be written. --insecure-registry Whether to skip TLS verification on the registry -j, --jobs int The maximum number of concurrent builds (default GOMAXPROCS) -L, --local Load into images to local docker daemon. --oci-layout-path string Path to save the OCI image layout of the built images --platform strings Which platform to use when pulling a multi-platform base. Format: all | <os>* -P, --preserve-import-paths Whether to preserve the full import path after KODOCKERREPO. --push Push images to KODOCKERREPO (default true) -R, --recursive Process the directory used in -f, --filename recursively. Useful when you want to manage related manifests organized within the same directory. --sbom string The SBOM media type to use (none will disable SBOM synthesis and upload, also supports: spdx, cyclonedx, go.version-m). (default "spdx") --sbom-dir string Path to file where the SBOM will be written. -l, --selector string Selector (label query) to filter on, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2) --tag-only Include tags but not digests in resolved image references. Useful when digests are not preserved when images are repopulated. -t, --tags strings Which tags to use for the produced image instead of the default 'latest' tag (may not work properly with --base-import-paths or --bare). (default [latest]) --tarball string File to save images tarballs ``` ``` -v, --verbose Enable debug logs ``` - Rapidly iterate with Go, Containers, and Kubernetes.
Project: KubeVela (App Definition and Development / Application Definition & Image Build)
File: alibaba-redis.eg.md

```yaml apiVersion: core.oam.dev/v1beta1 kind: Application metadata: name: redis-cloud-source spec: components: name: sample-redis type: alibaba-redis properties: instance_name: oam-redis account_name: oam password: Xyfff83jfewGGfaked writeConnectionSecretToRef: name: redis-conn ```
Project: Stolon (App Definition and Development / Database)
File: stolonctl_version.md

Display the version Display the version ``` stolonctl version [flags] ``` ``` -h, --help help for version ``` ``` --cluster-name string cluster name --kube-context string name of the kubeconfig context to use --kube-namespace string name of the kubernetes namespace to use --kube-resource-kind string the k8s resource kind to be used to store stolon clusterdata and do sentinel leader election (only "configmap" is currently supported) --kubeconfig string path to kubeconfig file. Overrides $KUBECONFIG --log-level string debug, info (default), warn or error (default "info") --metrics-listen-address string metrics listen address i.e "0.0.0.0:8080" (disabled by default) --store-backend string store backend type (etcdv2/etcd, etcdv3, consul or kubernetes) --store-ca-file string verify certificates of HTTPS-enabled store servers using this CA bundle --store-cert-file string certificate file for client identification to the store --store-endpoints string a comma-delimited list of store endpoints (use https scheme for tls communication) (defaults: http://127.0.0.1:2379 for etcd, http://127.0.0.1:8500 for consul) --store-key string private key file for client identification to the store --store-prefix string the store base prefix (default "stolon/cluster") --store-skip-tls-verify skip store certificate verification (insecure!!!) --store-timeout duration store request timeout (default 5s) ``` - stolon command line client
Project: StarRocks (App Definition and Development / Database)
File: sharedDataUseIntro.md

The usage of shared-data StarRocks clusters is also similar to that of a classic shared-nothing StarRocks cluster, except that the shared-data cluster uses storage volumes and cloud-native tables to store data in object storage. You can use the built-in storage volumes that StarRocks automatically creates, or you can manually create and set the default storage volume. This section describes how to manually create and set the default storage volume. NOTE If your shared-data StarRocks cluster is upgraded from v3.0, you do not need to define a default storage volume because StarRocks created one with the object storage-related properties you specified in the FE configuration file fe.conf. You can still create new storage volumes with other object storage resources and set the default storage volume differently. To give your shared-data StarRocks cluster permission to store data in your object storage, you must reference a storage volume when you create databases or cloud-native tables. A storage volume consists of the properties and credential information of the remote data storage. If you have deployed a new shared-data StarRocks cluster and disallow StarRocks to create a built-in storage volume (by specifying `enableloadvolumefromconf` as `false`), you must define a default storage volume before you can create databases and tables in the cluster.
Project: Quay (Provisioning / Container Registry)
File: README.md

UI for Quay based on React and Patternfly framework It is assumed that you have a Quay instance running that you can point the UI to. Run the following commands to get started ```bash git clone https://github.com/quay/quay-ui.git cd quay-ui npm install ``` Start the dev server by running ```bash npm start ``` Runs the app in the development mode.\ Open to view it in the browser. The page will reload if you make edits.\ You will also see any lint errors in the console. By default the UI connects to the quay backend for API. If you don't have a backend or want to develop without a backend you can set the environment variable `MOCK_API=true` before running `npm start`. In order for you to use this with a Quay backend, you need to configure CORS on the Quay side. Add the following to your `config.yaml` in Quay ```yaml CORS_ORIGIN: "http://localhost:9000" ``` If you are using `docker-compose` for local development, you can add this to `local-dev/stack/config.yaml` in the Quay repo. ```bash npm test ``` Launches the test runner in the interactive watch mode.\ See the section about for more information. Integration tests are ran via . The URL under test defaults to `http://localhost:9000` and can be overriden with the `baseUrl` parameter in the `cypress.config.ts` file. To run tests locally: Start the application with `npm start` When application has started run the tests with `npm run test:integration` ```bash npm run build ``` Builds the app for production to the `dist` folder.\ It correctly bundles React in production mode and optimizes the build for the best performance. The build is minified and the filenames include the hashes. See the section about for more information.
Project: KICS (Provisioning / Security & Compliance)
File: 4c137350-7307-4803-8c04-17c09a7a9fcf.md

title: Root Account Has Active Access Keys hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 4c137350-7307-4803-8c04-17c09a7a9fcf Query name: Root Account Has Active Access Keys Platform: CloudFormation Severity: <span style="color:#bb2124">High</span> Category: Insecure Configurations URL: The AWS Root Account must not have active access keys associated, which means if there are access keys associated to the Root Account, they must be inactive.<br> ```yaml title="Positive test num. 1 - yaml file" hl_lines="6" AWSTemplateFormatVersion: '2010-09-09' Resources: CFNKeys: Type: AWS::IAM::AccessKey Properties: UserName: Root ``` ```json title="Positive test num. 2 - json file" hl_lines="7" { "AWSTemplateFormatVersion": "2010-09-09", "Resources": { "CFNKeys": { "Type": "AWS::IAM::AccessKey", "Properties": { "UserName": "Root" } } } } ``` ```yaml title="Negative test num. 1 - yaml file" AWSTemplateFormatVersion: '2010-09-09' Resources: CFNKeys: Type: AWS::IAM::AccessKey Properties: UserName: MyUser ``` ```json title="Negative test num. 2 - json file" { "AWSTemplateFormatVersion": "2010-09-09", "Resources": { "CFNKeys": { "Type": "AWS::IAM::AccessKey", "Properties": { "UserName": "MyUser" } } } } ```
Project: Dgraph (App Definition and Development / Database)
File: thoughts.md

How to generate a unique list of uids by querying list of posting lists? Sol 1: Say there're k posting lists involved. One way to do so is to have a heap of k elements. At each iteration, we pop() an element from the heap (log k) Advance the pointer of that posting list, and retrieve another element (involves mutex read lock) Push() that element into the heap (log k) This would give us O(N*log k), with mutex lock acquired N times. With N=1000 and k=5, this gives us 1000 * ln(5) ~ 1600 Performance Improvements (memory tradeoff) [Sol1a]: We can alleviate the need for mutex locks by copying over all the posting list uids in separate vectors. This would avoid N lock acquisitions, only requiring the best-case scenario of k locks. But this also means all the posting list uids would be stored in memory. Performance with Memory [Sol1b]: Use k channels, with each channel only maintaining a buffer of say 1000 uids. In fact, keep the read lock acquired during this process, to avoid the posting list from changing during a query. So, basically have a way for a posting list to stream uids to a blocking channel, after having acquired a read lock. Overall this process of merging uids shouldn't take that long anyways; so this won't starve writes, only delay them. Another way [Sol2]: Pick a posting list, copy all it's uids in one go (one mutex lock) Use a binary tree to store uids. Eliminate duplicates. Iterate over each element in the uids vector, and insert into binary tree. [O(log N) max per insert] Repeat with other posting lists. This would give us O(N log N) complexity, with mutex lock acquired k times. With N=1000 and k=5, this gives us 1000 * ln(1000) ~ 7000 Not choosing this path. Solution: Sol1b
Project: LinuxKit (Provisioning / Automation & Configuration)
File: security-events.md

The incomplete list below is an assessment of some CVEs, and LinuxKit's resilience (or not) to them. : Requires CONFIGIPSCTP=y, which we do not set. : Requires CONFIGIPDCCP=y, which we do not set. (However, we were vulnerable to the ipv6 pieces that this patch fixes.) : This CVE requires `CONFIG_PRINTER=y`, so we are not vulnerable. (): This CVE requires `CONFIGNHDLC={y|m}`, which LinuxKit does not specify, and so is not vulnerable. This CVE only applies to kernels `<= 4.5, <= 4.4.21`. By using recent kernels (specifically, kernels `=> 4.9, >= 4.4.21`, LinuxKit mitigates this bug. : Requires CONFIGDRMVMWGFX=y, which we do not set. : Requires CONFIG_SOUND=y, which we do not set. : Requires the KVM backend (CONFIGKVM=y), and we only have CONFIGKVM_GUEST=y. Requires CONFIGDRMVIRTIO_GPU, which we do not set. aka XSA-216: we only have the XEN frontend, and do not set CONFIGXENBLKDEV_BACKEND. All users can do `accept()`, mitigated for kernels `>= 4.9.31, >= 4.10.16, >= 4.11.2` now packaged by LinuxKit Same as CVE-2017-8890, but for ipv6. : Users have access to ipv6 sockets, mitigated for kernels `>= 4.9.31, >= 4.10.16, >= 4.11.2` now packaged by LinuxKit : Same as CVE-2017-9074. : Users have access to ipv6 sockets (note that part of this is mitigated as well, so listed above: we do not set CONFIGIPDCCP), mitigated for kernels `>= 4.9.31, >= 4.10.16, >= 4.11.2` now packaged by LinuxKit : . Fixed in kernels `>= 4.9.35, >= 4.11.8`, now packaged by LinuxKit.
Project: KICS (Provisioning / Security & Compliance)
File: 5eccd62d-8b4d-46d3-83ea-1879f3cbd3ce.md

title: CA Certificate Identifier Is Outdated hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 5eccd62d-8b4d-46d3-83ea-1879f3cbd3ce Query name: CA Certificate Identifier Is Outdated Platform: Ansible Severity: <span style="color:#ff7213">Medium</span> Category: Encryption URL: The CA certificate Identifier must be 'rds-ca-2019'.<br> ```yaml title="Positive test num. 1 - yaml file" hl_lines="10 12" name: create minimal aurora instance in default VPC and default subnet group community.aws.rds_instance: engine: aurora dbinstanceidentifier: ansible-test-aurora-db-instance instance_type: db.t2.small password: "{{ password }}" username: "{{ username }}" cluster_id: ansible-test-cluster cacertificateidentifier: rds-ca-2015 name: create a DB instance using the default AWS KMS encryption key community.aws.rds_instance: id: test-encrypted-db state: present engine: mariadb storage_encrypted: True dbinstanceclass: db.t2.medium username: "{{ username }}" password: "{{ password }}" allocatedstorage: "{{ allocatedstorage }}" ``` ```yaml title="Negative test num. 1 - yaml file" name: create minimal aurora instance in default VPC and default subnet group community.aws.rds_instance: engine: aurora dbinstanceidentifier: ansible-test-aurora-db-instance instance_type: db.t2.small password: '{{ password }}' username: '{{ username }}' cluster_id: ansible-test-cluster cacertificateidentifier: rds-ca-2019 name: Create a DB instance using the default AWS KMS encryption key community.aws.rds_instance: id: test-encrypted-db state: present engine: mariadb storage_encrypted: true dbinstanceclass: db.t2.medium username: '{{ username }}' password: '{{ password }}' allocatedstorage: '{{ allocatedstorage }}' cacertificateidentifier: rds-ca-2019 ```
Project: LinuxKit (Provisioning / Automation & Configuration)
File: coding-style.md

Shell scripts should loosely follow the general Alpine style which is derived from the Linux Kernel guidelines, i.e. tabs for indentation etc. It's also useful to run `shellcheck` on the scripts. New Go code should be formatted with `gofmt` C code written from scratch should follow the as much as it makes sense for userspace code. You can check your code with like this: ``` checkpatch.pl --no-tree --file <sourcefile> ```