cwielenb/CNCF-Dataset huggingface dataset:
Project: FOSSA (Provisioning / Security & Compliance)
File: detect-vendored.md

Some projects, especially C or C++ projects, vendor open source libraries in directories adjacent to the first party code being developed in-house. FOSSA's Vendored Source Identification is intended to support identifying and categorizing such vendored libraries. FOSSA fingerprints all files in your project and uploads those fingerprints to our analysis service. The analysis service then uses a proprietary algorithm to compare those fingerprints with the fingerprints in our database of open source projects. In the FOSSA CLI, this feature is called "VSI", for "vendored source identification". VSI can be enabled with the `--detect-vendored` flag when running `fossa analyze`. For example: ``` fossa analyze --detect-vendored ``` By default, VSI ignores the following directory: `{scandir}/.git` This feature does not have a concrete limit, however at this time FOSSA considers projects that have more than 35,000 files "unsupported". That being said, they may work. [!NOTE] Files are counted recursively, meaning that every archive contained in your project is unpacked and its contents are scanned, and each of those interior files count when talking about the 35,000 file supported limit. If you find that this is a significant issue in your usage of this feature, we'd love to hear more about your use case. Please consider dropping us a message through the ! VSI fingerprints your first party source code but does not send it to the server. Currently this is implemented with two SHA-256 hashes: A raw SHA-256 hash of the file content. For text files, a SHA-256 hash of the file content with lightweight comment and whitespace removal. Additionally, the project file paths are uploaded to FOSSA; this is required for the statistical analysis to function properly. The code to perform this is open source in this CLI; FOSSA can also provide a binary that displays the data FOSSA CLI uploads to the backend. VSI is more computationally intensive, and therefore takes longer to run. For this reason, we recommend only enabling it when you are reasonably confident that you will obtain useful information by enabling it. To explain, enabling VSI causes the CLI to: Fingerprint all files in the project Send those fingerprints to the FOSSA analysis service for vendored source identification Wait for remote analysis to complete This process has undergone a lot of optimization to improve its performance (and this is an area were continually investing in), but it will never be as fast as scanning local files for dependency information. Vendored dependencies can be manually specified in `fossa-deps.yml`. For more information, see in `features`.
Project: ShardingSphere (App Definition and Development / Database)
File: show-migration-list.en.md

+++ title = "SHOW MIGRATION LIST" weight = 7 +++ The `SHOW MIGRATION LIST` syntax is used to query migration job list. {{< tabs >}} {{% tab name="Grammar" %}} ```sql ShowMigrationList ::= 'SHOW' 'MIGRATION' 'LIST' ``` {{% /tab %}} {{% tab name="Railroad diagram" %}} <iframe frameborder="0" name="diagram" id="diagram" width="100%" height="100%"></iframe> {{% /tab %}} {{< /tabs >}} | Columns | Description | |-|--| | id | migration job id | | tables | migration tables | | jobitemcount | migration job sharding number | | active | migration job states | | create_time | migration job create time | | stop_time | migration job stop time | Query migration job list ```sql SHOW MIGRATION LIST; ``` ```sql mysql> SHOW MIGRATION LIST; +++-+--+++ | id | tables | jobitemcount | active | createtime | stoptime | +++-+--+++ | j01013a38b0184e07c864627b5bb05da09ee0 | t_order | 1 | false | 2022-10-31 18:18:24 | 2022-10-31 18:18:31 | +++-+--+++ 1 row in set (0.28 sec) ``` `SHOW`, `MIGRATION`, `LIST`
Project: YDB (App Definition and Development / Database)
File: group_by.md

title: "Overview of the GROUP BY operator in {{ ydb-full-name }}" description: "The article describes which functions are used with the GROUP BY operator in {{ ydb-full-name }}. We review examples of queries using the GROUP BY operator." {% if select_command == "SELECT STREAM" %} {% include %} {% include %} {% else %} {% include %} {% include %} {% if featuregroupbyrollupcube %} {% include %} {% endif %} {% include %} {% include %} {% include %} {% endif %}
Project: Goldilocks (Provisioning / Security & Compliance)
File: README.md

<div align="center" class="no-border"> <img src="/img/goldilocks.svg" height="150" alt="Goldilocks" style="padding-bottom: 20px" /> <br> <h3>Get your resource requests "Just Right"</h3> <a href="https://github.com/FairwindsOps/goldilocks/releases"> <img src="https://img.shields.io/github/v/release/FairwindsOps/goldilocks"> </a> <a href="https://goreportcard.com/report/github.com/FairwindsOps/goldilocks"> <img src="https://goreportcard.com/badge/github.com/FairwindsOps/goldilocks"> </a> <a href="https://circleci.com/gh/FairwindsOps/goldilocks.svg"> <img src="https://circleci.com/gh/FairwindsOps/goldilocks.svg?style=svg"> </a> <a href="https://codecov.io/gh/FairwindsOps/goldilocks"> <img src="https://codecov.io/gh/FairwindsOps/goldilocks/branch/master/graph/badge.svg"> </a> </div> Goldilocks is a utility that can help you identify a starting point for resource requests and limits. By using the kubernetes in recommendation mode, we can see a suggestion for resource requests on each of our apps. This tool creates a VPA for each workload in a namespace and then queries them for information. Once your VPAs are in place, you'll see recommendations appear in the Goldilocks dashboard: <div align="center"> <img src="/img/screenshot.png" alt="Goldilocks Screenshot" /> </div> urce Community The goal of the Fairwinds Community is to exchange ideas, influence the open source roadmap, and network with fellow Kubernetes users. or to get involved! <a href="https://insights.fairwinds.com/auth/register/"> <img src="https://www.fairwinds.com/hubfs/DocBanners/FairwindsOSSUserGroup740x125v6.png" alt="Love Fairwinds Open Source? Automate Fairwinds Open Source for free with Fairwinds Insights. Click to learn more" /> </a> Enjoying Goldilocks? Check out some of our other projects: - Audit, enforce, and build policies for Kubernetes resources, including over 20 built-in checks for best practices - Detect Kubernetes resources that have been deprecated or removed in future versions - Check to see if any of your Helm charts have updates available - Simplify the management of RBAC in your Kubernetes clusters Or If you're interested in running Goldilocks in multiple clusters, tracking the results over time, integrating with Slack, Datadog, and Jira, or unlocking other functionality, check out , a platform for auditing and enforcing policy in Kubernetes clusters. <a href="https://fairwinds.com/pricing"> <img src="https://www.fairwinds.com/hubfs/DocBanners/FairwindsGoldilocks_Ad.png" alt="Fairwinds Insights" /> </a>
Project: Meshery (Provisioning / Automation & Configuration)
File: v0.6.180.md

layout: release date: 2023-11-21 tag: v0.6.180 [Connection] Add connection & credential definitions for `Meshery` model. @MUzairS15 (#9409) [Server] Perform transitions in a separate routine @MUzairS15 (#9407) [chore] Run `gofmt`. @MUzairS15 (#9408) [chore] rm comments @theBeginner86 (#9404) Thank you to our contributors for making this release possible: @MUzairS15, @leecalcote and @theBeginner86
Project: EDB (App Definition and Development / Database)
File: wal_archiving.md

WAL archiving is the process that feeds a in CloudNativePG. !!! Important CloudNativePG currently only supports WAL archives on object stores. Such WAL archives serve for both object store backups and volume snapshot backups. The WAL archive is defined in the `.spec.backup.barmanObjectStore` stanza of a `Cluster` resource. Please proceed with the same instructions you find in the to set up the WAL archive. !!! Info Please refer to in the API reference for a full list of options. If required, you can choose to compress WAL files as soon as they are uploaded and/or encrypt them: ```yaml apiVersion: postgresql.cnpg.io/v1 kind: Cluster [...] spec: backup: barmanObjectStore: [...] wal: compression: gzip encryption: AES256 ``` You can configure the encryption directly in your bucket, and the operator will use it unless you override it in the cluster configuration. PostgreSQL implements a sequential archiving scheme, where the `archive_command` will be executed sequentially for every WAL segment to be archived. !!! Important By default, CloudNativePG sets `archive_timeout` to `5min`, ensuring that WAL files, even in case of low workloads, are closed and archived at least every 5 minutes, providing a deterministic time-based value for your Recovery Point Objective (RPO). Even though you change the value of the , our experience suggests that the default value set by the operator is suitable for most use cases. When the bandwidth between the PostgreSQL instance and the object store allows archiving more than one WAL file in parallel, you can use the parallel WAL archiving feature of the instance manager like in the following example: ```yaml apiVersion: postgresql.cnpg.io/v1 kind: Cluster [...] spec: backup: barmanObjectStore: [...] wal: compression: gzip maxParallel: 8 encryption: AES256 ``` In the previous example, the instance manager optimizes the WAL archiving process by archiving in parallel at most eight ready WALs, including the one requested by PostgreSQL. When PostgreSQL will request the archiving of a WAL that has already been archived by the instance manager as an optimization, that archival request will be just dismissed with a positive status.
Project: ClickHouse (App Definition and Development / Database)
File: lowcardinality.md

slug: /en/sql-reference/data-types/lowcardinality sidebar_position: 42 sidebar_label: LowCardinality(T) Changes the internal representation of other data types to be dictionary-encoded. ``` sql LowCardinality(data_type) ``` Parameters `data_type` , , , , and numbers excepting . `LowCardinality` is not efficient for some data types, see the setting description. `LowCardinality` is a superstructure that changes a data storage method and rules of data processing. ClickHouse applies to `LowCardinality`-columns. Operating with dictionary encoded data significantly increases performance of queries for many applications. The efficiency of using `LowCardinality` data type depends on data diversity. If a dictionary contains less than 10,000 distinct values, then ClickHouse mostly shows higher efficiency of data reading and storing. If a dictionary contains more than 100,000 distinct values, then ClickHouse can perform worse in comparison with using ordinary data types. Consider using `LowCardinality` instead of when working with strings. `LowCardinality` provides more flexibility in use and often reveals the same or higher efficiency. Create a table with a `LowCardinality`-column: ``` sql CREATE TABLE lc_t ( `id` UInt16, `strings` LowCardinality(String) ) ENGINE = MergeTree() ORDER BY id ``` Settings: Functions: . Blog: Blog:
Project: Fluvio (App Definition and Development / Streaming & Messaging)
File: TESTS.md

SPU = 1 Bring up SC Bring up SPU Create custom SPU 5001 SPU status should be online Bring down SPU SPU status should be offline Bring up SC and SPU SPU status should be up Kill SC first then SPU SPU status should still be up since SC didn't have change to receive status from SPU Bring back up SC SPU status should be change to offline after health check resync period Bring up SC and SPU Create 2 custom SPU 2nd SPU status is initially empty After while, 2nd SPU's status should be offline since we didn't bring up 2nd SPU yet
Project: KubeArmor (Provisioning / Security & Compliance)
File: workload_visibility.md

KubeArmor has visibility into systems and application behavior. KubeArmor summarizes/aggregates the information and provides a user-friendly view to figure out the application behavior. <img src="../.gitbook/assets/app-behavior.png" width="400" class="center" alt="App Behavior"> <img src="../.gitbook/assets/app-behavior2.png" width="784" class="center" alt="App Behavior2"> Process data*: What are the processes executing in the pods? What processes are executing through which parent processes? File data*: What are the file system accesses made by different processes? Network Accesses*: What are the Ingress/Egress connections from the pod? What server binds are done in the pod? ``` karmor logs -n default --json --logFilter all --operation process ``` Get visibility into process executions in `default` namespace. ```json { "Timestamp": 1686491023, "UpdatedTime": "2023-06-11T13:43:43.289380Z", "ClusterName": "default", "HostName": "ip-172-31-24-142", "NamespaceName": "default", "PodName": "nginx-8f458dc5b-fl42t", "Labels": "app=nginx", "ContainerID": "8762eafc25a35ab90089f79703b86659989e8e547c2c029fb60f55d884355000", "ContainerName": "nginx", "ContainerImage": "docker.io/library/nginx:latest@sha256:af296b188c7b7df99ba960ca614439c99cb7cf252ed7bbc23e90cfda59092305", "ParentProcessName": "/x86_64-bottlerocket-linux-gnu/sys-root/usr/bin/runc", "ProcessName": "/bin/sh", "HostPPID": 3488352, "HostPID": 3488357, "PPID": 3488352, "PID": 832, "Type": "ContainerLog", "Source": "/x86_64-bottlerocket-linux-gnu/sys-root/usr/bin/runc", "Operation": "Process", "Resource": "/bin/sh -c cat /run/secrets/kubernetes.io/serviceaccount/token", "Data": "syscall=SYS_EXECVE", "Result": "Passed" } { "Timestamp": 1686491023, "UpdatedTime": "2023-06-11T13:43:43.291471Z", "ClusterName": "default", "HostName": "ip-172-31-24-142", "NamespaceName": "default", "PodName": "nginx-8f458dc5b-fl42t", "Labels": "app=nginx", "ContainerID": "8762eafc25a35ab90089f79703b86659989e8e547c2c029fb60f55d884355000", "ContainerName": "nginx", "ContainerImage": "docker.io/library/nginx:latest@sha256:af296b188c7b7df99ba960ca614439c99cb7cf252ed7bbc23e90cfda59092305", "ParentProcessName": "/bin/dash", "ProcessName": "/bin/cat", "HostPPID": 3488357, "HostPID": 3488363, "PPID": 832, "PID": 838, "Type": "ContainerLog", "Source": "/bin/dash", "Operation": "Process", "Resource": "/bin/cat /run/secrets/kubernetes.io/serviceaccount/token", "Data": "syscall=SYS_EXECVE", "Result": "Passed" } ```
Project: Chef InSpec (Provisioning / Security & Compliance)
File: inspec_yml.md

+++ title = "inspec.yml File" draft = false gh_repo = "inspec" aliases = ["/inspec/profiles/inspec_yaml"] [menu] [menu.inspec] title = "inspec.yml" identifier = "inspec/profiles/inspec.yml" parent = "inspec/profiles" weight = 20 +++ The `inspec.yml` metadata file describes a profile. This file is required. The `inspec.yml` file defines the following information: `name` : A unique name for the profile. Required `title` : A human-readable name for the profile. Optional `maintainer` : The profile maintainer. Optional `copyright` : The copyright holder. Optional `copyright_email` : The support contact information for the profile, typically an email address. Optional `license` : The license for the profile. Optional `summary` : A one line summary for the profile. Optional `description` : A multiple line description of the profile. Optional `version` : The profile version. Optional `inspec_version` : The SemVer version of Chef InSpec that the profile can run under. Optional `supports` : A list of targets. Optional `depends` : A list of . Optional `inputs` : A list of you can use in your controls. Optional `gem_dependencies` : A list of profile that must be installed for the profile to function correctly. Optional ```yaml name: ssh title: Basic SSH maintainer: Chef Software, Inc. copyright: Chef Software, Inc. copyright_email: support@chef.io license: Proprietary, All rights reserved summary: Verify that SSH Server and SSH Client are configured securely version: 1.0.0 supports: platform-family: linux inputs: name: input_name required: true description: 'Input description.' value: 10 priority: 50 depends: name: profile path: ../path/to/profile gem_dependencies: name: "gem-name" version: ">= 2.0.0" inspec_version: "~> 2.1" ``` The `inspec.yml` also supports embedded ERB. ```yaml name: dummy title: InSpec Profile maintainer: The Authors copyright: The Authors copyright_email: you@example.com license: Apache-2.0 summary: An InSpec Compliance Profile version: 0.1.0 depends: name: inherit url: "https://artifactory.com/artifactory/example-repo-local/inspec/0.4.1.tar.gz" username: <%= ENV['USERNAME'] %> password: <%= ENV['API_KEY'] %> ```
Project: Kapitan (Provisioning / Automation & Configuration)
File: kap_9_bring_your_own_helm.md

Currently the helm binding can't be run on Mac OSX. Attempts to fix this have been made on several occasions: <https://github.com/kapicorp/kapitan/pull/414> <https://github.com/kapicorp/kapitan/pull/547> <https://github.com/kapicorp/kapitan/pull/568> There are some issues with the current bindings besides the lack of Mac OSX support. The golang runtime (1.14) selected will effect older versions helm templates: <https://github.com/helm/helm/issues/7711>. Users can't select the version of helm they'd like to use for templating. Users supply their own helm binary. This allows them to control the version of golang runtime and version of helm they'd like to use. In Kapitan we could rewrite the interface to use and perform commands. The cli of helm 2 vs helm 3 is slightly different but shouldn't be difficult to codify. This would be great to get rid of cffi and golang which will reduce complexity and build time of the project. Depending on how this goes, this could pave the way for a "bring your own binary" input type.
Project: Lagoon (App Definition and Development / Application Definition & Image Build)
File: triggering-deployments.md

In order to automatically trigger new deployments using follow these instructions: Add your deployment SSH private key to Azure as a secure file as `idrsalagoon`. For more information about secure files have a look at the . Add the following configuration to your `azure-pipelines.yml`: ```yaml title="azure-pipelines.yml" pool: vmImage: 'ubuntu-latest' stages: stage: Deploy condition: and(succeeded(), in(variables['Build.SourceBranch'], 'refs/heads/staging', 'refs/heads/develop')) jobs: job: DeployLagoon steps: task: DownloadSecureFile@1 name: lagoonSshKey displayName: 'Download Lagoon SSH key' inputs: secureFile: idrsalagoon script: | curl -L "https://github.com/amazeeio/lagoon-cli/releases/download/0.9.2/lagoon-cli-0.9.2-linux-amd64" -o ./lagoon chmod +x ./lagoon displayName: 'Download lagoon-cli' script: ./lagoon login -i $(lagoonSshKey.secureFilePath) displayName: 'Log into Lagoon' script: ./lagoon deploy branch -e $(Build.SourceBranchName) -p my-awesome-project -b $(Build.SourceBranchName) --force displayName: 'Trigger deployment using lagoon-cli' ``` This will trigger a new deployment whenever changes are made on the `develop` or `staging` branch. Adjust these values accordingly so they fit your deployment strategy and configuration. There may be a case where you want to push without a deployment. Make sure your commit message contains "`[skip deploy]`" or "`[deploy skip]`" and Lagoon will not trigger a deployment from that commit.
Project: YugabyteDB (App Definition and Development / Database)
File: performances.md

Performances =============================================================================== Any anonymization process has a price as it will consume CPU time, RAM space and probably a bunch of disk I/O... Here's a a quick overview of the question depending on what strategy you are using.... In a nutshell, the anonymization performances will mainly depend on 2 important factors: The size of the database The number of masking rules Static Masking Basically what static masking does it rewrite entirely the masked tables on disk. This may be slow depending on your environment. And during this process, the tables will be locked. As an example: Anonymizing a 44GB database with 29 masking rules on an AWS EC2 instance takes approximately 25 minutes (see [MR 107] for more details). In this case, the cost of anonymization is "paid" by all the users but it is paid once and for all. Dynamic Masking With dynamic masking, the real data is replaced on-the-fly every time a masked user sends a query to the database. This means that the masking users will have slower response time than regular (unmasked) users. This is generally ok because usually masked users are not considered as important as the regular ones. If you apply 3 or 4 rules to a table, the response time for the masked users should approx. 20% to 30% slower than for the normal users. As the masking rules are applied for each queries of the masked users, the dynamic masking is appropriate when you have a limited number of masked users that connect only from time to time to the database. For instance, a data analyst connecting once a week to generate a business report. If there are multiple masked users or if a masked user is very active, you should probably export the masked data once-a-week on a secondary instance and let these users connect to this secondary instance. In this case, the cost of anonymization is "paid" only by the masked users. Anonymous Dumps Some benchmarks made in march 2022 suggest that the `pgdumpanon` wrapper is twice as slow as the regular `pg_dump` tool. If the backup process of your database takes 1 hour with `pg_dump`, then anonymizing and exporting the entire database with `pgdumpanon` will probably take 2 hours. In this case, the cost of anonymization is "paid" by the user asking for the anonymous export. Other users of the database will not be affected. How to speed things up ? It is always faster to replace the original data with a static value instead of calling a masking function. If you need to anonymize data for testing purpose, chances are that a smaller subset of your database will be enough. In that case, you can easily speed up the anonymization by downsizing the volume of data. Checkout the [Sampling] section for more details. Dynamic masking is not always required! In some cases, it is more efficient to build [Materialized Views] instead. For instance: ```sql CREATE MATERIALIZED VIEW masked_customer AS SELECT id, anon.randomlastname() AS name, anon.randomdatebetween('1920-01-01'::DATE,now()) AS birth, fklastorder, store_id FROM customer; ```
Project: StarRocks (App Definition and Development / Database)
File: SHOW_REPOSITORIES.md

displayed_sidebar: "English" Views the repositories created in StarRocks. ```SQL SHOW REPOSITORIES ``` | Return | Description | | - | -- | | RepoId | Repository ID. | | RepoName | Repository name. | | CreateTime | Repository creation time. | | IsReadOnly | If the repository is read-only. | | Location | Location of the repository in the remote storage system. | | Broker | Broker used to create the repository. | | ErrMsg | Error message during connection to the repository. | Example 1: Views the repositories created in StarRocks. ```SQL SHOW REPOSITORIES; ```