cwielenb/CNCF-Dataset huggingface dataset:
Project: ArangoDB (App Definition and Development / Database)
File: blockStreaming_lineByLine.md

by Takayuki Matsuoka `blockStreaming_lineByLine.c` is LZ4 Straming API example which implements line by line incremental (de)compression. Please note the following restrictions : Firstly, read "LZ4 Streaming API Basics". This is relatively advanced application example. Output file is not compatible with lz4frame and platform dependent. Line by line incremental (de)compression. Handle huge file in small amount of memory Generally better compression ratio than Block API Non-uniform block size First of all, allocate "Ring Buffer" for input and LZ4 compressed data buffer for output. ``` (1) Ring Buffer +--+ | Line#1 | ++-+ | v {Out#1} (2) Prefix Mode Dependency +-+ | | v | +--+-++ | Line#1 | Line#2 | +--++-+ | v {Out#2} (3) Prefix Prefix +-+ +-+ | | | | v | v | +--+-++-++ | Line#1 | Line#2 | Line#3 | +--+--++-+ | v {Out#3} (4) External Dictionary Mode +-+ +-+ | | | | v | v | +--+-++-+--+ | .... | Line#X | Line#X+1 | +--+--+--+-+ ^ | | v | {Out#X+1} | Reset (5) Prefix +--+ | | v | +--+--+-+--+-+ | .... | Line#X | Line#X+1 | Line#X+2 | +--+--+-+--+-+ ^ | | v | {Out#X+2} | Reset ``` Next (see (1)), read first line to ringbuffer and compress it by `LZ4compresscontinue()`. For the first time, LZ4 doesn't know any previous dependencies, so it just compress the line without dependencies and generates compressed line {Out#1} to LZ4 compressed data buffer. After that, write {Out#1} to the file and forward ringbuffer offset. Do the same things to second line (see (2)). But in this time, LZ4 can use dependency to Line#1 to improve compression ratio. This dependency is called "Prefix mode". Eventually, we'll reach end of ringbuffer at Line#X (see (4)). This time, we should reset ringbuffer offset. After resetting, at Line#X+1 pointer is not adjacent, but LZ4 still maintain its memory. This is called "External Dictionary Mode". In Line#X+2 (see (5)), finally LZ4 forget almost all memories but still remains Line#X+1. This is the same situation as Line#2. Continue these procedure to the end of text file. Decompression will do reverse order. Read compressed line from the file to buffer. Decompress it to the ringbuffer. Output decompressed plain text line to the file. Forward ringbuffer offset. If offset exceedes end of the ringbuffer, reset it. Continue these procedure to the end of the compressed file.
Project: KubeDL (Provisioning / Automation & Configuration)
File: hostnetwork.md

Network bandwidth is a bottleneck resource for communication-intensive jobs. However, native container network model causes a loss to network performance, `hostnetwork` is an available solution to address this. In addition, other scenarios (e.g: nvlink communications between containerized gpu processes) may depend on host-network as well. KubeDL provide a feature-gate to enable `hostnetwork` mode for jobs, users only need to add an annotation `kubedl.io/network-mode: host` to job specifications, for example: ```yaml apiVersion: "training.kubedl.io/v1alpha1" kind: "TFJob" metadata: name: "mnist" namespace: kubedl annotations: kubedl.io/network-mode: 'host' spec: cleanPodPolicy: None tfReplicaSpecs: Worker: replicas: 3 ... ``` The critical path of implementing `hostnetwork-mode` is to randomize container ports replicas and make replicas accessible to each other. KubeDL achieves by following steps: enable `hostnetwork` in `Pod` spec and set DNS policy as `ClusterFirstWithHostNet`; choose a random port as container port. amend target port of corresponding `Service`, and set `CluterIP` as empty string(instead of `None`), so that kube-proxy will be able to forward traffic from `Port` to `TargetPort`. amend environments in job cluster spec. handling replica fail-over and patch latest available port to keep connection alive. here is a `TFJob` example: <div align="center"> <img src="./img/tf_hostnetwork.png" width="700" title="TF-HostNetwork"> </div> <br/>
Project: Lagoon (App Definition and Development / Application Definition & Image Build)
File: update-lagoon.md

Download newest charts using Helm. ```bash title="Download newest charts" helm repo update ``` Check with `helm diff` for changes (). ```bash title="Check for changes" helm diff upgrade --install --create-namespace --namespace lagoon-core \ -f values.yml lagoon-core lagoon/lagoon-core ``` the Lagoon databases prior to any Helm actions. We also suggest scaling the API to a single pod, to aid the database migration scripts running in the initContainers. Run the upgrade using Helm. ```bash title="Run upgrade" helm upgrade --install --create-namespace --namespace lagoon-core \ -f values.yaml lagoon-core lagoon/lagoon-core ``` (Note that as of Lagoon v2.11.0, this step is no longer required.) If upgrading Lagoon Core, ensure you run the `rerun_initdb.sh` script to perform post upgrade migrations. ```bash title="Run script" kubectl --namespace lagoon-core exec -it lagoon-core-api-db-0 -- \ sh -c /rerun_initdb.sh ``` Re-scale the API pods back to their original level. If upgrading Lagoon Core, and you have enabled groups/user syncing for OpenSearch, you may additionally need to run the `sync:opendistro-security` script to update the groups in OpenSearch. This command can also be prefixed with a `GROUP_REGEX=<group-to-sync` to sync a single group at a time, as syncing the entire group structure may take a long time. ```bash title="Run script" kubectl --namespace lagoon-core exec -it deploy/lagoon-core-api -- \ sh -c yarn sync:opendistro-security ``` Check for additional upgrades. You may want to back up the databases before upgrading Lagoon Core, the following will create backups you can use to restore from if required. You can delete them afterwards. ```bash title="Back up API DB" kubectl --namespace lagoon-core exec -it lagoon-core-api-db-0 -- \ sh -c 'mysqldump --max-allowed-packet=500M --events \ --routines --quick --add-locks --no-autocommit \ --single-transaction infrastructure | gzip -9 > \ /var/lib/mysql/backup/$(date +%Y-%m-%d_%H%M%S).infrastructure.sql.gz' ``` ```bash title="Back up Keycloak DB" kubectl --namespace lagoon-core exec -it lagoon-core-keycloak-db-0 -- \ sh -c 'mysqldump --max-allowed-packet=500M --events \ --routines --quick --add-locks --no-autocommit \ --single-transaction keycloak | gzip -9 > \ /var/lib/mysql/backup/$(date +%Y-%m-%d_%H%M%S).keycloak.sql.gz' ```
Project: StarRocks (App Definition and Development / Database)
File: stddev_samp.md

displayed_sidebar: "English" Returns the sample standard deviation of an expression. Since v2.5.10, this function can also be used as a window function. ```Haskell STDDEV_SAMP(expr) ``` `expr`: the expression. If it is a table column, it must evaluate to TINYINT, SMALLINT, INT, BIGINT, LARGEINT, FLOAT, DOUBLE, or DECIMAL. Returns a DOUBLE value. The formula is as follows, where `n` represents the row count of the table: ```plain text MySQL > select stddevsamp(scanrows) from log_statis group by datetime; +--+ | stddevsamp(`scanrows`) | +--+ | 2.372044195280762 | +--+ ``` STDDEV_SAMP,STDDEV,SAMP
Project: KubeDB by AppsCode (App Definition and Development / Database)
File: kubectl-dba_data_verify_mariadb.md

title: Kubectl-Dba Data Verify Mariadb menu: docs_{{ .version }}: identifier: kubectl-dba-data-verify-mariadb name: Kubectl-Dba Data Verify Mariadb parent: reference-cli menuname: docs{{ .version }} sectionmenuid: reference Verify rows to a mariadb resource Use this cmd to verify data in a mariadb object ``` kubectl-dba data verify mariadb [flags] ``` ``` kubectl dba data verify mariadb -n demo sample-mariadb --rows 1000 ``` ``` -h, --help help for mariadb -r, --rows int number of rows to verify (default 100) ``` ``` --as string Username to impersonate for the operation. User could be a regular user or a service account in a namespace. --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups. --as-uid string UID to impersonate for the operation. --cache-dir string Default cache directory (default "/home/runner/.kube/cache") --certificate-authority string Path to a cert file for the certificate authority --client-certificate string Path to a client certificate file for TLS --client-key string Path to a client key file for TLS --cluster string The name of the kubeconfig cluster to use --context string The name of the kubeconfig context to use --default-seccomp-profile-type string Default seccomp profile --disable-compression If true, opt-out of response compression for all requests to the server --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure --kubeconfig string Path to the kubeconfig file to use for CLI requests. --match-server-version Require server version to match client version -n, --namespace string If present, the namespace scope for this CLI request --password string Password for basic authentication to the API server --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0") -s, --server string The address and port of the Kubernetes API server --tls-server-name string Server name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used --token string Bearer token for authentication to the API server --user string The name of the kubeconfig user to use --username string Username for basic authentication to the API server ``` - Verify data in a database
Project: Carvel (App Definition and Development / Application Definition & Image Build)
File: README_Cognito_UserPools_PreSignup.md

The following is a sample Lambda function that receives Amazon Cognito User Pools pre-signup event as an input and writes some of the record data to CloudWatch Logs. (Note that anything written to stdout or stderr will be logged as CloudWatch Logs events.) Please see instructions for setting up the Cognito triggers at https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-identity-pools-working-with-aws-lambda-triggers.html . ```go package main import ( "fmt" "github.com/aws/aws-lambda-go/events" "github.com/aws/aws-lambda-go/lambda" ) // handler is the lambda handler invoked by the `lambda.Start` function call func handler(event events.CognitoEventUserPoolsPreSignup) (events.CognitoEventUserPoolsPreSignup, error) { fmt.Printf("PreSignup of user: %s\n", event.UserName) event.Response.AutoConfirmUser = true return event, nil } func main() { lambda.Start(handler) } ```
Project: KICS (Provisioning / Security & Compliance)
File: 32ecd6eb-0711-421f-9627-1a28d9eff217.md

title: OSLogin Disabled hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 32ecd6eb-0711-421f-9627-1a28d9eff217 Query name: OSLogin Disabled Platform: Terraform Severity: <span style="color:#ff7213">Medium</span> Category: Access Control URL: Verifies that the OSLogin is enabled<br> ```tf title="Positive test num. 1 - tf file" hl_lines="8 3" resource "googlecomputeproject_metadata" "positive1" { metadata = { enable-oslogin = false } } resource "googlecomputeproject_metadata" "positive2" { metadata = { foo = "bar" } } ``` ```tf title="Negative test num. 1 - tf file" resource "googlecomputeproject_metadata" "negative1" { metadata = { enable-oslogin = true } } ```
Project: Meshery (Provisioning / Automation & Configuration)
File: aws-vpc-cni.md

layout: integration title: AWS VPC CNI subtitle: Collaborative and visual infrastructure as code for AWS VPC CNI image: /assets/img/integrations/aws-vpc-cni/icons/color/aws-vpc-cni-color.svg permalink: extensibility/integrations/aws-vpc-cni docURL: https://docs.meshery.io/extensibility/integrations/aws-vpc-cni description: integrations-category: Cloud Native Network integrations-subcategory: Cloud Native Network registrant: Artifact Hub components: name: eni-config colorIcon: assets/img/integrations/aws-vpc-cni/components/eni-config/icons/color/eni-config-color.svg whiteIcon: assets/img/integrations/aws-vpc-cni/components/eni-config/icons/white/eni-config-white.svg description: name: policy-endpoint colorIcon: assets/img/integrations/aws-vpc-cni/components/policy-endpoint/icons/color/policy-endpoint-color.svg whiteIcon: assets/img/integrations/aws-vpc-cni/components/policy-endpoint/icons/white/policy-endpoint-white.svg description: featureList: [ "Keep revision history and audit trail of all configuration changes", "Use MeshMap&lsquo;s visual designer to explore your App Mesh configuration", "Deploy Meshery on your EKS clusters for App Mesh management" ] howItWorks: "Collaborative Infrastructure as Code" howItWorksDetails: "Collaboratively manage infrastructure with your coworkers synchronously sharing the same designs." language: en list: include type: extensibility category: integrations
Project: ArangoDB (App Definition and Development / Database)
File: outcome_print.md

+++ title = "`std::string print(const basic_outcome<T, EC, EP, NoValuePolicy> &)`" description = "Returns a string containing a human readable rendition of the `basic_outcome`." +++ Returns a string containing a human readable rendition of the `basic_outcome`. Overridable: Not overridable. Requires: Always available. Namespace: `BOOSTOUTCOMEV2_NAMESPACE` Header: `<boost/outcome/iostream_support.hpp>` (must be explicitly included manually).
Project: ArangoDB (App Definition and Development / Database)
File: bad_outcome_access.md

+++ title = "`badoutcomeaccess`" description = "Exception type publicly inheriting from `std::logic_error` indicating an incorrect observation of value or error or exception occurred." +++ Exception type publicly inheriting from {{% api "std::logicerror" %}} indicating an incorrect observation of value or error or exception occurred by {{% api "basicoutcome<T, EC, EP, NoValuePolicy>" %}}. No member functions are added in addition to `std::logic_error`. Typical `.what()` strings are: `"no value"` `"no error"` `"no exception"` Requires: Nothing. Namespace: `BOOSTOUTCOMEV2_NAMESPACE` Header: `<boost/outcome/bad_access.hpp>`
Project: ShardingSphere (App Definition and Development / Database)
File: stop-migration-check.en.md

+++ title = "STOP MIGRATION CHECK" weight = 13 +++ The `STOP MIGRATION CHECK` syntax is used to stop migration check process. {{< tabs >}} {{% tab name="Grammar" %}} ```sql StopMigrationCheck ::= 'STOP' 'MIGRATION' 'CHECK' migrationJobId migrationJobId ::= string ``` {{% /tab %}} {{% tab name="Railroad diagram" %}} <iframe frameborder="0" name="diagram" id="diagram" width="100%" height="100%"></iframe> {{% /tab %}} {{< /tabs >}} `migrationJobId` needs to be obtained through syntax query Stop migration check process ```sql STOP MIGRATION CHECK 'j010180026753ef0e25d3932d94d1673ba551'; ``` `STOP`, `MIGRATION`, `CHECK`
Project: ORY Hydra (Provisioning / Key Management)
File: CreateJsonWebKeySet.md

Name | Type | Description | Notes | - | - | - Alg | string | JSON Web Key Algorithm The algorithm to be used for creating the key. Supports &#x60;RS256&#x60;, &#x60;ES256&#x60;, &#x60;ES512&#x60;, &#x60;HS512&#x60;, and &#x60;HS256&#x60;. | Kid | string | JSON Web Key ID The Key ID of the key to be created. | Use | string | JSON Web Key Use The \&quot;use\&quot; (public key use) parameter identifies the intended use of the public key. The \&quot;use\&quot; parameter is employed to indicate whether a public key is used for encrypting data or verifying the signature on data. Valid values are \&quot;enc\&quot; and \&quot;sig\&quot;. | `func NewCreateJsonWebKeySet(alg string, kid string, use string, ) *CreateJsonWebKeySet` NewCreateJsonWebKeySet instantiates a new CreateJsonWebKeySet object This constructor will assign default values to properties that have it defined, and makes sure properties required by API are set, but the set of arguments will change when the set of required properties is changed `func NewCreateJsonWebKeySetWithDefaults() *CreateJsonWebKeySet` NewCreateJsonWebKeySetWithDefaults instantiates a new CreateJsonWebKeySet object This constructor will only assign default values to properties that have it defined, but it doesn't guarantee that properties required by API are set `func (o *CreateJsonWebKeySet) GetAlg() string` GetAlg returns the Alg field if non-nil, zero value otherwise. `func (o CreateJsonWebKeySet) GetAlgOk() (string, bool)` GetAlgOk returns a tuple with the Alg field if it's non-nil, zero value otherwise and a boolean to check if the value has been set. `func (o *CreateJsonWebKeySet) SetAlg(v string)` SetAlg sets Alg field to given value. `func (o *CreateJsonWebKeySet) GetKid() string` GetKid returns the Kid field if non-nil, zero value otherwise. `func (o CreateJsonWebKeySet) GetKidOk() (string, bool)` GetKidOk returns a tuple with the Kid field if it's non-nil, zero value otherwise and a boolean to check if the value has been set. `func (o *CreateJsonWebKeySet) SetKid(v string)` SetKid sets Kid field to given value. `func (o *CreateJsonWebKeySet) GetUse() string` GetUse returns the Use field if non-nil, zero value otherwise. `func (o CreateJsonWebKeySet) GetUseOk() (string, bool)` GetUseOk returns a tuple with the Use field if it's non-nil, zero value otherwise and a boolean to check if the value has been set. `func (o *CreateJsonWebKeySet) SetUse(v string)` SetUse sets Use field to given value.
Project: Apache Spark (App Definition and Development / Streaming & Messaging)
File: sql-ref-identifier.md

layout: global title: Identifiers displayTitle: Identifiers license: | Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. An identifier is a string used to identify a database object such as a table, view, schema, column, etc. Spark SQL has regular identifiers and delimited identifiers, which are enclosed within backticks. Both regular identifiers and delimited identifiers are case-insensitive. ```sql { letter | digit | '_' } [ , ... ] ``` Note: If `spark.sql.ansi.enforceReservedKeywords` is set to true, ANSI SQL reserved keywords cannot be used as identifiers. For more details, please refer to . ```sql `c [ ... ]` ``` letter* Any letter from A-Z or a-z. digit* Any numeral from 0 to 9. c* Any character from the character set. Use <code>`</code> to escape special characters (e.g., <code>`</code>). ```sql -- This CREATE TABLE fails with ParseException because of the illegal identifier name a.b CREATE TABLE test (a.b int); Error in query: [PARSESYNTAXERROR] Syntax error at or near '.': extra input '.'(line 1, pos 20) == SQL == CREATE TABLE test (a.b int) --^^^ -- This CREATE TABLE works CREATE TABLE test (`a.b` int); -- This CREATE TABLE fails with ParseException because special character ` is not escaped CREATE TABLE test1 (`a`b` int); Error in query: [PARSESYNTAXERROR] Syntax error at or near '`'(line 1, pos 24) == SQL == CREATE TABLE test1 (`a`b` int) ^^^ -- This CREATE TABLE works CREATE TABLE test (`a``b` int); ```
Project: YDB (App Definition and Development / Database)
File: 01_init.md

To interact with {{ ydb-short-name }}, create an instance of the driver, client, and session: The {{ ydb-short-name }} driver lets the app and {{ ydb-short-name }} interact at the transport layer. The driver must exist throughout the {{ ydb-short-name }} access lifecycle and be initialized before creating a client or session. The {{ ydb-short-name }} client runs on top of the {{ ydb-short-name }} driver and enables the handling of entities and transactions. The {{ ydb-short-name }} session contains information about executed transactions and prepared queries, and is part of the {{ ydb-short-name }} client context.
Project: KubeDB by AppsCode (App Definition and Development / Database)
File: kubectl-dba_data_insert_mongodb.md

title: Kubectl-Dba Data Insert Mongodb menu: docs_{{ .version }}: identifier: kubectl-dba-data-insert-mongodb name: Kubectl-Dba Data Insert Mongodb parent: reference-cli menuname: docs{{ .version }} sectionmenuid: reference Insert data to mongodb Use this cmd to insert data into a mongodb database. ``` kubectl-dba data insert mongodb [flags] ``` ``` kubectl dba data insert mg -n demo mg-rs --rows 500 ``` ``` -h, --help help for mongodb -r, --rows int number of rows to insert (default 100) ``` ``` --as string Username to impersonate for the operation. User could be a regular user or a service account in a namespace. --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups. --as-uid string UID to impersonate for the operation. --cache-dir string Default cache directory (default "/home/runner/.kube/cache") --certificate-authority string Path to a cert file for the certificate authority --client-certificate string Path to a client certificate file for TLS --client-key string Path to a client key file for TLS --cluster string The name of the kubeconfig cluster to use --context string The name of the kubeconfig context to use --default-seccomp-profile-type string Default seccomp profile --disable-compression If true, opt-out of response compression for all requests to the server --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure --kubeconfig string Path to the kubeconfig file to use for CLI requests. --match-server-version Require server version to match client version -n, --namespace string If present, the namespace scope for this CLI request --password string Password for basic authentication to the API server --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0") -s, --server string The address and port of the Kubernetes API server --tls-server-name string Server name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used --token string Bearer token for authentication to the API server --user string The name of the kubeconfig user to use --username string Username for basic authentication to the API server ``` - Insert random data in a database