cwielenb/CNCF-Dataset huggingface dataset:
Project: Beam (App Definition and Development / Streaming & Messaging)
File: hllcount.md

title: "HllCount" <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> <table align="left"> <a target="_blank" class="button" href="https://beam.apache.org/releases/javadoc/current/index.html?org/apache/beam/sdk/extensions/zetasketch/HllCount.html"> <img src="/images/logos/sdks/java.png" width="20px" height="20px" alt="Javadoc" /> Javadoc </a> </table> <br><br> Estimates the number of distinct elements in a data stream using the . The respective transforms to create and merge sketches, and to extract from them, are: `HllCount.Init` aggregates inputs into HLL++ sketches. `HllCount.MergePartial` merges HLL++ sketches into a new sketch. `HllCount.Extract` extracts the estimated count of distinct elements from HLL++ sketches. You can read more about what a sketch is at https://github.com/google/zetasketch. Example 1: creates a long-type sketch for a `PCollection<Long>` with a custom precision: {{< highlight java >}} PCollection<Long> input = ...; int p = ...; PCollection<byte[]> sketch = input.apply(HllCount.Init.forLongs().withPrecision(p).globally()); {{< /highlight >}} Example 2: creates a bytes-type sketch for a `PCollection<KV<String, byte[]>>`: {{< highlight java >}} PCollection<KV<String, byte[]>> input = ...; PCollection<KV<String, byte[]>> sketch = input.apply(HllCount.Init.forBytes().perKey()); {{< /highlight >}} Example 3: merges existing sketches in a `PCollection<byte[]>` into a new sketch, which summarizes the union of the inputs that were aggregated in the merged sketches: {{< highlight java >}} PCollection<byte[]> sketches = ...; PCollection<byte[]> mergedSketch = sketches.apply(HllCount.MergePartial.globally()); {{< /highlight >}} Example 4: estimates the count of distinct elements in a `PCollection<String>`: {{< highlight java >}} PCollection<String> input = ...; PCollection<Long> countDistinct = input.apply(HllCount.Init.forStrings().globally()).apply(HllCount.Extract.globally()); {{< /highlight >}} Example 5: extracts the count distinct estimate from an existing sketch: {{< highlight java >}} PCollection<byte[]> sketch = ...; PCollection<Long> countDistinct = sketch.apply(HllCount.Extract.globally()); {{< /highlight >}} estimates the number of distinct elements or values in key-value pairs (but does not expose sketches; also less accurate than `HllCount`).
Project: StarRocks (App Definition and Development / Database)
File: Query_management.md

displayed_sidebar: "English" `Property` is set for user granularity. To set the maximum number of connections between Client and FE, use the following command. ```sql SET PROPERTY [FOR 'user'] 'key' = 'value' [, 'key' = 'value'] ``` User properties include the resources assigned to the user. The properties set here are for the user, not `user_identity`. That is, if two users `jack'@'%` and `jack'@'192.%` are created by the `CREATE USER` statement, then the `SET PROPERTY` statement can work on the user `jack`, not `jack'@'%` or `jack'@'192.%`. Example 1: ```sql For the user `jack`, change the maximum number of connections to 1000 SET PROPERTY FOR 'jack' 'maxuserconnections' = '1000'; Check the connection limit for the root user SHOW PROPERTY FOR 'root'; ``` The session variables can be set by 'key' = 'value', which can limit the concurrency, memory and other query parameters in the current session. For example: parallelfragmentexecinstancenum The parallelism of the query with a default value of 1. It indicates the number of fragment instances on each BE. You can set this to half the number of CPU cores of the BE to improve query performance. querymemlimit Memory limit of a query on each BE node, can be adjusted when a query reports insufficient memory. loadmemlimit Memory limit for import, can be adjusted when an import job reports insufficient memory. Example 2: ```sql set parallelfragmentexecinstancenum = 8; set querymemlimit = 137438953472; ``` The capacity quota of database storage is unlimited by default. And you can change quota value by using `alter database`. ```sql ALTER DATABASE db_name SET DATA QUOTA quota; ``` The quota units are: B/K/KB/M/MB/G/GB/T/TB/P/PB Example 3: ```sql ALTER DATABASE example_db SET DATA QUOTA 10T; ``` To terminate a query on a particular connection with the following command: ```sql kill connection_id; ``` The `connectionid` can be seen by `show processlist;` or `select connectionid();`. ```plain text show processlist; ++++--++++-++ | Id | User | Host | Cluster | Db | Command | Time | State | Info | ++++--++++-++ | 1 | starrocksmgr | 172.26.34.147:56208 | defaultcluster | starrocksmonitor | Sleep | 8 | | | | 129 | root | 172.26.92.139:54818 | default_cluster | | Query | 0 | | | | 114 | test | 172.26.34.147:57974 | defaultcluster | ssb100g | Query | 3 | | | | 3 | starrocksmgr | 172.26.34.147:57268 | defaultcluster | starrocksmonitor | Sleep | 8 | | | | 100 | root | 172.26.34.147:58472 | defaultcluster | ssb100 | Sleep | 637 | | | | 117 | starrocksmgr | 172.26.34.147:33790 | defaultcluster | starrocksmonitor | Sleep | 8 | | | | 6 | starrocksmgr | 172.26.34.147:57632 | defaultcluster | starrocksmonitor | Sleep | 8 | | | | 119 | starrocksmgr | 172.26.34.147:33804 | defaultcluster | starrocksmonitor | Sleep | 8 | | | | 111 | root | 172.26.92.139:55472 | default_cluster | | Sleep | 2758 | | | ++++--++++-++ 9 rows in set (0.00 sec) mysql> select connection_id(); +--+ | CONNECTION_ID() | +--+ | 98 | +--+ mysql> kill 114; Query OK, 0 rows affected (0.02 sec) ```
Project: Meshery (Provisioning / Automation & Configuration)
File: cloudevents.md

layout: integration title: Cloudevents subtitle: Collaborative and visual infrastructure as code for Cloudevents image: /assets/img/integrations/cloudevents/icons/color/cloudevents-color.svg permalink: extensibility/integrations/cloudevents docURL: https://docs.meshery.io/extensibility/integrations/cloudevents description: integrations-category: App Definition and Development integrations-subcategory: Streaming & Messaging registrant: Artifact Hub components: featureList: [ "Provides SDKs for Go, JavaScript, Java, C#, Ruby, PHP, PowerShell, Rust, and Python that can be used to build event routers, tracing systems, and other tools.", "Simplifies event declaration and delivery across services, platforms, and beyond.", "Provides interoperability across services, platforms and systems." ] howItWorks: "Collaborative Infrastructure as Code" howItWorksDetails: "Collaboratively manage infrastructure with your coworkers synchronously sharing the same designs." language: en list: include type: extensibility category: integrations
Project: KICS (Provisioning / Security & Compliance)
File: 67fd0c4a-68cf-46d7-8c41-bc9fba7e40ae.md

title: Last User Is 'root' hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 67fd0c4a-68cf-46d7-8c41-bc9fba7e40ae Query name: Last User Is 'root' Platform: Dockerfile Severity: <span style="color:#bb2124">High</span> Category: Best Practices URL: Leaving the last user as root can cause security risks. Change to another user after running the commands the need privileges<br> ```dockerfile title="Positive test num. 1 - dockerfile file" hl_lines="2" FROM alpine:2.6 USER root RUN npm install ``` ```dockerfile title="Negative test num. 1 - dockerfile file" FROM alpine:2.6 USER root RUN npm install USER guest ``` ```dockerfile title="Negative test num. 2 - dockerfile file" FROM golang:1.16 AS builder WORKDIR /go/src/github.com/foo/href-counter/ RUN go get -d -v golang.org/x/net/html COPY app.go ./ RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app . USER root FROM alpine:latest RUN apk --no-cache add ca-certificates WORKDIR /root/ COPY --from=builder /go/src/github.com/foo/href-counter/app ./ CMD ["./app"] RUN useradd -ms /bin/bash patrick USER patrick ```
Project: CockroachDB (App Definition and Development / Database)
File: 20150729_range_replica_naming.md

Feature Name: rangereplicanaming Status: completed Start Date: 2015-07-29 RFC PR: Cockroach Issue: Consistently use the word range to refer to a portion of the global sorted map (i.e. an entire raft consensus group, and replica to refer to a single copy of the range's data. We currently use the word "range" informally to refer to both the consensus group and the data owned by one member of the group, while in code the `storage.Range` type refers to the latter. This was a deliberate at the time because (at least in code) the latter is more common than the former. However, resolving certain issues related to replication and splits requires us to be precise about the difference, so I propose separating the two usages to improve clarity. Rename the type `storage.Range` to `storage.Replica`, and `range_*.go` to `replica_*.go`. Rename `RaftID` to `RangeID` (the name `RaftID` was chosen to avoid the ambiguity inherent in our old use of "range"). A new `Range` type may be created to house pieces of `Replica` that do in fact belong at the level of the range (for example, the `AdminSplit`, `AdminMerge`, and `ChangeReplicas` methods) This reverses a previously-made decision and moves/renames a lot of code. Keep `Range` for a single replica and use `Raft` or `Group` when naming things that relate to the whole group.
Project: KICS (Provisioning / Security & Compliance)
File: e227091e-2228-4b40-b046-fc13650d8e88.md

title: User With Privilege Escalation By Actions 'iam:AttachRolePolicy' hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: e227091e-2228-4b40-b046-fc13650d8e88 Query name: User With Privilege Escalation By Actions 'iam:AttachRolePolicy' Platform: Terraform Severity: <span style="color:#ff7213">Medium</span> Category: Access Control URL: User with privilege escalation by actions 'iam:AttachRolePolicy' and Resource set to '*'. For more information see https://rhinosecuritylabs.com/aws/aws-privilege-escalation-methods-mitigation/.<br> ```tf title="Positive test num. 1 - tf file" hl_lines="1" resource "awsiamuser" "cosmic" { name = "cosmic" } resource "awsiamuserpolicy" "testinline_policy" { name = "testinlinepolicy" user = awsiamuser.cosmic.name policy = jsonencode({ Version = "2012-10-17" Statement = [ { Action = [ "iam:AttachRolePolicy", ] Effect = "Allow" Resource = "*" }, ] }) } resource "awsiampolicy_attachment" "test-attach" { name = "test-attachment" users = [awsiamuser.cosmic.name] roles = [awsiamrole.role.name] groups = [awsiamgroup.group.name] } ``` ```tf title="Negative test num. 1 - tf file" resource "awsiamuser" "cosmic2" { name = "cosmic2" } resource "awsiamuserpolicy" "inlinepolicyruninstances2" { name = "inlinepolicyrun_instances" user = awsiamuser.cosmic2.name policy = jsonencode({ Version = "2012-10-17" Statement = [ { Action = [ "ec2:Describe*", ] Effect = "Allow" Resource = "*" }, ] }) } ```
Project: Chef InSpec (Provisioning / Security & Compliance)
File: google_compute_network_endpoint_group.md

+++ title = "googlecomputenetworkendpointgroup resource" draft = false gh_repo = "inspec" platform = "gcp" [menu] [menu.inspec] title = "googlecomputenetworkendpointgroup" identifier = "inspec/resources/gcp/googlecomputenetworkendpointgroup.md googlecomputenetworkendpointgroup resource" parent = "inspec/resources/gcp" +++ A `googlecomputenetworkendpointgroup` is used to test a Google NetworkEndpointGroup resource This resource has beta fields available. To retrieve these fields, include `beta: true` in the constructor for the resource ```ruby describe googlecomputenetworkendpointgroup(project: 'chef-gcp-inspec', zone: 'zone', name: 'inspec-gcp-endpoint-group') do it { should exist } its('default_port') { should cmp '90' } end describe googlecomputenetworkendpointgroup(project: 'chef-gcp-inspec', zone: 'zone', name: 'nonexistent') do it { should_not exist } end ``` Properties that can be accessed from the `googlecomputenetworkendpointgroup` resource: `id` : The unique identifier for the resource. `name` : Name of the resource; provided by the client when the resource is created. The name must be 1-63 characters long, and comply with RFC1035. Specifically, the name must be 1-63 characters long and match the regular expression `?` which means the first character must be a lowercase letter, and all following characters must be a dash, lowercase letter, or digit, except the last character, which cannot be a dash. `description` : An optional description of this resource. Provide this property when you create the resource. `networkendpointtype` : Type of network endpoints in this network endpoint group. Possible values: GCEVMIP_PORT `size` : Number of network endpoints in the network endpoint group. `network` : The network to which all network endpoints in the NEG belong. Uses "default" project network if unspecified. `subnetwork` : Optional subnetwork to which all network endpoints in the NEG belong. `default_port` : The default port used if the port number is not specified in the network endpoint. `zone` : Zone where the network endpoint group is located. Ensure the is enabled for the current project.
Project: Tetragon (Provisioning / Security & Compliance)
File: CONTRIBUTING.md

Welcome to Kubernetes. We are excited about the prospect of you joining our ! The Kubernetes community abides by the CNCF . Here is an excerpt: As contributors and maintainers of this project, and in the interest of fostering an open and welcoming community, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities. We have full documentation on how to get started contributing here: <! If your repo has certain guidelines for contribution, put them here ahead of the general k8s resources --> Kubernetes projects require that you sign a Contributor License Agreement (CLA) before we can accept your pull requests - Main contributor documentation, or you can just jump directly to the - Common resources for existing developers - We have a diverse set of mentorship programs available that are always looking for volunteers! <! Custom Information - if you're copying this template for the first time you can add custom content here, for example: - Replace `kubernetes-users` with your slack channel string, this will send users directly to your channel. -->
Project: YugabyteDB (App Definition and Development / Database)
File: cr-dp-views.md

title: crdpviews.sql linkTitle: crdpviews.sql headerTitle: crdpviews.sql description: crdpviews.sql - Part of the code kit for the "Analyzing a normal distribution" section within the YSQL window functions documentation. menu: v2.18: identifier: cr-dp-views parent: analyzing-a-normal-distribution weight: 40 type: docs Save this script as `crdpviews.sql`. ```plpgsql create or replace view t4_view as select k, dp_score as score from t4; -- This very simple view allows updates. create or replace view results as select method, bucket, n, mins, maxs from dp_results; ```
Project: Apache Spark (App Definition and Development / Streaming & Messaging)
File: sql-ref-syntax-aux-set-var.md

layout: global title: SET VAR displayTitle: SET VAR license: | Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. The `SET VAR` command sets a temporary variable which has been previously declared in the current session. To set a config variable or a hive variable use . ```sql SET { VAR | VARIABLE } { { variable_name = { expression | DEFAULT } } [, ...] | ( variable_name [, ...] ) = ( query ) } ``` variable_name* Specifies an existing variable. If you specify multiple variables, there must not be any duplicates. expression* Any expression, including scalar subqueries. DEFAULT* If you specify `DEFAULT`, the default expression of the variable is assigned, or `NULL` if there is none. query* A that returns at most one row and as many columns as the number of specified variables. Each column must be implicitly castable to the data type of the corresponding variable. If the query returns no row `NULL` values are assigned. ```sql -- DECLARE VARIABLE var1 INT DEFAULT 7; DECLARE VARIABLE var2 STRING; -- A simple assignment SET VAR var1 = 5; SELECT var1; 5 -- A complex expression assignment SET VARIABLE var1 = (SELECT max(c1) FROM VALUES(1), (2) AS t(c1)); SELECT var1; 2 -- resetting the variable to DEFAULT SET VAR var1 = DEFAULT; SELECT var1; 7 -- A multi variable assignment SET VAR (var1, var2) = (SELECT max(c1), CAST(min(c1) AS STRING) FROM VALUES(1), (2) AS t(c1)); SELECT var1, var2; 2 1 -- Too many rows SET VAR (var1, var2) = (SELECT c1, CAST(c1 AS STRING) FROM VALUES(1), (2) AS t(c1)); Error: ROWSUBQUERYTOOMANYROWS -- No rows SET VAR (var1, var2) = (SELECT c1, CAST(c1 AS STRING) FROM VALUES(1), (2) AS t(c1) WHERE 1=0); SELECT var1, var2; NULL NULL ```
Project: Apache Heron (App Definition and Development / Streaming & Messaging)
File: compiling-osx.md

id: version-0.20.5-incubating-compiling-osx title: Compiling on OS X sidebar_label: Compiling on OS X original_id: compiling-osx <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> This is a step-by-step guide to building Heron on Mac OS X (versions 10.10 and 10.11). If isn't yet installed on your system, you can install it using this one-liner: ```bash $ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" ``` Bazelisk helps automate the management of Bazel versions ```bash brew install bazelisk ``` ```bash brew install automake brew install cmake brew install libtool brew install ant brew install pkg-config ``` ```bash $ export CC=/usr/bin/clang $ export CXX=/usr/bin/clang++ $ echo $CC $CXX ``` ```bash $ git clone https://github.com/apache/incubator-heron.git && cd incubator-heron ``` ```bash $ ./bazel_configure.py ``` If this configure script fails with missing dependencies, Homebrew can be used to install those dependencies. ```bash $ bazel build heron/... ``` This will build in the Bazel default `fastbuild` mode. Production release packages include additional performance optimizations not enabled by default. To enable production optimizations, include the `opt` flag. This defaults to optimization level `-O2`. The second option overrides the setting to bump it to `-CO3`. ```bash $ bazel build -c opt heron/... ``` ```bash $ bazel build -c opt --copt=-O3 heron/... ``` If you wish to add the code syntax style check, add `--config=stylecheck`. ```bash $ bazel build scripts/packages:binpkgs $ bazel build scripts/packages:tarpkgs ``` This will install Heron packages in the `bazel-bin/scripts/packages/` directory.
Project: ArangoDB (App Definition and Development / Database)
File: disabling_catchall.md

+++ title = "`basic_result(Args...) = delete`" description = "Disabling catchall constructor used to give useful diagnostic error when trying to use non-inplace constructors when `predicate::constructors_enabled` is false." categories = ["constructors", "disabling-constructors"] weight = 160 +++ Disabling catchall constructor used to give useful diagnostic error when trying to use non-inplace constructors when `predicate::constructors_enabled` is false. Requires: `predicate::constructors_enabled` is false. Complexity: N/A.
Project: DevSpace (App Definition and Development / Application Definition & Image Build)
File: cli.md

title: "devspace --help" sidebar_label: devspace Welcome to the DevSpace! DevSpace accelerates developing, deploying and debugging applications with Docker and Kubernetes. Get started by running the init command in one of your projects: devspace init devspace dev DEVSPACE_CONFIG=other-config.yaml devspace dev ``` --debug Prints the stack trace if an error occurs --disable-profile-activation If true will ignore all profile activations -h, --help help for devspace --inactivity-timeout int Minutes the current user is inactive (no mouse or keyboard interaction) until DevSpace will exit automatically. 0 to disable. Only supported on windows and mac operating systems --kube-context string The kubernetes context to use --kubeconfig string The kubeconfig path to use -n, --namespace string The kubernetes namespace to use --no-colors Do not show color highlighting in log output. This avoids invisible output with different terminal background colors --no-warn If true does not show any warning when deploying into a different namespace or kube-context than before --override-name string If specified will override the DevSpace project name provided in the devspace.yaml -p, --profile strings The DevSpace profiles to apply. Multiple profiles are applied in the order they are specified --silent Run in silent mode and prevents any devspace log output except panics & fatals -s, --switch-context Switches and uses the last kube context and namespace that was used to deploy the DevSpace project --var strings Variables to override during execution (e.g. --var=MYVAR=MYVALUE) ``` ``` ```
Project: OpenAPI (App Definition and Development / Application Definition & Image Build)
File: pull_request_template.md

<!-- Thank you for contributing to the OpenAPI Specification! Please make certain you are submitting your PR on the correct branch and file: 3.0.x spec: v3.0.4-dev branch, versions/3.0.4.md 3.1.x spec: v3.1.1-dev branch, versions/3.1.1.md 3.2.0 spec: v3.2.0-dev branch, versions/3.2.0.md 3.0 schema: main branch, schemas/v3.0/... 3.1 schema: main branch, schemas/v3.1/... registry templates: gh-pages branch, registry/... registry contents: gh-pages branch, registries/... Note that we do not accept changes to published specifications. -->