cwielenb/CNCF-Dataset huggingface dataset:
Project: Beam (App Definition and Development / Streaming & Messaging)
File: 02_basic_pipelines.md

Prompt: What is a Pipeline in Apache Beam? Response: In Apache Beam, a encapsulates the entirety of a data processing task, including all the data and each step of the process. Essentially, it's a of transformations (known as `PTransform`) applied to data collections (`PCollection`). The simplest pipelines in Apache Beam follow a linear flow of operations, typically adhering to a read-process-write pattern. However, pipelines can also be significantly more complex, featuring multiple input sources, output sinks, and operations (known as a `PTransform`) that can both read from and output to multiple PCollections. For more information about pipeline design and best practices, see . To use Beam, your driver program must first create an instance of the Beam SDK class`Pipeline`(typically in the`main()`function). ```python import apache_beam as beam with beam.Pipeline() as pipeline: pass # Build your pipeline here. ``` When you create yourpipeline, you need to set . You can set your pipelines configuration options programmatically. You can also set the options ahead of time, or read them from the command line, and then pass them to the`Pipeline`object when you create the object. This option is often easier than setting the pipeline configuration options programmatically.
Project: YugabyteDB (App Definition and Development / Database)
File: ddl_drop_table.md

title: DROP TABLE statement [YSQL] headerTitle: DROP TABLE linkTitle: DROP TABLE description: Use the DROP TABLE statement to remove one or more tables (with all of their data) from the database. menu: v2.18: identifier: ddldroptable parent: statements type: docs Use the `DROP TABLE` statement to remove one or more tables (with all of their data) from the database. {{%ebnf%}} drop_table {{%/ebnf%}} Under normal operation, an error is raised if the table does not exist. Adding `IF EXISTS` will quietly ignore any non-existent tables specified. Specify the name of the table to be dropped. Objects associated with the table, such as prepared statements, will be eventually invalidated after the `DROP TABLE` statement is completed. `RESTRICT` is the default and it will not drop the table if any objects depend on it. `CASCADE` will drop any objects that transitively depend on the table. Do this: ```plpgsql set clientminmessages = warning; drop table if exists children, parents cascade; create table parents(k int primary key, v text); create table children( k int, parents_k int, v text, constraint childrenpk primary key(k, parentsk), constraint childrenfk foreign key(parentsk) references parents(k)); \d children ``` The `\d` meta-command output includes this information: ``` Foreign-key constraints: "childrenfk" FOREIGN KEY (parentsk) REFERENCES parents(k) ``` Now do this: ```plpgsql \set VERBOSITY verbose drop table parents restrict; ``` It causes this error: ``` 2BP01: cannot drop table parents because other objects depend on it ``` with this detail: ``` constraint children_fk on table children depends on table parents ``` Now do this: ```plpgsql drop table parents cascade; \d children ``` The 'DROP' now succeeds and the `\d` meta-command shows that the table "children" still exists but that it now as no foreign key constraint to the now-dropped "parents" table.
Project: KubeVela (App Definition and Development / Application Definition & Image Build)
File: SECURITY.md

If you've found a security issue that you'd like to disclose confidentially please send email to security@mail.kubevela.io . Only a restricted and carefully chosen group of KubeVela maintainers will have access to material sent to the security@mail.kubevela.io address. No outside users can subscribe to this list. Please provide as much information about your system and the issue as possible when contacting the list. Email sent to security@mail.kubevela.io is read and acknowledged with a non-automated response within three working days. For issues that are complicated and require significant attention, we will open an investigation and will provide you with a mechanism to check the status of our progress at any time.
Project: StarRocks (App Definition and Development / Database)
File: transform_values.md

displayed_sidebar: "English" Transforms values in a map using a and produces a new value for each entry in the map. This function is supported from v3.1 onwards. ```Haskell MAP transformvalues(lambdafunc, any_map) ``` `lambdafunc` can also be placed after `anymap`: ```Haskell MAP transformvalues(anymap, lambda_func) ``` `any_map`: the Map. `lambdafunc`: the Lambda expression you want to apply to `anymap`. Returns a map value where the data types of values are determined by the result of the Lambda expression and the data types of keys are the same as keys in `any_map`. If any input parameter is NULL, NULL is returned. If a key or value in the original map is NULL, NULL is processed as a normal value. The Lambda expression must have two parameters. The first parameter represents the key. The second parameter represents the value. The following example uses to generate a map value `{1:"ab",3:"cdd",2:null,null:"abc"}`. Then the Lambda expression is applied to each value of the map. The first example changes the value of each key-value pair to 1. The second example changes the value of each key-value pair to null. ```SQL mysql> select transformvalues((k,v)->1, colmap) from (select mapfromarrays([1,3,null,2,null],['ab','cdd',null,null,'abc']) as col_map)A; +-+ | transformvalues((k, v) -> 1, colmap) | +-+ | {1:1,3:1,2:1,null:1} | +-+ 1 row in set (0.02 sec) mysql> select transformvalues((k,v)->null, colmap) from (select mapfromarrays([1,3,null,2,null],['ab','cdd',null,null,'abc']) as col_map)A; +--+ | transformvalues((k, v) -> NULL, colmap) | +--+ | {1:null,3:null,2:null,null:null} | +--+ 1 row in set (0.01 sec) ```
Project: ArangoDB (App Definition and Development / Database)
File: tidylib.md

+++ title = "The HTMLTidy library" weight = 15 +++ {{% snippet "finale.cpp" "tidylib" %}} A C API may not initially appear to be a `T|E` based API, but if failure returns some domained error code and causes no other effects, and success returns some value, then it is effectively a "split" `T|E` API. The above is an example of exactly that form of "split" `T|E` API.
Project: KICS (Provisioning / Security & Compliance)
File: 9ba198e0-fef4-464a-8a4d-75ea55300de7.md

title: Neptune Cluster Instance is Publicly Accessible hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 9ba198e0-fef4-464a-8a4d-75ea55300de7 Query name: Neptune Cluster Instance is Publicly Accessible Platform: Terraform Severity: <span style="color:#bb2124">High</span> Category: Access Control URL: Neptune Cluster Instance should not be publicly accessible<br> ```tf title="Positive test num. 1 - tf file" hl_lines="7" resource "awsneptunecluster_instance" "example" { count = 2 clusteridentifier = awsneptune_cluster.default.id engine = "neptune" instance_class = "db.r4.large" apply_immediately = true publicly_accessible = true } ``` ```tf title="Negative test num. 1 - tf file" resource "awsneptunecluster_instance" "negative" { count = 2 clusteridentifier = awsneptune_cluster.default.id engine = "neptune" instance_class = "db.r4.large" apply_immediately = true publicly_accessible = false } ```
Project: Meshery (Provisioning / Automation & Configuration)
File: kured.md

layout: integration title: Kured subtitle: Collaborative and visual infrastructure as code for Kured image: /assets/img/integrations/kured/icons/color/kured-color.svg permalink: extensibility/integrations/kured docURL: https://docs.meshery.io/extensibility/integrations/kured description: integrations-category: Orchestration & Management integrations-subcategory: Scheduling & Orchestration registrant: Artifact Hub components: featureList: [ "Drag-n-drop cloud native infrastructure designer to configure, model, and deploy your workloads.", "Invite anyone to review and make changes to your private designs.", "Ongoing synchronization of Kubernetes configuration and changes across any number of clusters." ] howItWorks: "Collaborative Infrastructure as Code" howItWorksDetails: "Collaboratively manage infrastructure with your coworkers synchronously sharing the same designs." language: en list: include type: extensibility category: integrations
Project: KICS (Provisioning / Security & Compliance)
File: a6cd52a1-3056-4910-96a5-894de9f3f3b3.md

title: Cloud Storage Anonymous or Publicly Accessible hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: a6cd52a1-3056-4910-96a5-894de9f3f3b3 Query name: Cloud Storage Anonymous or Publicly Accessible Platform: Terraform Severity: <span style="color:#ff0000">Critical</span> Category: Access Control URL: Cloud Storage Buckets must not be anonymously or publicly accessible, which means the attribute 'members' must not possess 'allUsers' or 'allAuthenticatedUsers'<br> ```tf title="Positive test num. 1 - tf file" hl_lines="17 11 5" resource "googlestoragebucketiambinding" "positive1" { bucket = googlestoragebucket.default.name role = "roles/storage.admin" members = [] } resource "googlestoragebucketiambinding" "positive2" { bucket = googlestoragebucket.default.name role = "roles/storage.admin" members = ["user:jane@example.com","allUsers"] } resource "googlestoragebucketiambinding" "positive3" { bucket = googlestoragebucket.default.name role = "roles/storage.admin" members = ["user:jane@example.com", "allAuthenticatedUsers"] } ``` ```tf title="Negative test num. 1 - tf file" resource "googlestoragebucketiambinding" "negative1" { bucket = googlestoragebucket.default.name role = "roles/storage.admin" members = [ "user:jane@example.com", ] } ```
Project: Harbor (Provisioning / Container Registry)
File: 5-05-OVA-LDAP-integration.md

Test 5-05 - OVA LDAP integration ======= To verify that an OVA version of Harbor can work with AD and LDAP. The LDAP/AD configuration can be updated after power-off/on. User guide, installation guide of Harbor OVA version. This test requires an OVA binary of Harbor. A vCenter, at least an ESX host, and a network that supports DHCP. A linux host with Docker CLI installed (Docker client). TBD TBD None
Project: StarRocks (App Definition and Development / Database)
File: bin.md

displayed_sidebar: "English" Transforms the input `arg` into a binary. ```Shell bin(arg) ``` `arg`: the input you want to transform into a binary. It supports the BIGINT data type. Returns a value of the VARCHAR data type. ```Plain mysql> select bin(3); +--+ | bin(3) | +--+ | 11 | +--+ 1 row in set (0.02 sec) ```
Project: Operator Framework (App Definition and Development / Application Definition & Image Build)
File: backport-policy.md

title: Backport Policy weight: 2 Mainly critical issue fixes are backported to the most recent minor release. Special backport requests can be discussed during the weekly Triage meeting; this does not guarantee an exceptional backport will be created. Occasionally non-critical issue fixes will be backported, either at an approver's discretion or by request as noted above. For information on contacting maintainers and attending meetings, check the repository. Typically an issue will be fixed in the `master` branch, which will then be cherry-picked to the most recent release's branch. Those with approver permissions and above can create a cherry-pick PR, assuming no conflicts, by commenting `/cherry-pick <release branch>` in the PR fixing the issue in master. Fixes that are only relevant to a specific release branch can be made against that branch directly.
Project: Teller (Provisioning / Key Management)
File: CONTRIBUTING.md

Teller is open source and we love to receive contributions from our community you! There are many ways to contribute, from writing tutorials or blog posts, improving the documentation, submitting bug reports and feature requests or writing code. You can get in touch with us through , feedback and ideas are always welcome. If you have a bugfix or new feature that you would like to contribute, please find or open an issue about it first. Talk about what you would like to do. It may be that somebody is already working on it, or that there are particular issues that you should know about before implementing the change. Generally, we require that you test any code you are adding or modifying. Once your changes are ready , submit for review and we will make sure will review it , your effort is much appreciated! All feature development and most bug fixes hit the master branch first. Pull requests should be reviewed by someone with commit access. Once approved, the author of the pull request, or reviewer if the author does not have commit access, should "Squash and merge".
Project: ClickHouse (App Definition and Development / Database)
File: v21.3.9.83-lts.md

sidebar_position: 1 sidebar_label: 2022 Backported in : Fixed `quantile(s)TDigest`. Added special handling of singleton centroids according to tdunning/t-digest 3.2+. Also a bug with over-compression of centroids in implementation of earlier version of the algorithm was fixed. (). Backported in : Fixed very rare race condition on background cleanup of old blocks. It might cause a block not to be deduplicated if it's too close to the end of deduplication window. (). Backported in : Don't relax NOT conditions during partition pruning. This fixes and . (). Backported in : Fix bug in dict join with join_algorithm = 'auto'. Close . (). Backported in : Allow to move more conditions to `PREWHERE` as it was before version 21.1. Insufficient number of moved condtions could lead to worse performance. (). Backported in : Fixed `Cannot unlink file` error on unsuccessful creation of ReplicatedMergeTree table with multidisk configuration. This closes . (). Backported in : Fix corner cases in vertical merges with `ReplacingMergeTree`. In rare cases they could lead to fails of merges with exceptions like `Incomplete granules are not allowed while blocks are granules size`. (). Backported in : Fix restart / stop command hanging. Closes . (). Backported in : Added an exception in case of completely the same values in both samples in aggregate function `mannWhitneyUTest`. This fixes . ().
Project: Backstage (App Definition and Development / Application Definition & Image Build)
File: adr010-luxon-date-library.md

id: adrs-adr010 title: 'ADR010: Use the Luxon Date Library' description: Architecture Decision Record (ADR) for Luxon Date Library Date formatting (e.g. `a day ago`) and calculations are common within Backstage. Some of these useful features are not supported by the standard JavaScript `Date` object. The popular library has been commonly used to fill this gap but is affected by large bundle sizes and mutable state issues. On top of this, `momentjs` is and the project recommends using one of the more modern alternative libraries. See . We will use as the standard date library within Backstage. `Luxon` provides a similar feature set and API to `Moment.js`, but improves on its design through immutability and the usage of modern JavaScript APIs (e.g. `Intl`). This results in smaller bundle sizes while providing a full feature set and avoids the need for using additional libraries for common date & time tasks. All core packages and plugins within Backstage should use `Luxon` for any date manipulation or formatting that cannot be easily accomplished with the native JavaScript `Date` object. Using a single date library avoids having to learn multiple library APIs Having a single date library will reduce bundle sizes
Project: Devfile (App Definition and Development / Application Definition & Image Build)
File: user_story.md

name: User Story about: User story for new enhancement title: '' labels: '' assignees: '' /kind user-story <!-- Uncomment appropriate `/area` lines, and delete the rest. For example, `> /area api` would simply become: `/area api` --> /area ci /area api /area library /area registry /area alizer /area devworkspace /area integration-tests /area test-automation /area releng /area landing-page As a ... I want to ..., so that ... <!-- Decide which specific steps need to be completed, outline subtasks. --> [ ] .... [ ] ....
Project: StarRocks (App Definition and Development / Database)
File: be_bvars.md

displayed_sidebar: "English" `be_bvars` provides statistical information regarding bRPC. You can view RPC latency, QPS, and other statistics for some components of StarRocks. The following fields are provided in `be_bvars`: | Field | Description | | | | | BE_ID | ID of the BE where the bvar is located. | | NAME | Name of the bvar. | | DESC | Description of the bvar, including important statistical information. |
Project: ClickHouse (App Definition and Development / Database)
File: categoricalinformationvalue.md

slug: /en/sql-reference/aggregate-functions/reference/categoricalinformationvalue sidebar_position: 250 title: categoricalInformationValue Calculates the value of `(P(tag = 1) - P(tag = 0))(log(P(tag = 1)) - log(P(tag = 0)))` for each category. ``` sql categoricalInformationValue(category1, category2, ..., tag) ``` The result indicates how a discrete (categorical) feature `[category1, category2, ...]` contribute to a learning model which predicting the value of `tag`.
Project: ShardingSphere (App Definition and Development / Database)
File: alter-sql-parser-rule.en.md

+++ title = "Alter SQL_PARSER Rule" weight = 5 +++ The `ALTER SQL_PARSER RULE` syntax is used to alter the SQL parser rule configuration. {{< tabs >}} {{% tab name="Grammar" %}} ```sql AlterSqlParserRule ::= 'ALTER' 'SQL_PARSER' 'RULE' '(' sqlParserRuleDefinition ')' sqlParserRuleDefinition ::= parseTreeCacheDefinition? (',' sqlStatementCacheDefinition)? parseTreeCacheDefinition ::= 'PARSETREECACHE' '(' cacheOption ')' sqlStatementCacheDefinition ::= 'SQLSTATEMENTCACHE' '(' cacheOption ')' cacheOption ::= ('INITIALCAPACITY' '=' initialCapacity)? (','? 'MAXIMUMSIZE' '=' maximumSize)? initialCapacity ::= int maximumSize ::= int ``` {{% /tab %}} {{% tab name="Railroad diagram" %}} <iframe frameborder="0" name="diagram" id="diagram" width="100%" height="100%"></iframe> {{% /tab %}} {{< /tabs >}} `PARSETREECACHE`: local cache configuration of the syntax tree. `SQLSTATEMENTCACHE`: the local cache of SQL statement. Alter SQL parser rule ```sql ALTER SQL_PARSER RULE ( PARSETREECACHE(INITIALCAPACITY=128, MAXIMUMSIZE=1024), SQLSTATEMENTCACHE(INITIALCAPACITY=2000, MAXIMUMSIZE=65535) ); ``` `ALTER`, `SQLPARSER`, `RULE`, `PARSETREECACHE`, `INITIALCAPACITY`, `MAXIMUMSIZE`, `SQLSTATEMENT_CACHE`
Project: Meshery (Provisioning / Automation & Configuration)
File: mesheryctl-exp.md

layout: default title: mesheryctl-exp permalink: reference/mesheryctl/exp redirect_from: reference/mesheryctl/exp/ type: reference display-title: "false" language: en command: exp subcommand: nil Experimental commands for mesheryctl Commands under the Experimental group are for testing and evaluation prior to promotion to general availability. Experimental commands are subject to change. <pre class='codeblock-pre'> <div class='codeblock'> mesheryctl exp [flags] </div> </pre> <pre class='codeblock-pre'> <div class='codeblock'> -h, --help help for exp </div> </pre> <pre class='codeblock-pre'> <div class='codeblock'> --config string path to config file (default "/home/runner/.meshery/config.yaml") -v, --verbose verbose output </div> </pre> Go back to , if you want to add content manually to the CLI documentation, please refer to the for guidance.
Project: KICS (Provisioning / Security & Compliance)
File: 8cf4671a-cf3d-46fc-8389-21e7405063a2.md

title: StatefulSet Requests Storage hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 8cf4671a-cf3d-46fc-8389-21e7405063a2 Query name: StatefulSet Requests Storage Platform: Kubernetes Severity: <span style="color:#edd57e">Low</span> Category: Build Process URL: A StatefulSet requests volume storage.<br> ```yaml title="Positive test num. 1 - yaml file" hl_lines="73 33 66" apiVersion: apps/v1 kind: StatefulSet metadata: name: web spec: serviceName: "nginx" replicas: 2 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: name: nginx image: k8s.gcr.io/nginx-slim:0.8 ports: containerPort: 80 name: web volumeMounts: name: www mountPath: /usr/share/nginx/html volumeClaimTemplates: metadata: name: www spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 1Gi apiVersion: apps/v1 kind: StatefulSet metadata: name: web2 spec: serviceName: "nginx" replicas: 2 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: name: nginx image: k8s.gcr.io/nginx-slim:0.8 ports: containerPort: 80 name: web volumeMounts: name: www mountPath: /usr/share/nginx/html volumeClaimTemplates: metadata: name: www spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 1Gi metadata: name: www2 spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 2Gi ``` ```yaml title="Negative test num. 1 - yaml file" apiVersion: apps/v1 kind: StatefulSet metadata: name: web2 spec: serviceName: "nginx" replicas: 2 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: name: nginx image: k8s.gcr.io/nginx-slim:0.8 ports: containerPort: 80 name: web volumeMounts: name: www mountPath: /usr/share/nginx/html volumeClaimTemplates: metadata: name: www spec: accessModes: [ "ReadWriteOnce" ] ```
Project: YugabyteDB (App Definition and Development / Database)
File: create_user,role_option.grammar.md

```output.ebnf createuser ::= CREATE USER rolename [ [ WITH ] role_option [ , ... ] ] role_option ::= SUPERUSER | NOSUPERUSER | CREATEDB | NOCREATEDB | CREATEROLE | NOCREATEROLE | INHERIT | NOINHERIT | LOGIN | NOLOGIN | CONNECTION LIMIT connlimit | [ ENCRYPTED ] PASSWORD ' password ' | PASSWORD NULL | VALID UNTIL ' timestamp ' | IN ROLE role_name [ , ... ] | IN GROUP role_name [ , ... ] | ROLE role_name [ , ... ] | ADMIN role_name [ , ... ] | USER role_name [ , ... ] | SYSID uid ```
Project: OAuth2 Proxy (Provisioning / Key Management)
File: nextcloud.md

id: nextcloud title: NextCloud The Nextcloud provider allows you to authenticate against users in your Nextcloud instance. When you are using the Nextcloud provider, you must specify the urls via configuration, environment variable, or command line argument. Depending on whether your Nextcloud instance is using pretty urls your urls may be of the form `/index.php/apps/oauth2/` or `/apps/oauth2/`. Refer to the [OAuth2 documentation](https://docs.nextcloud.com/server/latest/adminmanual/configurationserver/oauth2.html) to set up the client id and client secret. Your "Redirection URI" will be `https://internalapp.yourcompany.com/oauth2/callback`. ``` -provider nextcloud -client-id <from nextcloud admin> -client-secret <from nextcloud admin> -login-url="<your nextcloud url>/index.php/apps/oauth2/authorize" -redeem-url="<your nextcloud url>/index.php/apps/oauth2/api/v1/token" -validate-url="<your nextcloud url>/ocs/v2.php/cloud/user?format=json" ``` Note: in all cases the validate-url will not have the `index.php`.