cwielenb/CNCF-Dataset huggingface dataset:
Project: Hazelcast Jet (App Definition and Development / Streaming & Messaging)
File: observables.md

title: Receive Results on the Client description: How to monitor Jet job results on the client side. id: version-4.5-observables original_id: observables Usually Jet jobs make use of distributed data Sinks to output results, in order to not bottleneck the process for massive datasets. While this is efficient, it's not always convenient. For certain jobs, especially batch ones, which produce a reasonably small amount of output, some clients would prefer to get results directly, at the point where they have submitted the job. In order to make this happen Jet implements reactive style `Observables`, which can be used in the following way: ```java JetInstance jet = Jet.bootstrappedInstance(); Observable<Long> observable = jet.newObservable(); observable.addObserver(System.out::println); Pipeline pipeline = Pipeline.create(); pipeline.readFrom(TestSources.items(0L, 1L, 2L, 3L, 4L)) .writeTo(Sinks.observable(observable)); jet.newJob(pipeline).join(); observable.destroy(); ``` As we can see Observables can be obtained from any Jet member or client. An Observable can be observed by registering `Observers` on it , which are basically callbacks for handling results, errors and completion events. Handling errors and completion is optional. In order to make sure that no results are lost it is recommended that the Observers are registered with their Observables before Job submission. Due to their reactive nature Observers and Observables work both in batch and streaming jobs, but for batch jobs they have even more convenient ways to be used. They can for example be iterated over, without needing to register any explicit Observers up-front. They can even be converted into a `Future` form, with a build in transformation function for the results. For example just counting the results and printing the count is as simple as: ```java JetInstance jet = Jet.bootstrappedInstance(); Observable<Long> observable = jet.getObservable("results"); Pipeline pipeline = Pipeline.create(); pipeline.readFrom(TestSources.items(0L, 1L, 2L, 3L, 4L)) .writeTo(Sinks.observable("results")); observable.toFuture(Stream::count).thenAccept(System.out::println); jet.newJob(pipeline).join(); observable.destroy(); ``` Observables are backed by stored in the cluster which should be cleaned up by the client, once they are no longer necessary. They have a `destroy()` method which does just that. If the Observable isnt destroyed, its memory will be leaked in the cluster forever. In case the client crashes, or looses track of its Observables for any reason its still possible to identify all live instances, by using `JetInstance.getObservables()` and then the ones no longer needed can be cleaned-up via their destroy methods.
Project: KICS (Provisioning / Security & Compliance)
File: b23e9b98-0cb6-4fc9-b257-1f3270442678.md

title: Deployment Without PodDisruptionBudget hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: b23e9b98-0cb6-4fc9-b257-1f3270442678 Query name: Deployment Without PodDisruptionBudget Platform: Kubernetes Severity: <span style="color:#edd57e">Low</span> Category: Availability URL: Deployments should be assigned with a PodDisruptionBudget to ensure high availability<br> ```yaml title="Positive test num. 1 - yaml file" hl_lines="20" apiVersion: policy/v1beta1 kind: PodDisruptionBudget metadata: name: nginx-pdb spec: minAvailable: 2 selector: matchLabels: app: xpto apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: name: nginx image: nginx:1.14.2 ports: containerPort: 80 ``` ```yaml title="Negative test num. 1 - yaml file" apiVersion: policy/v1beta1 kind: PodDisruptionBudget metadata: name: nginx-pdb spec: minAvailable: 2 selector: matchLabels: app: nginx32 apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx32 template: metadata: labels: app: nginx spec: containers: name: nginx image: nginx:1.14.2 ports: containerPort: 80 ```
Project: SPIFFE (Provisioning / Key Management)
File: ELI_NESTEROV.md

Eli is a long-time SPIFFE community member and previously designed and operated the world's largest SPIRE deployment, exceeding 1,000,000 nodes. Over the years, he has had an outsized influence on the projects, and has seen them used in places and ways that few other people have. As such, he has a very strong understanding of the workload identity landscape overall, as well as how and where SPIFFE shines (and where it does not). Eli cares deeply about SPIFFE and the community that we have built here, and it is difficult for me to imagine a nominee more qualified than himself. GitHub Handle: @elinesterov Email Address: eli@spirl.com LinkedIn Profile: https://www.linkedin.com/in/elinesterov Current Affiliation: SPIRL, Inc.
Project: Chef InSpec (Provisioning / Security & Compliance)
File: google_logging_project_sinks.md

+++ title = "googleloggingproject_sinks resource" draft = false gh_repo = "inspec" platform = "gcp" [menu] [menu.inspec] title = "googleloggingproject_sinks" identifier = "inspec/resources/gcp/googleloggingprojectsinks.md googleloggingprojectsinks resource" parent = "inspec/resources/gcp" +++ A `googleloggingproject_sinks` is used to test a Google ProjectSink resource ```ruby describe googleloggingproject_sinks(project: 'chef-gcp-inspec') do its('names') { should include 'inspec-gcp-org-sink' } end ``` describe googleloggingproject_sinks(project: 'chef-inspec-gcp') do its('count') { should be <= 100} end describe googleloggingproject_sinks(project: 'chef-inspec-gcp') do its('sink_names') { should include "my-sink" } end describe googleloggingproject_sinks(project: 'chef-inspec-gcp') do its('sink_destinations') { should include "storage.googleapis.com/a-logging-bucket" } end googleloggingprojectsinks(project: 'chef-inspec-gcp').where(sinkname: /project/).sinknames.each do |sinkname| describe googleloggingprojectsink(project: 'chef-inspec-gcp', sink: sinkname) do its('writer_identity') { should eq "serviceAccount:my-logging-service-account.iam.gserviceaccount.com" } end end Properties that can be accessed from the `googleloggingproject_sinks` resource: See the resource for more information. `projects` : an array of `googleloggingproject_sink` project `names` : an array of `googleloggingproject_sink` name `filters` : an array of `googleloggingproject_sink` filter `destinations` : an array of `googleloggingproject_sink` destination `writer_identities` : an array of `googleloggingprojectsink` writeridentity `include_children` : an array of `googleloggingprojectsink` includechildren This resource supports all of the above properties as filter criteria, which can be used with `where` as a block or a method. Ensure the is enabled for the current project.
Project: Meshery (Provisioning / Automation & Configuration)
File: 06-15-2023-18-54-06_46a7a527.md

timestamp: "2023-06-15 18:54:31 UTC Thu" meshery-component: meshery-consul meshery-component-version: edge meshery-server-version: "*name" k8s-distro: minikube k8s-version: "v1.21.5" service-mesh: Consul service-mesh-version: "v1.2.0-rc1" tests: consul-client: null consul-server: null overall-status: "failing"
Project: YugabyteDB (App Definition and Development / Database)
File: dcl_set_session_authorization.md

title: SET SESSION AUTHORIZATION statement [YSQL] headerTitle: SET SESSION AUTHORIZATION linkTitle: SET SESSION AUTHORIZATION description: Use the SET SESSION AUTHORIZATION statement to set the current user and session user of the current session to be the specified user. menu: v2.18: identifier: dclsetsession_authorization parent: statements type: docs Use the `SET SESSION AUTHORIZATION` statement to set the current user and session user of the current session to be the specified user. {{%ebnf%}} setsessionauthorization, resetsessionauthorization {{%/ebnf%}} Session user can only be changed by superusers. Once the session user is set to `role_name`, any further SQL commands will use the privileges available to that role. To reset the session user back to current authenticated user, `RESET SESSION AUTHORIZATION` or `SET SESSION AUTHORIZATION DEFAULT` can be used. Set session user to John. ```plpgsql yugabyte=# select sessionuser, currentuser; sessionuser | currentuser --+-- yugabyte | yugabyte (1 row) yugabyte=# set session authorization john; SET yugabyte=# select sessionuser, currentuser; sessionuser | currentuser --+-- john | john (1 row) ```
Project: Druid (App Definition and Development / Database)
File: build.md

id: build title: "Build from source" <!-- ~ Licensed to the Apache Software Foundation (ASF) under one ~ or more contributor license agreements. See the NOTICE file ~ distributed with this work for additional information ~ regarding copyright ownership. The ASF licenses this file ~ to you under the Apache License, Version 2.0 (the ~ "License"); you may not use this file except in compliance ~ with the License. You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, ~ software distributed under the License is distributed on an ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY ~ KIND, either express or implied. See the License for the ~ specific language governing permissions and limitations ~ under the License. --> You can build Apache Druid directly from source. Use the version of this page that matches the version you want to build. For building the latest code in master, follow the latest version of this page : make sure it has `/master/` in the URL. JDK 8, 8u92+ or JDK 11. See our for information about obtaining a JDK. Distribution builds require Python 3.x and the `pyyaml` module. Integration tests require `pyyaml` version 5.1 or later. ```bash git clone git@github.com:apache/druid.git cd druid ``` The basic command to build Druid from source is: ```bash mvn clean install ``` This will run static analysis, unit tests, compile classes, and package the projects into JARs. It will not generate the source or binary distribution tarball. In addition to the basic stages, you may also want to add the following profiles and properties: -Pdist - Distribution profile: Generates the binary distribution tarball by pulling in core extensions and dependencies and packaging the files as `distribution/target/apache-druid-x.x.x-bin.tar.gz` -Papache-release - Apache release profile: Generates GPG signature and checksums, and builds the source distribution tarball as `distribution/target/apache-druid-x.x.x-src.tar.gz` -Prat - Apache Rat profile: Runs the Apache Rat license audit tool -DskipTests - Skips unit tests (which reduces build time) -Dweb.console.skip=true - Skip front end project Putting these together, if you wish to build the source and binary distributions with signatures and checksums, audit licenses, and skip the unit tests, you would run: ```bash mvn clean install -Papache-release,dist,rat -DskipTests ``` You are building Druid from source following the instructions on this page but you get ``` [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.6.0:exec (generate-binary-license) on project distribution: Command execution failed.: Process exited with an error: 1 (Exit value: 1) -> [Help 1] ``` Resolution: Make sure you have Python installed as well as the `yaml` module: ```bash pip install pyyaml ``` On some systems, ensure you use the Python 3.x version of `pip`: ```bash pip3 install pyyaml ```
Project: StarRocks (App Definition and Development / Database)
File: Authentication.md

displayed_sidebar: "English" In addition to the authentication method of "username+password", StarRocks also supports LDAP. To use LDAP authentication, you need to add the LDAP service into the FE node configuration first. `authenticationldapsimpleserverhost`: Specify the service IP. `authenticationldapsimpleserverport`: Specify the service port, with a default value of 389. When creating a user, specify the authentication method as LDAP authentication by `IDENTIFIED WITH authenticationldapsimple AS 'xxx'`. xxx is the DN (Distinguished Name) of the user in LDAP. Example 1: ~~~sql CREATE USER tom IDENTIFIED WITH authenticationldapsimple AS 'uid=tom,ou=company,dc=example,dc=com' ~~~ It is possible to create a user without specifying the user's DN in LDAP. When the user logs in, StarRocks will go to the LDAP system to retrieve the user information. if there is one and only one match, the authentication is successful. Example 2: ~~~sql CREATE USER tom IDENTIFIED WITH authenticationldapsimple ~~~ In this case, additional configuration needs to be added to the FE `authenticationldapsimplebindbase_dn`: The base DN of the user, specifying the retrieval range of the user. `authenticationldapsimpleusersearch_attr`: The name of the attribute in the LDAP object that identifies the user, uid by default. `authenticationldapsimplebindroot_dn`: The DN of the administrator account used to retrieve the user information. `authenticationldapsimplebindroot_pwd`: The password of the administrator account used when retrieving the user information. LDAP authentication requires the client to pass on a clear-text password to StarRocks. There are three ways to pass on a clear-text password: MySQL command line* Add `--default-auth mysqlclearpassword --enable-cleartext-plugin` when executing: ~~~sql mysql -utom -P8030 -h127.0.0.1 -p --default-auth mysqlclearpassword --enable-cleartext-plugin ~~~ JDBC* Since JDBCs default MysqlClearPasswordPlugin requires SSL transport, a custom plugin is required. ~~~java public class MysqlClearPasswordPluginWithoutSSL extends MysqlClearPasswordPlugin { @Override public boolean requiresConfidentiality() { return false; } } ~~~ Once connected, configure the custom plugin into the property. ~~~java ... Properties properties = new Properties();// replace xxx.xxx.xxx to your package name properties.put("authenticationPlugins", "xxx.xxx.xxx.MysqlClearPasswordPluginWithoutSSL"); properties.put("defaultAuthenticationPlugin", "xxx.xxx.xxx.MysqlClearPasswordPluginWithoutSSL"); properties.put("disabledAuthenticationPlugins", "com.mysql.jdbc.authentication.MysqlNativePasswordPlugin");DriverManager.getConnection(url, properties); ~~~ ODBC* Add `default\auth=mysqlclearpassword` and `ENABLECLEARTEXT\_PLUGIN=1` in the DSN of ODBC: , along with username and password.
Project: YDB (App Definition and Development / Database)
File: topology.md

A {{ ydb-short-name }} cluster consists of static and dynamic nodes. Static nodes enable data storage, implementing one of the supported redundancy schemes depending on the established operating mode. Dynamic nodes enable query execution, transaction coordination, and other data control functionality. Cluster topology is determined by the fault tolerance requirements. The following operating modes are available: | Mode | Storage<br>volume multiplier | Minimum<br>number<br>of nodes | Description | | | | | `none` | 1 | 1 | There is no redundancy.<br>Any hardware failure causes the storage pool to become unavailable.<br>This mode is only recommended for functional testing. | | `block-4-2` | 1.5 | 8 | with two blocks of redundancy added to the four blocks of source data is applied. Storage nodes are placed in at least 8 failure domains (usually racks).<br>The storage pool is available if any two domains fail, continuing to record all 6 data parts in the remaining domains.<br>This mode is recommended for storage pools within a single availability zone (usually a data processing center). | | `mirror-3-dc` | 3 | 9 | Data is replicated to 3 availability zones using 3 failure domains (usually racks) within each zone.<br>The storage pool is available if one availability zone and one failure domain fail in the remaining zones.<br>This mode is recommended for multi-data center installations. | | `mirror-3dc-3-nodes` | 3 | 3 | It is a simplified version of `mirror-3dc`. This mode requires at least 3 servers with 3 disks each. Each server must be located in an independent data center in order to provide the best fault tolerance.<br>Health in this mode is maintained if no more than 1 node fails.<br>This mode is only recommended for functional testing. | {% note info %} Node failure means both its total and partial unavailability, for example, failure of a single disk on a node. The storage volume multiplier specified above only applies to the fault tolerance factor. Other influencing factors (for example, slot fragmentation and granularity) must be taken into account for storage size planning. {% endnote %} For information about how to set the {{ ydb-short-name }} cluster topology, see .
Project: ClickHouse (App Definition and Development / Database)
File: v22.11.5.15-stable.md

sidebar_position: 1 sidebar_label: 2023 Backported in : Another fix for `Cannot read all data` error which could happen while reading `LowCardinality` dictionary from remote fs. Fixes . (). Backported in : Fix `SELECT ... FROM system.dictionaries` exception when there is a dictionary with a bad structure (e.g. incorrect type in xml config). (). Automatically merge green backport PRs and green approved PRs (). Improve release scripts (). Fix wrong approved_at, simplify conditions (). Get rid of artifactory in favor of r2 + ch-repos-manager (). Trim refs/tags/ from GITHUB_TAG in release workflow ().
Project: YugabyteDB (App Definition and Development / Database)
File: yba_provider.md

Manage YugabyteDB Anywhere providers Manage YugabyteDB Anywhere providers ``` yba provider [flags] ``` ``` -h, --help help for provider ``` ``` -a, --apiToken string YugabyteDB Anywhere api token. --config string Config file, defaults to $HOME/.yba-cli.yaml --debug Use debug mode, same as --logLevel debug. --disable-color Disable colors in output. (default false) -H, --host string YugabyteDB Anywhere Host (default "http://localhost:9000") -l, --logLevel string Select the desired log level format. Allowed values: debug, info, warn, error, fatal. (default "info") -o, --output string Select the desired output format. Allowed values: table, json, pretty. (default "table") --timeout duration Wait command timeout, example: 5m, 1h. (default 168h0m0s) --wait Wait until the task is completed, otherwise it will exit immediately. (default true) ``` - yba - Command line tools to manage your YugabyteDB Anywhere (Self-managed Database-as-a-Service) resources. - Manage a YugabyteDB Anywhere AWS provider - Manage a YugabyteDB Anywhere Azure provider - Delete a YugabyteDB Anywhere provider - Describe a YugabyteDB Anywhere provider - Manage a YugabyteDB Anywhere GCP provider - Manage a YugabyteDB Anywhere K8s provider - List YugabyteDB Anywhere providers - Manage a YugabyteDB Anywhere on-premises provider
Project: Packer (App Definition and Development / Application Definition & Image Build)
File: plugin_integration.md

name: Plugin Integration Request about: Open request to add your plugin as a Packer integration (https://developer.hashicorp.com/packer/integrations) labels: integration-request A written description of your plugin along with a link to the plugin repository. <! By default all integrations are registered as community integrations. HashiCorp Technology partners https://www.hashicorp.com/partners/find-a-partner will be registered as a partner once verified. > file in plugin repository. to plugin repository. [ ] Has added top-level integration README.md file to plugin `docs` directory. [ ] All plugins components have one README.md describing their usage. [ ] Has a fully synced `.web-docs` directory ready for publishing to the integrations portal.
Project: YugabyteDB (App Definition and Development / Database)
File: ddl_alter_foreign_data_wrapper.md

title: ALTER FOREIGN DATA WRAPPER statement [YSQL] headerTitle: ALTER FOREIGN DATA WRAPPER linkTitle: ALTER FOREIGN DATA WRAPPER description: Use the ALTER FOREIGN DATA WRAPPER statement to alter a foreign-data wrapper. menu: v2.18: identifier: ddlalterforeigndatawrapper parent: statements type: docs Use the `ALTER FOREIGN DATA WRAPPER` command to alter the definition of the foreign-data wrapper. This command can be used to alter the handler/validator functions or the options of the foreign-data wrapper. It can also be used to change the owner or rename the foreign-data wrapper. {{%ebnf%}} alterforeigndata_wrapper {{%/ebnf%}} Alter the foreign-data wrapper named fdw_name. The `HANDLER` clause can be used to specify the handler function. The `NO HANDLER` clause can be used to specify that the foreign-data wrapper has no handler function. The `VALIDATOR` clause can be used to specify the validator function. The `NO VALIDATOR` can be used to specify that the foreign-data wrapper has no validator function. The `OPTIONS` clause can be used to specify the new options of the foreign-data wrapper. `ADD`, `SET`, and `DROP` specify the action to be performed. `ADD` is assumed if no operation is explicitly specified. The new owner of the FDW can be specified using new_owner The new name of the FDW can be specified using new_name Change the handler to `newhandler`. ```plpgsql yugabyte=# ALTER FOREIGN DATA WRAPPER my_wrapper HANDLER; ``` Alter the foreign-data wrapper to have no validator. ```plpgsql yugabyte=# ALTER FOREIGN DATA WRAPPER my_wrapper NO VALIDATOR; ``` Alter the foreign-data wrapper's options: add `new` and set it to `'1'`, change the value of `old` to `'2'`. ```plpgsql yugabyte=# ALTER FOREIGN DATA WRAPPER my_wrapper OPTIONS(ADD new '1', SET old '2'); ```
Project: ClickHouse (App Definition and Development / Database)
File: hdfsCluster.md

slug: /en/sql-reference/table-functions/hdfsCluster sidebar_position: 81 sidebar_label: hdfsCluster Allows processing files from HDFS in parallel from many nodes in a specified cluster. On initiator it creates a connection to all nodes in the cluster, discloses asterisks in HDFS file path, and dispatches each file dynamically. On the worker node it asks the initiator about the next task to process and processes it. This is repeated until all tasks are finished. Syntax ``` sql hdfsCluster(cluster_name, URI, format, structure) ``` Arguments `cluster_name` Name of a cluster that is used to build a set of addresses and connection parameters to remote and local servers. `URI` URI to a file or a bunch of files. Supports following wildcards in readonly mode: ``, `*`, `?`, `{'abc','def'}` and `{N..M}` where `N`, `M` numbers, `abc`, `def` strings. For more information see . `format` The of the file. `structure` Structure of the table. Format `'column1name column1type, column2name column2type, ...'`. Returned value A table with the specified structure for reading data in the specified file. Examples Suppose that we have a ClickHouse cluster named `cluster_simple`, and several files with following URIs on HDFS: hdfs://hdfs1:9000/somedir/somefile_1 hdfs://hdfs1:9000/somedir/somefile_2 hdfs://hdfs1:9000/somedir/somefile_3 hdfs://hdfs1:9000/anotherdir/somefile_1 hdfs://hdfs1:9000/anotherdir/somefile_2 hdfs://hdfs1:9000/anotherdir/somefile_3 Query the amount of rows in these files: ``` sql SELECT count(*) FROM hdfsCluster('clustersimple', 'hdfs://hdfs1:9000/{some,another}dir/somefile{1..3}', 'TSV', 'name String, value UInt32') ``` Query the amount of rows in all files of these two directories: ``` sql SELECT count(*) FROM hdfsCluster('clustersimple', 'hdfs://hdfs1:9000/{some,another}dir/*', 'TSV', 'name String, value UInt32') ``` :::note If your listing of files contains number ranges with leading zeros, use the construction with braces for each digit separately or use `?`. ::: See Also