cwielenb/CNCF-Dataset huggingface dataset:
Project: OAuth2 Proxy (Provisioning / Key Management)
File: request_signatures.md

id: request_signatures title: Request Signatures If `signature_key` is defined, proxied requests will be signed with the `GAP-Signature` header, which is a [Hash-based Message Authentication Code (HMAC)](https://en.wikipedia.org/wiki/Hash-basedmessageauthentication_code) of selected request information and the request body [see `SIGNATURE_HEADERS` in `oauthproxy.go`](https://github.com/oauth2-proxy/oauth2-proxy/blob/master/oauthproxy.go). `signaturekey` must be of the form `algorithm:secretkey`, (ie: `signaturekey = "sha1:secret0"`) For more information about HMAC request signature validation, read the following: [Amazon Web Services: Signing and Authenticating REST Requests](https://docs.aws.amazon.com/AmazonS3/latest/dev/RESTAuthentication.html) [rc3.org: Using HMAC to authenticate Web service requests](http://rc3.org/2011/12/02/using-hmac-to-authenticate-web-service-requests/)
Project: EMQ Technologies (App Definition and Development / Streaming & Messaging)
File: feat-12189.en.md

Enhanced JWT ACL Claim Format. The JWT ACL claim has been upgraded to support a more versatile format. It now accepts an array structure, which resembles the file-based ACL rules. For example: ```json [ { "permission": "allow", "action": "pub", "topic": "${username}/#", "qos": [0,1], "retain": true }, { "permission": "allow", "action": "sub", "topic": "eq ${username}/#", "qos": [0,1] }, { "permission": "deny", "action": "all", "topics": ["#"] } ] ``` In this new format, when no matching rule is found, the action is not automatically denied. This allows the authorization process to proceed to other configured authorization sources. If no match is found throughout the chain, the final decision defers to the default permission set in `authorization.no_match`.
Project: Flink (App Definition and Development / Streaming & Messaging)
File: python_config.md

title: "Configuration" weight: 121 type: docs aliases: /dev/python/python_config.html <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> Depending on the requirements of a Python API program, it might be necessary to adjust certain parameters for optimization. For Python DataStream API program, the config options could be set as following: ```python from pyflink.common import Configuration from pyflink.datastream import StreamExecutionEnvironment config = Configuration() config.set_integer("python.fn-execution.bundle.size", 1000) env = StreamExecutionEnvironment.getexecutionenvironment(config) ``` For Python Table API program, all the config options available for Java/Scala Table API program could also be used in the Python Table API program. You could refer to the for more details on all the available config options for Table API programs. The config options could be set as following in a Table API program: ```python from pyflink.table import TableEnvironment, EnvironmentSettings envsettings = EnvironmentSettings.instreaming_mode() tenv = TableEnvironment.create(envsettings) tenv.getconfig().set("python.fn-execution.bundle.size", "1000") ``` The config options could also be set when creating EnvironmentSettings: ```python from pyflink.common import Configuration from pyflink.table import TableEnvironment, EnvironmentSettings config = Configuration() config.set_string("python.fn-execution.bundle.size", "1000") env_settings = EnvironmentSettings \ .new_instance() \ .instreamingmode() \ .with_configuration(config) \ .build() tableenv = TableEnvironment.create(envsettings) table_env = TableEnvironment.create(config) ``` {{< generated/python_configuration >}}
Project: Beam (App Definition and Development / Streaming & Messaging)
File: 55_direct_runner.md

Prompt: How to run Apache Beam pipelines locally? Response: You can run Apache Beam pipelines locally on your machine using the . It is designed for testing and development, ensuring that pipelines adhere closely to the Apache Beam model and are robust across different Beam runners. This option is suitable for testing and debugging pipelines, allowing quick iteration on code with immediate result visualization and unit testing using preferred local debugging tools. The Direct Runner performs additional checks to ensure users do not rely on semantics not guaranteed by the model. These checks include: Enforcing immutability of elements. Enforcing encodability of elements. Ensuring elements are processed in an arbitrary order at all points. Ensuring serialization of user functions (`DoFn`, `CombineFn`, etc.). To execute your Apache Beam pipeline locally using the Direct Runner, follow these steps: *1. Specify Dependencies (Java Only):* When using the Apache Beam Java SDK, specify your dependency on the Direct Runner in the `pom.xml` file of your Java project directory: ```java <dependency> <groupId>org.apache.beam</groupId> <artifactId>beam-runners-direct-java</artifactId> <version>2.54.0</version> <scope>runtime</scope> </dependency> ``` Make sure to include all necessary dependencies to create a self-contained application and compile your Java code into a JAR file. *2. Configure Pipeline Options:* While you can configure your pipeline by creating a `PipelineOptions` object and setting fields directly, the Beam SDKs include a command-line parser for setting fields using command-line arguments. To read options from the command line, construct your `PipelineOptions` object as shown: Java SDK: ```java PipelineOptions options = PipelineOptionsFactory.fromArgs(args).withValidation().create(); ``` Python SDK: ```python from apachebeam.options.pipelineoptions import PipelineOptions beam_options = PipelineOptions() ``` This method enables specifying options as command-line arguments in the format `--<option>=<value>`. Set the `runner` to `direct` or `DirectRunner`. Default values for other pipeline options are usually sufficient. For additional options, refer to the reference documentation for the `DirectOptions` interface for and . *3. Run the Pipeline:* Execute your pipeline on the Direct Runner from the command line. For Python SDK, use the `python -m` syntax to invoke the Python module containing your pipeline code: ```python python -m yourmodulename ``` For Java SDK, run your Java JAR file using the `java -jar` command: ```java java -jar your-project.jar ``` For additional information and considerations, refer to the section in the Apache Beam documentation.
Project: ShardingSphere (App Definition and Development / Database)
File: show-sharding-table-rules-used-algorithm.en.md

+++ title = "SHOW SHARDING TABLE NODES" weight = 10 +++ `SHOW SHARDING TABLE RULES USED ALGORITHM` syntax is used to query sharding rules used specified sharding algorithm in specified logical database {{< tabs >}} {{% tab name="Grammar" %}} ```sql ShowShardingTableRulesUsedAlgorithm::= 'SHOW' 'SHARDING' 'TABLE' 'RULES' 'USED' 'ALGORITHM' algorithmName ('FROM' databaseName)? algorithmName ::= identifier databaseName ::= identifier ``` {{% /tab %}} {{% tab name="Railroad diagram" %}} <iframe frameborder="0" name="diagram" id="diagram" width="100%" height="100%"></iframe> {{% /tab %}} {{< /tabs >}} When databaseName is not specified, the default is the currently used DATABASE. If DATABASE is not used, No database selected will be prompted. | Columns | Descriptions | ||--| | type | Sharding rule type | | name | Sharding rule name | Query sharding table rules for the specified sharding algorithm in spicified logical database ```sql SHOW SHARDING TABLE RULES USED ALGORITHM tableinline FROM shardingdb; ``` ```sql mysql> SHOW SHARDING TABLE RULES USED ALGORITHM tableinline FROM shardingdb; +-+--+ | type | name | +-+--+ | table | torderitem | +-+--+ 1 row in set (0.00 sec) ``` Query sharding table rules for specified sharding algorithm in the current logical database ```sql SHOW SHARDING TABLE RULES USED ALGORITHM table_inline; ``` ```sql mysql> SHOW SHARDING TABLE RULES USED ALGORITHM table_inline; +-+--+ | type | name | +-+--+ | table | torderitem | +-+--+ 1 row in set (0.01 sec) ``` `SHOW`, `SHARDING`, `TABLE`, `RULES`, `USED`, `ALGORITHM`, `FROM`
Project: StarRocks (App Definition and Development / Database)
File: SHOW_ANALYZE_JOB.md

displayed_sidebar: "English" Views the information and status of custom collection tasks. By default, StarRocks automatically collects full statistics of a table. It checks for any data updates every 5 minutes. If data change is detected, data collection will be automatically triggered. If you do not want to use automatic full collection, you can set the FE configuration item `enablecollectfull_statistic` to `false` and customize a collection task. This statement is supported from v2.4. ```SQL SHOW ANALYZE JOB [WHERE] ``` You can filter results by using the WHERE clause. The statement returns the following columns. | Column | Description | | | | | Id | The ID of the collection task. | | Database | The database name. | | Table | The table name. | | Columns | The column names. | | Type | The type of statistics, including `FULL` and `SAMPLE`. | | Schedule | The type of scheduling. The type is `SCHEDULE` for an automatic task. | | Properties | Custom parameters. | | Status | The task status, including PENDING, RUNNING, SUCCESS, and FAILED. | | LastWorkTime | The time of the last collection. | | Reason | The reason why the task failed. NULL is returned if task execution was successful. | ```SQL -- View all the custom collection tasks. SHOW ANALYZE JOB -- View custom collection tasks of database `test`. SHOW ANALYZE JOB where `database` = 'test'; ``` : customize an automatic collection task. : delete a custom collection task. : cancel a custom collection task that is running. For more information about collecting statistics for CBO, see .
Project: StarRocks (App Definition and Development / Database)
File: log10.md

displayed_sidebar: "English" Calculates the base 10 logarithm of a number. ```SQL log10(arg) ``` `arg`: The value whose logarithm you want to calculate. Only DOUBLE data type is supported. NOTE StarRocks returns NULL if `arg` is specified as a negative or 0. Returns a value of the DOUBLE data type. Example 1: Calculate the base 10 logarithm of 100. ```Plain select log10(100); ++ | log10(100) | ++ | 2 | ++ 1 row in set (0.02 sec) ```
Project: Chef InSpec (Provisioning / Security & Compliance)
File: systemd_service.md

+++ title = "systemd_service resource" draft = false gh_repo = "inspec" platform = "linux" [menu] [menu.inspec] title = "systemd_service" identifier = "inspec/resources/os/systemdservice.md systemdservice resource" parent = "inspec/resources/os" +++ Use the `systemd_service` Chef InSpec audit resource to test a service using SystemD. {{< readfile file="content/inspec/reusable/md/inspec_installation.md" >}} This resource first became available in v1.0.0 of InSpec. A `systemd_service` resource block declares the name of a service and then one (or more) matchers to test the state of the service: describe systemdservice('servicename') do it { should be_installed } it { should be_enabled } it { should be_running } end where `('service_name')` must specify a service name `beinstalled`, `beenabled`, and `be_running` are valid matchers for this resource; all matchers available to the `service` resource may be used The path to the service manager's control may be specified for situations where the path isn't available in the current `PATH`. For example: describe systemdservice('servicename', '/path/to/control') do it { should be_enabled } it { should be_installed } it { should be_running } end {{< readfile file="content/inspec/reusable/md/inspecmatcherslink.md" >}} This resource has the following special matchers. The `be_enabled` matcher tests if the named service is enabled: it { should be_enabled } The `be_installed` matcher tests if the named service is installed: it { should be_installed } The `be_running` matcher tests if the named service is running: it { should be_running }
Project: RabbitMQ (App Definition and Development / Streaming & Messaging)
File: rabbitmq-stream-14798.md

The stream protocol has been introduced in RabbitMQ 3.9.0, and is meant to be used in conjuction with Streams. Streams are a new persistent and replicated data structure which models an append-only log with non-destructive consumer semantics. Learn more about . These blog posts expand on the documentation: (includes slides) (includes video) (includes diagrams) Stream publishers Stream messages received / s Stream messages confirmed to publishers / s Stream consumers Stream messages delivered / s Errors since boot Namespace RabbitMQ Cluster `rabbitmq-stream` plugin to be enabled `rabbitmq-prometheus` plugin to be enabled
Project: YugabyteDB (App Definition and Development / Database)
File: sampling.md

Sampling =============================================================================== Principle The GDPR introduces the concept of principle of "[data minimisation]" which means that the collection of personal information must be limited to what is directly relevant and necessary to accomplish a specified purpose. If you're writing an anonymization policy for a dataset, chances are that you don't need to anonymize the entire database. In most cases, extract a subset of the table is sufficient. For example, if you want to export an anonymous dumps of the data for testing purpose in a CI workflow, extracting and masking only 10% of the database may be enough. Furthermore, anonymizing a smaller portion (i.e a "sample") of the dataset will be way faster. Example Let's say you have a huge amounts of http logs stored in a table. You want to remove the ip addresses and extract only 10% of the table: ```sql CREATE TABLE http_logs ( id integer NOT NULL, date_opened DATE, ip_address INET, url TEXT ); SECURITY LABEL FOR anon ON COLUMN httplogs.ipaddress IS 'MASKED WITH VALUE NULL'; SECURITY LABEL FOR anon ON TABLE http_logs IS 'TABLESAMPLE BERNOULLI(10)'; ``` Now you can either do static masking, dynamic masking or an anonymous dumps. The mask data will represent a 10% portion of the real data. Syntax The syntax is exactly the same as the [TABLESAMPLE clause] which can be placed at the end of a [SELECT] statement. You can also defined a sampling ratio at the database-level and it will be applied to all the tables that don't have their own `TABLESAMPLE` rule. ```sql SECURITY LABEL FOR anon ON DATABASE app IS 'TABLESAMPLE SYSTEM(33)'; ``` Maintaining Referential Integrity NOTE :The sampling method describe above WILL FAIL if you have foreign keys pointing at the table you want to sample. Extracting a subset of a database while maintaining referential integrity is tricky and it is not supported by this extension. If you really need to keep referential integrity in an anonymized dataset, you need to do it in 2 steps: First, extract a sample with [pg_sample] Second, anonymize that sample There may be other sampling tools for PostgreSQL but [pg_sample] is probably the best one.
Project: KICS (Provisioning / Security & Compliance)
File: 32ecd76e-7bbf-402e-bf48-8b9485749558.md

title: Token Auth File Is Set hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 32ecd76e-7bbf-402e-bf48-8b9485749558 Query name: Token Auth File Is Set Platform: Kubernetes Severity: <span style="color:#bb2124">High</span> Category: Access Control URL: When using kube-apiserver command, the 'token-auth-file' flag should not be set<br> ```yaml title="Positive test num. 1 - yaml file" hl_lines="11" apiVersion: v1 kind: Pod metadata: name: command-demo labels: purpose: demonstrate-command spec: containers: name: command-demo-container image: gcr.io/google_containers/kube-apiserver-amd64:v1.6.0 command: ["kube-apiserver", "--token-auth-file=/path/to/any/file"] restartPolicy: OnFailure ``` ```yaml title="Positive test num. 2 - yaml file" hl_lines="11" apiVersion: v1 kind: Pod metadata: name: command-demo labels: purpose: demonstrate-command spec: containers: name: command-demo-container image: gcr.io/google_containers/kube-apiserver-amd64:v1.6.0 command: ["kube-apiserver"] args: ["--token-auth-file=/path/to/any/file"] restartPolicy: OnFailure ``` ```yaml title="Negative test num. 1 - yaml file" apiVersion: v1 kind: Pod metadata: name: command-demo labels: purpose: demonstrate-command spec: containers: name: command-demo-container image: gcr.io/google_containers/kube-apiserver-amd64:v1.6.0 command: ["kube-apiserver"] restartPolicy: OnFailure ``` ```yaml title="Negative test num. 2 - yaml file" apiVersion: v1 kind: Pod metadata: name: command-demo labels: purpose: demonstrate-command spec: containers: name: command-demo-container image: gcr.io/google_containers/kube-apiserver-amd64:v1.6.0 command: ["kube-apiserver"] args: [] restartPolicy: OnFailure ```
Project: Rudder (Provisioning / Security & Compliance)
File: installation.md

`rudderc` is built into the Rudder server. The binary is available in `/opt/rudder/bin/rudderc`. On other systems, like workstations, you can download a for Linux or Windows x86_64 in the repository. To be able to check and compile techniques, the `rudderc` program needs access to the method library of the target systems. When running on a Rudder server, which has a built-in `rudderc` binary, the local library will be used. The standalone binary from the repository includes the default library. You can override the default library by passing a `--library` argument.
Project: YDB (App Definition and Development / Database)
File: public-talks.md

Subject: Scale it easy: YDB's high performance in a nutshell. . . . Description: Implementing a distributed database with strong consistency isnt difficult; the challenge lies in ensuring speed and scalability. YDB excels in these aspects. In this talk, well discuss YDBs architecture and its high performance, present results of benchmarks, and compare YDB to top competitors. Subject: Scalability and Fault Tolerance in YDB. . Description: In this talk we will cover two layers of YDB: Tablet and BlobStorage layers that together provide fault tolerance, scalability, and user isolation. Subject: Parallel Asynchronous Replication between YDB Database Instances. . . . Description: In this talk we present an approach to asynchronous replication in YDB that provides the following characteristics: changefeed from source database is sharded among multiple persistent queues, sharded changefeed is applied to the target database in a manner to guarantee target database consistency.
Project: openGemini (App Definition and Development / Database)
File: MAINTAINERS.md

The official list of maintainers and reviewers for the project maintenance Please keep the below list sorted in ascending order. | Maintainer | GitHub ID | Affiliation | Email | | - | | -- | -- | | Shilin Li | @shilinlee | Huawei | | | Yu Xiang | @xiangyu5632 | Huawei | | | Xiang Fan | @fx408 | Huawei | | | Jack Liu | @MrSanZhi | Huawei | | | Zhangjian He| @shoothzj | Huawei | |
Project: Stolon (App Definition and Development / Database)
File: twonodes.md

If you're going to run stolon on an IaaS or any kind of two nodes (baremetal, VM) this document provides some example configurations. These examples provide a two nodes setup that can be extended to a number greater than two nodes. The related store should live outside the instances hosting the stolon cluster for multiple reasons: Stores like etcd/consul uses the raft consensus protocol that requires at least 3 nodes to provides high availability (it can survive to the death of (N/2)-1 members) Disk I/O of postgres must not impact the etcd/consul I/O You can run multiple stolon clusters using the same store Every nodes will run one keeper, one sentinel and one proxy. The load balancer will balance between the stolon proxies. Clients will connect to the load balancer VIP. Every nodes will run one keeper, one sentinel and one proxy. Every nodes also runs a instance that will mantain a VIP assigned to one of the nodes. If one of the nodes dies/become partitioned keepealived will switch the VIP to the other node. Clients will connect to the VIP managed by keepalived.
Project: Shifu (Provisioning / Automation & Configuration)
File: CHANGELOG-v0.42.0.md

Allow Auth with Username and Password for TelemetryService MQTT by @rhoninl in https://github.com/Edgenesis/shifu/pull/873 Rebase v0.41.0 by @rhoninl in https://github.com/Edgenesis/shifu/pull/872 Full Changelog: https://github.com/Edgenesis/shifu/compare/v0.41.0...v0.42.0
Project: Apache Hadoop (App Definition and Development / Database)
File: RELEASENOTES.0.17.3.md

<! --> These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements. | Blocker | Checksum verification is disabled for LocalFS* Checksum verification was mistakenly disabled for LocalFileSystem. | Blocker | Bug in FSInputChecker makes it possible to read from an invalid buffer* Checksum input stream can sometimes return invalid data to the user. | Major | Chinese translation of core docs* Chinese translation for hadoop 0.17.x core docs.
Project: KubeVela (App Definition and Development / Application Definition & Image Build)
File: export-service.eg.md

```yaml apiVersion: core.oam.dev/v1beta1 kind: Application metadata: name: app-collect-service-endpoint-and-export spec: components: type: webservice name: busybox properties: image: busybox imagePullPolicy: IfNotPresent cmd: sleep '1000000' traits: type: expose properties: port: [8080] type: LoadBalancer policies: type: topology name: local properties: clusters: ["local"] type: topology name: worker properties: clusters: ["cluster-worker"] workflow: steps: type: deploy name: deploy properties: policies: ["local"] type: collect-service-endpoints name: collect-service-endpoints outputs: name: host valueFrom: value.endpoint.host name: port valueFrom: value.endpoint.port type: export-service name: export-service properties: name: busybox topology: worker inputs: from: host parameterKey: ip from: port parameterKey: port ```