cwielenb/CNCF-Dataset huggingface dataset:
Project: Beam (App Definition and Development / Streaming & Messaging)
File: committer-onboarding.md

<!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> A brief checklist of things to know and do for new committers. [ ] Sign an ICLA and send it to secretary@apache.org if you haven't before. [ ] Read the ASF new committer guide: https://www.apache.org/dev/new-committers-guide.html - [ ] Log in to https://whimsy.apache.org; that will confirm a working ASF account you can edit email routing for the account, and add other emails that you own you can directly edit mailing list subscriptions (for example, you might switch them to your ASF account - you can still post from any of your registered emails)\ [ ] Link your GitHub account with your ASF account at https://gitbox.apache.org/. Once you see the big green "Merge" button on pull requests, this is working. [ ] (PMC Chair) Add the new committer to https://whimsy.apache.org/roster/committee/beam [ ] Announce the new committer
Project: Teleport (Provisioning / Security & Compliance)
File: 0082-session-tracker-resource-rbac.md

authors: Joel Wejdenstl (jwejdenstal@goteleport.com) state: implemented Introduce a new resource kind `session_tracker` that may be used in the RBAC verb system to manage access for existing session tracker resources using list/read resource verbs in addition to dynamic `where` conditions in the same manner as RFD 44. Session tracker visibility is currently tightly coupled with the permission to join a given session. They are invisible by default and may only be read by users with permission to join a session in some mode using the `join_sessions` primitive from Moderated Sessions. The functionality proposed by this RFD will allow one to configure access to list/read session tracker resources without being able to join the sessions in question. Only the `list` and `read` verbs are supported for session tracker resources. Supporting other verbs makes little sense as tracker resources may only be safely managed by the node that created them. The `list` verb will allow access to the `GetActiveSessionTrackers` RPC while `read` will provide access to the `GetSessionTracker` RPC. The `where` clause may be used to specify an expression that will be used to selectively filter or allow/deny a request based on tracker contents. The environment in which the expression will execute defines the following variables derived from the tracker object and request user: `user`: standard user object as defined in RFD 44 `tracker`: object `session_id`: string `kind`: string `participants`: array of strings `state`: string `hostname`: string `address`: string `login`: string `cluster`: string `kube_cluster`: string `host_user`: string `host_roles`: array of strings Configuration example that denies list/read access to all sessions you are a participant of: ```yaml spec: deny: rules: resources: [session_tracker] verbs: [list, read] where: 'contains(tracker.participants, user.metadata.name)' ``` To preserve earlier functionality from the legacy session metadata system, we should allow the auditor to list all active sessions by default but without the ability to join them. This involves adding this section to the predefined role definition: ```yaml spec: allow: rules: resources: [session_tracker] verbs: [list, read] ``` For existing clusters, we preserve the current auditor role if the user has modified it from the preset definition. If it has not been modified, we append the above configuration rule to the role to which will put the functionality on par with Teleport 9. We will check this by comparing the deserialized role definition against the template minus the new rule.
Project: Open Policy Agent (OPA) (Provisioning / Security & Compliance)
File: opa-playground.md

title: OPA Playground subtitle: Online Rego Playground software: [] inventors: styra labels: category: tooling layer: rego code: https://play.openpolicyagent.org/ blogs: https://blog.openpolicyagent.org/the-rego-playground-977566855cec https://blog.openpolicyagent.org/rego-playground-new-features-ec0345a73b9e docs_features: learning-rego: note: | Sometimes if you're working on a Rego policy in an integrated system it can help to debug it in isolation first. The playground is a great place to do that. Get started with a . language-tooling: note: | The playground is a great place to get started with writing and testing your first policies. You can also share links if you're asking for help in the community Slack. Get started with a simple . debugging-rego: note: | The playground is a great place to debug Rego policies as you can quickly iterate on the policy and data at the same time before sharing links to collaborate on a fix with others. Interactive online Rego playground for writing and sharing policies
Project: Shifu (Provisioning / Automation & Configuration)
File: telemetry-service-SQLServer_MySQL.md

Telemetry Service is a standalone service that takes telemetry data collected by `deviceShifu` and fan-out it to designated endpoints for future process. Let telemetry service support pushing telemetries to SQL endpoints. Let telemetry service support any random endpoints Let telemetry service serve as SQL service. We will predefine the schema for the database, since the uncertainty of schema and data format will make the feature hard to use and debug. The predefined schema looks like: ```SQL CREATE TABLE ${DbTableName} ( TelemetryID INT IDENTITY(1,1) PRIMARY KEY, DeviceName VARCHAR(255), TelemetryData TEXT, TelemetryTimeStamp DATETIME ); ``` `TelemetryID` is generated by the telemetry service, `DeviceName` is get from `EdgeDeviceSpec`'s SKU', `TelemetryData` is the raw data collected from device and `TelemetryTimeStamp` is the time of data collected. We will add `EdgeDeviceSpec` into our `TelemetryRequest` in order to get device related info to our database: ```go type TelemetryRequest struct { Spec EdgeDeviceSpec `json:spec,omitempty` RawData []byte `json:"rawData,omitempty"` MQTTSetting *MQTTSetting `json:"mqttSetting,omitempty"` SQLConnectionSetting *SQLConnectionSetting `json:"sqlConnectionSetting,omitempty"` MinIOSetting *MinIOSetting `json:"minIOSetting,omitempty"` } ``` Thanks to our good design for TDEngine, we can simply add a new `dbType` in our `SQLConnection` field to establish the connection to SQL Server. ```go const ( DBTypeTDengine DBType = "TDengine" DBTypeSQLServer DBType = "SQLServer" DBTypeMySQL DBType = "MySQL" ) ``` We then create a new package called `SQLServer` and `MySQL` alone with `TDengine`, and similar to `TDengine`, we can implement the functions of `ConnectDB` and `SendToDB`. However, the sql statement for `SQLServer` and `MySQL` should be: ```go func sendToSQLServer(rawData []byte, sqlcs *v1alpha1.SQLConnectionSetting) err { // Send rawData to TDengine // sample code: qlURL := constructTDengineUri(sqlcs) db, err := sql.Open("SQLServer", taosUri) if err != nil { klog.Error("failed to connect SQLServer, err:", err) return err } defer db.Close() ... } func insertIntoDB() { ... // DeviceId is retrieved from edgeDeviceSpec, primary id is generated by database result, err := db.DB.Exec(fmt.Sprintf("Insert Into %s (DeviceName,TelemetryData,TelemetryTimeStamp) Values('%s','%s','%s')", *db.Settings.DBTable, deviceName, string(rawData), time.Now().Format("2006-01-02 15:04:05"))) ... } ```
Project: Podman (App Definition and Development / Application Definition & Image Build)
File: PULL_REQUEST_TEMPLATE.md

<!-- Thanks for sending a pull request! Please make sure you've read our contributing guidelines and how to submit a pull request (https://github.com/containers/podman/blob/main/CONTRIBUTING.md#submitting-pull-requests). In case you're only changing docs, make sure to prefix the pull-request title with "[CI:DOCS]". That will prevent functional tests from running and save time and energy. Finally, be sure to sign commits with your real name. Since by opening a PR you already have commits, you can add signatures if needed with something like `git commit -s --amend`. --> <!-- If no, just write `None` in the release-note block below. If yes, a release note is required: Enter your extended release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required". For more information on release notes, please follow the Kubernetes model: https://git.k8s.io/community/contributors/guide/release-notes.md --> ```release-note ```
Project: YugabyteDB (App Definition and Development / Database)
File: edit-universe.md

title: Use YugabyteDB Anywhere to modify a universe headerTitle: Modify universe linkTitle: Modify universe description: Use YugabyteDB Anywhere to modify a universe. menu: v2.18_yugabyte-platform: identifier: edit-universe parent: manage-deployments weight: 60 type: docs YugabyteDB Anywhere allows you to expand a universe to add more nodes and shrink the universe to reduce the number of nodes. Typically, you do this by navigating to Universes > UniverseName > Actions > Edit Universe, as per the following illustration: Using the Edit Universe page, you can specify the new intent for the universe. This may include a new configuration of nodes powered by a different instance type. YugabyteDB Anywhere performs these modifications through the YB-Masters powering the universe. The YB-Masters ensure that the new nodes start hosting the tablet leaders for a set of tablets in such a way that the tablet leader count remains evenly balanced across all the available nodes. Expansion of universes created with an on-premises cloud provider and secured with third-party certificates obtained from external certification authorities follows a different workflow. For details, see . For universes that use Google Cloud Provider (GCP), Amazon Web Services (AWS), or Microsoft Azure, YBA allows you to change the VM images and increase the volume size without moving the data from the old nodes to the new nodes. This is known as smart resize and is subject to the following: For Azure universes, you can't increase the volume size for ultra SSDs. To avoid potential data loss, you can't do a smart resize of instances with ephemeral disks. Smart resize to ephemeral disks is supported. Smart resize cannot decrease the volume size. You can't do a smart resize if you change any options on the Edit Universe page other than the Instance Type and the size portion of the Volume Info field. If you change the Instance Type or both the Instance Type and the Volume Info size and then click Save, YBA gives you the option to either do a full migration of the universe and its data to new nodes, or do a smart resize, as per the following illustrations: If you change only the Volume Info size and click Save, YBA automatically performs a smart resize.
Project: YDB (App Definition and Development / Database)
File: table-ttl-reset.md

Use the `table ttl reset` subcommand to reset for the specified table. General format of the command: ```bash {{ ydb-cli }} [global options...] table ttl reset [options...] <table path> ``` `global options`: . `options`: . `table path`: The table path. View the description of the TTL reset command: ```bash {{ ydb-cli }} table ttl reset --help ``` {% include %} Reset TTL for the `series` table: ```bash {{ ydb-cli }} -p quickstart table ttl reset \ series ```
Project: Apache Heron (App Definition and Development / Streaming & Messaging)
File: observability-scribe.md

id: version-0.20.0-incubating-observability-scribe title: Observability with Scribe sidebar_label: Scribe original_id: observability-scribe <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> You can integrate Heron with to monitor and gather runtime metrics exported by Heron topologies. Heron supports custom metric exporters from the Metrics Manager. You can either build your own Scribe metrics sink or use the . To set up your Heron cluster to export to Scribe, you need to make two changes to the `metrics_sinks.yaml` configuration file: Add `scribe-sink` to the `sinks` list Add a `scribe-sink` map to the file that sets values for the listed below. You can uncomment the existing `prometheus-sink` map to get the default configuration. Parameter | Description | Default :|:|:- `class` | The Java class used to control Prometheus sink behavior | `flush-frequency-ms` | How frequently, in milliseconds, the `flush()` method is called | 60000 (one minute) `sink-restart-attempts` | How many times Heron should attempt to publish metrics to Scribe before no longer attempting | -1 (forever) `scribe-host` | The Scribe host to export metrics to | 127.0.0.1 `scribe-port` | The Scribe port to export metrics to | 1463 `scribe-category` | The Scribe category to export metrics to | `scribe-category` `service-namespace` | The service name for the category in `scribe-category` | `heron` `scribe-timeout-ms` | The timeout, in millisconds, when attempting to export metrics to Scribe | 200 `scribe-connect-server-attempts` | The maximum number of retry attempts when connecting to Scribe on the configured host and port | 2 `scribe-retry-attempts` | The maximum number of retry attempts when writing metrics to Scribe | 5 `scribe-retry-interval-ms` | The time interval, in milliseconds, at which Heron attempts to write metrics to Scribe | 100
Project: Beam (App Definition and Development / Streaming & Messaging)
File: cognite.md

title: "Cognite" icon: /images/logos/powered-by/cognite.png hasLink: "https://www.cognite.com/en/" <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->
Project: SeaTunnel (App Definition and Development / Streaming & Messaging)
File: PULL_REQUEST_TEMPLATE.md

<!-- Thank you for contributing to SeaTunnel! Please make sure that your code changes are covered with tests. And in case of new features or big changes remember to adjust the documentation. Feel free to ping committers for the review! Make sure that the pull request corresponds to a . Name the pull request in the form "[Feature] [component] Title of the pull request", where Feature can be replaced by `Hotfix`, `Bug`, etc. Minor fixes should be named following this pattern: `[hotfix] [docs] Fix typo in README.md doc`. --> <!-- Describe the purpose of this pull request. For example: This pull request adds checkstyle plugin.--> <!-- Note that it means any user-facing change including all aspects such as the documentation fix. If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible. If possible, please also clarify if this is a user-facing change compared to the released SeaTunnel versions or within the unreleased branches such as dev. If no, write 'No'. If you are adding/modifying connector documents, please follow our new specifications: https://github.com/apache/seatunnel/issues/4544. --> <!-- If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible. If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future. If tests were not added, please describe why they were not added and/or why it was difficult to add. If you are adding E2E test cases, maybe refer to https://github.com/apache/seatunnel/blob/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-mysql-e2e/src/test/resources/mysqlcdctomysql.conf, here is a good example. --> [ ] If any new Jar binary package adding in your PR, please add License Notice according [ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs [ ] If you are contributing the connector code, please check that the following files are updated: Update change log that in connector document. For more details you can refer to Update and add new connector information in it Update the pom file of .
Project: ClickHouse (App Definition and Development / Database)
File: redis.md

slug: /en/sql-reference/table-functions/redis sidebar_position: 170 sidebar_label: redis This table function allows integrating ClickHouse with . Syntax ```sql redis(host:port, key, structure[, dbindex[, password[, poolsize]]]) ``` Arguments `host:port` Redis server address, you can ignore port and default Redis port 6379 will be used. `key` any column name in the column list. `structure` The schema for the ClickHouse table returned from this function. `db_index` Redis db index range from 0 to 15, default is 0. `password` User password, default is blank string. `pool_size` Redis max connection pool size, default is 16. `primary` must be specified, it supports only one column in the primary key. The primary key will be serialized in binary as a Redis key. columns other than the primary key will be serialized in binary as Redis value in corresponding order. queries with key equals or in filtering will be optimized to multi keys lookup from Redis. If queries without filtering key full table scan will happen which is a heavy operation. are not supported for `redis` table function at the moment. Returned Value A table object with key as Redis key, other columns packaged together as Redis value. Read from Redis: ```sql SELECT * FROM redis( 'redis1:6379', 'key', 'key String, v1 String, v2 UInt32' ) ``` Insert into Redis: ```sql INSERT INTO TABLE FUNCTION redis( 'redis1:6379', 'key', 'key String, v1 String, v2 UInt32') values ('1', '1', 1); ``` See Also
Project: YDB (App Definition and Development / Database)
File: load-actors-pdisk-read.md

Tests the performance of reads from the PDisk. The load is generated on behalf of a VDisk. The actor creates chunks on the specified PDisk, writes random data to them, and reads the data from them using the specified parameters. After the load stops, the data written by the actor is deleted. You can generate two types of load: Continuous: The actor ensures the specified number of requests are run concurrently. To generate continuous load, set a zero interval between requests (e.g., `IntervalMsMin: 0` and `IntervalMsMax: 0`), while keeping the `InFlightReads` parameter different from zero. Interval: The actor runs requests at preset intervals. To generate interval load, set a non-zero interval between requests, e.g., `IntervalMsMin: 10` and `IntervalMsMax: 100`. You can set the maximum number of in-flight requests using the `InFlightReads` parameter. If its value is `0`, their number is unlimited. {% include %} | Parameter | Description | | | `PDiskId` | ID of the Pdisk being loaded on the node. | | `PDiskGuid` | Globally unique ID of the PDisk being loaded. | | `VDiskId` | The load is generated on behalf of a VDisk with the following parameters:<ul><li>`GroupID`: Group ID.</li><li>`GroupGeneration`: Group generation.</li><li>`Ring`: Group ring ID.</li><li>`Domain`: Ring fail domain ID.</li><li>`VDisk`: Index of the VDisk in the fail domain.</li></ul> | | `Chunks` | Chunk parameters.<br>`Slots`: Number of slots per chunk, determines the write size.<br>You can specify multiple `Chunks`, in which case a specific chunk to read data from is selected based on its `Weight`. | | `DurationSeconds` | Load duration in seconds. | | `IntervalMsMin`,<br>`IntervalMsMax` | Minimum and maximum intervals between requests under interval load, in milliseconds. The interval value is selected randomly from the specified range. | | `InFlightReads` | Number of simultaneously processed read requests. | | `Sequential` | Type of reads.<ul><li>`True`: Sequential.</li><li>`False`: Random.</li></ul> | | `IsWardenlessTest` | Set it to `False` in case the PDiskReadLoad actor is run on the cluster; otherwise, e.g. when it is run during unit tests, set it to `True`. | The following actor reads data blocks of `32`MB during `120` seconds with `64` in-flight requests (continuous load): ```proto PDiskReadLoad: { PDiskId: 1000 PDiskGuid: 2258451612736857634 VDiskId: { GroupID: 11234 GroupGeneration: 5 Ring: 1 Domain: 1 VDisk: 3 } Chunks: { Slots: 4096 Weight: 1 } Chunks: { Slots: 4096 Weight: 1 } Chunks: { Slots: 4096 Weight: 1 } Chunks: { Slots: 4096 Weight: 1 } Chunks: { Slots: 4096 Weight: 1 } Chunks: { Slots: 4096 Weight: 1 } Chunks: { Slots: 4096 Weight: 1 } Chunks: { Slots: 4096 Weight: 1 } DurationSeconds: 120 IntervalMsMin: 0 IntervalMsMax: 0 InFlightReads: 64 Sequential: false IsWardenlessTest: false } ``` When viewing test results, the following value should be of most interest to you: `Average speed since start`: Average read speed since start, in MB/s, e.g., `1257.148154`.
Project: Kraken (Provisioning / Container Registry)
File: README.md

Kraken Shadow Datastore ======================== This backend is designed for transitioning Kraken to a new backend by allowing admins to specify an "active" and "shadow" backend. The term shadow is used because the other backend "shadows" the active backend. Writes are sent to both backends, but reads only occur from the active. This ensures data consistency between the backends, and allows the old, proven backed to act as a safety net in the case where the new backend fails or has to be taken offline due to some unforeseen problems. Because all reads occur from the active backend, data needs to be manually migrated from the old backend to the new, before bringing the shadow backend online. This means there will be some downtime incurred during the transition. Supported Backends This currently supports SQL, HDFS, S3, and testfs backends. Note that this backend is only enabled as a tag datastore for the build-index. However this could be used as the blob datastore for the origin servers, provided it is not configured to use the SQL backend in active or shadow modes. Configuration The config has two required items "activebackend" and "shadowbackend", each of which function the same as the standard "backend" in that the backend type and normal configuration is defined under each, depending on which role that backend will play. For example, here is a configuration that uses the SQL backend as the active, and the testfs backend as the shadow: ```yaml backends: namespace: .* backend: shadow: active_backend: sql: dialect: mysql connection_string: "kraken:kraken@tcp(kraken-mysql:3306)/kraken?parseTime=True" debug_logging: true shadow_backend: testfs: addr: localhost:7357 root: tags namepath: dockertag ```