cwielenb/CNCF-Dataset huggingface dataset:
Project: ClickHouse (App Definition and Development / Database)
File: backupview.md

slug: /en/operations/utilities/backupview title: clickhouse_backupview Python module to help analyzing backups made by the command. The main motivation was to allows getting some information from a backup without actually restoring it. This module provides functions to enumerate files contained in a backup read files from a backup get useful information in readable form about databases, tables, parts contained in a backup check integrity of a backup ```python from clickhousebackupview import openbackup, S3, FileInfo backup = openbackup(S3("uri", "accesskeyid", "secretaccess_key")) print(backup.get_databases())) for db in backup.get_databases(): for tbl in backup.get_tables(database=db): print(backup.getcreatequery(database=db, table=tbl)) print(backup.get_partitions(database=db, table=tbl)) print(backup.get_parts(database=db, table=tbl)) backup.extractall(table="mydb.mytable", out='/tmp/mybackup_1/all/') backup.extracttabledata(table="mydb.mytable", out='/tmp/mybackup1/mytable/') backup.extracttabledata(table="mydb.mytable", partition="202201", out='/tmp/mybackup1/202201/') backup.extracttabledata(table="mydb.mytable", part="2022011002003", out='/tmp/mybackup1/2022011002003/') ``` For more examples see the .
Project: LinuxKit (Provisioning / Automation & Configuration)
File: platform-rpi3.md

LinuxKit supports building and booting images on a Raspberry Pi 3b using the mainline arm64 bit kernels. The LinuxKit arm64 kernel has support for some of the devices on the Raspberry Pi 3b, but notably it does not support: WLAN Bluetooth Graphics (though text console should work) It is unlikely, that we going to add support for these in the main LinuxKit kernels in the near future. The LinuxKit kernel is more targeted at VMs and baremetal servers where support for these type of devices is typically not needed. However, it should be possible to easily extend the LinuxKit kernel build process add the required kernel options, in a similar fashion to how `-dbg` kernels are build. See the documentation for details. We use the mainline Linux kernels for the Raspberry Pi and it is booted via . The `moby` tool, via `linuxkit build -format rpi3 <YAML>`, currently produces a `tar` archive which can be extracted onto a FAT32 formatted SD card to boot your Raspberry Pi. Currently, the root filesystem is provided as a RAM disk via the `initrd`. The LinuxKit images support console access via HDMI and USB keyboard as well as via serial. For serial console, you need a suitable cable to connect to the GPIO pins as described . There currently is no support for persistent disks for the Raspberry Pi. It may be possible to partition the SD card, format a second partition as `ext4` (or similar), and use it for persistent storage. TODO: Experiment with and document this set up. To enable and external USB stick as disk, add the following to the onboot section in your YAML: ``` name: usb-storage image: linuxkit/modprobe:<hash> command: ["modprobe", "usb_storage"] ``` The onboard, USB connected network interface is supported by the LinuxKit kernel, but for some unknown reason the driver is not cold plugged by `mdev`. To use the network interface, the driver needs to be `modprobe`d before the network interface can be used. The easiest way is to add the following section to the `onboot` section in your LinuxKit YAML file: ``` name: netdev image: linuxkit/modprobe:<hash> command: ["modprobe", "smsc95xx"] ``` For Raspberry Pi 3b+ use: ``` name: netdev image: linuxkit/modprobe:<hash> command: ["modprobe", "lan78xx"] ``` TODO: Figure out why mdev is not loading the driver.
Project: KCL (Provisioning / Automation & Configuration)
File: 1.about_this_guide.md

This guide is intended to help document how `KCL` (the KCL compiler) works, and to help new contributors get involved in KCL development. This guide consists of 4 parts: Building and Testing `KCL`: Contains information that should be useful no matter how you are contributing, about building, testing, debugging, profiling, etc. Contributing to `KCL`: Contains information that should be useful no matter how you are contributing, about procedures for contribution, using git and Github, etc. `KCL` Architecture: Contains an introduction to the architecture of the compiler and a detailed introduction to each compilation process. Appendices: There are a few of these with different information, including a glossary. The Guide itself is of course open-source as well, and the sources can be found at the . If you find any mistakes in the guide, please file an issue about it. Even better, open a Github Pull Request (PR) with a correction! If you do contribute to the guide, please see the corresponding subsection on writing documentation in this guide. Read to start.
Project: StarRocks (App Definition and Development / Database)
File: array_contains_seq.md

displayed_sidebar: "English" Checks whether all the elements of `arr2` appear in `arr1` in the same exact order, that is, whether `arr2` is a subset of `arr1` and the elements in `arr2` observe the same exact order as elements in `arr1`. If yes, this function will return 1. For example: `select arraycontainsseq([1,2,3,4], [1,2,3]);` returns 1. `select arraycontainsseq([1,2,3,4], [4,3]);` returns 0. This function is supported from v3.3 onwards. ~~~Haskell BOOLEAN arraycontainsall(arr1, arr2) ~~~ `arr`: the two arrays to compare. This syntax checks whether `arr2` is a subset of `arr1` and in the same exact order. The data types of elements in the two arrays must be the same. For the data types of array elements supported by StarRocks, see . Returns a value of the BOOLEAN type. `1` is returned if `arr2` is a subset of `arr1` and the elements in `arr2` observe the same order as those in `arr1`. Otherwise, `0` is returned. An empty array is a subset of any array. Therefore, `1` is returned if `arr2` is empty but `arr1` is a valid array. NULL is returned if any input array is NULL. Nulls in arrays are processed as normal values. For example, `SELECT arraycontainsseq([1, 2, NULL, 3, 4], [2,3])` will return 0. However, `SELECT arraycontainsseq([1, 2, NULL, 3, 4], [2,NULL,3])` will return 1. ```Plaintext MySQL > select arraycontainsseq([1,2,3,4], [1,2,3]); ++ | arraycontainsseq([1, 2, 3, 4], [1, 2, 3]) | ++ | 1 | ++ MySQL > select arraycontainsseq([1,2,3,4], [3,2]); ++ | arraycontainsseq([1, 2, 3, 4], [3, 2]) | ++ | 0 | ++ MySQL > select arraycontainsseq([1, 2, NULL, 3, 4], ['a']); +--+ | arraycontainsall([1, 2, NULL, 3, 4], ['a']) | +--+ | 0 | +--+ MySQL > select arraycontainsseq([1,2,3,4,null], null); ++ | array_contains([1, 2, 3, 4, NULL], NULL) | ++ | NULL | ++ MySQL > select arraycontainsseq([1,2,3,4], []); +--+ | arraycontainsseq([1, 2, 3, 4], []) | +--+ | 1 | +--+ ```
Project: Open Policy Agent (OPA) (Provisioning / Security & Compliance)
File: kong-authorization.md

title: API Gateway Authorization with Kong labels: layer: network category: gateway software: kong code: https://github.com/TravelNest/kong-authorization-opa https://github.com/open-policy-agent/contrib/tree/master/kongapiauthz inventors: travelnest wada-ama Kong is a microservice API Gateway. OPA provides fine-grained, context-aware control over the requests that Kong receives.
Project: YugabyteDB (App Definition and Development / Database)
File: yba_storage-config_nfs_update.md

Update an NFS YugabyteDB Anywhere storage configuration Update an NFS storage configuration in YugabyteDB Anywhere ``` yba storage-config nfs update [flags] ``` ``` --new-name string [Optional] Update name of the storage configuration. -h, --help help for update ``` ``` -a, --apiToken string YugabyteDB Anywhere api token. --config string Config file, defaults to $HOME/.yba-cli.yaml --debug Use debug mode, same as --logLevel debug. --disable-color Disable colors in output. (default false) -H, --host string YugabyteDB Anywhere Host (default "http://localhost:9000") -l, --logLevel string Select the desired log level format. Allowed values: debug, info, warn, error, fatal. (default "info") -n, --name string [Optional] The name of the storage configuration for the operation. Required for create, delete, describe, update. -o, --output string Select the desired output format. Allowed values: table, json, pretty. (default "table") --timeout duration Wait command timeout, example: 5m, 1h. (default 168h0m0s) --wait Wait until the task is completed, otherwise it will exit immediately. (default true) ``` - Manage a YugabyteDB Anywhere NFS storage configuration
Project: YugabyteDB (App Definition and Development / Database)
File: zrange.md

title: ZRANGE linkTitle: ZRANGE description: ZRANGE menu: preview: parent: api-yedis weight: 2385 aliases: /preview/api/redis/zrange /preview/api/yedis/zrange type: docs `ZRANGE key start stop [WITHSCORES]` This command returns `members` ordered from lowest to highest score in the specified range at sorted set `key`. `start` and `stop` represent the low and high index bounds respectively and are zero-indexed. They can also be negative numbers indicating offsets from the end of the sorted set, with -1 being the last element of the sorted set, -2 the penultimate element, and so on. If `key` does not exist, an empty list is returned. If `key` is associated with non sorted-set data, an error is returned. Returns a list of members found in the range specified by `start`, `stop`, unless the WITHSCORES option is specified (see below). WITHSCORES: Makes the command return both the `member` and its `score`. You can do this as shown below. ```sh $ ZADD z_key 1.0 v1 2.0 v2 3.0 v3 ``` ``` (integer) 3 ``` ```sh $ ZRANGE z_key 0 2 ``` ``` 1) "v1" 2) "v2" 3) "v3" ``` With negative indices. ```sh $ ZRANGE z_key -2 -1 ``` ``` 1) "v2" 2) "v3" ``` Both positive and negative indices. ```sh $ ZRANGE z_key 1 -1 WITHSCORES ``` ``` 1) "v2" 2) "2.0" 3) "v3" 4) "3.0" ``` Empty list returned if key doesn't exist. ```sh $ ZRANGE zkeyno_exist 0 2 WITHSCORES ``` ``` (empty list or set) ``` , , , ,
Project: YDB (App Definition and Development / Database)
File: docker.md

| Version | Release date | Download | Changelog | : | : | : | : v23.4 v.23.4.11 | 14/05/2024 | `cr.yandex/crptqonuodf51kdj7a7d/ydb:23.4.11` v23.3 v.23.3.17 | 14/10/2023 | `cr.yandex/crptqonuodf51kdj7a7d/ydb:23.3.17` v.23.3.13 | 12/10/2023 | `cr.yandex/crptqonuodf51kdj7a7d/ydb:23.3.13` | v23.2 v.23.2.12 | 14/08/2023 | `cr.yandex/crptqonuodf51kdj7a7d/ydb:23.2.12` | v23.1 v.23.1.26 | 05/05/2023 | `cr.yandex/crptqonuodf51kdj7a7d/ydb:23.1.26` | v22.5 v.22.5.10 | 09/02/2023 | `cr.yandex/crptqonuodf51kdj7a7d/ydb:22.5.10` | v22.4 v.22.4.44 | 28/11/2022 | `cr.yandex/crptqonuodf51kdj7a7d/ydb:22.4.44` | v.22.4.31 | 23/10/2022 | `cr.yandex/crptqonuodf51kdj7a7d/ydb:22.4.31` v.22.4.30 | 09/10/2022 | `cr.yandex/crptqonuodf51kdj7a7d/ydb:22.4.30` v.22.4.21 | 03/10/2022 | `cr.yandex/crptqonuodf51kdj7a7d/ydb:22.4.21` v22.2 v.22.2.47 | 07/07/2022 | `cr.yandex/crptqonuodf51kdj7a7d/ydb:22.2.47` v.22.2.40 | 16/06/2022 | `cr.yandex/crptqonuodf51kdj7a7d/ydb:22.2.40` v.22.2.39 | 15/06/2022 | `cr.yandex/crptqonuodf51kdj7a7d/ydb:22.2.39` v.22.2.31 | 07/06/2022 | `cr.yandex/crptqonuodf51kdj7a7d/ydb:22.2.31` v.22.2.22 | 18/04/2022 | `cr.yandex/crptqonuodf51kdj7a7d/ydb:22.2.22` v.22.2.5 | 14/02/2022 | `cr.yandex/crptqonuodf51kdj7a7d/ydb:22.2.5`
Project: StarRocks (App Definition and Development / Database)
File: BACKUP.md

displayed_sidebar: "English" Backs up data in a specified database, table, or partition. Currently, StarRocks only supports backing up data in OLAP tables. BACKUP is an asynchronous operation. You can check the status of a BACKUP job status using , or cancel a BACKUP job using . You can view the snapshot information using . CAUTION Only users with the ADMIN privilege can back up data. In each database, only one running BACKUP or RESTORE job is allowed each time. Otherwise, StarRocks returns an error. StarRocks does not support specifying data compression algorithm for data backup. ```SQL BACKUP SNAPSHOT <dbname>.<snapshotname> TO <repository_name> [ ON ( <tablename> [ PARTITION ( <partitionname> [, ...] ) ] [, ...] ) ] [ PROPERTIES ("key"="value" [, ...] ) ] ``` | Parameter | Description | | | | | db_name | Name of the database that stores the data to be backed up. | | snapshot_name | Specify a name for the data snapshot. Globally unique. | | repository_name | Repository name. You can create a repository using . | | ON | Name of the tables to be backed up. The whole database is backed up if this parameter is not specified. | | PARTITION | Name of the partitions to be backed up. The whole table is backed up if this parameter is not specified. | | PROPERTIES | Properties of the data snapshot. Valid keys:`type`: Backup type. Currently, only full backup `FULL` is supported. Default: `FULL`.`timeout`: Task timeout. Unit: second. Default: `86400`. | Example 1: Backs up the database `exampledb` to the repository `examplerepo`. ```SQL BACKUP SNAPSHOT exampledb.snapshotlabel1 TO example_repo PROPERTIES ("type" = "full"); ``` Example 2: Backs up the table `exampletbl` in `exampledb` to `example_repo`. ```SQL BACKUP SNAPSHOT exampledb.snapshotlabel2 TO example_repo ON (example_tbl); ``` Example 2: Backs up the partitions `p1` and `p2` of `exampletbl` and the table `exampletbl2` in `exampledb` to `examplerepo`. ```SQL BACKUP SNAPSHOT exampledb.snapshotlabel3 TO example_repo ON( example_tbl PARTITION (p1, p2), example_tbl2 ); ```
Project: Backstage (App Definition and Development / Application Definition & Image Build)
File: adr006-avoid-react-fc.md

id: adrs-adr006 title: 'ADR006: Avoid React.FC and React.SFC' description: Architecture Decision Record (ADR) log on Avoid React.FC and React.SFC Facebook has removed `React.FC` from their base template for a TypeScript project. The reason for this was that it was found to be an unnecessary feature with next to no benefits in combination with a few downsides. The main reasons were: children props were implicitly added Generic Type was not supported on children Read more about the removal in . To keep our codebase up to date, we have decided that `React.FC` and `React.SFC` should be avoided in our codebase when adding new code. Here is an example: ```typescript / Avoid this: / type BadProps = { text: string }; const BadComponent: FC<BadProps> = ({ text, children }) => ( <div> <div>{text}</div> {children} </div> ); / Do this instead: / type GoodProps = { text: string; children?: React.ReactNode }; const GoodComponent = ({ text, children }: GoodProps) => ( <div> <div>{text}</div> {children} </div> ); / Or as a shorthand, if no specific child type is required / type GoodProps = PropsWithChildren<{ text: string }>; const GoodComponent = ({ text, children }: GoodProps) => ( <div> <div>{text}</div> {children} </div> ); ``` We will gradually remove the current usage of `React.FC` and `React.SFC` from our codebase.
Project: CockroachDB (App Definition and Development / Database)
File: 20161215_join_filters.md

Feature Name: pushing down filters through joins Status: completed Start Date: 2016-12-15 Authors: Radu RFC PR: Cockroach Issue: A short write-up of the rules for pushing down filters and ON conditions when doing JOINs. Optimization of queries that include JOINs. There are two expressions to consider: ON expression (e.g. `SELECT * FROM a JOIN b ON a.x = b.x`) filter expression (e.g. `SELECT * FROM a JOIN b WHERE a.x = b.x`) For the purpose of this discussion, we consider `USING` and `NATURAL JOIN` as an implicit form of an `ON` expression. We consider that the expression in question is made up of three expressions: ``` expr = (leftExpr AND rightExpr AND combinedExpr)` ``` where: `leftExpr` references only columns from the left table (I will use "table" to refer to any join operand); `rightExpr` `rightExpr` references only columns from the right table `combinedExpr` is an expression that references columns from both tables. Inner joins are simple: there is no semantic difference between the ON expression and the filter; any expression can be pushed down. The following are all equivalent: ```sql SELECT * FROM l JOIN r WHERE (leftExpr AND rightExpr AND combinedExpr) SELECT * FROM l JOIN r ON (leftExpr AND rightExpr AND combinedExpr) SELECT * FROM (SELECT * FROM l WHERE leftExpr) JOIN (SELECT * FROM r WHERE rightExpr) ON combinedExpr ``` In a left outer join, all rows from the left table end up in the output in some form, for any ON condition. Consider a general form: ```sql SELECT * FROM l LEFT OUTER JOIN r ON (onLeft AND onRight AND onCombined) WHERE (filterLeft AND filterRight AND filterCombined) ``` Observations: `onRight` can be pushed down to `r`: ```sql SELECT * FROM l LEFT OUTER JOIN (SELECT * from r WHERE onRight) ON (onLeft AND onCombined) WHERE (filterLeft AND filterRight AND filterCombined) ``` `filterLeft` can be pushed down to `l`: ```sql SELECT * FROM (SELECT * from l WHERE filterLeft) JOIN r ON (onLeft AND onRight AND onCombined) WHERE (filterRight AND filterCombined) ``` This is all we can do in general. If we know that the filter expression always evaluates to false if all columns on the right side are all NULLs, the query is equivalent to an inner join (and the inner join optimizations would apply). A right outer join is symmetrical to a left outer join, so all the above apply if we swap `left` and `right`. For a full outer join, we cannot push down any filters. As before, if we know that the filter excludes rows that have NULLs on all the left (or right, or both) columns, the query would be equivalent to a left outer (or right outer, or inner, respectively) join.
Project: Monokle (App Definition and Development / Application Definition & Image Build)
File: architecture.md

Monokle Desktop is an Electron desktop application built with React & TypeScript. Redux Toolkit is used for state management. Visuals are done using Ant Design. Other fairly common libraries/frameworks are used internally for common functionality (see components below). This project was bootstrapped from <https://github.com/yhirose/react-typescript-electron-sample-with-create-react-app-and-electron-builder>, which provides: TypeScript support for Electron main process source code. Hot-relaod support for Electron app. Electron-builder support. See <https://www.electron.build/> for more info on Electron-builder. /electron : Electron main thread - contains startup code and some ipc handlers invoked from the UI. /src : Root for Monokle Desktop UI application - contains App.tsx, Index.tsx, etc. /components : UI components - coarse to fine grained; organisms -> molecules -> atoms /constants : constants.. /models : type definitions for core objects (see below) and states /redux : redux-related code (selectors, hooks, store, reducers, thunks, services) /styles : styles.. /utils : common utility functions /assets : icons/images/etc Most logic revolves around objects which "encapsulate" all data associated with a parsed resource. objects correspond to a parsed file - which can contain 0..n K8s Resource objects. The holds the main state of the application (see inline comments), state changes are handled by the and corresponding The content of most high level and should be fairly self-explanatory. A few highlights: The renders the selected folder. The renders the main resource navigator, including sections for Helm Charts and Kustomizations. The component renders the source editor using the Monaco editor (same as used by VS Code). The component renders nice forms for K8sResources using the react-jsonschema-form component (see below), corresponding schemas/uiSchemas are in the folder. https://github.com/eemeli/yaml for yaml parsing. https://github.com/react-monaco-editor/react-monaco-editor for source editing. https://github.com/micromatch/micromatch for dynamic filtering in navigator and file exclusion matching. https://github.com/JSONPath-Plus/JSONPath for finding refs/selectors in resources. https://github.com/rjsf-team/react-jsonschema-form for generating forms for k8s resources. https://github.com/wbkd/react-flow for graph diagrams. https://github.com/tweenjs/es6-tween for animation tweening. https://github.com/pengx17/monaco-yaml for yaml support in the source editor. https://github.com/paulmillr/chokidar for file watching. https://github.com/ant-design/ant-design/ for UI. https://github.com/styled-components/styled-components for custom styling. https://github.com/gsoft-inc/craco for overriding CRA config for folder aliases, see https://www.npmjs.com/package/craco-alias#examples. Monokle Desktop currently uses the latest 1.22.1 schemas for validation and editor functionality - see for making schema version configurable. Schemas are currently taken from strict version is used.
Project: Meshery (Provisioning / Automation & Configuration)
File: tikv-operator.md

layout: integration title: Tikv Operator subtitle: Collaborative and visual infrastructure as code for Tikv Operator image: /assets/img/integrations/tikv-operator/icons/color/tikv-operator-color.svg permalink: extensibility/integrations/tikv-operator docURL: https://docs.meshery.io/extensibility/integrations/tikv-operator description: integrations-category: App Definition and Development integrations-subcategory: Database registrant: Artifact Hub components: featureList: [ "Drag-n-drop cloud native infrastructure designer to configure, model, and deploy your workloads.", "Invite anyone to review and make changes to your private designs.", "Ongoing synchronization of Kubernetes configuration and changes across any number of clusters." ] howItWorks: "Collaborative Infrastructure as Code" howItWorksDetails: "Collaboratively manage infrastructure with your coworkers synchronously sharing the same designs." language: en list: include type: extensibility category: integrations
Project: KubeBlocks by ApeCloud (App Definition and Development / Database)
File: switchover.md

title: Switch over a PostgreSQL cluster description: How to switch over a PostgreSQL cluster keywords: [postgresql, switch over a cluster, switchover] sidebar_position: 6 sidebar_label: Switchover import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; You can initiate a switchover for a PostgreSQL Replication Cluster by executing the kbcli or kubectl command. Then KubeBlocks modifies the instance roles. Make sure the cluster is running normally. Check whether the following role probe parameters exist to verify whether the role probe is enabled. ```bash kubectl get cd postgresql -o yaml > probes: roleProbe: failureThreshold: 3 periodSeconds: 2 timeoutSeconds: 1 ``` You can switch over a secondary of a PostgreSQL PrimaeySecondary database to the primary role, and the former primary instance to a secondary. <Tabs> <TabItem value="kbcli" label="kbcli" default> Switchover with no primary instance specified ```bash kbcli cluster promote mycluster ``` Switchover with a specified new primary instance ```bash kbcli cluster promote mycluster --instance='mycluster-postgresql-2' ``` If there are multiple components, you can use `--component` to specify a component. ```bash kbcli cluster promote mycluster --instance='mycluster-postgresql-2' --component='postgresql' ``` </TabItem> <TabItem value="kubectl" label="kubectl"> The value of `instanceName` decides whether a new primary instance is specified for the switchover. Switchover with no specified primary instance ```yaml kubectl apply -f -<<EOF apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: name: mycluster-switchover-jhkgl spec: clusterRef: mycluster type: Switchover switchover: componentName: postgresql instanceName: '*' >> ``` Switchover with a specified new primary instance ```yaml kubectl apply -f -<<EOF apiVersion: apps.kubeblocks.io/v1alpha1 kind: OpsRequest metadata: name: mycluster-switchover-jhkgl spec: clusterRef: mycluster type: Switchover switchover: componentName: postgresql instanceName: 'mycluster-postgresql-2' >> ``` </TabItem> </Tabs> Check the instance status to verify whether the switchover is performed successfully. ```bash kbcli cluster list-instances ``` If an error occurs, refer to to troubleshoot the operation.