cwielenb/CNCF-Dataset huggingface dataset:
Project: YugabyteDB (App Definition and Development / Database)
File: special_fn_and_proc_attribute.diagram.md

<svg class="rrdiagram" version="1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg" width="515" height="230" viewbox="0 0 515 230"><path class="connector" d="M0 22h35m71 0h10m36 0h10m133 0h205m-475 25q0 5 5 5h5m66 0h10m36 0h30m84 0h54m-148 25q0 5 5 5h5m117 0h6q5 0 5-5m-138 30q0 5 5 5h5m118 0h5q5 0 5-5m-143-55q5 0 5 5v80q0 5 5 5h5m116 0h7q5 0 5-5v-80q0-5 5-5m5 0h170q5 0 5-5m-465 120q0 5 5 5h5m43 0h10m71 0h10m108 0h208q5 0 5-5m-470-145q5 0 5 5v170q0 5 5 5h25m38 0h20m-73 0q5 0 5 5v8q0 5 5 5h48q5 0 5-5v-8q0-5 5-5m5 0h10m78 0h10m38 0h10m91 0h10m120 0h5q5 0 5-5v-170q0-5 5-5m5 0h15"/><polygon points="0,29 5,22 0,15" style="fill:black;stroke-width:0"/><rect class="literal" x="35" y="5" width="71" height="25" rx="7"/><text class="text" x="45" y="22">RENAME</text><rect class="literal" x="116" y="5" width="36" height="25" rx="7"/><text class="text" x="126" y="22">TO</text><a xlink:href="../../syntaxresources/grammardiagrams#subprogram-name"><rect class="rule" x="162" y="5" width="133" height="25"/><text class="text" x="172" y="22">subprogramname</text></a><rect class="literal" x="35" y="35" width="66" height="25" rx="7"/><text class="text" x="45" y="52">OWNER</text><rect class="literal" x="111" y="35" width="36" height="25" rx="7"/><text class="text" x="121" y="52">TO</text><a xlink:href="../../syntaxresources/grammardiagrams#role-name"><rect class="rule" x="177" y="35" width="84" height="25"/><text class="text" x="187" y="52">rolename</text></a><rect class="literal" x="177" y="65" width="117" height="25" rx="7"/><text class="text" x="187" y="82">CURRENTROLE</text><rect class="literal" x="177" y="95" width="118" height="25" rx="7"/><text class="text" x="187" y="112">CURRENTUSER</text><rect class="literal" x="177" y="125" width="116" height="25" rx="7"/><text class="text" x="187" y="142">SESSIONUSER</text><rect class="literal" x="35" y="155" width="43" height="25" rx="7"/><text class="text" x="45" y="172">SET</text><rect class="literal" x="88" y="155" width="71" height="25" rx="7"/><text class="text" x="98" y="172">SCHEMA</text><a xlink:href="../../syntaxresources/grammardiagrams#schema-name"><rect class="rule" x="169" y="155" width="108" height="25"/><text class="text" x="179" y="172">schemaname</text></a><rect class="literal" x="55" y="185" width="38" height="25" rx="7"/><text class="text" x="65" y="202">NO</text><rect class="literal" x="123" y="185" width="78" height="25" rx="7"/><text class="text" x="133" y="202">DEPENDS</text><rect class="literal" x="211" y="185" width="38" height="25" rx="7"/><text class="text" x="221" y="202">ON</text><rect class="literal" x="259" y="185" width="91" height="25" rx="7"/><text class="text" x="269" y="202">EXTENSION</text><a xlink:href="../../syntaxresources/grammardiagrams#extension-name"><rect class="rule" x="360" y="185" width="120" height="25"/><text class="text" x="370" y="202">extension_name</text></a><polygon points="511,29 515,29 515,15 511,15" style="fill:black;stroke-width:0"/></svg>
Project: MongoDB (App Definition and Development / Database)
File: owners_format.md

This is loosely based on and OWNERS files. `version` is the current version of the `OWNERS.yml` file format. This should always be 1.0.0. `aliases` point to yaml files files that list aliases that can be used in this OWNERS.yml file. `filters` are a list of globs that match . The filter must match at least once file and be unique to the file. Each filter must have a list of `approvers`. An approval from any single approver will allow the code to be merged. Each filter may have a list of `emeritusapprovers`. `emeritusapprovers` are folks that used to be approvers that no longer have approver privileges. This allows us to keep track of folks who built up a knowledge base of this code that might need to be consulted in a critical situation. Both `approvers` and `emeritus_approvers` should be either github usernames, emails, or aliases. `options` are not required and are various options about how to use this OWNERS.yml file. Currently there is only a single option `noparentowners` which is defaulted to false. If this option is set to true it will stop upwards OWNERS resolution. ```yaml version: 1.0.0 # Should always be 1.0.0 aliases: //buildscripts/resmokelib/devprodcorrectnessaliases.yml filters: # List of all filters "*": # Select all files (will apply recursively) approvers: # Anyone on this list can approve PRs devprod-correctness # alias for a group of users IamXander # github username user.name@mongodb.com # email address emeritus_approvers: # This list is just for historical reference userB "*.md": approvers: markdown-approvers options: # All options for this file noparentowners: false # See above for noparentowners. Defaulted to false so this line is not needed. ``` `version` is the current version of the aliases file format. This should always be 1.0.0. `aliases` are a list of group names. Each group name must have one or more reviewers. Reviewers should be github usernames. ```yaml version: 1.0.0 aliases: devprod-build: IamXander # github username user.name@mongodb.com # email address ``` a/b/c/OWNERS.yml ```yaml version: 1.0.0 aliases: //aliases.yml filters: "*.py": approvers: teamC "*.md": approvers: teamMD ``` a/b/OWNERS.yml ```yaml version: 1.0.0 aliases: //aliases.yml filters: "*.json": approvers: teamB "*.py": approvers: teamPY options: noparentowners: true ``` a/OWNERS.yml ```yaml version: 1.0.0 aliases: //aliases.yml filters: "*": approvers: teamA "*.yaml": approvers: teamYAML ``` If someone changes `a/b/c/file.py` the owner resolution will select teamC since the first file searched is `a/b/c/OWNERS.yml` First we compare if `file.py` matches `.md`. It does not so we now check if `file.py` matches ``. It does match so teamC is selected for review. If someone changes `a/b/c/file.yaml` the owner resolution will not find a team. The first file searched is `a/b/c/OWNERS.yml`. No filters match file.yaml. Next we search in `a/b/OWNERS.yml`. No filters match there either. We stop searching up because `noparentowners` is set to true.
Project: StarRocks (App Definition and Development / Database)
File: least.md

displayed_sidebar: "English" Returns the smallest value from a list of one or more parameters. Generally, the return value has the same data type as the input. The comparison rules are the same as the function. ```Haskell LEAST(expr1,...); ``` `expr1`: the expression to compare. It supports the following data types: SMALLINT TINYINT INT BIGINT LARGEINT FLOAT DOUBLE DECIMALV2 DECIMAL32 DECIMAL64 DECIMAL128 DATETIME VARCHAR Example 1: Return the smallest value for a single input. ```Plain select least(3); +-+ | least(3) | +-+ | 3 | +-+ 1 row in set (0.00 sec) ``` Example 2: Return the smallest value from a list of values. ```Plain select least(3,4,5,5,6); +-+ | least(3, 4, 5, 5, 6) | +-+ | 3 | +-+ 1 row in set (0.01 sec) ``` Example 3: One parameter is of the DOUBLE type and a DOUBLE value is returned. ```Plain select least(4,4.5,5.5); +--+ | least(4, 4.5, 5.5) | +--+ | 4.0 | +--+ ``` Example 4: The input parameters are a mix of number and string but the string can be converted into a number. The parameters are compared as numbers. ```Plain select least(7,'5'); ++ | least(7, '5') | ++ | 5 | ++ 1 row in set (0.01 sec) ``` Example 5: The input parameters are a mix of number and string but the string cannot be converted into a number. The parameters are compared as strings. The string `'1'` is less than `'at'`. ```Plain select least(1,'at'); +-+ | least(1, 'at') | +-+ | 1 | +-+ ``` Example 6: The input parameters are characters. ```Plain mysql> select least('A','B','Z'); +-+ | least('A', 'B', 'Z') | +-+ | A | +-+ 1 row in set (0.00 sec) ``` LEAST, least
Project: Curiefense (Provisioning / Security & Compliance)
File: GEO.md

Curiefense support using or to perform ip address lookup. By default, curiefense uses Maxmind GeoLite IP service. It is not possible to uses the two providers at the same time. | Variable name | Default value | Description | | | -- | | | MAXMIND_ROOT | `/cf-config/current/config/maxmind` | Path to the root directory containing maxmind geoip databases | | MAXMINDCOUNTRY | `GeoLite2-Country.mmdb` | Path to the database inside `MAXMINDROOT` directory | | MAXMINDCITY | `GeoLite2-City.mmdb` | Path to the database inside `MAXMINDROOT` directory | | MAXMINDASN | `GeoLite2-ASN.mmdb` | Path to the ASN database inside `MAXMINDROOT` directory | A version of GeoLite2 IP databases are already included in this repository so geographic ip address lookup works by default. ipinfo can be enabled by setting the environment variable `USE_IPINFO` to `true`. | Variable name | Default value | Description | | | - | - | | USE_IPINFO | `false` | Set to `true` to enable using Ipinfo instead of GeoIP | | IPINFO_ROOT | - | Path to the root directory containing ipinfo databases | | IPINFOLOCATION | - | Path to the database inside `IPINFOROOT` directory | | IPINFOCOMPANY | - | Path to the database inside `IPINFOROOT` directory | | IPINFOASN | - | Path to the database inside `IPINFOROOT` directory | | IPINFOPRIVACY | - | Path to the database inside `IPINFOROOT` directory | | IPINFOCARRIER | - | Path to the database inside `IPINFOROOT` directory |
Project: YDB (App Definition and Development / Database)
File: topic-pipeline.md

The use of the `topic read` and `topic write` commands with standard I/O devices and support for reading messages in streaming mode lets you build full-featured integration scenarios with message transfer across topics and their conversion. This section describes a number of these scenarios. Transferring a single message from `topic1` in the `quickstart` database to `topic2` in `db2`, waiting for it to appear in the source topic ```bash {{ ydb-cli }} -p quickstart topic read topic1 -c c1 -w | {{ ydb-cli }} -p db2 topic write topic2 ``` Transferring all one-line messages that appear in `topic1` in the `quickstart` database to `topic2` in `db2` in background mode. You can use this scenario if it's guaranteed that there are no `0x0A` bytes (newline) in source messages. ```bash {{ ydb-cli }} -p quickstart topic read topic1 -c c1 --format newline-delimited -w | \ {{ ydb-cli }} -p db2 topic write topic2 --format newline-delimited ``` Transferring an exact binary copy of all messages that appear in `topic1` in the `quickstart` database to `topic2` in `db2` in background mode with base64-encoding of messages in the transfer stream. ```bash {{ ydb-cli }} -p quickstart topic read topic1 -c c1 --format newline-delimited -w --transform base64 | \ {{ ydb-cli }} -p quickstart topic write topic2 --format newline-delimited --transform base64 ``` Transferring a limited batch of one-line messages filtered by the `ERROR` substring ```bash {{ ydb-cli }} -p quickstart topic read topic1 -c c1 --format newline-delimited | \ grep ERROR | \ {{ ydb-cli }} -p db2 topic write topic2 --format newline-delimited ``` Writing YQL query results as messages to `topic1` ```bash {{ ydb-cli }} -p quickstart yql -s "select * from series" --format json-unicode | \ {{ ydb-cli }} -p quickstart topic write topic1 --format newline-delimited ``` Running a YQL, passing each message read from `topic1` as a parameter ```bash {{ ydb-cli }} -p quickstart topic read topic1 -c c1 --format newline-delimited -w | \ {{ ydb-cli }} -p quickstart table query execute -q 'declare $s as String;select Len($s) as Bytes' \ --stdin-format newline-delimited --stdin-par s --stdin-format raw ``` Running a YQL query involving adaptive batching of parameters from messages read from `topic1` ```bash {{ ydb-cli }} -p quickstart topic read topic1 -c c1 --format newline-delimited -w | \ {{ ydb-cli }} -p quickstart table query execute \ -q 'declare $s as List<String>;select ListLength($s) as Count, $s as Items' \ --stdin-format newline-delimited --stdin-par s --stdin-format raw \ --batch adaptive ```
Project: ClickHouse (App Definition and Development / Database)
File: v24.2.2.71-stable.md

sidebar_position: 1 sidebar_label: 2024 Backported in : Update tzdata to 2024a. (). PartsSplitter invalid ranges for the same part (). Try to avoid calculation of scalar subqueries for CREATE TABLE. (). Fix deadlock in parallel parsing when lots of rows are skipped due to errors (). Fixmaxquerysizeforkqlcompound_operator: (). Reduce the number of read rows from `system.numbers` (). Don't output number tips for date types (). Fix buffer overflow in CompressionCodecMultiple (). Remove nonsense from SQL/JSON (). Prevent setting custom metadata headers on unsupported multipart upload operations (). Fix crash in arrayEnumerateRanked (). Fix crash when using input() in INSERT SELECT JOIN (). Fix crash with different allowexperimentalanalyzer value in subqueries (). Remove recursion when reading from S3 (). Fix multiple bugs in groupArraySorted (). Fix Keeper reconfig for standalone binary (). Backported in : Decoupled changes from . (). Backported in : Eliminates the need to provide input args to docker server jobs to clean yml files. (). Backported in : Debug and fix markreleaseready. (). Backported in : Fix buildreport job so that it's defined by ciconfig only (not yml file). (). Backported in : Do not await ci pending jobs on release branches decrease wait timeout to fit into gh job timeout. (). Backported in : Set limited number of builds for "special build check" report in backports. (). Backported in : ... (). Backported in : ... (). Backported in : ... (). Backported in : Just a preparation for the merge queue support. (). Backported in : ... (). Backported in : ... (). Backported in : ... (). Backported in : ... (). Backported in :. (). CI: make workflow yml abstract (). Cancel PipelineExecutor properly in case of exception in spawnThreads (). General sanity in function `seriesOutliersDetectTukey` (). Speed up cctools building ().
Project: secureCodeBox (Provisioning / Security & Compliance)
File: integrating-a-hook.md

title: "Integrating A New Hook" sidebar_position: 1 In the secureCodeBox we created new Custom Resource Definitions (CRD) for Kubernetes to manage scanners (ScanType) and hooks (see ). To add a new Hook you need to add a new ScanCompletionHook (see ). In the secureCodeBox we use Helm Charts for all our Resources to make the installation of new scanners/hooks as easy as possible and enable us to install only the scanners/hooks we actually need. The directory structure of a hook Helm Chart will look something like this: ```bash docs README.ArtifactHub.md README.DockerHub-Hook.md hook Dockerfile hook.js hook.test.js package.json package-lock.json .dockerignore templates finding-post-processing-hook.yaml _helpers.tpl NOTES.txt Chart.yaml values.yaml Makefile README.md .helm-docs.gotmpl .helmignore .gitignore ``` To create a new Helm Chart for your hook you can use the following command (replace new-hook with the name of the hook): ```bash helm create new-hook ``` This command will create a new directory named new-hook and some template files provided by `helm` to simplify the creation of Helm Charts (see ). The following pages describe the purpose of all files and how to configure them.
Project: Backstage (App Definition and Development / Application Definition & Image Build)
File: scaling.md

id: scaling title: Scaling Backstage Deployments sidebar_label: Scaling description: Scaling Backstage production deployments There are several different methods for scaling Backstage deployments. The most straight-forward one is to simply deploy multiple identical instances and distributing incoming requests across them. The one requirement for this to work is that all instances need to share the same external resources, such as the database, and optional caching or search services. The Backstage backend plugins will coordinate through the database to share state and coordinate work. Another method for scaling Backstage deployments is to break apart the backend into multiple different services, each running a different set of plugins. This is a more advanced approach and requires you to be able to route requests to the appropriate backends based on the plugin ID. Both for ingress, but also internal traffic between Backstage backends, which is done by creating a custom implementation of the interface. See the for more details on how to separate your deployment into multiple backend instances. Lastly, you can also replicate the Backstage deployments across multiple regions. This is not a pattern that there is built-in support for and typically only makes sense to do for individual backend plugins.
Project: StarRocks (App Definition and Development / Database)
File: map_filter.md

displayed_sidebar: "English" Filters key-value pairs in a map by applying a Boolean array or a to each key-value pair. The pair that evaluates to `true` is returned. This function is supported from v3.1 onwards. ```Haskell MAP mapfilter(anymap, array<boolean>) MAP mapfilter(lambdafunc, any_map) ``` `mapfilter(anymap, array<boolean>)` Evaluates key-value pairs in `any_map` one by one against `array<boolean>` and returns key-value pairs that evaluate to `true`. `mapfilter(lambdafunc, any_map)` Applies `lambdafunc` to the key-value pairs in `anymap` one by one and returns key-value pairs whose result is `true`. `any_map`: the map value. `array<boolean>`: the Boolean array used to evaluate the map value. `lambda_func`: the Lambda expression used to evaluate the map value. Returns a map whose data type is the same as `any_map`. If `any_map` is NULL, NULL is returned. If `array<boolean>` is null, an empty map is returned. If a key or value in the map value is NULL, NULL is processed as a normal value. The Lambda expression must have two parameters. The first parameter represents the key. The second parameter represents the value. The following example uses to generate a map value `{1:"ab",3:"cdd",2:null,null:"abc"}`. Then each key-value pair is evaluated against `array<boolean>` and the pair whose result is `true` is returned. ```SQL mysql> select mapfilter(colmap, array<boolean>[0,0,0,1,1]) from (select mapfromarrays([1,3,null,2,null],['ab','cdd',null,null,'abc']) as col_map)A; +-+ | mapfilter(colmap, ARRAY<BOOLEAN>[0, 0, 0, 1, 1]) | +-+ | {null:"abc"} | +-+ 1 row in set (0.02 sec) mysql> select map_filter(null, array<boolean>[0,0,0,1,1]); +-+ | map_filter(NULL, ARRAY<BOOLEAN>[0, 0, 0, 1, 1]) | +-+ | NULL | +-+ 1 row in set (0.02 sec) mysql> select mapfilter(colmap, null) from (select mapfromarrays([1,3,null,2,null],['ab','cdd',null,null,'abc']) as col_map)A; ++ | mapfilter(colmap, NULL) | ++ | {} | ++ 1 row in set (0.01 sec) ``` The following example uses mapfromarrays() to generate a map value `{1:"ab",3:"cdd",2:null,null:"abc"}`. Then each key-value pair is evaluated against the Lambda expression and the key-value pair whose value is not null is returned. ```SQL mysql> select mapfilter((k,v) -> v is not null,colmap) from (select mapfromarrays([1,3,null,2,null],['ab','cdd',null,null,'abc']) as col_map)A; ++ | mapfilter((k,v) -> v is not null, colmap) | ++ | {1:"ab",3:"cdd",null:'abc'} | ++ 1 row in set (0.02 sec) ```
Project: Checkov (Provisioning / Security & Compliance)
File: EncryptionCalculation.md

When building queries, we noticed we were constantly asking a common question - Is the resource encrypted? To avoid having to know the correct configuration of each resource, and to make queries more concise, we decided to add `encryption` as a custom attribute to the relevant resource types. You can skip directly to: To support the different configurations of the different terraform resources with encryption, we've created 2 important objects defined : `GenericResourceEncryption` - this class executes the configuration logic based on the constructor parameters to decide whether the resource is encrypted or not. `ENCRYPTIONBYRESOURCETYPE` - a map of <resourcetype, GenericResourceEncryption<resource_type>>. This means that for every resource type (i.e. `awss3bucket`, `awsrdscluster` etc) there's either an entry in this map or there isn't. If there is an entry - the GenericResourceEncryption class will decide whether it is encrypted according to the . If there is no entry - the attribute will not exist for that resource type. ENCRYPTIONBYRESOURCE_TYPE receives as a second parameter a dict, consisting of the attribute paths as keys & the possible matching values as value list, i.e.: ```python { "encryptatrest.enabled": [True], "kmskeyid": [], "nodetonode_encryption.enabled": [True] } ``` Please note the empty list is supported - it means ANY. So in the case above, if the attribute `kmskeyid` exists in the resource it will be marked as encrypted, no matter what the actual value is for `kmskeyid`. However, we do expect `encryptatrest.enabled` to be set to `True` - otherwise it will be marked as unencrypted. This field can be leveraged in policies, i.e. query the field `encryption_` for the strings "ENCRYPTED" / "UNENCRYPTED". Example , and the expected resources can be found in the matching . To add support for a new resource type, for example `foo_bar`, a new entry needs to be added to `ENCRYPTIONBYRESOURCE_TYPE`, which maps the resource type string to an instance of `GenericResourceEncryption`. The constructor first parameter is the resource type, and the second is the dict as described in the
Project: Carvel (App Definition and Development / Application Definition & Image Build)
File: README_KinesisFirehose.md

The following is a sample Lambda function that transforms Kinesis Firehose records by doing a ToUpper on the data. ```go package main import ( "fmt" "strings" "github.com/aws/aws-lambda-go/events" "github.com/aws/aws-lambda-go/lambda" ) func handleRequest(evnt events.KinesisFirehoseEvent) (events.KinesisFirehoseResponse, error) { fmt.Printf("InvocationID: %s\n", evnt.InvocationID) fmt.Printf("DeliveryStreamArn: %s\n", evnt.DeliveryStreamArn) fmt.Printf("Region: %s\n", evnt.Region) var response events.KinesisFirehoseResponse for _, record := range evnt.Records { fmt.Printf("RecordID: %s\n", record.RecordID) fmt.Printf("ApproximateArrivalTimestamp: %s\n", record.ApproximateArrivalTimestamp) // Transform data: ToUpper the data var transformedRecord events.KinesisFirehoseResponseRecord transformedRecord.RecordID = record.RecordID transformedRecord.Result = events.KinesisFirehoseTransformedStateOk transformedRecord.Data = []byte(strings.ToUpper(string(record.Data))) response.Records = append(response.Records, transformedRecord) } return response, nil } func main() { lambda.Start(handleRequest) } ```