cwielenb/CNCF-Dataset huggingface dataset:
Project: FOSSA (Provisioning / Security & Compliance)
File: container-scanning-generic-ci.md

- - Scenario: As a development team we want to analyze container image for vulnerability and compliance issues as part of the CI process. Use the `fossa container analyze <IMAGE>` command. With Docker: ```bash docker build . -t <IMAGE> fossa container analyze <IMAGE> ``` With Podman: ```bash podman build . -t <IMAGE> fossa container analyze <IMAGE> ``` With Buildah: ```bash buildah bud --format=docker -f Dockerfile -t <IMAGE> . buildah push <IMAGE> docker-archive:image.tar fossa container analyze image.tar ``` Use the `fossa container test <IMAGE>` command to test a previously analyzed image. This command exits with a non-zero exit code if issues are discovered. ```bash fossa container test <IMAGE> ``` Prerequisite Dockerfile: ```Dockerfile FROM alpine:latest RUN apk add tree python3 py3-pip COPY ./app ./app RUN pip install -r ./app/reqs.txt ``` `app/reqs.txt`: ```text flask ``` With Docker ```bash VERSION="1.0.0" # or git commit hash: VERSION=$(git log -1 --pretty=%h) REPO="core-app:" TAG="$REPO$VERSION" BUILDTIMESTAMP=$( date '+%F%H:%M:%S' ) docker build -t "$TAG" -build-arg VERSION="$VERSION" --build-arg BUILDTIMESTAMP="$BUILDTIMESTAMP" . fossa container analyze "$TAG" fossa container test "$TAG" docker push "$TAG" ``` With Podman ```bash VERSION="1.0.0" # or git commit hash: VERSION=$(git log -1 --pretty=%h) REPO="core-app:" TAG="$REPO$VERSION" BUILDTIMESTAMP=$( date '+%F%H:%M:%S' ) podman build . -t "$TAG" --build-arg VERSION="$VERSION" --build-arg BUILDTIMESTAMP="$BUILDTIMESTAMP" . fossa container analyze "$TAG" fossa container test "$TAG" podman push "$TAG" ``` With Buildah ```bash VERSION="1.0.0" # or git commit hash: VERSION=$(git log -1 --pretty=%h) REPO="core-app:" TAG="$REPO$VERSION" BUILDTIMESTAMP=$( date '+%F%H:%M:%S' ) buildah bud --format=docker -f Dockerfile --build-arg VERSION="$VERSION" --build-arg BUILDTIMESTAMP="$BUILDTIMESTAMP" -t "$TAG" . buildah push "$TAG" docker-archive:image.tar fossa container analyze image.tar fossa container test image.tar rm image.tar ```
Project: YDB (App Definition and Development / Database)
File: auth-static.md

{% include %} Below are examples of the code for authentication based on a username and token in different {{ ydb-short-name }} SDKs. {% list tabs %} C++ ```c++ auto driverConfig = NYdb::TDriverConfig() .SetEndpoint(endpoint) .SetDatabase(database) .SetCredentialsProviderFactory(NYdb::CreateLoginCredentialsProviderFactory({ .User = "user", .Password = "password", })); NYdb::TDriver driver(driverConfig); ``` Go (native) You can pass the username and password in the connection string. For example: ```shell "grpcs://login:password@localohost:2135/local" ``` You can also explicitly pass them using the `ydb.WithStaticCredentials` parameter: {% include %} Go (database/sql) You can pass the username and password in the connection string. For example: {% include %} You can also explicitly pass the username and password at driver initialization via a connector using the `ydb.WithStaticCredentials` parameter: {% include %} Java ```java public void work(String connectionString, String username, String password) { AuthProvider authProvider = new StaticCredentials(username, password); GrpcTransport transport = GrpcTransport.forConnectionString(connectionString) .withAuthProvider(authProvider) .build()); TableClient tableClient = TableClient.newClient(transport).build(); doWork(tableClient); tableClient.close(); transport.close(); } ``` Node.js {% include %} Python {% include %} Python (asyncio) {% include %} C# (.NET) ```C# using Ydb.Sdk; using Ydb.Sdk.Auth; const string endpoint = "grpc://localhost:2136"; const string database = "/local"; var config = new DriverConfig( endpoint: endpoint, // Database endpoint, "grpcs://host:port" database: database, // Full database path credentials: new StaticCredentialsProvider(user, password) ); await using var driver = await Driver.CreateInitialized(config); ``` PHP ```php <?php use YdbPlatform\Ydb\Ydb; use YdbPlatform\Ydb\Auth\Implement\StaticAuthentication; $config = [ // Database path 'database' => '/local', // Database endpoint 'endpoint' => 'localhost:2136', // Auto discovery (dedicated server only) 'discovery' => false, // IAM config 'iam_config' => [ 'insecure' => true, // 'rootcertfile' => './CA.pem', // Root CA file (uncomment for dedicated server) ], 'credentials' => new StaticAuthentication($user, $password) ]; $ydb = new Ydb($config); ``` {% endlist %}
Project: StarRocks (App Definition and Development / Database)
File: from_binary.md

displayed_sidebar: "English" Converts a binary value to a VARCHAR string based on the specified binary format (`binarytype`). The following binary formats are supported: `hex`, `encode64`, and `utf8`. If no `binarytype` is specified, `hex` is the default. ```Haskell frombinary(binary[, binarytype]) ``` `binary`: the input binary to convert, required. `binary_type`: the binary format for conversion, optional. `hex` (default): `from_binary` uses the `hex` method to encode the input binary to a VARCHAR string. `encode64`: `from_binary` uses the `base64` method to encode the input binary to a VARCHAR string. `utf8`: `from_binary` converts the input binary to a VARCHAR string without any transformation. Returns a VARCHAR string. ```Plain mysql> select frombinary(tobinary('ABAB', 'hex'), 'hex'); +-+ | frombinary(tobinary('ABAB', 'hex'), 'hex') | +-+ | ABAB | +-+ 1 row in set (0.02 sec) mysql> select frombase64(frombinary(to_binary('U1RBUlJPQ0tT', 'encode64'), 'encode64')); +--+ | frombase64(frombinary(to_binary('U1RBUlJPQ0tT', 'encode64'), 'encode64')) | +--+ | STARROCKS | +--+ 1 row in set (0.01 sec) mysql> select frombinary(tobinary('STARROCKS', 'utf8'), 'utf8'); +--+ | frombinary(tobinary('STARROCKS', 'utf8'), 'utf8') | +--+ | STARROCKS | +--+ 1 row in set (0.01 sec) ```
Project: Pulsar (App Definition and Development / Streaming & Messaging)
File: pip-335 Oxia metadata plugin.md

Oxia is a scalable metadata store and coordination system that can be used as the core infrastructure to build large scale distributed systems. Oxia was created with the primary goal of providing an alternative Pulsar to replace ZooKeeper as the long term preferred metadata store, overcoming all the current limitations in terms of metadata access throughput and data set size. Add a Pulsar MetadataStore plugin that uses Oxia client SDK. Users will be able to start a Pulsar cluster using just Oxia, without any ZooKeeper involved. It's not in the scope of this proposal to change any default behavior or configuration of Pulsar. Oxia semantics and client SDK were already designed with Pulsar and MetadataStore plugin API in mind, so there is not much integration work that needs to be done here. Just few notes: Oxia client already provides support for transparent batching of read and write operations, so there will be no use of the batching logic in `AbstractBatchedMetadataStore` Oxia does not treat keys as a walkable file-system like interface, with directories and files. Instead all the keys are independent. Though Oxia sorting of keys is aware of '/' and provides efficient key range scanning operations to identify the first level children of a given key Oxia, unlike ZooKeeper, doesn't require the parent path of a key to exist. eg: we can create `/a/b/c` key without `/a/b` and `/a` existing. In the Pulsar integration for Oxia we're forcing to create all parent keys when they are not there. This is due to several places in BookKeeper access where it does not create the parent keys, though it will later make `getChildren()` operations on the parents. Unlike in the ZooKeeper implementation, the notification of events is guaranteed in Oxia, because the Oxia client SDK will use the transaction offset after server reconnections and session restarted events. This will ensure that brokers cache will always be properly invalidated. We will then be able to remove the current 5minutes automatic cache refresh which is in place to prevent the ZooKeeper missed watch issue. Oxia: https://github.com/streamnative/oxia Oxia Java Client SDK: https://github.com/streamnative/oxia-java
Project: ArangoDB (App Definition and Development / Database)
File: acknowledgements.md

Acknowledgements ================ : A very powerful library for function objects in C++98. Shunsuke Sogame : A metaprogramming library with many functional constructs Louis Dionne : A library to create named operators Konrad Rudolph - Jamboree - Sean Parent - Eric Niebler : Functional Programming in C++ Brian McNamara and Yannis Smaragdakis - Joel de Guzman, Dan Marsden, Thomas Heller, and John Fletcher
Project: YDB (App Definition and Development / Database)
File: debug-prometheus.md

{% include %} Below are examples of the code for enabling metrics in Prometheus in different {{ ydb-short-name }} SDKs. {% list tabs %} Go (native) ```go package main import ( "context" "github.com/prometheus/client_golang/prometheus" metrics "github.com/ydb-platform/ydb-go-sdk-prometheus/v2" "github.com/ydb-platform/ydb-go-sdk/v3" "github.com/ydb-platform/ydb-go-sdk/v3/trace" ) func main() { ctx := context.Background() registry := prometheus.NewRegistry() db, err := ydb.Open(ctx, os.Getenv("YDBCONNECTIONSTRING"), metrics.WithTraces( registry, metrics.WithDetails(trace.DetailsAll), metrics.WithSeparator("_"), ), ) if err != nil { panic(err) } defer db.Close(ctx) ... } ``` Go (database/sql) ```go package main import ( "context" "database/sql" "github.com/prometheus/client_golang/prometheus" metrics "github.com/ydb-platform/ydb-go-sdk-prometheus/v2" "github.com/ydb-platform/ydb-go-sdk/v3" "github.com/ydb-platform/ydb-go-sdk/v3/trace" ) func main() { ctx := context.Background() registry := prometheus.NewRegistry() nativeDriver, err := ydb.Open(ctx, os.Getenv("YDBCONNECTIONSTRING"), metrics.WithTraces( registry, metrics.WithDetails(trace.DetailsAll), metrics.WithSeparator("_"), ), ) if err != nil { panic(err) } defer nativeDriver.Close(ctx) connector, err := ydb.Connector(nativeDriver) if err != nil { panic(err) } db := sql.OpnDB(connector) defer db.Close() ... } ``` {% endlist %}
Project: Shifu (Provisioning / Automation & Configuration)
File: CHANGELOG-v0.22.0.md

<Feature>implement write in opcua protocol by @leepala in https://github.com/Edgenesis/shifu/pull/644 <bug Fix>Fix the opcua protocol creating too many clients, resulting in excessive session usage by @leepala in https://github.com/Edgenesis/shifu/pull/643 <BUG Fix>fix wrong type when write to opcua server by @leepala in https://github.com/Edgenesis/shifu/pull/650 Added design for FSM by @BtXin in https://github.com/Edgenesis/shifu/pull/633 Create CODEOFCONDUCT.md by @saiyan86 in https://github.com/Edgenesis/shifu/pull/645 Create CONTRIBUTING.md by @saiyan86 in https://github.com/Edgenesis/shifu/pull/646 Full Changelog: https://github.com/Edgenesis/shifu/compare/v0.21.0...v0.22.0
Project: Backstage (App Definition and Development / Application Definition & Image Build)
File: 06-utility-apis.md

id: utility-apis title: Utility APIs sidebar_label: Utility APIs description: Utility APIs NOTE: The new frontend system is in alpha and is only supported by a small number of plugins. Utility APIs are pieces of standalone functionality, interfaces that can be requested by plugins to use. They are defined by a TypeScript interface as well as a reference (an "API ref") used to access its implementation. They can be provided both by plugins and the core framework, and are themselves that can have inputs, be replaced, and be declaratively configured in your app-config. A common example of a utility API is a client interface to interact with the backend part of a plugin, such as the catalog client. Any frontend plugin can then request an implementation of that interface to make requests through. The following diagram shows a hypothetical application, which depends on two plugins and also provides some extra overrides. Note that both the plugins and the core framework provide utility APIs, and that they depend on each other. The app also chooses to use its overrides mechanism to supply a replacement implementation of one API, which takes precedence over the default one. Thus, all consumers of that API will be sure to get that new implementation provided to them. All utility APIs implement the `createApiExtension.factoryDataRef` output data type, and must attach exclusively to the `core` extension's `apis` input no matter who provided them. These defaults are provided out of the box by the `createApiExtension` framework function. Since utility APIs are extensions, they can also have inputs in advanced use cases. This is occasionally useful for complex APIs that can themselves be extended with additional programmatic functionality by adopters. The of the plugin docs The legacy docs on
Project: Monokle (App Definition and Development / Application Definition & Image Build)
File: cluster-mode.md

The Cluster Mode in Monokle Desktop provides real-time insights into the state of your clusters. The initial overview provides general information about its state and contained resources: The navigation on the left makes it easy to select types of resources and see their individual state and properties: And the YAML for each resource can be easily viewed, modified and applied back to your cluster for quick fixes: All existing validation functionality already available in Monokle is applied to the resources found in your cluster - making it easy to find resources that have validation warning or errors - for example, in regard to the built-in OPA policies: These errors are of course also highlighted in the source view - allowing you to immediately fix them and deploy the fixed resource back to your cluster. After selecting a pod resource in the cluster dashboard, the log tab allows you to see logs in a specific pod. On the shell tab, you can interact with a pod, you can run shell commands in this pod and see the results.
Project: Operator Framework (App Definition and Development / Application Definition & Image Build)
File: v1.24.0.md

title: v1.24.0 weight: 998976000 In Ansible & Helm-based operators, if you need to support `arm64`, add the following `| sed 's/aarch64/arm64/'` on the following line `ARCH := $(shell uname -m | sed 's/x86_64/amd64/')` The resulting line should look like this `ARCH := $(shell uname -m | sed 's/x86_64/amd64/' | sed 's/aarch64/arm64/')` See for more details.
Project: StarRocks (App Definition and Development / Database)
File: get_json_string.md

displayed_sidebar: "English" Analyzes and gets strings from the specified path (`jsonpath`) in a JSON string. If the format of `jsonstring` or `json_path` is wrong or if no matching value is found, this function will return NULL. The alias is getjsonobject. ```Haskell VARCHAR getjsonstring(VARCHAR jsonstr, VARCHAR jsonpath) ``` `json_str`: the JSON string. Supported data type is VARCHAR. `jsonpath`: the JSON path. Supported data type is VARCHAR. `jsonpath` starts with `$` and uses `.` as the path separator. `[ ]` is used as the array subscripts which starts from 0. For example, `$."my.key"[1]` indicates to obtain the second value form element `my.key`. Returns a value of the VARCHAR type. If no matching object is found, NULL is returned. Example 1: Get the value whose key is `k1`. ```Plain Text MySQL > SELECT getjsonstring('{"k1":"v1", "k2":"v2"}', "$.k1"); ++ | getjsonstring('{"k1":"v1", "k2":"v2"}', '$.k1') | ++ | v1 | ++ ``` Example 2: Get the value whose key is `a` from the first element. ```Plain Text MySQL > SELECT getjsonobject('[{"a":"123", "b": "456"},{"a":"23", "b": "56"}]', '$[0].a'); ++ | getjsonobject('[{"a":"123", "b": "456"},{"a":"23", "b": "56"}]', '$[0].a') | ++ | 123 | ++ ``` Example 3: Get the second element in the array whose key is `my.key` ```Plain Text MySQL > SELECT getjsonstring('{"k1":"v1", "my.key":["e1", "e2", "e3"]}', '$."my.key"[1]'); ++ | getjsonstring('{"k1":"v1", "my.key":["e1", "e2", "e3"]}', '$."my.key"[1]') | ++ | e2 | ++ ``` Example 4: Get the first element in the array whose path is `k1.key -> k2`. ```Plain Text MySQL > SELECT getjsonstring('{"k1.key":{"k2":["v1", "v2"]}}', '$."k1.key".k2[0]'); +--+ | getjsonstring('{"k1.key":{"k2":["v1", "v2"]}}', '$."k1.key".k2[0]') | +--+ | v1 | +--+ ``` Example 5: Get all values whose key is `k1` from the array. ```Plain Text MySQL > SELECT getjsonstring('[{"k1":"v1"}, {"k2":"v2"}, {"k1":"v3"}, {"k1":"v4"}]', '$.k1'); ++ | getjsonstring('[{"k1":"v1"}, {"k2":"v2"}, {"k1":"v3"}, {"k1":"v4"}]', '$.k1') | ++ | ["v1","v3","v4"] | ++ ``` GETJSONSTRING,GET,JSON,STRING
Project: Lagoon (App Definition and Development / Application Definition & Image Build)
File: install-lagoon-remote.md

Now we will install Lagoon Remote into the Lagoon namespace. The service is the broker. Create `lagoon-remote-values.yml` in your config directory as you did the previous two files, and update the values. rabbitMQPassword* ``` bash title="Get RabbitMQ password" kubectl -n lagoon-core get secret lagoon-core-broker -o jsonpath="{.data.RABBITMQ_PASSWORD}" | base64 --decode ``` rabbitMQHostname* ```bash title="lagoon-remote-values.yml" lagoon-core-broker.lagoon-core.svc.local ``` taskSSHHost* ```bash title="Update SSH Host" kubectl get service lagoon-core-broker-amqp-ext \ -o custom-columns="NAME:.metadata.name,IP ADDRESS:.status.loadBalancer.ingress[].ip,HOSTNAME:.status.loadBalancer.ingress[].hostname" ``` harbor-password* ```bash title="Get Harbor secret" kubectl -n harbor get secret harbor-harbor-core -o jsonpath="{.data.HARBORADMINPASSWORD}" | base64 --decode ``` Add the Harbor configuration from the step. ```yaml title="lagoon-remote-values.yml" lagoon-build-deploy: enabled: true extraArgs: "--enable-harbor=true" "--harbor-url=https://harbor.lagoon.example.com" "--harbor-api=https://harbor.lagoon.example.com/api/" "--harbor-username=admin" "--harbor-password=<from harbor-harbor-core secret>" rabbitMQUsername: lagoon rabbitMQPassword: <from lagoon-core-broker secret> rabbitMQHostname: lagoon-core-broker.lagoon-core.svc.cluster.local lagoonTargetName: <name of lagoon remote, can be anything> taskSSHHost: <IP of ssh service loadbalancer> taskSSHPort: "22" taskAPIHost: "api.lagoon.example.com" dbaas-operator: enabled: true mariadbProviders: production: environment: production hostname: 172.17.0.1.nip.io readReplicaHostnames: 172.17.0.1.nip.io password: password port: '3306' user: root development: environment: development hostname: 172.17.0.1.nip.io readReplicaHostnames: 172.17.0.1.nip.io password: password port: '3306' user: root ``` Enable Install Lagoon Remote: ```bash title="Install Lagoon remote" helm upgrade --install --create-namespace \ --namespace lagoon \ -f lagoon-remote-values.yml \ lagoon-remote lagoon/lagoon-remote ```
Project: KubeBlocks by ApeCloud (App Definition and Development / Database)
File: connect-database-in-testing-environment.md

title: Connect database in testing environment description: How to connect to a database in testing environment keywords: [connect to a database, testing environment, test environment] sidebar_position: 2 sidebar_label: Testing environment You can use the `kbcli cluster connect` command and specify the cluster name to be connected. ```bash kbcli cluster connect ${cluster-name} ``` The lower-level command is actually `kubectl exec`. The command is functional as long as the K8s API server is accessible. Execute the following command to get the network information of the targeted database and connect it with the printed IP address. ```bash kbcli cluster connect --show-example --show-password ${cluster-name} ``` Information printed includes database addresses, port No., username, password. The figure below is an example of MySQL database network information. Address: -h specifies the server address. In the example below it is 127.0.0.1 Port: -P specifies port No. , In the example below it is 3306. User: -u is the user name. Password: -p shows the password. In the example below, it is hQBCKZLI. :::note The password does not include -p. :::
Project: MongoDB (App Definition and Development / Database)
File: README_range_deleter.md

The `config.rangeDeletions` collection is a shard-local internal collection containing a document for each range needing to be eventually cleared up; documents in `config.rangeDeletions` are usually referred as "range deletion tasks" or "range deleter documents". The complete format of range deletion tasks is defined in , with the relevant fields being the following: `collectioUUID`: the UUID of the collection the range belongs to `range`: the `[min, max)` shard key range to be deleted `pending`: boolean flag present if the range is not yet ready for deletion When a migration starts, a `pending` range deletion task is created both on donor and recipient side and such documents are later modified depending on the migration outcome: Commit: donor range deletion document flagged as ready (remove `pending` flag) AND recipient range deletion document deleted. Abort: donor range deletion document deleted AND recipient range deletion document flagged as ready (remove `pending` flag). The is a primary only service living on shards that is driven by the state persisted on `config.rangeDeletions`: for each collection with at least one range deletion task, the service keeps track in memory of the mapping `<collection uuid, [list of ranges containing orphaned documents]>`. Its main functions are: Scheduling for deletion an orphaned range only when ongoing reads retaining such range have locally drained. Performing the actual deletion of orphaned docs belonging to a range (this happens on a dedicated thread only performing range deletions). Providing a way to know when orphans deletions on specific ranges have completed (callers can get a future to wait for a range to become orphans free, such future gets notified only when all range deletion tasks overlapping with the specified range have completed). When a shard node steps up, a thread to asynchronously recover range deletions is spawned (being asynchronous, it does not block step-up); recovery happens in the following steps: Disallow writes on config.rangeDeletions For each document in config.rangeDeletions: Register a task on the range deleter service Reallow writes on config.rangeDeletions The is taking care of keeping in sync the in-memory state with the persistent state by reacting to documents modifications on `config.rangeDeletions` in the following ways: Register a task on the service when a range deletion task document is flagged as ready (insert document without pending flag or update existing document to remove such flag) De-register a task from the service when a range deletion task is completed (document deleted) When a shard primary steps down, the whole service is destroyed: Potentially ongoing orphans cleanup are stopped In-memory state is cleared up The shutdown of the service is structured in the following way in order not to block step-down: Call for shutdown of the thread performing range deletions On next step-up, join such thread (it's very improbable for the shutdown to have not completed yet following a step-down+up, hence this should be a no-op)
Project: StarRocks (App Definition and Development / Database)
File: named_struct.md

displayed_sidebar: "English" Creates a struct with the specified field names and values. This function is supported from v3.1 onwards. ```Haskell STRUCT named_struct({STRING name1, ANY val1} [, ...] ) ``` `nameN`: A STRING field. `valN`: An expression of any type that specifies the value for field N. Values are nullable. The expressions of names and values must be in pairs. Otherwise, the struct cannot be created. You must pass at least one pair of field name and value, separated by a comma (`,`). Returns a STRUCT value. ```plain SELECT named_struct('a', 1, 'b', 2, 'c', 3); +--+ | named_struct('a', 1, 'b', 2, 'c', 3) | +--+ | {"a":1,"b":2,"c":3} | +--+ SELECT named_struct('a', null, 'b', 2, 'c', 3); +--+ | named_struct('a', null, 'b', 2, 'c', 3) | +--+ | {"a":null,"b":2,"c":3} | +--+ ```