cwielenb/CNCF-Dataset huggingface dataset:
Project: Flink (App Definition and Development / Streaming & Messaging)
File: job.md

title: "JOB Statements" weight: 16 type: docs aliases: /dev/table/sql/job.html <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> Job statements are used for management of Flink jobs. Flink SQL supports the following JOB statements for now: SHOW JOBS STOP JOB {{< tabs "show jobs statement" >}} {{< tab "SQL CLI" >}} The following examples show how to run `JOB` statements in . {{< /tab >}} {{< /tabs >}} {{< tabs "show jobs" >}} {{< tab "SQL CLI" >}} ```sql Flink SQL> SHOW JOBS; +-+-++-+ | job id | job name | status | start time | +-+-++-+ | 228d70913eab60dda85c5e7f78b5782c | myjob | RUNNING | 2023-02-11T05:03:51.523 | +-+-++-+ Flink SQL> SET 'state.savepoints.dir'='file:/tmp/'; [INFO] Execute statement succeeded. Flink SQL> STOP JOB '228d70913eab60dda85c5e7f78b5782c' WITH SAVEPOINT; +--+ | savepoint path | +--+ | file:/tmp/savepoint-3addd4-0b224d9311e6 | +--+ ``` {{< /tab >}} {{< /tabs >}} ```sql SHOW JOBS ``` Show the jobs in the Flink cluster. <span class="label label-danger">Attention</span> SHOW JOBS statements only work in or . ```sql STOP JOB '<job_id>' [WITH SAVEPOINT] [WITH DRAIN] ``` Stop the specified job. WITH SAVEPOINT Perform a savepoint right before stopping the job. The savepoint path could be specified with either in the cluster configuration or via `SET` statements (the latter would take precedence). WITH DRAIN Increase the watermark to the maximum value before the last checkpoint barrier. Use it when you want to terminate the job permanently. <span class="label label-danger">Attention</span> STOP JOB statements only work in or . {{< top >}}
Project: Metal³ (Provisioning / Automation & Configuration)
File: ironic-endpoint-keepalived-configuration.md

The motivation behind maintaining Ironic Endpoint with Keepalived is to ensure that the Ironic Endpoint IP is also passed onto the target cluster control plane. This also guarantees that once pivoting is done and the management cluster is taken down, target cluster controlplane can re-claim the ironic endpoint IP through keepalived. The end goal is to make ironic endpoint reachable in the target cluster. ```bash kustomize build $BMOPATH/ironic-deployment/keepalived | kubectl apply -f - ``` where $BMOPATH points to the baremetal-operator path. Ironic Endpoint IP is maintained with Keepalived which now runs in a separate docker container in bmo deployment. The container is named `ironic-endpoint-keepalived`. The container files reside in `resources/keepalived-docker` path. ```bash tree resources/keepalived-docker/ Dockerfile manage-keepalived.sh OWNERS sample.keepalived.conf ``` It is assumed that the docker image is uploaded in some registry and the image URL is used with `ironic-deployment/keepalived/keepalived_patch.yaml` to replace the default image URL with the correct URL through kustomization. Important Note When the baremetal-operator is deployed through metal3-dev-env, this container inherits the following environment variables through configmap: ```bash $PROVISIONING_IP $PROVISIONING_INTERFACE ``` In case you are deploying baremetal-operator locally, make sure to populate and export these environment variables before deploying.
Project: ShardingSphere (App Definition and Development / Database)
File: Feb_23_Stack_Overflow.en.md

+++ title = "Apache ShardingSphere is on the Stack Overflow Podcast, InfoQ and FOSDEM!" weight = 36 chapter = true +++ Our PMC Trista Pan, and our community contributor Yacine Si Tayeb, joined the podcast to share about the Apache ShardingSphere ecosystem. They shared the advantages of data sharding and how it splits a database into smaller distributed databases to spread query load across multiple servers. During the conversation, they introduced the complexities that arise with large databases, the rise of open source in China, and how Tristas position as the only female Apache member in China can help other women get into technology. Our community thanks the Stack Overflow team for this opportunity, and for being such great hosts. Being hosted on such a great platform is both an achievement and an opportunity for us. Our PMC teams community building efforts continue on what has so far been a great path that included being published on InfoQ with a piece titled and also joining fantastic events such as . During FOSDEM our community shared how to build a distributed & secure ecosystem with . Check out below the episode notes, and the links to the podcast and FOSDEMs recording of the talk. ShardingSphere builds distributed data systems, making it easier for organizations to load balance massive data stores across multiple servers. Now that open source software has taken over Western software, its Chinas turn. Even big companies like and are opening up their projects. Trista is the in China, which is both an honor and a demonstration of how much work needs to be done to support women in STEM. Link: We thank the InfoQ editorial team for their support, and offering to share ShardingSpheres features and architecture with their readers. In this piece we introduced the strengths of data sharding, and Database Plus our concept for creating a distributed database system for more than sharding, positioned above DBMS. Link: The FOSDEM session focused on introducing how to empower PostgreSQL thanks to the ecosystem provided by Apache ShardingSphere. ShardingSpheres capability to transform any DBMS into a distributed database system, and enhance it with data sharding, elastic scaling, and encryption features is unique among current open source projects. Link: Yacine Si Tayeb SphereEx Head of International Operations Apache ShardingSphere Contributor Passionate about technology and innovation, Yacine moved to Beijing to pursue his Ph.D. in Business Administration, and fell in awe of the local startup and tech scene. His career path has so far been shaped by opportunities at the intersection of technology and business. Recently he took on a keen interest in the development of the ShardingSphere database middleware ecosystem and Open-Source community building.
Project: ArangoDB (App Definition and Development / Database)
File: basic_outcome_failure_exception_from_error.md

+++ title = "`auto basicoutcomefailureexceptionfrom_error(const EC &)`" description = "ADL discovered free function synthesising an exception type from an error type, used by the `.failure()` observers." +++ Synthesises an exception type from an error type, used by the {{% api "exception_type failure() const noexcept" %}} observers. ADL discovered. Default overloads for this function are defined in Outcome for {{% api "std::error_code" %}} and `boost::system::errorcode`, these return `std::makeexceptionptr(std::systemerror(ec))` and `boost::copyexception(boost::system::systemerror(ec))` respectively. Overridable: Argument dependent lookup. Requires: Nothing. Namespace: Namespace of `EC` type. Header: `<boost/outcome/stdoutcome.hpp>`, `<boost/outcome/boostoutcome.hpp>`
Project: Hazelcast Jet (App Definition and Development / Streaming & Messaging)
File: 2020-01-28-new-website.md

title: Announcing New Documentation Website author: Emin Demirci authorURL: http://twitter.com/emndmrc authorImageURL: https://3l0wd94f0qdd10om8642z9se-wpengine.netdna-ssl.com/wp-content/uploads/2019/07/emin-demirci-170x170.png We are excited to tell you that we've just launched a new website for Hazelcast Jet, where we share technical content regarding the project including how-to-guides and technical design documents. We'll keep this website frequently updated. Stay tuned!
Project: Meshery (Provisioning / Automation & Configuration)
File: v0.6.115.md

layout: release date: 2023-08-03 tag: v0.6.115 Update README.md @theBeginner86 (#8358) Fix connection status endpoint @abdullah1308 (#8354) Bump github.com/docker/docker from 20.10.21+incompatible to 20.10.24+incompatible in /scripts/component_updater @dependabot (#8153) update relationship def and policy @MUzairS15 (#8327) [MeshSync] Package update v0.6.11 @leecalcote (#8325) Specify additional specific load generator options in mesheryctl perf @abdullah1308 (#8148) [Performance Profile] - Specify flags in performance profile @UsmanDev09 (#8219) [bug] Fix broken filters page @theBeginner86 (#8359) [Meshmodels] - Displaying duplicate components and models @UsmanDev09 (#8275) [Responsiveness] - Made Configurations design page responsive @UsmanDev09 (#8357) [UI] Added required prop in Publish Modal to publish catalog from MeshMap @Yashsharma1911 (#8349) Bump word-wrap from 1.2.3 to 1.2.4 in /ui @dependabot (#8213) Bump @mui/material from 5.13.7 to 5.14.3 in /provider-ui @dependabot (#8336) Bump react-dom from 17.0.2 to 18.2.0 in /ui @dependabot (#8329) Bump eslint from 8.45.0 to 8.46.0 in /provider-ui @dependabot (#8337) Bump @cypress/code-coverage from 3.10.7 to 3.11.0 in /provider-ui @dependabot (#8334) Bump @babel/types from 7.21.3 to 7.22.5 in /ui @dependabot (#8330) Bump tar from 6.1.12 to 6.1.15 in /ui @dependabot (#8328) Bump next from 13.4.7 to 13.4.12 in /provider-ui @dependabot (#8333) Bump @mui/styles from 5.12.0 to 5.14.3 in /ui @dependabot (#8331) Bump @mui/icons-material from 5.11.16 to 5.14.3 in /provider-ui @dependabot (#8335) Update MeshMap Snapshot Service @Abhishek-kumar09 (#8332) [bug] Fix broken filters page @theBeginner86 (#8359) [CI] Update Meshery Helm Chart @theBeginner86 (#8355) Bump word-wrap from 1.2.3 to 1.2.4 in /ui @dependabot (#8213) Bump @mui/material from 5.13.7 to 5.14.3 in /provider-ui @dependabot (#8336) Bump react-dom from 17.0.2 to 18.2.0 in /ui @dependabot (#8329) Bump eslint from 8.45.0 to 8.46.0 in /provider-ui @dependabot (#8337) Bump @cypress/code-coverage from 3.10.7 to 3.11.0 in /provider-ui @dependabot (#8334) Bump github.com/layer5io/service-mesh-performance from 0.3.4 to 0.6.1 @dependabot (#8338) Bump @babel/types from 7.21.3 to 7.22.5 in /ui @dependabot (#8330) Bump tar from 6.1.12 to 6.1.15 in /ui @dependabot (#8328) Bump next from 13.4.7 to 13.4.12 in /provider-ui @dependabot (#8333) Bump @mui/styles from 5.12.0 to 5.14.3 in /ui @dependabot (#8331) Bump @mui/icons-material from 5.11.16 to 5.14.3 in /provider-ui @dependabot (#8335) Bump github.com/onsi/ginkgo/v2 from 2.9.5 to 2.11.0 @dependabot (#8341) Update MeshMap Snapshot Service @Abhishek-kumar09 (#8332) [CI] Enhance Edge Release Channel with Versioning @vaibhav-xt (#8259) Update MeshMap Snapshot Service @Abhishek-kumar09 (#8332) Thank you to our contributors for making this release possible: @Abhishek-kumar09, @MUzairS15, @UsmanDev09, @Yashsharma1911, @abdullah1308, @dependabot, @dependabot[bot], @leecalcote, @theBeginner86 and @vaibhav-xt
Project: KICS (Provisioning / Security & Compliance)
File: 3db3f534-e3a3-487f-88c7-0a9fbf64b702.md

title: AmazonMQ Broker Encryption Disabled hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 3db3f534-e3a3-487f-88c7-0a9fbf64b702 Query name: AmazonMQ Broker Encryption Disabled Platform: Terraform Severity: <span style="color:#bb2124">High</span> Category: Encryption URL: AmazonMQ Broker should have Encryption Options defined<br> ```tf title="Positive test num. 1 - tf file" hl_lines="1" resource "awsmqbroker" "positive1" { broker_name = "example" configuration { id = awsmqconfiguration.test.id revision = awsmqconfiguration.test.latest_revision } engine_type = "ActiveMQ" engine_version = "5.15.9" hostinstancetype = "mq.t2.micro" securitygroups = [awssecurity_group.test.id] user { username = "ExampleUser" password = "MindTheGap" } } ``` ```tf title="Negative test num. 1 - tf file" resource "awsmqbroker" "negative1" { broker_name = "example" configuration { id = awsmqconfiguration.test.id revision = awsmqconfiguration.test.latest_revision } engine_type = "ActiveMQ" engine_version = "5.15.9" hostinstancetype = "mq.t2.micro" securitygroups = [awssecurity_group.test.id] user { username = "ExampleUser" password = "MindTheGap" } encryption_options { kmskeyid = "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab" useawsowned_key = false } } resource "awsmqbroker" "negative2" { broker_name = "example" configuration { id = awsmqconfiguration.test.id revision = awsmqconfiguration.test.latest_revision } engine_type = "ActiveMQ" engine_version = "5.15.9" hostinstancetype = "mq.t2.micro" securitygroups = [awssecurity_group.test.id] user { username = "ExampleUser" password = "MindTheGap" } encryption_options { } } ```
Project: Vald (App Definition and Development / Database)
File: index-correction.md

In the Vald cluster, the same Index is replicated to multiple agents due to the `index_replica` setting. However, inconsistencies between replicas may occur due to pod eviction or the occurrence of OOM killer during vector insertions. For example, The timestamp of the index differs between agents (some agents have an old index saved and it has not been updated). The number of replicas does not meet the value set in `index_replica`. To resolve these inconsistencies, you can use the `Index Correction` feature. `Index Correction` is implemented as a , checking the consistency between replicas regularly and resolving any inconsistencies. enabled Turns the index correction feature on/off. schedule Sets the interval for the job start in cron notation (the default value is `3 6 *`, which means 3:06 AM every day). suspend for CronJob. ```yaml manager: index: corrector: enabled: true schedule: "3 6 *" suspend: false ``` Processing time Under conditions of 10 million identical vectors(not including `indexreplica`) and 10 agent replicas, the processing takes about 30~40 minutes (this is only a reference, and the actual execution time may vary depending on the infrastructure). Time complexity of the process is `O(MN)` where M is the number of identical vector items and N is the number of agent replicas. `indexreplica` does not matter for the processing time. concurrencyPolicy `Forbid` is set internally, so a new job will not be created while an existing job is running. In other words, if the process does not finish within the interval specified by the schedule, the next job will not be scheduled. Index operations during correction Vector operations performed after the start of the index correction job are not considered in that job.
Project: KICS (Provisioning / Security & Compliance)
File: fae52418-bb8b-4ac2-b287-0b9082d6a3fd.md

title: EFS With Vulnerable Policy hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: fae52418-bb8b-4ac2-b287-0b9082d6a3fd Query name: EFS With Vulnerable Policy Platform: Terraform Severity: <span style="color:#ff7213">Medium</span> Category: Access Control URL: EFS (Elastic File System) policy should avoid wildcard in 'Action' and 'Principal'.<br> ```tf title="Positive test num. 1 - tf file" hl_lines="16" provider "aws" { region = "us-east-1" } resource "awsefsfilesystem" "notsecure" { creation_token = "efs-not-secure" tags = { Name = "NotSecure" } } resource "awsefsfilesystempolicy" "notsecurepolicy" { filesystemid = awsefsfilesystem.notsecure.id policy = <<POLICY { "Version": "2012-10-17", "Id": "ExamplePolicy01", "Statement": [ { "Sid": "ExampleStatement01", "Effect": "Allow", "Principal": { "AWS": "*" }, "Action": [ "elasticfilesystem:*" ] } ] } POLICY } ``` ```tf title="Negative test num. 1 - tf file" resource "awsefsfile_system" "fs" { creation_token = "my-product" } resource "awsefsfilesystempolicy" "policy" { filesystemid = awsefsfile_system.fs.id policy = <<POLICY { "Version": "2012-10-17", "Id": "ExamplePolicy01", "Statement": [ { "Sid": "ExampleStatement01", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::111122223333:user/Carlos" } "Resource": "awsefsfile_system.test.arn", "Action": [ "elasticfilesystem:ClientMount", "elasticfilesystem:ClientWrite" ], "Condition": { "Bool": { "aws:SecureTransport": "true" } } } ] } POLICY } ```
Project: Dapr (App Definition and Development / Application Definition & Image Build)
File: README.md

This project aims to test the performance of Dapr workflows under various conditions. VU (Virtual User): The number of concurrent workflows running at a time Iterations: Total number of workflow runs Req_Duration: Time taken to complete a workflow run. Sidecar: Dapr Sidecar For the test, a single instance of an application with Dapr sidecar was deployed on AKS and then different workflows were created using K6 framework. For each of the tests, app memory as well as Dapr sidecar memory limit was capped at 800 Mb. Workflow Description Contains 5 chained activities Each activity performs numeric calculation and the result is sent back to workflow Test Description Run workflow with concurrency of 50 and total runs 500 Calculate memory and cpu usage after the run Run the whole set again (total 5 times) without restarting the application Monitor sidecar memory and cpu usage after each set Workflow Description Contains 5 chained activities Each activity performs numeric calculation and the result is sent back to workflow. Test Description Run workflow with concurrency of 50 and total runs 500 Restart the application Run worklfow with concurrency of 100 and total runs 500 Restart the application Run worklfow with concurrency of 150 and total runs 500 Monitor memory/cpu usage as well as req_duration(time taken to complete a workflow) after each set Workflow Description Contains 5 chained activities Each activity performs numeric calculation and the result is sent back to workflow Test Description Run workflow with concurrency of 500 and for total runs 3000 Verify there are no errors and all workflow runs pass Monitor memory/cpu usage as well as req_duration(time taken to complete a workflow) after the run Workflow Description Contains 5 activities in parallel Each activity performs numeric calculation and the result is aggregated at the workflow Test Description Run workflow with concurrency of 110 and total runs 550 Verify there are no errors and all workflow runs pass Monitor memory/cpu usage as well as req_duration(time taken to complete a workflow) after the run Contains 3 activities in series Activity 1 saves data of given size in the configured statestore Activity 2 gets saved data from the statestore using key Activity 3 deletes data from the statestore Workflow takes data size as input and passes a string of that size as payload to the activity Test Description Run workflow with payload size of 10KB with concurrency of 50 and total runs 500 Restart the application Run workflow with payload size of 50KB with concurrency of 50 and total runs 500 Restart the application Run workflow with payload size of 100KB with concurrency of 50 and total runs 500 Monitor memory usage after each run
Project: YugabyteDB (App Definition and Development / Database)
File: scaling-transactions.md

title: Transactions headerTitle: Transactions linkTitle: Transactions description: Scaling concurrent transactions in YugabyteDB. headcontent: Transaction performance when scaling out menu: v2.18: name: Transactions identifier: scaling-transactions-bench parent: explore-scalability weight: 60 type: docs The is the gold standard for measuring the transaction processing capacity of a database. It simulates order entry for a wholesale parts supplier. It includes a mixture of transaction types, including the following: Entering and delivering orders Recording payments Checking order status Monitoring stock levels The performance metric measures the number of new orders that can be processed per minute and is expressed in transactions per minute (TPM-C). The benchmark is designed to simulate business expansion by increasing the number of warehouses. The following shows the results for the TPC-C benchmark using the following configuration: YugabyteDB version 2.18.0 AWS, us-west region, c5d.9xlarge instances Replication factor 3 The 100K benchmark was run on a universe with 59 nodes. | Efficiency | TPMC | Average NewOrder Latency (ms) | YSQL Ops/sec | CPU USage (%) | | : | - | | | - | | 99.83 | 1283804.18 | 51.86 | 348602.48 | 58.22 | With only 59 nodes, YugabyteDB breezed through the 100K warehouse at a 99.83% efficiency and clocked 1.3 million transactions per minute. The 100K benchmark was run on a universe scaled out to 75 nodes. | Efficiency | TPMC | Average NewOrder Latency (ms) | YSQL Ops/sec | CPU USage (%) | | : | --| | | - | | 99.3 | 1M | 123.33 | 950K | 80 | The latency and IOPS during the test execution are shown in the following illustration.
Project: Meshery (Provisioning / Automation & Configuration)
File: v0.5.24.md

layout: release date: 2021-06-18 tag: v0.5.24 Fix Meshery logo click behaviour @sudo-NithishKarthik (#3225) fix edge channel deployments @navendu-pottekkat (#3233) [Bug fix] Fixed `dataFetch` catching error when getting Grafana Boards @dhruv0000 (#3220) change service port @s1ntaxe770r (#3224) [mesheryctl] Wait for pods to terminate on `system stop` @navendu-pottekkat (#3227) meshery fixes for kubeconfig @kumarabd (#3228) switch provider package to edge-latest @kumarabd (#3221) Kumarabd/feature/broker @kumarabd (#3219) changes for kubeconfig validation @kumarabd (#3191) Moved swagger.yaml to /helpers @piyushsingariya (#3211) [ui] Update grammar in operator removal message @josephmidura (#3188) Fix #3192: Fix installation steps in mesheryctl bash file @Abhishek-kumar09 (#3193) Update README.md: Fix meshery badges @amino19 (#3194) Add dynamic Management Pane UI @utkarsh-pro (#3202) Added imports for remote provider extensions @Ayush-Khandelwal-007 (#3203) Added Application UI @Chadha93 (#3186) Reverted `gqlpraser`, Addon Resolvers Fix and added Make commands @dhruv0000 (#3181) [mesheryctl] `mesheryctl perf` subcommands @piyushsingariya (#3082) [mesheryctl] Added `mesheryctl exp application` @piyushsingariya (#3170) Fix Netlify builds @ramrodo (#3179) operator panic fix @kumarabd (#3172) [Docs] - Fixed CI Failing Issue Due to Error in Syntax of Main FIle @adithyaakrishna (#3190) Update CodeCov report configuration @ramrodo (#3177) docs and make updated for error utils @kumarabd (#3216) [Docs] `mesheryctl system mesh validate` page added @amino19 (#3149) Added the new releases of meshery to Docs @iambami (#3196) [Docs] - Fixed CI Failing Issue Due to Error in Syntax of Main FIle @adithyaakrishna (#3190) [Docs] - Added status command files @umairnsr87 (#3086) Thank you to our contributors for making this release possible: @Abhishek-kumar09, @Ayush-Khandelwal-007, @Chadha93, @adithyaakrishna, @amino19, @dhruv0000, @iambami, @josephmidura, @kumarabd, @leecalcote, @navendu-pottekkat, @piyushsingariya, @ramrodo, @s1ntaxe770r, @sudo-NithishKarthik, @umairnsr87 and @utkarsh-pro