cwielenb/CNCF-Dataset huggingface dataset:
Project: YugabyteDB (App Definition and Development / Database)
File: network-access.md

<!-- +++ private = true +++ --> YugabyteDB Managed only allows access to clusters from trusted IP addresses. For applications in peered VPCs to be able to connect, you need to add the CIDR of the peered VPC to the cluster . You can also assign IP allow lists to your cluster any time after the cluster is created. You can add IP addresses using any combination of the following options. | Option | Description | | :-- | :- | | Add Current IP Address | Creates an allow list using the public IP address of your computer and adds it to the cluster IP allow list. | | Add Peered VPC Networks | Only available for clusters being deployed in a VPC. VPCs must be peered, and the peering connection active for the peered networks to be added to the IP allow list.<br>Choose Add All Peered Networks to create an IP allow list from every network peered with the cluster VPC, and add it to the cluster.<br>Choose Add Individual Peered Networks to select specific peered networks to add to the cluster IP allow list. | | Add Existing IP Allow List | Choose from a list of IP allow lists already created for your account. | | Create New IP Allow List | Create a new IP allow list and manually enter the CIDR and public IP addresses. | Enable Public Access for this Cluster - To connect to a cluster deployed in a VPC from a public IP address (including your current address), you must enable Public Access for the cluster. When enabled, a public IP address is added to each region of the cluster. You can view the private and public host addresses under Connection Parameters on the cluster Settings > Infrastructure tab. For clusters in AWS and Azure, you can connect your cluster to your application VPC using a private link. To do this, you add a private service endpoint (PSE) to each region in your cluster. You can also after your cluster is created. To use a private link for network isolation and security, choose Create Private Service Endpoint, then add security principals. A security principal is a cloud provider account with permissions to manage endpoints. In AWS, this would be Amazon resource names, and in Azure, Subscription IDs. After the cluster is created, you still need to complete the private link setup by adding an endpoint on your application VPC for each region in your cluster, and linking them to the PSEs on your cluster. For more information, refer to .
Project: Backstage (App Definition and Development / Application Definition & Image Build)
File: v1.4.0.md

id: v1.4.0 title: v1.4.0 description: Backstage Release v1.4.0 These are the release notes for the v1.4.0 release of . A huge thanks to the whole team of maintainers and contributors as well as the amazing Backstage Community for the hard work in getting this release developed and done. Backstage Search is now promoted to v1.0 with a stable Search API for plugin developers to add search to their plugins, and app integrators to expose that to their users. No additional features are released as part of the major release as the maintainers of the search packages focused on the following for this release: Bug fixes and clean up of deprecations. Improved error handling for missing indices. Separation of AWS Elastic and Open Elastic support. Refactor the packages by moving reusable components from `@backstage/plugin-search` to `@backstage/plugin-search-react`. Improved documentation including tutorials for how to . With moving to v1.0 we also recommend using Elasticsearch for production usage. Moving forward, the search maintainers will prioritize the Elasticsearch engine while well rely on the community to maintain the Postgres engine. This release adds the new `@backstage/backend-app-api` and `@backstage/backend-plugin-api` packages, both part of the . These packages are highly experimental and we do not recommend using them for any purpose, yet. The `@backstage/cli` had a large number of deprecated commands removed, for example, `app:build` is now gone and should be replaced with `package build`. For a full list of commands that were removed, see the . Many deprecated symbols have now been removed from `@backstage/plugin-auth-backend`, most notably the provider factories such as `createGithubProvider` have been removed, which should be replaced with `providers.github.create`. See the for more details. This is a new module for `@backstage/plugin-api-docs` that exports a widget implementation for rendering gRPC docs generated by `protoc-gen-doc`. Contributed by . This is a new module for `@backstage/plugin-catalog-backend` that exports a catalog processor which de-references `$ref` fields in OpenAPI definitions. Contributed by . A plugin to embed the inside your Backstage instance to run queries against GraphQL endpoints. Contributed by . `@backstage/plugin-scaffolder-backend`, please upgrade to the latest version if you are using this module. We recommend that you keep your Backstage project up to date with this latest release. For more guidance on how to upgrade, check out the documentation for . Below you can find a list of links and references to help you learn about and start using this new release. , , and Backstage's for discussions and support Backstage , , and Sign up for our if you want to be informed about what is happening in the world of Backstage.
Project: ClickHouse (App Definition and Development / Database)
File: monitoring.md

slug: /en/operations/monitoring sidebar_position: 45 sidebar_label: Monitoring description: You can monitor the utilization of hardware resources and also ClickHouse server metrics. keywords: [monitoring, observability, advanced dashboard, dashboard, observability dashboard] import SelfManaged from '@site/docs/en/snippets/selfmanagedonly_automated.md'; <SelfManaged /> You can monitor: Utilization of hardware resources. ClickHouse server metrics. <img width="400" alt="Screenshot 2023-11-12 at 6 08 58 PM" src="https://github.com/ClickHouse/ClickHouse/assets/3936029/2bd10011-4a47-4b94-b836-d44557c7fdc1" /> ClickHouse comes with a built-in advanced observability dashboard feature which can be accessed by `$HOST:$PORT/dashboard` (requires user and password) that shows the following metrics: Queries/second CPU usage (cores) Queries running Merges running Selected bytes/second IO wait CPU wait OS CPU Usage (userspace) OS CPU Usage (kernel) Read from disk Read from filesystem Memory (tracked) Inserted rows/second Total MergeTree parts Max parts for partition ClickHouse also monitors the state of hardware resources by itself such as: Load and temperature on processors. Utilization of storage system, RAM and network. This data is collected in the `system.asynchronousmetriclog` table. ClickHouse server has embedded instruments for self-state monitoring. To track server events use server logs. See the section of the configuration file. ClickHouse collects: Different metrics of how the server uses computational resources. Common statistics on query processing. You can find metrics in the , , and tables. You can configure ClickHouse to export metrics to . See the in the ClickHouse server configuration file. Before configuring export of metrics, you should set up Graphite by following their official . You can configure ClickHouse to export metrics to . See the in the ClickHouse server configuration file. Before configuring export of metrics, you should set up Prometheus by following their official . Additionally, you can monitor server availability through the HTTP API. Send the `HTTP GET` request to `/ping`. If the server is available, it responds with `200 OK`. To monitor servers in a cluster configuration, you should set the parameter and use the HTTP resource `/replicasstatus`. A request to `/replicasstatus` returns `200 OK` if the replica is available and is not delayed behind the other replicas. If a replica is delayed, it returns `503 HTTPSERVICEUNAVAILABLE` with information about the gap.
Project: StarRocks (App Definition and Development / Database)
File: EXPLAIN_ANALYZE.md

displayed_sidebar: "English" Executes the specified SQL statement, and shows the query profile of this statement. For more information about query profile, see . This feature is supported from v3.1 onwards. CAUTION Only users with the SELECT or INSERT privilege on a specific table can perform this operation. ```SQL EXPLAIN ANALYZE <statement> ``` `statement`: the SQL statement of which to show the query profile. Both and are supported. Please note that when you analyze the profiles of an INSERT INTO statement, no data will actually be inserted. By default, the transaction is aborted, ensuring that no unintended changes are made to the data in the process of profile analysis. Example 1: Simulate and analyze a SELECT statement. The query result are discarded. Example 2: Simulate and analyze an INSERT INTO statement. The loading transaction will be aborted.
Project: Numaflow (App Definition and Development / Streaming & Messaging)
File: releasing.md

Always create a release branch for the releases, for example branch `release-0.5` is for all the v0.5.x versions release. If it's a new release branch, simply create a branch from `main`. Cherry-pick fixes to the release branch, skip this step if it's the first release in the branch. Run `make test` to make sure all test cases pass locally. Push to remote branch, and make sure all the CI jobs pass. Run `make prepare-release VERSION=v{x.y.z}` to update version in manifests, where `x.y.x` is the expected new version. Follow the output of last step, to confirm if all the changes are expected, and then run `make release VERSION=v{x.y.z}`. Follow the output, push a new tag to the release branch, GitHub actions will automatically build and publish the new release, this will take around 10 minutes. Test the new release, make sure everything is running as expected, and then recreate a `stable` tag against the latest release. ```shell git tag -d stable git tag -a stable -m stable git push -d {your-remote} stable git push {your-remote} stable ``` Find the new release tag, and edit the release notes.
Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.1.0.2.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Integrate Snappy compression | Major | . | Eli Collins | Alejandro Abdelnur | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Make streaming to handle non-utf8 byte array | Major | . | Runping Qi | Klaas Bosteels | | | Add support for application-specific typecodes to typed bytes | Blocker | . | Klaas Bosteels | Klaas Bosteels | | | Cleanup FS\* processIOError methods | Major | namenode | Eli Collins | Eli Collins | | | Backport HADOOP-4885 to branch-1 | Major | namenode | Brandon Li | Brandon Li | | | rename hadoop 64 bit rpm/deb package name | Major | . | Giridharan Kesavan | Giridharan Kesavan | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | TypedBytesInput's readRaw() does not preserve custom type codes | Blocker | contrib/streaming | Klaas Bosteels | Klaas Bosteels | | | removedStorageDirs is not updated everywhere we remove a storage dir | Major | namenode | Eli Collins | Eli Collins | | | A single failed name dir can cause the NN to exit | Critical | namenode | Eli Collins | Eli Collins | | | Distributed caches are not removed properly | Critical | distributed-cache | Allen Wittenauer | Thomas Graves | | | Deadlock in metrics | Major | metrics | Kihwal Lee | Kihwal Lee | | | ProcfsBasedProcessTree#constructProcessInfo() may throw NumberFormatException | Critical | . | Ted Yu | Ted Yu | | | Webhdfs "SETOWNER" call returns incorrect content-type | Major | webhdfs | bc Wong | Tsz Wo Nicholas Sze | | | cannot read empty file using webhdfs | Major | webhdfs | Zhanwei Wang | Tsz Wo Nicholas Sze | | | Allow more aggressive action on detection of the jetty issue | Major | tasktracker | Kihwal Lee | Thomas Graves | | | 64bit secure datanodes do not start as the jsvc path is wrong | Major | . | Arpit Gupta | Arpit Gupta | | | create the configure script for native compilation as part of the build | Blocker | build | Giridharan Kesavan | Giridharan Kesavan |
Project: StarRocks (App Definition and Development / Database)
File: md5sum_numeric.md

displayed_sidebar: "English" Calculates the 128-bit checksum of multiple strings. The checksum is represented by a decimal string of digits. ```Haskell LARGEINT md5sum_numeric(VARCHAR expr,...); ``` `expr`: the string to calculate. It must be of the VARCHAR type. Returns a checksum of the LARGEINT type. If the input is empty, an error is returned. ```Plain Text mysql> select md5sum_numeric("starrocks"); +--+ | md5sum_numeric('starrocks') | +--+ | 328761383472799310362963866384446095221 | +--+ 1 row in set (0.01 sec) mysql> select md5sum_numeric("starrocks","star"); +--+ | md5sum_numeric('starrocks', 'star') | +--+ | 163436627872604162110984707546327199448 | +--+ 1 row in set (0.01 sec) mysql> select md5sum_numeric(null); +--+ | md5sum_numeric(NULL) | +--+ | 281949768489412648962353822266799178366 | +--+ 1 row in set (0.01 sec) ```
Project: YDB (App Definition and Development / Database)
File: from_as_table.md

Accessing named expressions as tables using the `AS_TABLE` function. `AS_TABLE($variable)` lets you use the value of `$variable` as the data source for the query. In this case, the variable `$variable` must have the type `List<Struct<...>>`. Example ```yql $data = AsList( AsStruct(1u AS Key, "v1" AS Value), AsStruct(2u AS Key, "v2" AS Value), AsStruct(3u AS Key, "v3" AS Value)); SELECT Key, Value FROM AS_TABLE($data); ```
Project: OAuth2 Proxy (Provisioning / Key Management)
File: CONTRIBUTING.md

To develop on this project, please fork the repo and clone into your `$GOPATH`. Dependencies are not checked in so please download those separately. Download the dependencies using `go mod download`. ```bash cd $GOPATH/src/github.com # Create this directory if it doesn't exist git clone git@github.com:<YOUR_FORK>/oauth2-proxy oauth2-proxy/oauth2-proxy cd oauth2-proxy/oauth2-proxy go mod download ``` We track bugs and issues using Github. If you find a bug, please open an Issue. If you want to fix a bug, please fork, create a feature branch, fix the bug and open a PR back to this repo. Please mention the open bug issue number within your PR if applicable.
Project: StarRocks (App Definition and Development / Database)
File: CREATE_RESOURCE.md

displayed_sidebar: "English" Creates resources. The following types of resources can be created: Apache Spark, Apache Hive, Apache Iceberg, Apache Hudi, and JDBC. Spark resources are used in to manage loading information, such as YARN configurations, storage path of intermediate data, and Broker configurations. Hive, Iceberg, Hudi, and JDBC resources are used for managing data source access information involved in querying . :::tip Only users with the SYSTEM-level CREATE RESOURCE privilege can perform this operation. You can create JDBC resources only in StarRocks v2.3 and later. ::: ```sql CREATE [EXTERNAL] RESOURCE "resource_name" PROPERTIES ("key"="value", ...) ``` `resource_name`: the name of the resource to create. For the naming conventions, see . `PROPERTIES`: specifies the properties of the resource type. The PROPERTIES vary depending on the resource type. See Examples for details. Create a Spark resource named spark0 in yarn Cluster mode. ```sql CREATE EXTERNAL RESOURCE "spark0" PROPERTIES ( "type" = "spark", "spark.master" = "yarn", "spark.submit.deployMode" = "cluster", "spark.jars" = "xxx.jar,yyy.jar", "spark.files" = "/tmp/aaa,/tmp/bbb", "spark.executor.memory" = "1g", "spark.yarn.queue" = "queue0", "spark.hadoop.yarn.resourcemanager.address" = "127.0.0.1:9999", "spark.hadoop.fs.defaultFS" = "hdfs://127.0.0.1:10000", "working_dir" = "hdfs://127.0.0.1:10000/tmp/starrocks", "broker" = "broker0", "broker.username" = "user0", "broker.password" = "password0" ); ``` Parameters related to Spark are as follows: ```plain text spark.master: required. Currently, yarn and spark ://host:port. are supported. spark.submit.deployMode: deployment mode of the Spark program is required. Support cluster and client. spark.hadoop.yarn.resourcemanager.address: required when master is yarn. spark.hadoop.fs.defaultFS: required when master is yarn. Other parameters are optional. Please refer to http://spark.apache.org/docs/latest/configuration.html ``` If Spark is used for ETL, working_DIR and broker need to be specified. The instructions are as follows: ```plain text working_dir: Directory used by ETL. It is required when spark is used as ETL resource. For example: hdfs://host:port/tmp/starrocks. broker: Name of broker. It is required when spark is used as ETL resource and needs to be configured beforehand by using `ALTER SYSTEM ADD BROKER` command. broker.property_key: It is the property information needed to be specified when broker reads the intermediate files created by ETL. ``` Create a Hive resource named hive0. ```sql CREATE EXTERNAL RESOURCE "hive0" PROPERTIES ( "type" = "hive", "hive.metastore.uris" = "thrift://10.10.44.98:9083" ); ```
Project: KICS (Provisioning / Security & Compliance)
File: 5c6b727b-1382-4629-8ba9-abd1365e5610.md

title: Redshift Publicly Accessible hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 5c6b727b-1382-4629-8ba9-abd1365e5610 Query name: Redshift Publicly Accessible Platform: Ansible Severity: <span style="color:#bb2124">High</span> Category: Insecure Configurations URL: AWS Redshift Clusters must not be publicly accessible. Check if 'publicly_accessible' field is true (default is false)<br> ```yaml title="Positive test num. 1 - yaml file" hl_lines="9 17 25" name: Basic cluster provisioning example04 community.aws.redshift: command: create node_type: ds1.xlarge identifier: new_cluster username: cluster_admin password: 1nsecur3 publicly_accessible: yes name: Basic cluster provisioning example05 community.aws.redshift: command: create node_type: ds1.xlarge identifier: new_cluster username: cluster_admin password: 1nsecur3 publicly_accessible: True name: Basic cluster provisioning example06 redshift: command: create node_type: ds1.xlarge identifier: new_cluster username: cluster_admin password: 1nsecur3 publicly_accessible: Yes ``` ```yaml title="Negative test num. 1 - yaml file" name: Basic cluster provisioning example01 community.aws.redshift: command: create node_type: ds1.xlarge identifier: new_cluster username: cluster_admin password: 1nsecur3 publicly_accessible: no name: Basic cluster provisioning example02 community.aws.redshift: command: create node_type: ds1.xlarge identifier: new_cluster username: cluster_admin password: 1nsecur3 name: Basic cluster provisioning example03 redshift: command: create node_type: ds1.xlarge identifier: new_cluster username: cluster_admin password: 1nsecur3 publicly_accessible: false ```
Project: external-secrets (Provisioning / Security & Compliance)
File: SECURITY.md

- - <a name="reporting"></a> DO NOT CREATE AN ISSUE to report a security problem. Instead, please send an email to cncf-ExternalSecretsOp-maintainers@lists.cncf.io <a name="vulnerability-management"></a> We learn about critical software updates and security threats from these sources GitHub Security Alerts Dependency Updates Please follow the guide .
Project: KICS (Provisioning / Security & Compliance)
File: acb6b4e2-a086-4f35-aefd-4db6ea51ada2.md

title: Elasticsearch Log Disabled hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: acb6b4e2-a086-4f35-aefd-4db6ea51ada2 Query name: Elasticsearch Log Disabled Platform: Terraform Severity: <span style="color:#ff7213">Medium</span> Category: Observability URL: AWS Elasticsearch should have logs enabled<br> ```tf title="Positive test num. 1 - tf file" hl_lines="6" resource "awselasticsearchdomain" "positive1" { logpublishingoptions { cloudwatchloggrouparn = awscloudwatchloggroup.example.arn logtype = "INDEXSLOW_LOGS" enabled = false } } ``` ```tf title="Positive test num. 2 - tf file" hl_lines="1" resource "awselasticsearchdomain" "positive2" { domain_name = "example" elasticsearch_version = "1.5" cluster_config { instance_type = "r4.large.elasticsearch" } snapshot_options { automatedsnapshotstart_hour = 23 } tags = { Domain = "TestDomain" } } ``` ```tf title="Negative test num. 1 - tf file" resource "awselasticsearchdomain" "negative1" { logpublishingoptions { cloudwatchloggrouparn = awscloudwatchloggroup.example.arn logtype = "INDEXSLOW_LOGS" enabled = true //for default its true } } ```
Project: ArangoDB (App Definition and Development / Database)
File: std_result.md

+++ title = "`stdresult<T, E = std::errorcode, NoValuePolicy = policy::default_policy<T, E, void>>`" description = "A type alias to a `basicresult` configured with `std::errorcode` and `policy::default_policy`." +++ A type alias to a {{% api "basicresult<T, E, NoValuePolicy>" %}} configured with {{% api "std::errorcode" %}} and `policy::`{{% api "default_policy" %}}. This type alias always references the `std` edition of things, unlike {{% api "result<T, E = varies, NoValuePolicy = policy::defaultpolicy<T, E, void>>" %}} which references either this alias or {{% api "boostresult<T, E = boost::system::errorcode, NoValuePolicy = policy::defaultpolicy<T, E, void>>" %}}. Namespace: `BOOSTOUTCOMEV2_NAMESPACE` Header: `<boost/outcome/std_result.hpp>`
Project: Carvel (App Definition and Development / Application Definition & Image Build)
File: README_SES.md

The following is a sample class and Lambda function that receives Amazon SES event message data as input, writes some of the message data to CloudWatch Logs, and responds with a 200 status and the same body as the request. (Note that anything written to stdout or stderr will be logged as CloudWatch Logs events.) ```go package main import ( "context" "fmt" "github.com/aws/aws-lambda-go/events" "github.com/aws/aws-lambda-go/lambda" ) func handler(ctx context.Context, sesEvent events.SimpleEmailEvent) error { for _, record := range sesEvent.Records { ses := record.SES fmt.Printf("[%s - %s] Mail = %+v, Receipt = %+v \n", record.EventVersion, record.EventSource, ses.Mail, ses.Receipt) } return nil } func main() { lambda.Start(handler) } ```