cwielenb/CNCF-Dataset huggingface dataset:
Project: YugabyteDB (App Definition and Development / Database)
File: foreign-data-wrappers.md

title: Foreign data wrappers linkTitle: Foreign data wrappers description: Foreign data wrappers in YSQL menu: v2.18: identifier: advanced-features-foreign-data-wrappers parent: advanced-features weight: 245 type: docs A foreign data wrapper is a library that you can use to access and interact with an external data (foreign data) source. They allow you to query foreign objects from remote servers as if they were local objects. To access foreign data, you first create a foreign data wrapper. Then, you create a foreign server, which specifies how to connect to the external data source. You may also need to create a user mapping to map a specific user to authorization credentials in the foreign server. Finally, you create foreign tables, which represent the structure of the data on the external source. Use the command to create foreign data wrappers. Example: ```plpgsql yugabyte=# CREATE FOREIGN DATA WRAPPER mywrapper HANDLER myhandler OPTIONS (dummy 'true'); ``` You use foreign servers to specify connection information for an external data source. Create foreign servers using the command. Example: ```plpgsql yugabyte=# CREATE SERVER myserver FOREIGN DATA WRAPPER mywrapper OPTIONS (host '197.0.2.1'); ``` User mappings associate a user with authorization credentials in the foreign server. You can create a user mapping with the command. Example: ```plpgsql yugabyte=# CREATE USER MAPPING FOR myuser SERVER myserver OPTIONS (user 'john', password 'password'); ``` Use the command to create foreign tables. ```sql yugabyte=# CREATE FOREIGN TABLE mytable (col1 int, col2 int) SERVER myserver OPTIONS (schema 'externalschema', table 'externaltable'); ``` The following foreign data wrappers are bundled with YugabyteDB:
Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.0.2.1.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | reduce copy progress not updating | Minor | . | Owen O'Malley | Owen O'Malley | | | hadoop dfs -report throws exception | Minor | . | Johan Oskarsson | Doug Cutting | | | Patch to HADOOP-96 uses long deprecated call | Critical | . | Bryan Pendleton | Hairong Kuang |
Project: Couler (Provisioning / Automation & Configuration)
File: enhancement_proposal.md

name: Enhancement proposal about: Propose an enhancement for this project labels: 'enhancement' What change needs making? When would you use this? <!-- Issue Author: Don't delete the following message to encourage other users to support your issue! --> Message from the maintainers: Impacted by this bug? Give it a . We prioritize the issues with the most .
Project: Druid (App Definition and Development / Database)
File: papers-and-talks.md

id: papers-and-talks title: "Papers" <!-- ~ Licensed to the Apache Software Foundation (ASF) under one ~ or more contributor license agreements. See the NOTICE file ~ distributed with this work for additional information ~ regarding copyright ownership. The ASF licenses this file ~ to you under the Apache License, Version 2.0 (the ~ "License"); you may not use this file except in compliance ~ with the License. You may obtain a copy of the License at ~ ~ http://www.apache.org/licenses/LICENSE-2.0 ~ ~ Unless required by applicable law or agreed to in writing, ~ software distributed under the License is distributed on an ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY ~ KIND, either express or implied. See the License for the ~ specific language governing permissions and limitations ~ under the License. --> - Discusses the Druid architecture in detail. - Discusses how Druid supports real-time and batch workflows. - Discusses the motivations behind Druid and the architecture of the system. - Discusses how real-time ingestion in Druid works and use cases at Netflix. - Discusses how approximate algorithms work in Druid. - Discusses Lambda architectures with Druid. - Discusses features that were added to scale Druid. - Discusses building applications on top of Druid.
Project: KICS (Provisioning / Security & Compliance)
File: knative-queries.md

This page contains all queries from Knative. | Query |Severity|Category|More info| ||--|--|--| |Serving Revision Spec Without Timeout Seconds<br/><sup><sub>e8bb41e4-2f24-4e84-8bea-8c7c070cf93d</sub></sup>|<span style="color:#5bc0de">Info</span>|Insecure Configurations|<a href="../knative-queries/e8bb41e4-2f24-4e84-8bea-8c7c070cf93d" target="_blank">Query details</a><br><a href="https://knative.dev/docs/reference/api/serving-api/#serving.knative.dev/v1.RevisionSpec">Documentation</a><br/>|
Project: ClickHouse (App Definition and Development / Database)
File: 30_unexpected-behaviour.md

name: Unexpected behaviour about: Some feature is working in non-obvious way title: '' labels: unexpected behaviour assignees: '' (you don't have to strictly follow this form) Describe the unexpected behaviour A clear and concise description of what works not as it is supposed to. How to reproduce Which ClickHouse server version to use Which interface to use, if matters Non-default settings, if any `CREATE TABLE` statements for all tables involved Sample data for all these tables, use if necessary Queries to run that lead to unexpected result Expected behavior A clear and concise description of what you expected to happen. Error message and/or stacktrace If applicable, add screenshots to help explain your problem. Additional context Add any other context about the problem here.
Project: KICS (Provisioning / Security & Compliance)
File: f53f16d6-46a9-4277-9fbe-617b1e24cdca.md

title: BOM - AWS EFS hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: f53f16d6-46a9-4277-9fbe-617b1e24cdca Query name: BOM - AWS EFS Platform: Terraform Severity: <span style="color:#CCCCCC">Trace</span> Category: Bill Of Materials URL: A list of EFS resources found. Amazon Elastic File System (Amazon EFS) automatically grows and shrinks as you add and remove files with no need for management or provisioning.<br> ```tf title="Positive test num. 1 - tf file" hl_lines="1" resource "awsefsfile_system" "positive1" { creation_token = "my-product" encrypted = true tags = { Name = "MyProduct" } } ``` ```tf title="Positive test num. 2 - tf file" hl_lines="1" resource "awsefsfile_system" "positive2" { creation_token = "my-product" encrypted = true tags = { Name = "MyProduct" } } resource "awsefsfilesystempolicy" "policy" { filesystemid = awsefsfile_system.positive2.id bypasspolicylockoutsafetycheck = true policy = <<POLICY { "Version": "2012-10-17", "Id": "ExamplePolicy01", "Statement": [ { "Sid": "ExampleStatement01", "Effect": "Allow", "Principal": { "AWS": "*" }, "Resource": "${awsefsfile_system.test.arn}", "Action": [ "elasticfilesystem:ClientMount", "elasticfilesystem:ClientWrite" ], "Condition": { "Bool": { "aws:SecureTransport": "true" } } } ] } POLICY } ``` ```tf title="Negative test num. 1 - tf file" module "efs" { source = "cloudposse/efs/aws" version = "0.31.1" namespace = "eg" stage = "test" name = "app" region = "us-west-1" vpcid = var.vpcid subnets = var.private_subnets securitygroups = [var.securitygroup_id] zoneid = var.awsroute53dnszone_id } ```
Project: Vald (App Definition and Development / Database)
File: bug_report.md

name: Bug report about: Create a report to help us improve title: "" labels: "type/bug, priority/medium, team/core" assignees: "" <!-- A clear and concise description of what the bug is. --> <!-- Please describe the steps to reproduce the behavior: --> <!-- A clear and concise description of what you expected to happen. --> <! Please change the versions below along with your environment --> Go Version: 1.22.3 Rust Version: 1.77.2 Docker Version: 20.10.8 Kubernetes Version: v1.30.0 NGT Version: 2.2.1
Project: Numaflow (App Definition and Development / Streaming & Messaging)
File: feature_request.md

name: Feature request about: Propose an enhancement for this project labels: 'enhancement' What change needs making? When would you use this? <!-- Issue Author: Don't delete this message to encourage other users to support your issue! --> Message from the maintainers: If you wish to see this enhancement implemented please add a reaction to this issue! We often sort issues this way to know what to prioritize.
Project: GreptimeDB (App Definition and Development / Database)
File: style-guide.md

This style guide is intended to help contributors to GreptimeDB write code that is consistent with the rest of the codebase. It is a living document and will be updated as the codebase evolves. It's mainly an complement to the . Formatting Modules Comments Place all `mod` declaration before any `use`. Use `unimplemented!()` instead of `todo!()` for things that aren't likely to be implemented. Add an empty line before and after declaration blocks. Place comment before attributes (`#[]`) and derive (`#[derive]`). Use the file with same name instead of `mod.rs` to define a module. E.g.: ``` . cache cache_size.rs write_cache.rs cache.rs ``` Add comments for public functions and structs. Prefer document comment (`///`) over normal comment (`//`) for structs, fields, functions etc. Add link (`[]`) to struct, method, or any other reference. And make sure that link works. Define a custom error type for the module if needed. Prefer `with_context()` over `context()` when allocation is needed to construct an error. Use `error!()` or `warn!()` macros in the `common_telemetry` crate to log errors. E.g.: ```rust error!(e; "Failed to do something"); ```
Project: ClickHouse (App Definition and Development / Database)
File: v23.6.2.18-stable.md

sidebar_position: 1 sidebar_label: 2023 Backported in : Update cargo dependencies. (). Fix reading from empty column in `parseSipHashKey` (). Allow parametric UDFs (). Remove the usage of Analyzer setting in the client (). Fix 02116tupleelement with Analyzer (). Fix SQLLogic docker images (). Fix source image for sqllogic (). Pin for docker-ce ().
Project: Dragonfly (Provisioning / Container Registry)
File: OWNERS.md

This page lists current and emeritus maintainers. This can be used for routing PRs, discussions, questions and overall coordination. See that describes governance guidelines and maintainer responsibilities.. See about contributors and maintainers standards in the community. See for general contributing guidelines. Please refer to our . <!-- markdownlint-disable --> | GitHub ID | Name | Email | Company | | :-: | :--: | :--: | :--: | | | Weipeng Sun | weipeng.swp@alibaba-inc.com | Alibaba Group | | | Jin Zhang | zj3142063@gmail.com | Alibaba Group | | | Yuxing Liu | yuxing.lyx@alibaba-inc.com | Alibaba Group | | | Eddie Huang | inoc603@gmail.com | ByteDance | | | Haibing Zhou | zhouhaibing089@gmail.com | eBay | | | Chaobing Chen | chenchaobing@126.com | Meitu | | | Dajun Cui | cuidajun.cdj@alibaba-inc.com | Alibaba Group | <!-- markdownlint-restore -->
Project: ShardingSphere (App Definition and Development / Database)
File: new_SQL.en.md

+++ title = "Exploring Apache ShardingSphere with the perspective of new SQL" weight = 4 chapter = true +++ NewSQL is popular in recent years, and companies all over the world have different interpretations of NewSQL. This sharing will bring you the interpretation of NewSQL concept and product architecture around the world. The architecture, characteristics, planning and open source community of Apache ShardingSphere, a popular open source distributed database middleware service platform, are analyzed in detail from the perspective of NewSQL. {{< bilibili BV12t4y117GC >}}
Project: Beam (App Definition and Development / Streaming & Messaging)
File: 40_ai_bigquery_ml.md

Prompt: How can I integrate BigQuery ML with Apache Beam? Response: allows you to develop and execute ML models using queries. You can seamlessly integrate these models into your Apache Beam pipeline using and perform inference on them using the RunInference API. For an example of training a basic BigQuery ML model, refer to the . Once you have trained your model, you will need to export it. Here is an example BigQuery command to export a model to a bucket: ``` bq extract -m <model name> gs://<cloud storage path> ``` To incorporate your BigQuery ML model into an Apache Beam pipeline using `tfx_bsl`, it must be saved in the format. You will need to download the model to your local directory to perform local predictions: ```python import apache_beam import tensorflow as tf from google.protobuf import text_format from tensorflow.python.framework import tensor_util from tfxbsl.beam import runinference from tfx_bsl.public.beam import RunInference from tfxbsl.public.proto import modelspec_pb2 inputs = tf.train.Example(...) model_path = <path to the directory where the model is stored> def extract_prediction(response): with beam.Pipeline() as p: res = ( p | beam.Create([inputs]) | RunInference( modelspecpb2.InferenceSpecType( savedmodelspec=modelspecpb2.SavedModelSpec( modelpath=modelpath, signaturename=['servingdefault']))) | beam.ParDo(extract_prediction) ) ``` This example uses the transform from the `tfxbsl` library, directing it to the local directory where the model is stored. The transform takes `tf.train.Example` type elements as inputs and produces `tensorflowserving.apis.predictionlogpb2.PredictionLog` type elements as outputs. Depending on the signature of your model, you can extract values from the output. For the prediction output format of exported models for each model type, refer to the section in the BigQuery documentation.
Project: Operator Framework (App Definition and Development / Application Definition & Image Build)
File: v1.28.0.md

title: v1.28.0 weight: 998972000 In the requirements.yaml file replace: ```yaml name: kubernetes.core version: "2.3.1" ``` with: ```yaml name: kubernetes.core version: "2.4.0" ``` See for more details. Regarding the upcoming major changes in the plugin, please note that `go/v3` is being deprecated by Kubebuilder hence Operator SDK would also be migrating to `go/v4` soon. Following are the changes to be addressed in your project structure, `Makefile` and `go.mod` files: 1) [go/v3, ansible/v1, helm/v1] Update your `go.mod` file to upgrade the dependencies and run `go mod tidy` to download them ```go k8s.io/api v0.26.2 k8s.io/apiextensions-apiserver v0.26.2 k8s.io/apimachinery v0.26.2 k8s.io/cli-runtime v0.26.2 k8s.io/client-go v0.26.2 k8s.io/kubectl v0.26.2 sigs.k8s.io/controller-runtime v0.14.5 sigs.k8s.io/controller-tools v0.11.3 sigs.k8s.io/kubebuilder/v3 v3.9.1 ``` 2) [go/v3, ansible/v1, helm/v1] With the bump to Kubebuilder 3.9.1, the `go/v3` projects are now scaffolded with 0.13.1 version of kube-rbac-proxy. Modify the version of kube-rbac-proxy in the following scaffolded files: - config/default/managerauthproxy_patch.yaml - bundle/manifests/memcached-operator.clusterserviceversion.yaml ```diff gcr.io/kubebuilder/kube-rbac-proxy:v0.13.0 gcr.io/kubebuilder/kube-rbac-proxy:v0.13.1 ``` 3) [gov4alpha] For scaffolds, replace scaffoldings of v1beta1 by using `k8s.io/api/admission/v1` instead of `k8s.io/api/admission/v1beta1` as v1beta1 is removed in latest k8s. ```diff admissionv1beta1 "k8s.io/api/admission/v1beta1" admissionv1 "k8s.io/api/admission/v1" ``` 4) [gov4alpha] The project structure has changed from `/controllers` to `internal/controller/`. Modify the Dockerfile of the `go/v4-alpha` plugin to below: ```diff COPY controllers/ controllers/ COPY internal/controller/ internal/controller/ ``` 5) [go/v3, gov4alpha] Update your Makefile by Replacing `ENVTESTK8SVERSION = 1.25` with `ENVTESTK8SVERSION = 1.26`. Using `manifests generate fmt vet` to build manager binary by replacing `generate fmt vet` in the build target to `manifests generate fmt vet`: ```diff build: generate fmt vet ## Build manager binary. build: manifests generate fmt vet ## Build manager binary. ``` 6) [gov4alpha] The gov4alpha plugin is now being scaffolded with Kubebuilder's go/v4-alpha which has a modified project file structure according to Go's best practices. Refer https://github.com/kubernetes-sigs/kubebuilder/pull/2985 for changes. Make the following changes in your scaffolded project if gov4aplha plugin was used: - Replacing `go run ./main.go` to `go run ./cmd/main.go` due to the change in project structure. Replacing ```diff `test -s $(LOCALBIN)/controller-gen || GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-tools/cmd/controller-gen@$(CONTROLLERTOOLSVERSION)` test -s $(LOCALBIN)/controller-gen && $(LOCALBIN)/controller-gen --version | grep -q $(CONTROLLERTOOLSVERSION) || \ GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-tools/cmd/controller-gen@$(CONTROLLERTOOLSVERSION) ``` See for more details.
Project: KICS (Provisioning / Security & Compliance)
File: ba40ace1-a047-483c-8a8d-bc2d3a67a82d.md

title: EKS node group remote access disabled hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: ba40ace1-a047-483c-8a8d-bc2d3a67a82d Query name: EKS node group remote access disabled Platform: Terraform Severity: <span style="color:#ff7213">Medium</span> Category: Networking and Firewall URL: EKS node group remote access is disabled when 'SourceSecurityGroups' is missing<br> ```tf title="Positive test num. 1 - tf file" hl_lines="13" resource "awseksnode_group" "positive" { clustername = awseks_cluster.example.name nodegroupname = "example" noderolearn = awsiamrole.example.arn subnetids = awssubnet.example[*].id scaling_config { desired_size = 1 max_size = 1 min_size = 1 } remote_access { ec2sshkey = "my-rsa-key" } depends_on = [ awsiamrolepolicyattachment.example-AmazonEKSWorkerNodePolicy, awsiamrolepolicyattachment.example-AmazonEKSCNIPolicy, awsiamrolepolicyattachment.example-AmazonEC2ContainerRegistryReadOnly, ] } ``` ```tf title="Negative test num. 1 - tf file" resource "awseksnode_group" "negative" { clustername = awseks_cluster.example.name nodegroupname = "example" noderolearn = awsiamrole.example.arn subnetids = awssubnet.example[*].id scaling_config { desired_size = 1 max_size = 1 min_size = 1 } remote_access { ec2sshkey = "my-rsa-key" sourcesecuritygroups_ids = "sg-213120ASNE" } depends_on = [ awsiamrolepolicyattachment.example-AmazonEKSWorkerNodePolicy, awsiamrolepolicyattachment.example-AmazonEKSCNIPolicy, awsiamrolepolicyattachment.example-AmazonEC2ContainerRegistryReadOnly, ] } ```
Project: EMQ Technologies (App Definition and Development / Streaming & Messaging)
File: v5.3.0.en.md

Upgraded ekka to 0.15.13, which incorporates the following changes: Upgraded Mria to 0.6.2. Introduced the ability to configure the bootstrap data sync batch size, as detailed in . Enhanced the reliability of mria_membership processes, as described in . Fix log message formatting error. Added `node.defaultbootstrapbatch_size` option to EMQX configuration. Increasing the value of this option can greatly reduce a replicant node startup time, especially when the EMQX cluster interconnect network latency is high and the EMQX built-in database holds a large amount of data, e.g. when the number of subscriptions is high. Added a new rule-engine SQL function `bytesize` to get the size of a byte-string. e.g. `SELECT * FROM "t/#" WHERE bytesize(payload) > 10`. Updated to quicer version 0.0.200 in preparation for enabling openssl3 support for QUIC transport. Fixed an issue where logging would stop if "Rotation Size" would be set to `infinity` on file log handlers. Improve EMQX graceful shutdown (`emqx stop` command): Increase timeout from 1 to 2 minutes. Printed an error message if EMQX can't stop gracefully within the configured timeout. Print periodic status messages while EMQX is shutting down. Fixed telemetry reporting error on Windows when os_mon module is unavailable. Lowered CMD_overridden log severity from warning to info. Upgraded rpc library gen_rpc from 2.8.1 to 3.1.0. Upgraded library `esockd` from 5.9.6 to 5.9.7. This upgrade included: Enhancements regarding proxy protocol error and timeout. Lowered `ssl_error` exceptions to info-level logging. Malformed MQTT packet parsing exception log level is lowered from `error` to `info`. In command `emqx ctl listeners` output, the `shutdown_count` counter is incremented when TLS handshake failure (`sslerror`) or Malformed packet (`frameerror`) happens. Fixed log formatter when log.HANDLER.formatter is set to 'json'. The bug was introduced in v5.0.4 where the log line was no longer a valid JSON, but prefixed with timestamp string and level name. Disable access to the `logout` endpoint by the API key, this endpoint is for the Dashboard only.
Project: Backstage (App Definition and Development / Application Definition & Image Build)
File: plugin-development.md

id: plugin-development title: Plugin Development description: Documentation on Plugin Development Backstage plugins provide features to a Backstage App. Each plugin is treated as a self-contained web app and can include almost any type of content. Plugins all use a common set of platform APIs and reusable UI components. Plugins can fetch data from external sources using the regular browser APIs or by depending on external modules to do the work. Consider writing plugins in `TypeScript`. Plan the directory structure of your plugin so that it becomes easy to manage. Prefer using the , otherwise go with . Check out the shared Backstage APIs before building a new one. Each plugin can export routable extensions, which are then imported into the app and mounted at a path. First you will need a `RouteRef` instance to serve as the mount point of your extensions. This can be used within your own plugin to create a link to the extension page using `useRouteRef`, as well as for other plugins to link to your extension. It is best to place these in a separate top-level `src/routes.ts` file, in order to avoid import cycles, for example like this: ```tsx / src/routes.ts / import { createRouteRef } from '@backstage/core-plugin-api'; // Note: This route ref is for internal use only, don't export it from the plugin export const rootRouteRef = createRouteRef({ id: 'Example Page', }); ``` Now that we have a `RouteRef`, we import it into `src/plugin.ts`, create our plugin instance with `createPlugin`, as well as create and wrap our routable extension using `createRoutableExtension` from `@backstage/core-plugin-api`: ```tsx / src/plugin.ts / import { createPlugin, createRouteRef } from '@backstage/core-plugin-api'; import ExampleComponent from './components/ExampleComponent'; // Create a plugin instance and export this from your plugin package export const examplePlugin = createPlugin({ id: 'example', routes: { root: rootRouteRef, // This is where the route ref should be exported for usage in the app }, }); // This creates a routable extension, which are typically full pages of content. // Each extension should also be exported from your plugin package. export const ExamplePage = examplePlugin.provide( createRoutableExtension({ name: 'ExamplePage', // The component needs to be lazy-loaded. It's what will actually be rendered in the end. component: () => import('./components/ExampleComponent').then(m => m.ExampleComponent), // This binds the extension to this route ref, which allows for routing within and across plugin extensions mountPoint: rootRouteRef, }), ); ``` This extension can then be imported and used in the app as follow, typically placed within the top-level `<FlatRoutes>`: ```tsx <Route path="/any-path" element={<ExamplePage />} /> ```