cwielenb/CNCF-Dataset huggingface dataset:
Project: ClickHouse (App Definition and Development / Database)
File: covarsamp.md

slug: /en/sql-reference/aggregate-functions/reference/covarsamp sidebar_position: 37 Calculates the value of `((x - x)(y - y)) / (n - 1)`. Returns Float64. When `n <= 1`, returns `nan`. :::note This function uses a numerically unstable algorithm. If you need in calculations, use the `covarSampStable` function. It works slower but provides a lower computational error. :::
Project: LinuxKit (Provisioning / Automation & Configuration)
File: 2017-04-20-summit.md

Moby tool will move out of LinuxKit init will get replaced and probably re-written in Rust package and kernel builds will leverage multi-stage builds How to combine kernel flavors? ex: Wireguard + Landlock proposal: Automated kernel patch merging for build recipes? proposal: "Security" kernel channel with combination of projects Can we have a LinuxKit mailing list or use Slack? prefer mailing list community `#linuxkit` channel currently exists Issue tracking: make the development process more transparent need to keep community up to date with timelines for big changes such as Moby tool movement, can use mailing list discussed creating an issue tracker/milestone with a monthly horizon for tracking progress Modules: current kernel wont change much Lessons learned from RancherOS: need robust testing across all platforms and use-cases with LinuxKit: instead, have to build custom OS yourself Roadmap: replace system daemons with type-safe languages, potentially in unikernels Collaborate with formal-verification groups at Microsoft Moby yml: should `outputs` be in the cmd-line vs. yml? Diagnostics: currently closed source but will become a public package Error logging: we've added yml validation, would love more input from the community with how we can improve this Outputs: currently support `bzImage+initrd`, `iso-bios`, `iso-uefi`, `qemu`, `gcp` tarball+upload, `vmdk`, `vhd`, `pxe` Coming soon: `ami`, `azure` Should Moby tool be able to upload outputs to clouds? yes: we might call that `moby push` or something TBD: can we collaborate here with packer? Who are the users? Docker: we want to build a lean and secure Linux layer for Docker to run on Custom kernel and appliance builders: using bleeding edge security features, minimal OSes for embedded use-cases, etc. Testers: can easily build and spin up custom Linux OS for reproducible and minimal test environments Tinkerers: folks who want to play, implement weekend projects Security: prove value of incubation projects and kernel patches - ultimately want to see our projects make it upstream
Project: external-secrets (Provisioning / Security & Compliance)
File: deprecation-policy.md

We follow the and : alpha, beta, GA. The project is currently in `beta` state. Please try the `beta` features and provide feedback. After the features exits beta, it may not be practical to make more changes. alpha The support for a feature may be dropped at any time without notice. The API may change in incompatible ways in a later software release without notice. The software is recommended for use only in short-lived testing clusters, due to increased risk of bugs and lack of long-term support. beta The software is well tested. Enabling a feature is considered safe. Features are enabled by default. The support for a feature will not be dropped, though the details may change. The schema and/or semantics of objects may change in incompatible ways in a subsequent beta or stable release. When this happens, migration instructions are provided. Schema changes may require deleting, editing, and re-creating API objects. The editing process may not be straightforward. The migration may require downtime for applications that rely on the feature. The software is not recommended for production uses. Subsequent releases may introduce incompatible changes. If you have multiple clusters which can be upgraded independently, you may be able to relax this restriction. GA The stable versions of features appear in released software for many subsequent versions. Use it in production ;) We define the following scope that is covered by our deprecation policy. We follow the . API Objects and fields: `.Spec`, `.Status` and `.Status.Conditions[]` Enums and constant values Controller Configuration: CLI flags & environment variables Metrics as defined in the a feature or specific behavior: `ExternalSecret` We do not provide stability guarantee for source code imports. The Interfaces and the behavior will change in a unexpected and backwards-incompatible way. However, The maintained helm chart is not part of this deprecation policy.
Project: ArangoDB (App Definition and Development / Database)
File: ISSUE_TEMPLATE.md

ArangoDB Version: <!-- e.g. 3.9.2 or self-compiled devel branch --> Deployment Mode: <!-- Single Server | Leader/Follower ("Master/Slave") | Active Failover | Cluster | DC2DC --> Deployment Strategy: <!-- Manual Start | Manual Start in Docker | ArangoDB Starter | ArangoDB Starter in Docker | Kubernetes --> Configuration: <!-- cluster setup details, notable server settings, etc. --> Infrastructure: <!-- AWS | Azure | ... | own --> Operating System: <!-- Ubuntu 20.04 | Windows 10 | MacOS 10.13.4 | ... --> Total RAM in your machine: <!-- e.g. 32Gb. If more machines are in use, provide info for each of your machines --> Disks in use: <!-- SSD | HDD. If more machines are in use, provide info for each of your machines --> Used Package: <!-- Debian or Ubuntu .deb | SUSE or RedHat .rpm | Docker - official Docker library | other --> Affected feature: <!-- e.g. Installation | Foxx | AQL query using web interface | arangosh | with driver | ... --> AQL query (if applicable): ``` ``` AQL explain and/or profile (if applicable): <!-- output of db.explain("<my aql query>") or db.profileQuery("<my aql query>") --> Dataset: <!-- description, or if possible, share an example dataset to reproduce the issue either as Gist with an arangodump, or an arangosh script with db.collection.save({my: "values"}) statements --> Size of your Dataset on disk: <!-- size of your dataset on disk--> Replication Factor & Number of Shards (Cluster only): <!-- please list these settings for each collection in question --> Problem: Expected result:
Project: Lagoon (App Definition and Development / Application Definition & Image Build)
File: options.md

description: Configuring Applications for use on Lagoon Project- and environment-level configuration for Lagoon is provided in the `.lagoon.yml` file in your repository. See . Service-level configuration for Lagoon in provided in the `docker-compose.yml` file in your repository. In particular, the `lagoon.type` and associated service labels are documented in the individual services. See Lagoon has the ability to provision storage for most services - the built-in Lagoon service types have a `-persistent` variant that can add in the necessary PVCs, volumes, etc. We have updated our examples to reflect this configuration locally. Lagoon has configurations available for: Mariadb - all supported versions PostgreSQL - all supported versions Lagoon also has the capability to utilize the to automatically provision these databases using an underlying managed database service (i.e. RDS, Google Cloud Databases, Azure Database). This will happen automatically when these services are provisioned and configured for your cluster. If these are not available, a pod will be provisioned as a fallback. Lagoon supports Redis as a cache backend. In production, some users provision a managed Redis service for their production environments to help them scale. Lagoon supports Elasticsearch, Solr and OpenSearch as search providers. External search providers can also be configured if required. Lagoon auto-generates routes for services that have ingress requirements. Custom routes can be provided in the `.lagoon.yml` on a per-service basis. Lagoon makes heavy use of environment variables, at build and runtime. Where these are used to provide critical configuration for your application (e.g. database config/credentials) - it is important that the local and Lagoon versions are named similarly. See .
Project: DragonflyDB (App Definition and Development / Database)
File: bug_report.md

name: Bug report about: Create a report to help Dragonfly DB improve title: '' labels: 'bug' assignees: '' Describe the bug A clear and concise description of what the bug is. To Reproduce Steps to reproduce the behavior: Insert records using `command` Query records using `command` Scroll down to '....' See error Expected behavior A clear and concise description of what you expected to happen. Screenshots If applicable, add screenshots to help explain your problem. Environment (please complete the following information): OS: [ubuntu 20.04] Kernel: # Command: `uname -a` Containerized?: [Bare Metal, Docker, Docker Compose, Docker Swarm, Kubernetes, Other] Dragonfly Version: [e.g. 0.3.0] Reproducible Code Snippet ``` ``` Additional context Add any other context about the problem here.
Project: Backstage (App Definition and Development / Application Definition & Image Build)
File: v1.5.0.md

id: v1.5.0 title: v1.5.0 description: Backstage Release v1.5.0 These are the release notes for the v1.5.0 release of . A huge thanks to the whole team of maintainers and contributors as well as the amazing Backstage Community for the hard work in getting this release developed and done. Added a new `GitHubEntityProvider` (), which allows for automatic discovery of catalog entity definition files out of your GitHub projects. This is an improvement upon the `GithubDiscoveryProcessor` that existed before, and we recommend using entity providers rather than processors for discovery and ingestion when possible. Contributed by This release adds an experimental API that allows plugin authors to define plugin wide options. These options can then be used by adopters of the plugin to reconfigure it to fit their app. Check out the docs for more information on how to get started. Feedback is welcome on this new feature! Contributed by This release adds the new `@backstage/backend-defaults` package, part of the . This package is highly experimental and we do not recommend using it for any purpose, yet. Interact with AWS Proton in Backstage. Contributed by This new plugin can be used to display GitHub issues for your entities. Contributed by This new backend for `@backstage/plugin-sonarqube` replaces the Sonarqube proxy configuration; once it is installed, you can remove the `/sonarqube` proxy entry. For more information, see the plugin . Contributed by This new module for the catalog backend adds the `BitbucketServerEntityProvider`, which allows discovery of entities out of Bitbucket Server installations. Contributed by This release does not contain any security fixes. We recommend that you keep your Backstage project up to date with this latest release. For more guidance on how to upgrade, check out the documentation for . Below you can find a list of links and references to help you learn about and start using this new release. , , and Backstage's for discussions and support Backstage , , and Sign up for our if you want to be informed about what is happening in the world of Backstage.
Project: Beam (App Definition and Development / Streaming & Messaging)
File: grouping-elements-for-efficient-external-service-calls.md

title: "Pattern for grouping elements for efficient external service calls" <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> {{< language-switcher java py >}} Usually, authoring an Apache Beam pipeline can be done with out-of-the-box tools and transforms like ParDo's, Window's and GroupByKey's. However, when you want more tight control, you can keep state in an otherwise stateless DoFn. State is kept on a per-key and per-windows basis, and as such, the input to your stateful DoFn needs to be keyed (e.g. by the customer identifier if you're tracking clicks from an e-commerce website). Examples of use cases are: assigning a unique ID to each element, joining streams of data in 'more exotic' ways, or batching up API calls to external services. In this section we'll go over the last one in particular. Make sure to check the for deeper understanding on state and timers. The `GroupIntoBatches`-transform uses state and timers under the hood to allow the user to exercise tight control over the following parameters: `maxBufferDuration`: limits the amount of waitingtime for a batch to be emitted. `batchSize`: limits the number of elements in one batch. `batchSizeBytes`: (in Java only) limits the bytesize of one batch (using input coder to determine elementsize). `elementByteSize`: (in Java only) limits the bytesize of one batch (using a user defined function to determine elementsize). while abstracting away the implementation details from users. The `withShardedKey()` functionality increases parallelism by spreading one key over multiple threads. The transforms are used in the following way in Java & Python: {{< highlight java >}} input.apply( "Batch Contents", GroupIntoBatches.<String, GenericJson>ofSize(batchSize) .withMaxBufferingDuration(maxBufferingDuration) .withShardedKey()) {{< /highlight >}} {{< highlight py >}} input | GroupIntoBatches.WithShardedKey(batchSize, maxBufferingDuration) {{< /highlight >}} Applying these transforms will output groups of elements in a batch on a per-key basis, which you can then use to call an external API in bulk rather than on a per-element basis, resulting in a lower overhead in your pipeline.
Project: Beam (App Definition and Development / Streaming & Messaging)
File: 33_ai_ml_pipelines.md

Prompt: How can I use Apache Beam in my AI/ML pipeline? Response: Apache Beam is an open-source, unified programming model for defining batch and streaming data-parallel processing pipelines. Each Apache Beam pipeline is represented as a Directed Acyclic Graph (DAG) of data processing operations constructed using the Apache Beam SDK in one of the . To execute a pipeline, you need to deploy it to one of the supported . You can use Apache Beam for various tasks within your AI/ML pipeline, including data validation, preprocessing, model validation, and model deployment and inference. Apache Beam offers a rich set of and that allow for reading and writing data from and to various data sources and sinks, as well as performing data validation. For data preprocessing, Apache Beam provides the class. This feature allows you to ensure data consistency by applying the same preprocessing steps for both training and inference. Additionally, Apache Beam allows integration with pre-trained models from , , and . To execute machine learning inference tasks, Apache Beam provides the RunInference API. is a that supports the usage of any custom models. For an example of running inference on a custom model loaded with , refer to the example in the Apache Beam GitHub repository. The `RunInference` transform efficiently handles models of any size, making it suitable for large language models (LLMs) and other complex architectures. You can find an example of deploying and performing inference on large language models (LLMs) in the section of the Apache Beam documentation. For more information on implementing AI/ML pipelines using Apache Beam, see the section in the Apache Beam documentation.
Project: Tremor (App Definition and Development / Streaming & Messaging)
File: Instrument.md

Notes: Time spent in the onramp in 'crossbeamchannelsender' is spent waiting for the pipeline. Heuristics: About 10% are spent in 'non hot functions', so if we approach 90% of weight on a single function it means contention on this. Onramp: In the time in 'Crossbeam channel Sender send' approaches zero (aka ~10%) we are limited by the onramp Pipeline: If the Weight (not self weight) of 'ExecutableGraph::run' approaches 100% the we are limited by the pipeline
Project: Apache Hadoop (App Definition and Development / Database)
File: CHANGELOG.1.1.2.md

<! --> | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Port conf servlet to dump running configuration to branch 1.x | Major | conf | Junping Du | Jing Zhao | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | Improve confusing log message that prints exception when editlog read is completed | Major | namenode | Suresh Srinivas | Jing Zhao | | | Fix failed testcases with @ignore annotation In branch-1 | Minor | test | Jing Zhao | Jing Zhao | | | Introduce HADOOP\PROXY\USER for secure impersonation in child hadoop client processes | Major | security | Luke Lu | Yu Gao | | | Introduce HADOOP\SECURITY\CONF\_DIR for task-controller | Major | task-controller | Luke Lu | Yu Gao | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | "Text File Busy" errors launching MR tasks | Major | . | Todd Lipcon | Andy Isaacson | | | Missing jersey jars as dependency in the pom causes hive tests to fail | Major | . | Giridharan Kesavan | Giridharan Kesavan | | | NameNode could be stuck in SafeMode due to never-created blocks | Critical | namenode | Brandon Li | Brandon Li | | | When using SPNEGO, NN should not try to log in using KSSL principal | Major | namenode | Aaron T. Myers | Aaron T. Myers | | | TaskTracker's heartbeat is out of control | Major | . | Liyin Liang | Liyin Liang | | | TestJobHistoryServer fails some times with

Address already in use' | Minor | jobhistoryserver, test | sam liu | sam liu | | | Deadlock in configuration when writing configuration to hdfs | Blocker | . | Arpit Gupta | Jing Zhao | | | TestMRServerPorts throws NullReferenceException | Minor | . | Gopal V | Gopal V | | | TestMapredHeartbeat fails assertion on HeartbeatInterval | Minor | . | Gopal V | Gopal V | | | TestFairScheduler & TestCapacityScheduler fails due to JobHistory exception | Minor | . | Gopal V | Gopal V | | | TestWebUIAuthorization fails on branch-1 | Major | . | Arun C Murthy | Arun C Murthy | | | TestRecoveryManager fails on branch-1 | Major | . | Arun C Murthy | Arun C Murthy | | | Make LocalJobRunner work with private distributed cache | Minor | client | Luke Lu | Yu Gao | | | NLineInputFormat drops data in 1.1 and beyond | Blocker | mrv1 | Robert Joseph Evans | Vinod Kumar Vavilapalli | | | SortedRanges.Range#compareTo is not spec compliant | Major | task | Luke Lu | Yu Gao | | | Checkpoint exception causes fatal damage to fsimage. | Blocker | namenode | ChenFolin | Chris Nauroth | | | Fix UGI for IBM JDK running on Windows | Major | security | Luke Lu | Yu Gao | | | GzipCodec NPE upon reset with IBM JDK | Major | io | Luke Lu | Yu Li | | | hadoop 1.1.2. hdfs write bug | Major | fuse-dfs | WangMeng | | | JIRA | Summary | Priority | Component | Reporter | Contributor | |:- |:- | : |:- |:- |:- | | | ant test will build failed for trying to delete a file | Minor | test | meng gong | Luke Lu |
Project: YugabyteDB (App Definition and Development / Database)
File: key-value-workload-ysql.md

title: Benchmark YSQL performance with key-value workloads headerTitle: Key-value workload linkTitle: Key-value workload description: Benchmark YSQL performance with key-value workloads. headcontent: Benchmark YSQL performance with key-value workloads menu: v2.18: identifier: key-value-workload-2-ysql parent: benchmark weight: 6 type: docs <ul class="nav nav-tabs-alt nav-tabs-yb"> <li > <a href="../key-value-workload-ysql/" class="nav-link active"> <i class="icon-postgres" aria-hidden="true"></i> YSQL </a> </li> <li > <a href="../key-value-workload-ycql/" class="nav-link"> <i class="icon-cassandra" aria-hidden="true"></i> YCQL </a> </li> </ul> Coming soon.
Project: KICS (Provisioning / Security & Compliance)
File: 2ae9d554-23fb-4065-bfd1-fe43d5f7c419.md

title: Public Security Group Rule Sensitive Port hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 2ae9d554-23fb-4065-bfd1-fe43d5f7c419 Query name: Public Security Group Rule Sensitive Port Platform: Terraform Severity: <span style="color:#bb2124">High</span> Category: Networking and Firewall URL: A sensitive port, such as port 23 or port 110, is open to the public in either TCP or UDP protocol<br> ```tf title="Positive test num. 1 - tf file" hl_lines="10" resource "alicloudsecuritygroup" "default" { name = "default" } resource "alicloudsecuritygrouprule" "allowall_tcp" { type = "ingress" ip_protocol = "tcp" nic_type = "internet" policy = "accept" port_range = "19/20" priority = 1 securitygroupid = alicloudsecuritygroup.default.id cidr_ip = "0.0.0.0/0" } ``` ```tf title="Positive test num. 2 - tf file" hl_lines="10" resource "alicloudsecuritygroup" "default" { name = "default" } resource "alicloudsecuritygrouprule" "allowall_tcp" { type = "ingress" ip_protocol = "udp" nic_type = "internet" policy = "accept" port_range = "4333/4334" priority = 1 securitygroupid = alicloudsecuritygroup.default.id cidr_ip = "0.0.0.0/0" } ``` ```tf title="Positive test num. 3 - tf file" hl_lines="10" resource "alicloudsecuritygroup" "default" { name = "default" } resource "alicloudsecuritygrouprule" "allowall_tcp" { type = "ingress" ip_protocol = "all" nic_type = "internet" policy = "accept" port_range = "444/445" priority = 1 securitygroupid = alicloudsecuritygroup.default.id cidr_ip = "0.0.0.0/0" } ``` ```tf title="Negative test num. 1 - tf file" resource "alicloudsecuritygroup" "default" { name = "default" } resource "alicloudsecuritygrouprule" "allowall_tcp" { type = "ingress" ip_protocol = "tcp" nic_type = "internet" policy = "accept" port_range = "1/65535" priority = 1 securitygroupid = alicloudsecuritygroup.default.id cidr_ip = "10.159.6.18/12" } ``` ```tf title="Negative test num. 2 - tf file" resource "alicloudsecuritygroup" "default" { name = "default" } resource "alicloudsecuritygrouprule" "allowall_tcp" { type = "ingress" ip_protocol = "icmp" nic_type = "internet" policy = "accept" port_range = "1/65535" priority = 1 securitygroupid = alicloudsecuritygroup.default.id cidr_ip = "0.0.0.0/0" } ```