cwielenb/CNCF-Dataset huggingface dataset:
Project: Apache RocketMQ (App Definition and Development / Streaming & Messaging)
File: Configuration_System.md

This section focuses on the configuration of the system (JVM/OS) The latest released version of JDK 1.8 is recommended. Set the same Xms and Xmx value to prevent the JVM from resizing the heap for better performance. A simple JVM configuration is as follows: -server -Xms8g -Xmx8g -Xmn4g Direct ByteBuffer memory size setting. Full GC will be triggered when the Direct ByteBuffer up to the specified size: -XX:MaxDirectMemorySize=15g If you dont care about the boot time of RocketMQ broker, pre-touch the Java heap to make sure that every page will be allocated during JVM initialization is a better choice. Those who dont care about the boot time can enable it: -XX:+AlwaysPreTouch Disable biased locking maybe reduce JVM pauses: -XX:-UseBiasedLocking As for garbage collection, G1 collector with JDK 1.8 is recommended: -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:G1ReservePercent=25 -XX:InitiatingHeapOccupancyPercent=30 These GC options looks a little aggressive, but its proved to have good performance in our production environment Dont set a too small value for -XX:MaxGCPauseMillis, otherwise JVM will use a small young generation to achieve this goal which will cause very frequent minor GC.So use rolling GC log file is recommended: -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=30m If write GC file will increase latency of broker, consider redirect GC log file to a memory file system: -Xloggc:/dev/shm/mqgc%p.log123 There is a os.sh script that lists a lot of kernel parameters in folder bin which can be used for production use with minor changes. Below parameters need attention, and more details please refer to documentation for /proc/sys/vm/*. vm.extra_free_kbytes, tells the VM to keep extra free memory between the threshold where background reclaim (kswapd) kicks in, and the threshold where direct reclaim (by allocating processes) kicks in. RocketMQ uses this parameter to avoid high latency in memory allocation. (It is specific to the kernel version vm.min_free_kbytes, if you set this to lower than 1024KB, your system will become subtly broken, and prone to deadlock under high loads. vm.max_map_count, limits the maximum number of memory map areas a process may have. RocketMQ will use mmap to load CommitLog and ConsumeQueue, so set a bigger value for this parameter is recommended. vm.swappiness, define how aggressive the kernel will swap memory pages. Higher values will increase aggressiveness, lower values decrease the amount of swap. 10 is recommended for this value to avoid swap latency. File descriptor limits, RocketMQ needs open file descriptors for files(CommitLog and ConsumeQueue) and network connections. We recommend setting 655350 for file descriptors. Disk scheduler, the deadline I/O scheduler is recommended for RocketMQ, which attempts to provide a guaranteed latency for requests.
Project: Apache Hadoop (App Definition and Development / Database)
File: RELEASENOTES.2.7.5.md

<! --> These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements. | Major | Make SO\_RCVBUF and SO\_SNDBUF size configurable for DataTransferProtocol sockets and allow configuring auto-tuning* HDFS-8829 introduces two new configuration settings: dfs.datanode.transfer.socket.send.buffer.size and dfs.datanode.transfer.socket.recv.buffer.size. These settings can be used to control the socket send buffer and receive buffer sizes respectively on the DataNode for client-DataNode and DataNode-DataNode connections. The default values of both settings are 128KB for backwards compatibility. For optimum performance it is recommended to set these values to zero to enable the OS networking stack to auto-tune buffer sizes. | Major | RM may allocate wrong AM Container for new attempt* ResourceManager will now record ResourceRequests from different attempts into different objects.
Project: Podman (App Definition and Development / Application Definition & Image Build)
File: userns-gid-map.md

Directly specifies a GID mapping to be used to set ownership, at the filesystem level, on the working container's contents. Commands run when handling `RUN` instructions defaults to being run in their own user namespaces, configured using the UID and GID maps. Entries in this map take the form of one or more triples of a starting in-container GID, a corresponding starting host-level GID, and the number of consecutive IDs which the map entry represents. This option overrides the remap-gids setting in the options section of /etc/containers/storage.conf. If this option is not specified, but a global --userns-gid-map setting is supplied, settings from the global option is used. If none of --userns-uid-map-user, --userns-gid-map-group, or --userns-gid-map are specified, but --userns-uid-map is specified, the GID map is set to use the same numeric values as the UID map.
Project: YugabyteDB (App Definition and Development / Database)
File: redis.md

title: Compare Redis in-memory store with YugabyteDB linkTitle: Redis in-memory store description: Compare Redis in-memory store with the YugabyteDB database. aliases: /comparisons/redis/ menu: preview_faq: parent: comparisons identifier: comparisons-redis weight: 1129 type: docs Following are the key areas of differences between YugabyteDB and . YugabyteDB is a persistent database rather than an in-memory cache. (While Redis has a check-pointing feature for persistence, it is a highly inefficient operation that does a process fork. It is also not an incremental operation; the entire memory state is written to disk causing serious overall performance impact.) YugabyteDB is an auto-sharded, clustered with built-in support for strongly consistent replication and multi-DC deployment flexibility. Operations such as add node, remove node are simple, throttled, and intent-based, and leverage YugabyteDB's architectural benefits. Unlike Redis, the entire data set does not need to fit in memory. In Yugabyte, the hot data lives in RAM. Colder data is automatically tiered to storage, and on-demand paged in at block granularity from storage much like traditional database. Applications that use Redis only as a cache and use a separate backing database as the main system of record, and need to deal with pain points around keeping the cache and DB consistent and operational pain points at two levels of infrastructure (sharding, load-balancing, geo-redundancy) can leverage YugabyteDB as a unified cache + database offering. Scan resistant block cache design ensures long scans (such as of older data) do not impact reads for recent data.
Project: Ansible (Provisioning / Automation & Configuration)
File: bug_wrong_repo.md

Hi! Thanks very much for your interest in Ansible. It means a lot to us. This appears to be something that should be filed against another project or bug tracker. Here's why: FILL IN << CHOOSE AS APPROPRIATE >> <https://github.com/ansible-community/ansible-lint> <https://github.com/ansible/ansible-runner> <https://github.com/ansible/ansible-navigator> <https://github.com/ansible-community/antsibull> <https://github.com/ansible-community/ara> <https://github.com/ansible/awx> <https://github.com/ansible-collections/community.general> <https://github.com/ansible-community/molecule> For AAP or Tower licensees report issues via your Red Hat representative or <https://issues.redhat.com> If you can stop by the tracker or forum for one of those projects, we'd appreciate it. Because this project is very active, we're unlikely to see comments made on closed tickets and we lock them after some time. Should you still wish to discuss things further, or if you disagree with our thought process, please stop by one of our two mailing lists: <https://groups.google.com/forum/#!forum/ansible-devel> Matrix: IRC: #ansible-devel on We'd be happy to discuss things. See this page for a complete list of communication channels and their purposes: <https://docs.ansible.com/ansible/latest/community/communication.html> Thank you once again!
Project: Trivy (Provisioning / Security & Compliance)
File: circleci.md

``` $ cat .circleci/config.yml jobs: build: docker: image: docker:stable-git steps: checkout setupremotedocker run: name: Build image command: docker build -t trivy-ci-test:${CIRCLE_SHA1} . run: name: Install trivy command: | apk add --update-cache --upgrade curl curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin run: name: Scan the local image with trivy command: trivy image --exit-code 0 --no-progress trivy-ci-test:${CIRCLE_SHA1} workflows: version: 2 release: jobs: build ```
Project: Podman (App Definition and Development / Application Definition & Image Build)
File: group-add.md

Assign additional groups to the primary user running within the container process. `keep-groups` is a special flag that tells Podman to keep the supplementary group access. Allows container to use the user's supplementary group access. If file systems or devices are only accessible by the rootless user's group, this flag tells the OCI runtime to pass the group access into the container. Currently only available with the `crun` OCI runtime. Note: `keep-groups` is exclusive, other groups cannot be specified with this flag. (Not available for remote commands, including Mac and Windows (excluding WSL2) machines)
Project: Carvel (App Definition and Development / Application Definition & Image Build)
File: README_Lambda.md

Lambda events consist of a request that was routed to a Lambda function by the Lambda Function URLs feature. When this happens, Lambda expects the result of the function to be the response that Lambda should respond with. The following is a sample class and Lambda function that receives Lambda Function URLs event record data as an input, writes some of the record data to CloudWatch Logs, and responds with a 200 status and the same body as the request. (Note that anything written to stdout or stderr will be logged as CloudWatch Logs events.) ```go package main import ( "context" "fmt" "github.com/aws/aws-lambda-go/events" "github.com/aws/aws-lambda-go/lambda" ) func handleRequest(ctx context.Context, request events.LambdaFunctionURLRequest) (events.LambdaFunctionURLResponse, error) { fmt.Printf("Processing request data for request %s.\n", request.RequestContext.RequestID) fmt.Printf("Body size = %d.\n", len(request.Body)) fmt.Println("Headers:") for key, value := range request.Headers { fmt.Printf(" %s: %s\n", key, value) } return events.LambdaFunctionURLResponse{Body: request.Body, StatusCode: 200}, nil } func main() { lambda.Start(handleRequest) } ```
Project: KICS (Provisioning / Security & Compliance)
File: 12726829-93ed-4d51-9cbe-13423f4299e1.md

title: SQS With SSE Disabled hide: toc: true navigation: true <style> .highlight .hll { background-color: #ff171742; } .md-content { max-width: 1100px; margin: 0 auto; } </style> Query id: 12726829-93ed-4d51-9cbe-13423f4299e1 Query name: SQS With SSE Disabled Platform: CloudFormation Severity: <span style="color:#ff7213">Medium</span> Category: Encryption URL: Amazon Simple Queue Service (SQS) queue should protect the contents of their messages using Server-Side Encryption (SSE)<br> ```yaml title="Positive test num. 1 - yaml file" hl_lines="8 4" Resources: MyQueue: Type: AWS::SQS::Queue Properties: QueueName: "SampleQueue" MyQueue2: Type: AWS::SQS::Queue Properties: QueueName: "SampleQueue" SqsManagedSseEnabled: false ``` ```json title="Positive test num. 2 - json file" hl_lines="11 5" { "Resources": { "MyQueue": { "Type": "AWS::SQS::Queue", "Properties": { "QueueName": "SampleQueue" } }, "MyQueue2": { "Type": "AWS::SQS::Queue", "Properties": { "QueueName": "SampleQueue", "SqsManagedSseEnabled": "false" } } } } ``` ```yaml title="Negative test num. 1 - yaml file" Resources: MyQueue: Type: AWS::SQS::Queue Properties: QueueName: "SampleQueue" KmsMasterKeyId: wewewewewewe MyQueue2: Type: AWS::SQS::Queue Properties: QueueName: "SampleQueue" SqsManagedSseEnabled: true ``` ```json title="Negative test num. 2 - json file" { "Resources": { "MyQueue": { "Type": "AWS::SQS::Queue", "Properties": { "QueueName": "SampleQueue", "KmsMasterKeyId": "wewewewewewe" } }, "MyQueue2": { "Type": "AWS::SQS::Queue", "Properties": { "QueueName": "SampleQueue", "SqsManagedSseEnabled": "true" } } } } ```
Project: Apache RocketMQ (App Definition and Development / Streaming & Messaging)
File: CONTRIBUTING.md

We are always very happy to have contributions, whether for trivial cleanups or big new features. We want to have high quality, well documented codes for each programming language, as well as the surrounding of integration tools that people use with RocketMQ. Nor is code the only way to contribute to the project. We strongly value documentation, integration with other project, and gladly accept improvements for these aspects. Recommend reading: * To submit a change for inclusion, please do the following: If your have a pull request on GitHub, and updated more than once, it's better to squash all commits. Identify how many commits you made since you began: ``git log``. Squash these commits by N: ``git rebase -i HEAD~N`` . Leave "pick" tag in the first line. Change all other commits from "pick" to "fixup". Then do "force push" to overwrite remote history: ``git push -u origin ROCKETMQ-9999 --force`` All your changes are now in a single commit, that would be much better for review. More details of squash can be found at . We are always interested in adding new contributors. What we look for are series of contributions, good taste and ongoing interest in the project. If you are interested in becoming a committer, please let one of the existing committers know and they can help you walk through the process. Nowadays,we have several important contribution points: If you want to contribute the above listing points, you must abide our some prerequisites:
Project: YugabyteDB (App Definition and Development / Database)
File: DEVELOPER.md

Cutting a new release NOTE: This assumes you've run the tests and they pass. Increase the version number. [ ] Introduce `hll--<old>--<new>.sql`. [ ] Change `CHANGELOG.md`. [ ] Change `hll.control`. [ ] Change `Makefile`. [ ] Change `README.md` [ ] Change `setup.sql` and `setup.out` Commit changes and open a PR. Tag the commit on master after the PR is merged. 'vX.Y.Z' is the name format.
Project: RabbitMQ (App Definition and Development / Streaming & Messaging)
File: 3.10.16.md

RabbitMQ `3.10.16` is a maintenance release in the `3.10.x` . This series . Please refer to the upgrade section from if upgrading from a version prior to 3.10.0. This release requires Erlang 24.3 and supports Erlang 25. has more details on Erlang version requirements for RabbitMQ. Erlang 23 and 24 versions older than 24.3 have reached . This release of RabbitMQ requires Erlang 24.3 or later versions. Nodes will fail to start on older Erlang releases. Erlang 25 is recommended: it offers much improved performance on ARM64 architectures, across all architectures, and the most recent TLS 1.3 implementation. Release notes can be found on GitHub at . `rabbitmqctl hash_password` is a new command that produces a of the provided password. Contributed by @SimonUnge (AWS). GitHub issue: `rabbitmq-diagnostics checkportconnectivity` now supports a new optional flag, `--address`, that makes the check connect to a specific IP address instead of resolving node's hostname. This is useful when target node is configured to only listen for connections on one interface but not others: ``` shell rabbitmq-diagnostics checkportconnectivity --address 127.0.0.1 rabbitmq-diagnostics checkportconnectivity --address "::1" ``` GitHub issue: User filtering combined with pagination in the management UI did not work as expected. GitHub issue: Correctly format JSON field value in channel detail API response. Contributed @Syuparn. GitHub issue: AMQP 1.0 connection churn resulted in a memory leak. GitHub issue: None in this release. To obtain source code of the entire distribution, please download the archive named `rabbitmq-server-3.10.16.tar.xz` instead of the source tarball produced by GitHub.
Project: ArangoDB (App Definition and Development / Database)
File: partialfunctions.md

<!-- Copyright 2018 Paul Fultz II Distributed under the Boost Software License, Version 1.0. (http://www.boost.org/LICENSE10.txt) --> Partial function evaluation =========================== Many of the adaptors(such as or ) in the library supports optional partial evaluation of functions. For example, if we have the `sum` function adapted with the `partial` adaptor: auto sum = partial( { return x+y; }); So if we write `sum(1, 2)` it will return 3, however, if we write `sum(1)` it will return a new function, which when called again, it will evaluate the function and return 3: int i = sum(1, 2); // Returns 3 auto f = sum(1); int j = f(2); // returns 3 Of course due to limitations in C++, deciding whether evaluate the function or to partially evaluated it, is based on the callability of the function and not arity. So if we call `sum(1, 2, 3)`, it will return a function: auto f = sum(1, 2, 3); However, this can get out of hande as the function `f` will never be evaluated. Plus, it would be nice to produce an error at the point of calling the function rather than a confusing error of trying to use a partial function. The decorator lets us annotate the function with the max arity: auto sum = partial(limit_c<2>( { return x+y; })); So now if we call `sum(1, 2, 3)`, we will get a compiler error. So this improves the situation, but it is not without its limitations. For example if we were to define a triple sum using the adaptor: auto sum = pipable(limit_c<3>( { return x+y+z; })); So if we call `sum(1)`, there is no compiler error, not until we try to pipe in a value: auto f = sum(1); // No error here auto i = 2 | f; // Compile error Of course, the goal may not be to use the pipable call, which could lead to some confusing errors. Currently, there is not a good solution to this.
Project: Trivy (Provisioning / Security & Compliance)
File: aws-security-hub.md

{ width=50 } In the following example using the template `asff.tpl`, file can be generated. ``` $ AWSREGION=us-west-1 AWSACCOUNT_ID=123456789012 trivy image --format template --template "@contrib/asff.tpl" -o report.asff golang:1.12-alpine ``` ASFF template needs AWSREGION and AWSACCOUNT_ID from environment variables. The Product field follows the pattern below to match what AWS requires for the . {% raw %} ``` "ProductArn": "arn:aws:securityhub:{{ env "AWS_REGION" }}::product/aquasecurity/aquasecurity", ``` {% endraw %} In order to upload results you must first run like: ``` aws securityhub enable-import-findings-for-product --product-arn arn:aws:securityhub:<AWS_REGION>::product/aquasecurity/aquasecurity ``` The findings are with a key of `Findings` and a value of the array of findings. In order to upload via the CLI the outer wrapping must be removed being left with only the array of findings. The easiest way of doing this is with the using the command ``` cat report.asff | jq '.Findings' ``` Then, you can upload it with AWS CLI. ``` $ aws securityhub batch-import-findings --findings file://report.asff ``` The command limits the number of findings uploaded to 100 per request. The best known workaround to this problem is using to run the following command ``` jq '.[:100]' report.asff 1> short_report.asff ``` You can customize ``` $ export AWS_REGION=us-west-1 $ export AWSACCOUNTID=123456789012 $ trivy image --format template --template "@your-asff.tpl" -o report.asff golang:1.12-alpine ```
Project: secureCodeBox (Provisioning / Security & Compliance)
File: scanners-and-hooks.md

title: "Community Scanners and Hooks" sidebar_label: Scanners and Hooks path: "docs/community-features/scanners-and-hooks" sidebar_position: 0 :::note Thanks to our community, there are even more features available for the secureCodeBox than in the main branch and official releases. This site gives an overview about all community features that are currently known to us. If you want your feature to get listed here or to have it removed, please . Features added to this list have been checked at least once by the core development team to fit the general concept of the secureCodeBox. However, we cannot guarantee that they are compatible with the latest release. You have to use them on your own responsibility and risk. ::: - Added 16.05.2022 Following...
Project: GraphScope (App Definition and Development / Database)
File: bug_report.md

name: Bug report about: Create a report to help us improve title: "[BUG] " labels: '' assignees: '' Describe the bug A clear and concise description of what the bug is. To Reproduce Steps to reproduce the behavior: Go to '...' Click on '....' Scroll down to '....' See error Expected behavior A clear and concise description of what you expected to happen. Screenshots If applicable, add screenshots to help explain your problem. Environment (please complete the following information): GraphScope version: [e.g., v0.1, v0.2.2, master] OS: [e.g. macOS, Linux] Version [e.g. 10.15] Kubernetes Version [e.g., 1.19] Additional context Add any other context about the problem here.
Project: ArangoDB (App Definition and Development / Database)
File: assume_error_rvalue.md

+++ title = "`errortype &&assumeerror() && noexcept`" description = "Narrow contract rvalue reference observer of the stored error. Constexpr propagating, never throws." categories = ["observers"] weight = 720 +++ Narrow contract rvalue reference observer of the stored error. `NoValuePolicy::narrowerrorcheck()` is first invoked, then the reference to the error is returned. As a valid default constructed error is always present, no undefined behaviour occurs unless `NoValuePolicy::narrowerrorcheck()` does that. Note that if `error_type` is `void`, only a `const` overload returning `void` is present. Requires: Always available. Complexity: Depends on `NoValuePolicy::narrowerrorcheck()`. Guarantees: An exception is never thrown.
Project: ArangoDB (App Definition and Development / Database)
File: exception.md

+++ title = "`static auto &&_exception(Impl &&) noexcept`" description = "Returns a reference to the exception in the implementation passed in. Constexpr, never throws." categories = ["observers"] weight = 270 +++ Returns a reference to the exception in the implementation passed in. No checking is done to ensure there is an error. Constexpr where possible. Requires: Always available. Complexity: Constant time. Guarantees: Never throws an exception.