cwielenb/CNCF-Dataset huggingface dataset:
Project: CubeFS (Runtime / Cloud Native Storage)
File: LICENSE.md

The MIT License (MIT) Copyright (c) 2013 Fatih Arslan Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Project: Kata Containers (Runtime / Container Runtime)
File: how-to-use-virtio-fs-nydus-with-kata.md

Refer to for introduction and `nydus` has supported Kata Containers with hypervisor `QEMU` and `CLH` currently. You can use Kata Containers with `nydus` as follows, Use ; Deploy `nydus` environment as ; Start `nydus-snapshotter` with `enablenydusoverlayfs` enabled; Use `latest` branch to compile and build `kata-containers.img`; Update `configuration-qemu.toml` or `configuration-clh.toml`to include: ```toml shared_fs = "virtio-fs-nydus" virtiofsdaemon = "<nydusd binary path>" virtiofsextra_args = [] ``` run `crictl run -r kata nydus-container.yaml nydus-sandbox.yaml`; The `nydus-sandbox.yaml` looks like below: ```yaml metadata: attempt: 1 name: nydus-sandbox uid: nydus-uid namespace: default log_directory: /tmp linux: security_context: namespace_options: network: 2 annotations: "io.containerd.osfeature": "nydus.remoteimage.v1" ``` The `nydus-container.yaml` looks like below: ```yaml metadata: name: nydus-container image: image: localhost:5000/ubuntu-nydus:latest command: /bin/sleep args: 600 log_path: container.1.log ```
Project: Cilium (Runtime / Cloud Native Network)
File: cilium-dbg_recorder.md

<!-- This file was autogenerated via cilium cmdref, do not edit manually--> Introspect or mangle pcap recorder ``` -h, --help help for recorder ``` ``` --config string Config file (default is $HOME/.cilium.yaml) -D, --debug Enable debug messages -H, --host string URI to server-side API ``` - CLI - Delete individual pcap recorder - Display individual pcap recorder - List current pcap recorders - Update individual pcap recorder
Project: lxd (Runtime / Container Runtime)
File: storage_cephobject.md

(storage-cephobject)= % Include content from ```{include} storage_ceph.md :start-after: <!-- Include start Ceph intro --> :end-before: <!-- Include end Ceph intro --> ``` is an object storage interface built on top of to provide applications with a RESTful gateway to . It provides object storage functionality with an interface that is compatible with a large subset of the Amazon S3 RESTful API. % Include content from ```{include} storage_ceph.md :start-after: <!-- Include start Ceph terminology --> :end-before: <!-- Include end Ceph terminology --> ``` A Ceph Object Gateway consists of several OSD pools and one or more Ceph Object Gateway daemon (`radosgw`) processes that provide object gateway functionality. ```{note} The `cephobject` driver can only be used for buckets. For storage volumes, use the {ref}`Ceph <storage-ceph>` or {ref}`CephFS <storage-cephfs>` drivers. ``` % Include content from ```{include} storage_ceph.md :start-after: <!-- Include start Ceph driver cluster --> :end-before: <!-- Include end Ceph driver cluster --> ``` You must set up a `radosgw` environment beforehand and ensure that its HTTP/HTTPS endpoint URL is reachable from the Incus server or servers. See for information on how to set up a Ceph cluster and on how to set up a `radosgw` environment. The `radosgw` URL can be specified at pool creation time using the option. Incus uses the `radosgw-admin` command to manage buckets. So this command must be available and operational on the Incus servers. % Include content from ```{include} storage_ceph.md :start-after: <!-- Include start Ceph driver remote --> :end-before: <!-- Include end Ceph driver remote --> ``` % Include content from ```{include} storage_ceph.md :start-after: <!-- Include start Ceph driver control --> :end-before: <!-- Include end Ceph driver control --> ``` The following configuration options are available for storage pools that use the `cephobject` driver and for storage buckets in these pools. (storage-cephobject-pool-config)= Key | Type | Default | Description :-- | : | : | :- `cephobject.bucket.name_prefix` | string | - | Prefix to add to bucket names in Ceph `cephobject.cluster_name` | string | `ceph` | The Ceph cluster to use `cephobject.radosgw.endpoint` | string | - | URL of the `radosgw` gateway process `cephobject.radosgw.endpointcertfile` | string | - | Path to the file containing the TLS client certificate to use for endpoint communication `cephobject.user.name` | string | `admin` | The Ceph user to use `volatile.pool.pristine` | string | `true` | Whether the `radosgw` `incus-admin` user existed at creation time Key | Type | Default | Description :-- | : | : | :- `size` | string | - | Quota of the storage bucket
Project: JuiceFS (Runtime / Cloud Native Storage)
File: clone.md

title: Clone Files or Directories sidebar_position: 6 This command makes a 1:1 clone of your data by creating a mere metadata copy, without creating any new data in the object storage, thus cloning is very fast regardless of target file / directory size. Under JuiceFS, this command is a better alternative to `cp`, moreover, for Linux clients using kernels with support, then the `cp` command achieves the same result as `juicefs clone`. The clone result is a metadata copy only, where all the files are still referencing the same underlying object storage blocks, that's why a clone behaves the same in every way as its originals. When either of them go through actual file data modification, the affected data blocks will be copied on write, and become new blocks after write, while the unchanged part of the files remains the same, still referencing the original blocks. Please note that system tools like disk-free or disk-usage (`df`, `du`) will report the space used by the cloned data, but the underlying object storage space will not grow as blocks remains the same. On the same way, as metadata is actually replicated, the clone will take the same metadata engine storage space as the original. Clones takes up both file system storage space, inodes and metadata engine storage space. Pay special attention when making clones on large size directories. ```shell juicefs clone SRC DST juicefs clone /mnt/jfs/file1 /mnt/jfs/file2 juicefs clone /mnt/jfs/dir1 /mnt/jfs/dir2 ``` In terms of transaction consistency, cloning behaves as follows: Before `clone` command finishes, destination file is not visible. For file: The `clone` command ensures atomicity, meaning that the cloned file will always be in a correct and consistent state. For directory: The `clone` command does not guarantee atomicity for directories. In other words, if the source directory changes during the cloning process, the target directory may be different from the source directory. Only one `clone` can be successfully created from the same location at the same time. The failed clone will clean up the temporarily created directory tree. The clone is done by the mount process, it will be interrupted if `clone` command is terminated. If the clone fails or is interrupted, `mount` process will cleanup any created inodes. If the mount process fails to do that, there could be some leaking the metadata engine and object storage, because the dangling tree still hold the references to underlying data blocks. They could be cleaned up by the command.
Project: CubeFS (Runtime / Cloud Native Storage)
File: inode.md

``` bash curl -v http://192.168.0.22:17220/getInode?pid=100&ino=1024 ``` Request Parameters: | Parameter | Type | Description | |--||-| | pid | Integer | Shard ID | | ino | Integer | Inode ID | ``` bash curl -v http://192.168.0.22:17220/getExtentsByInode?pid=100&ino=1024 ``` Request Parameters: | Parameter | Type | Description | |--||-| | pid | Integer | Shard ID | | ino | Integer | Inode ID | ``` bash curl -v http://192.168.0.22:17220/getAllInodes?pid=100 ``` Request Parameters: | Parameter | Type | Description | |--||-| | pid | Integer | Shard ID | ``` bash curl -v '192.168.0.22:17220/getEbsExtentsByInode?pid=282&ino=16797167' ``` Request Parameters: | Parameter | Type | Description | |--||-| | pid | Integer | Shard ID | | ino | Integer | Inode ID |
Project: Kilo (Runtime / Cloud Native Network)
File: building_website.md

You may have noticed that the `markdown` files in the `/docs` directory are also displayed on . If you want to add documentation to Kilo, you can start a local webserver to check out how the website would look like. Install . The markdown files for the website are located in `/website/docs` and are generated from the like-named markdown files in the `/docs` directory and from the corresponding header files without the `.md` extension in the `/website/docs` directory. To generate the markdown files in `/website/docs`, run: ```shell make website/docs/README.md ``` Next, build the website itself by installing the `node_modules` and building the website's HTML from the generated markdown: ```shell make website/build/index.html ``` Now, start the website server with: ```shell yarn --cwd website start ``` This command should have opened a browser window with the website; if not, open your browser and point it to `http://localhost:3000`. If you make changes to any of the markdown files in `/docs` and want to reload the local `node` server, run: ```shell make website/docs/README.md -B ``` You can execute the above while the node server is running and the website will be rebuilt and reloaded automatically. If you add a new file to the `/docs` directory, you also need to create a corresponding header file containing the front-matter in `/website/docs/`. Then, regenerate the markdown for the website with the command: ```shell make website/docs/README.md ``` Edit `/website/sidebars.js` accordingly. Note: The `id` in the header file `/website/docs/<new file>` must match the `id` specified in `website/sidebars.js`.
Project: JuiceFS (Runtime / Cloud Native Storage)
File: ADOPTERS.md

title: Adopters sidebar_position: 1 slug: /adopters | Company/Team | Environment | Use Cases | User Story | |-|-|--|--| | | Production | AI | | | | Production | Big Data, AI | | | | Production | Big Data | | | | Production | Big Data | | | | Production | Big Data | | | | Production | Big Data | | | | Production | Big Data | | | | Production | Big Data, File Sharing | | | | Production | AI | | | | Production | File Sharing | | | | Production | File Sharing | | | | Production | AI | | | | Production | DevOps | | | | Production | AI | | | | Production | Big Data, File Sharing | | | | Production | Big Data | | | | Production | Big Data, File Sharing | | | | Production | AI, File Sharing | | | | Production | AI | | | | Production | File Sharing | | | | Production | AI | | | | Production | File Sharing, VFX Rendering | | | | Production | AI | | | | Production | Big Data | | | | Production | AI, File Sharing | | | | Production | AI | | | | Production | File Sharing | | | | Production | AI, File Sharing | | | | Production | AI, File Sharing | | | | Production | AI, File Sharing | | | | Production | Big Data | | | | Production | File Sharing, VFX Rendering | | | | Production | AI, File Sharing | | | | Production | Big Data | | | | Production | Big Data, File Sharing | | | | Testing | File Sharing | | | | Production | Big Data, File Sharing | | | | Production | File Sharing | | | | Production | AI | | | | Production | Big Data | | | | Production | Big Data | | | | Production | HPC, File Sharing | | | | Production | AI | | | | Production | AI | | You are welcome to share your experience after using JuiceFS, either by submitting a Pull Request directly to this list, or by contacting us at .
Project: OpenEBS (Runtime / Cloud Native Storage)
File: bug-report.md

name: Bug Report about: Report a bug if something isn't working as expected <!-- Make sure to include as much information as possible so we can fix it as quickly as possible. --> <!-- If you know how to fix this bug, please open a pull request on https://github.com/openebs/openebs/compare/?template=bugs.md --> <!-- If you can't answer some sections, please delete them --> <!-- Provide a description of this bug --> <!-- Tell us what should happen --> <!-- Tell us what happens instead --> <!-- Suggest a fix or reason for this bug --> <!-- Provide a link to a live example or steps to reproduce this bug --> <!-- Add screenshots of this bug --> <!-- Include as many relevant details about the environment you experienced the bug in --> `kubectl get nodes`: `kubectl get pods --all-namespaces`: `kubectl get services`: `kubectl get sc`: `kubectl get pv`: `kubectl get pvc`: OS (from `/etc/os-release`): Kernel (from `uname -a`): Install tools: Others:
Project: Project Calico (Runtime / Cloud Native Network)
File: release-checklist.md

In order to cut a new release, a few things must be done: auto-generate the CHANGELOG using the provided script bump version.go and docs/installation.md to the new release push a tag for the new release draft a bump version.go to the next release, appending `-dev` For the former, you can use the following script: $ echo -e "$(./contrib/generate-changelog.sh v$LATEST_RELEASE)\n" | cat - CHANGELOG | sponge CHANGELOG You can find `sponge` in the `moreutils` package on Ubuntu. This script will generate all merged changes since $LATEST_RELEASE and append it to the top of the CHANGELOG. However, this will show up as "HEAD" at the top: $ ./contrib/generate-changelog.sh v$LATEST_RELEASE abc123 Some merged PR summary ... You'll need to manually modify "HEAD" to show up as the latest release. When drafting a new release, you must make sure that a `darwin` and `linux` build of confd have been uploaded. If you have cross-compile support, you can use the following command to generate those binaries: $ CONFDCROSSPLATFORMS="darwin/amd64 linux/amd64" NEWRELEASE="x.y.z" $ for platform in $CONFD_CROSSPLATFORMS; do \ GOOS=${platform%/} GOARCH=${platform##/} ./build; \ mv bin/confd bin/confd-$NEW_RELEASE-${platform%/}-${platform##/}; \ done You can then drag and drop these binaries into the release draft.
Project: FD.io (Runtime / Cloud Native Network)
File: dpdk_crypto_ipsec_doc.md

This document is meant to contain all related information about implementation and usability. DPDK Cryptodev is an asynchronous crypto API that supports both Hardware and Software implementations (for more details refer to ). When there are enough Cryptodev resources for all workers, the node graph is reconfigured by adding and changing the default next nodes. The following nodes are added: dpdk-crypto-input : polling input node, dequeuing from crypto devices. dpdk-esp-encrypt : internal node. dpdk-esp-decrypt : internal node. dpdk-esp-encrypt-post : internal node. dpdk-esp-decrypt-post : internal node. Set new default next nodes: for esp encryption: esp-encrypt -> dpdk-esp-encrypt for esp decryption: esp-decrypt -> dpdk-esp-decrypt When building DPDK with VPP, Cryptodev support is always enabled. Additionally, on x86_64 platforms, DPDK is built with SW crypto support. VPP allocates crypto resources based on a best effort approach: first allocate Hardware crypto resources, then Software. if there are not enough crypto resources for all workers, the graph node is not modified and the default VPP IPsec implementation based in OpenSSL is used. The following message is displayed: 0: dpdkipsecinit: not enough Cryptodevs, default to OpenSSL IPsec To enable DPDK Cryptodev the user just need to provide cryptodevs in the startup.conf. Below is an example startup.conf, it is not meant to be a default configuration: ``` dpdk { dev 0000:81:00.0 dev 0000:81:00.1 dev 0000:85:01.0 dev 0000:85:01.1 vdev cryptoaesnimb0,socket_id=1 vdev cryptoaesnimb1,socket_id=1 } ``` In the above configuration: 0000:81:01.0 and 0000:81:01.1 are Ethernet device BDFs. 0000:85:01.0 and 0000:85:01.1 are Crypto device BDFs and they require the same driver binding as DPDK Ethernet devices but they do not support any extra configuration options. Two AESNI-MB Software (Virtual) Cryptodev PMDs are created in NUMA node 1. For further details refer to The following CLI command displays the Cryptodev/Worker mapping: show crypto device mapping [verbose] Building the DPDK Crypto Libraries requires the open source project nasm (The Netwide Assembler) to be installed. Recommended version of nasm is 2.12.02. Minimum supported version of nasm is 2.11.06. Use the following command to determine the current nasm version: nasm -v CentOS 7.3 and earlier and Fedora 21 and earlier use unsupported versions of nasm. Use the following set of commands to build a supported version: wget http://www.nasm.us/pub/nasm/releasebuilds/2.12.02/nasm-2.12.02.tar.bz2 tar -xjvf nasm-2.12.02.tar.bz2 cd nasm-2.12.02/ ./configure make sudo make install
Project: HwameiStor (Runtime / Cloud Native Storage)
File: issue_template.md

<!-- If you need help or think you have found a bug, please help us with your issue by entering the following information (otherwise you can delete this text): --> Output of `lsblk`: ``` ``` Output of `blkid`: ``` ``` Output of `kubectl version`: ``` ``` Output of `kubectl get lsn -o yaml`: ``` ``` Cloud Provider/Platform (AKS, GKE, Minikube etc.): ``` ```
Project: Ceph (Runtime / Cloud Native Storage)
File: nats_adapter.md

This directory contains examples on how to use together with a to add NATS to the list of bucket notifications endpoints. To test your setup: Install and start a nats-server. Subscribe to the NATS server using a , choosing the topic to be 'Bucket_Notification' (as defined in the ) ```bash nats-sub "Bucket_Notification" ``` . Alternatively, configure the script to point to an existing NATS broker by editing the following part in the script to match the parameters of your existing nats server. ``` nats_host = '{host}', nats_port = {port}, ``` Upload the : ```bash radosgw-admin script put --infile=nats_adapter.lua --context=postRequest ``` Add the packages used in the script: ```bash radosgw-admin script-package add --package=nats --allow-compilation radosgw-admin script-package add --package=lunajson --allow-compilation radosgw-admin script-package add --package='lua-cjson 2.1.0-1' --allow-compilation ``` Restart radosgw. create a bucket: ``` s3cmd --host=localhost:8000 --host-bucket="localhost:8000/%(bucket)" mb s3://mybucket ``` upload a file to the bucket and make sure that the nats server received the notification ``` s3cmd --host=localhost:8000 --host-bucket="localhost:8000/%(bucket)" put hello.txt s3://mybucket ``` Expected output: ``` Received on [Bucket_Notification]: {"Records":[ { "eventVersion":"2.1", "eventSource":"ceph:s3", "awsRegion":"default", "eventTime":"2019-11-22T13:47:35.124724Z", "eventName":"ObjectCreated:Put", "userIdentity":{ "principalId":"tester" }, "requestParameters":{ "sourceIPAddress":"" }, "responseElements":{ "x-amz-request-id":"503a4c37-85eb-47cd-8681-2817e80b4281.5330.903595", "x-amz-id-2":"14d2-zone1-zonegroup1" }, "s3":{ "s3SchemaVersion":"1.0", "configurationId":"mynotif1", "bucket":{ "name":"mybucket", "ownerIdentity":{ "principalId":"tester" }, "arn":"arn:aws:s3:us-east-1::mybucket1", "id":"503a4c37-85eb-47cd-8681-2817e80b4281.5332.38" }, "object":{ "key":"hello.txt", "size":"1024", "eTag":"", "versionId":"", "sequencer": "F7E6D75DC742D108", "metadata":[], "tags":[] } }, "eventId":"", "opaqueData":"me@example.com" } ]} ``` Lua 5.3 (or higher) Luarocks
Project: Rook (Runtime / Cloud Native Storage)
File: INSTALL.md

Rook is composed of a golang project and can be built directly with standard `golang` tools, and storage software (like Ceph) that are built inside containers. We currently support these platforms for building: Linux: most modern distributions should work although most testing has been done on Ubuntu Mac: macOS 10.6+ is supported Recommend 2+ cores, 8+ GB of memory and 128GB of SSD. Inside your build environment (Docker for Mac or a VM), 2+ GB memory is also recommended. The following tools are need on the host: curl docker (1.12+) or Docker for Mac (17+) git make golang helm You can build the Rook binaries and all container images for the host platform by simply running the command below. Building in parallel with the `-j` option is recommended. ```console make -j4 build ``` Run `make help` for more options. Every PR and every merge to master triggers the CI process in GitHub actions. On every commit to PR and master the CI will build, run unit tests, and run integration tests. If the build is for master or a release, the build will also be published to . You can also run the build for all supported platforms: ```console make -j4 build.all ```
Project: containerd (Runtime / Container Runtime)
File: rootless.md

A non-root user can execute containerd by using . For example can be used for setting up a user namespace (along with mount namespace and optionally network namespace). Please refer to RootlessKit documentation for further information. See also https://rootlesscontaine.rs/ . The easiest way is to use `containerd-rootless-setuptool.sh` included in . ```console $ containerd-rootless-setuptool.sh install $ nerdctl run -d --restart=always --name nginx -p 8080:80 nginx:alpine ``` See for further information. <details> <summary>Click here to show the "hard way"</summary> <p> ```console $ rootlesskit --net=slirp4netns --copy-up=/etc --copy-up=/run \ --state-dir=/run/user/1001/rootlesskit-containerd \ sh -c "rm -f /run/containerd; exec containerd -c config.toml" ``` `--net=slirp4netns --copy-up=/etc` is only required when you want to unshare network namespaces. See for further information about the network drivers. `--copy-up=/DIR` mounts a writable tmpfs on `/DIR` with symbolic links to the files under the `/DIR` on the parent namespace so that the user can add/remove files under `/DIR` in the mount namespace. `--copy-up=/etc` and `--copy-up=/run` are needed on typical setup. Depending on the containerd plugin configuration, you may also need to add more `--copy-up` options. `rm -f /run/containerd` removes the "copied-up" symbolic link to `/run/containerd` on the parent namespace (if exists), which cannot be accessed by non-root users. The actual `/run/containerd` directory on the host is not affected. `--state-dir` is set to a random directory under `/tmp` if unset. RootlessKit writes the PID to a file named `child_pid` under this directory. You need to provide `config.toml` with your own path configuration. e.g. ```toml version = 2 root = "/home/penguin/.local/share/containerd" state = "/run/user/1001/containerd" [grpc] address = "/run/user/1001/containerd/containerd.sock" ``` A client program such as `ctr` also needs to be executed inside the daemon namespaces. ```console $ nsenter -U --preserve-credentials -m -n -t $(cat /run/user/1001/rootlesskit-containerd/child_pid) $ export CONTAINERD_ADDRESS=/run/user/1001/containerd/containerd.sock $ export CONTAINERD_SNAPSHOTTER=native $ ctr images pull docker.io/library/ubuntu:latest $ ctr run -t --rm --fifo-dir /tmp/foo-fifo --cgroup "" docker.io/library/ubuntu:latest foo ``` The `overlayfs` snapshotter does not work inside user namespaces before kernel 5.11, except on Ubuntu and Debian kernels. However, can be used instead if running kernel >= 4.18. Enabling cgroup requires cgroup v2 and systemd, e.g. `ctr run --cgroup "user.slice:foo:bar" --runc-systemd-cgroup ...` . See also . </p> </details>