cwielenb/CNCF-Dataset huggingface dataset:
Project: Firecracker (Runtime / Container Runtime)
File: entropy.md

An entropy device is a that provides guests with "high-quality randomness for guest use". Guests issue requests in the form of a buffer that will be filled with random bytes from the device. The source of random bytes that the device will use to fill the buffers is an implementation decision. On the guest side, the kernel uses random bytes received through the device as an extra source of entropy. Moreover, the guest VirtIO driver exposes the `/dev/hwrng` character device. User-space applications can use this device to request random bytes from the device. Firecracker offers the option of attaching a single `virtio-rng` device. Users can configure it through the `/entropy` API endpoint. The request body includes a single (optional) parameter for configuring a rate limiter. For example, users can configure the entropy device with a bandwidth rate limiter of 10KB/sec like this: ```console curl --unix-socket $socket_location -i \ -X PUT 'http://localhost/entropy' \ -H 'Accept: application/json' \ -H 'Content-Type: application/json' \ -d "{ \"rate_limiter\": { \"bandwidth\": { \"size\": 1000, \"onetimeburst\": 0, \"refill_time\": 100 } } }" ``` If a configuration file is used for configuring a microVM, the same setup can be achieved by adding a section like this: ```json "entropy": { "rate_limiter": { "bandwidth" { "size": 1000, "onetimeburst": 0, "refill_time": 100 } } } ``` On the host side, Firecracker relies on to retrieve the random bytes. `aws-lc-rs` uses the . In order to use the entropy device, users must use a kernel with the `virtio-rng` front-end driver compiled in or loaded as a module. The relevant kernel configuration option is `CONFIGHWRANDOM_VIRTIO` (which depends on `CONFIGHWRANDOM` and `CONFIG_VIRTIO`).
Project: Cilium (Runtime / Cloud Native Network)
File: cilium-operator-azure_completion_bash.md

<!-- This file was autogenerated via cilium-operator --cmdref, do not edit manually--> Generate the autocompletion script for bash Generate the autocompletion script for the bash shell. This script depends on the 'bash-completion' package. If it is not installed already, you can install it via your OS's package manager. To load completions in your current shell session: source <(cilium-operator-azure completion bash) To load completions for every new session, execute once: cilium-operator-azure completion bash > /etc/bash_completion.d/cilium-operator-azure cilium-operator-azure completion bash > $(brew --prefix)/etc/bash_completion.d/cilium-operator-azure You will need to start a new shell for this setup to take effect. ``` cilium-operator-azure completion bash ``` ``` -h, --help help for bash --no-descriptions disable completion descriptions ``` - Generate the autocompletion script for the specified shell
Project: CNI-Genie (Runtime / Cloud Native Network)
File: external-build-dependencies.md

| Software | License | Repo Link |||| |k8s.io/api | Apache License 2.0 |https://github.com/kubernetes/api |k8s.io/apimachinery | Apache License 2.0 |https://github.com/kubernetes/apimachinery |k8s.io/client-go | Apache License 2.0 |https://github.com/kubernetes/client-go |k8s.io/kube-openapi | Apache License 2.0 |https://github.com/kubernetes/kube-openapi |fsnotify |BSD-3-Clause | https://github.com/fsnotify/fsnotify |inf | BSD-3-Clause |https://github.com/go-inf/inf |tomb.v1 | BSD-3-Clause |https://github.com/go-tomb/tomb/tree/v1 |yaml.v2 | Apache License 2.0 |https://github.com/go-yaml/yaml/tree/v2 |crypto | BSD-3-Clause | https://github.com/golang/crypto |exp | BSD-3-Clause | https://github.com/golang/exp |net | BSD-3-Clause | https://github.com/golang/net |sys | BSD-3-Clause | https://github.com/golang/sys |text | BSD-3-Clause | https://github.com/golang/text |time | BSD-3-Clause | https://github.com/golang/time |cni | Apache License 2.0 | https://github.com/containernetworking/cni |go-iptables| Apache License 2.0 | https://github.com/coreos/go-iptables |go-spew |ISC | https://github.com/davecgh/go-spew |yaml |MIT | https://github.com/ghodss/yaml |protobuf |BSD-3-Clause | https://github.com/gogo/protobuf golang/glog |Apache License 2.0 | https://github.com/golang/glog |golang/groupcache |Apache License 2.0 | https://github.com/golang/groupcache |golang/protobuf |Apache License 2.0 | https://github.com/golang/protobuf |cadvisor |Apache License 2.0 | https://github.com/google/cadvisor |gofuzz |Apache License 2.0 | https://github.com/google/gofuzz |btree |Apache License 2.0 | https://github.com/google/btree |gnostic |Apache License 2.0 | https://github.com/googleapis/gnostic |httpcache | MIT| https://github.com/gregjones/httpcache |golang-lru |MLP-2.0 | https://github.com/hashicorp/golang-lru |tail |MIT | https://github.com/hpcloud/tail |mergo |BSD-3-Clause | https://github.com/imdario/mergo |json-iterator/go |MIT | https://github.com/json-iterator/go |concurrent |Apache License 2.0 | https://github.com/modern-go/concurrent |reflect2 |Apache License 2.0 | https://github.com/modern-go/reflect2 |ginkgo |MIT | https://github.com/onsi/ginkgo |gomega |MIT | https://github.com/onsi/gomega |GoLLRB |BSD-3-Clause | https://github.com/petar/GoLLRB |diskv | MIT| https://github.com/peterbourgon/diskv |spf13/pflag |MIT | https://github.com/spf13/pflag
Project: CubeFS (Runtime / Cloud Native Storage)
File: mount-sequence.md

FUSE mounting is a little bit tricky. There's a userspace helper tool that performs the handshake with the kernel, and then steps out of the way. This helper behaves differently on different platforms, forcing a more complex API on us. On Linux, the mount is immediate and file system accesses wait until the requests are served. On OS X, the mount becomes visible only after `InitRequest` (and maybe more) have been served. Let's see what happens if `InitRequest` gets an error response. On Linux, the mountpoint is there but all operations will fail: On OS X, the mount never happened:
Project: Kata Containers (Runtime / Container Runtime)
File: what-is-vm-cache-and-how-do-I-use-it.md

VMCache is a new function that creates VMs as caches before using it. It helps speed up new container creation. The function consists of a server and some clients communicating through Unix socket. The protocol is gRPC in . The VMCache server will create some VMs and cache them by factory cache. It will convert the VM to gRPC format and transport it when gets requested from clients. Factory `grpccache` is the VMCache client. It will request gRPC format VM and convert it back to a VM. If VMCache function is enabled, `kata-runtime` will request VM from factory `grpccache` when it creates a new sandbox. Both and VMCache help speed up new container creation. When VM templating enabled, new VMs are created by cloning from a pre-created template VM, and they will share the same initramfs, kernel and agent memory in readonly mode. So it saves a lot of memory if there are many Kata Containers running on the same host. VMCache is not vulnerable to because each VM doesn't share the memory. VMCache can be enabled by changing your Kata Containers config file (`/usr/share/defaults/kata-containers/configuration.toml`, overridden by `/etc/kata-containers/configuration.toml` if provided) such that: `vmcachenumber` specifies the number of caches of VMCache: unspecified or == 0 VMCache is disabled `> 0` will be set to the specified number `vmcacheendpoint` specifies the address of the Unix socket. Then you can create a VM templating for later usage by calling: ``` $ sudo kata-runtime factory init ``` and purge it by `ctrl-c` it. Cannot work with VM templating. Only supports the QEMU hypervisor.
Project: Stash by AppsCode (Runtime / Cloud Native Storage)
File: RELEASE.md

The `yaml` Project is released on an as-needed basis. The process is as follows: An issue is proposing a new release with a changelog since the last release All must LGTM this release An OWNER runs `git tag -s $VERSION` and inserts the changelog and pushes the tag with `git push $VERSION` The release issue is closed An announcement email is sent to `kubernetes-dev@googlegroups.com` with the subject `[ANNOUNCE] kubernetes-template-project $VERSION is released`
Project: Kilo (Runtime / Cloud Native Network)
File: README.md

This website is built using , a modern static website generator. ``` $ yarn ``` ``` $ yarn start ``` This command starts a local development server and open up a browser window. Most changes are reflected live without having to restart the server. ``` $ yarn build ``` This command generates static content into the `build` directory and can be served using any static contents hosting service. ``` $ GITUSER=<Your GitHub username> USESSH=true yarn deploy ``` If you are using GitHub pages for hosting, this command is a convenient way to build the website and push to the `gh-pages` branch. iled overview of the rationale behind this method, . This package uses and therefore supports . Caveat #1: When using `yaml.Marshal` and `yaml.Unmarshal`, binary data should NOT be preceded with the `!!binary` YAML tag. If you do, go-yaml will convert the binary data from base64 to native binary data, which is not compatible with JSON. You can still use binary in your YAML files though - just store them without the `!!binary` tag and decode the base64 in your code (e.g. in the custom JSON methods `MarshalJSON` and `UnmarshalJSON`). This also has the benefit that your YAML and your JSON binary data will be decoded exactly the same way. As an example: ``` BAD: exampleKey: !!binary gIGC GOOD: exampleKey: gIGC ... and decode the base64 data in your code. ``` Caveat #2: When using `YAMLToJSON` directly, maps with keys that are maps will result in an error since this is not supported by JSON. This error will occur in `Unmarshal` as well since you can't unmarshal map keys anyways since struct fields can't be keys. To install, run: ``` $ go get sigs.k8s.io/yaml ``` And import using: ``` import "sigs.k8s.io/yaml" ``` Usage is very similar to the JSON library: ```go package main import ( "fmt" "sigs.k8s.io/yaml" ) type Person struct { Name string `json:"name"` // Affects YAML field names too. Age int `json:"age"` } func main() { // Marshal a Person struct to YAML. p := Person{"John", 30} y, err := yaml.Marshal(p) if err != nil { fmt.Printf("err: %v\n", err) return } fmt.Println(string(y)) /* Output: age: 30 name: John */ // Unmarshal the YAML back into a Person struct. var p2 Person err = yaml.Unmarshal(y, &p2) if err != nil { fmt.Printf("err: %v\n", err) return } fmt.Println(p2) /* Output: {John 30} */ } ``` `yaml.YAMLToJSON` and `yaml.JSONToYAML` methods are also available: ```go package main import ( "fmt" "sigs.k8s.io/yaml" ) func main() { j := []byte(`{"name": "John", "age": 30}`) y, err := yaml.JSONToYAML(j) if err != nil { fmt.Printf("err: %v\n", err) return } fmt.Println(string(y)) /* Output: age: 30 name: John */ j2, err := yaml.YAMLToJSON(y) if err != nil { fmt.Printf("err: %v\n", err) return } fmt.Println(string(j2)) /* Output: {"age":30,"name":"John"} */ } ```
Project: Weave Net (Runtime / Cloud Native Network)
File: LICENSE.md

MIT License =========== Copyright (C) 2015 Matt Layher Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Project: Swift (Runtime / Cloud Native Storage)
File: dockerhub_description.md

SAIO is a containerized instance of Openstack Swift object storage. It is running the main services of Swift, designed to provide an endpoint for application developers to test against both the Swift and AWS S3 API. It can also be used when integrating with a CI/CD system. These images are not configured to provide data durability and are not intended for production use. ``` docker pull openstackswift/saio docker run -d -p 8080:8080 openstackswift/saio ``` Example using swift client to target endpoint: ``` swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat ``` Example using s3cmd to test AWS S3: Create config file: ``` [default] access_key = test:tester secret_key = testing host_base = localhost:8080 host_bucket = localhost:8080 use_https = False ``` Test with s3cmd: ``` s3cmd -c s3cfg_saio mb s3://bucket ``` Image tags: `latest` automatically built/published by Zuul, follows master branch. Releases are also tagged in case you want to test against a specific release. Source Code: github.com/openstack/swift Maintained by: Openstack Swift community Feedback/Questions: #openstack-swift on OFTC
Project: Kanister (Runtime / Cloud Native Storage)
File: markdown-examples.md

This page demonstrates some of the built-in markdown extensions provided by VitePress. VitePress provides Syntax Highlighting powered by , with additional features like line-highlighting: Input ````md ```js{4} export default { data () { return { msg: 'Highlighted!' } } } ``` ```` Output ```js{4} export default { data () { return { msg: 'Highlighted!' } } } ``` Input ```md ::: info This is an info box. ::: ::: tip This is a tip. ::: ::: warning This is a warning. ::: ::: danger This is a dangerous warning. ::: ::: details This is a details block. ::: ``` Output ::: info This is an info box. ::: ::: tip This is a tip. ::: ::: warning This is a warning. ::: ::: danger This is a dangerous warning. ::: ::: details This is a details block. ::: Check out the documentation for the .
Project: JuiceFS (Runtime / Cloud Native Storage)
File: fuse_mount_options.md

title: FUSE Mount Options sidebar_position: 5 slug: /fusemountoptions JuiceFS provides several access methods, FUSE is the common one, which is the way to mount the file system locally using the `juicefs mount` command. Users can add FUSE mount options for more granular control. This guide describes the common FUSE mount options for JuiceFS, with two ways to add mount options: Run , and use `-o` to specify multiple options separated by commas. ```bash juicefs mount -d -o allowother,writebackcache sqlite3://myjfs.db ~/jfs ``` When writing `/etc/fstab` items, add FUSE options directly to the `options` field, with multiple options separated by commas. ``` redis://localhost:6379/1 /jfs juicefs netdev,writebackcache 0 0 ``` This option is automatically enabled when JuiceFS is mounted and does not need to be explicitly specified. It will enable the kernel's file access checks, which are performed outside the filesystem. When enabled, both the kernel checks and the file system checks must succeed before further operations. :::tip The kernel performs standard Unix permission checks based on mode bits, UID/GID, and directory entry ownership. ::: By default FUSE only allows access to the user mounting the file system. `allowother` option overrides this behavior to allow access for other users. When mounting JuiceFS using root, `allowother` is automatically assumed (search for `AllowOther` in ). When mounting by non-root users, you'll need to first modify `/etc/fuse.conf` and enable `userallowother`, and then add `allow_other` to the mount command. :::note This mount option requires at least version 3.15 Linux kernel ::: FUSE supports , which means the `write()` syscall can often complete rapidly. It's recommended to enable this mount option when write small data (e.g. 100 bytes) frequently. These two options are used to specify the owner ID and owner group ID of the mount point, but only allow to execute the mount command as root, e.g. `sudo juicefs mount -o userid=100,groupid=100`. This option will output Debug information from the low-level library (`go-fuse`) to `juicefs.log`. :::note This option will output debug information for the low-level library (`go-fuse`) to `juicefs.log`. Note that this option is different from the global `-debug` option for the JuiceFS client, where the former outputs debug information for the `go-fuse` library and the latter outputs debug information for the JuiceFS client. see the documentation . :::
Project: Gluster (Runtime / Cloud Native Storage)
File: write-behind.md

performance/write-behind translator =================================== Basic working -- Write behind is basically a translator to lie to the application that the write-requests are finished, even before it is actually finished. On a regular translator tree without write-behind, control flow is like this: application makes a `write()` system call. VFS ==> FUSE ==> `/dev/fuse`. fuse-bridge initiates a glusterfs `writev()` call. `writev()` is `STACK_WIND()`ed up to client-protocol or storage translator. client-protocol, on receiving reply from server, starts `STACK_UNWIND()` towards the fuse-bridge. On a translator tree with write-behind, control flow is like this: application makes a `write()` system call. VFS ==> FUSE ==> `/dev/fuse`. fuse-bridge initiates a glusterfs `writev()` call. `writev()` is `STACK_WIND()`ed up to write-behind translator. write-behind adds the write buffer to its internal queue and does a `STACK_UNWIND()` towards the fuse-bridge. write call is completed in application's percepective. after `STACK_UNWIND()`ing towards the fuse-bridge, write-behind initiates a fresh writev() call to its child translator, whose replies will be consumed by write-behind itself. Write-behind doesn't cache the write buffer, unless `option flush-behind on` is specified in volume specification file. Windowing With respect to write-behind, each write-buffer has three flags: `stackwound`, `writebehind` and `got_reply`. `stackwound`: if set, indicates that write-behind has initiated `STACKWIND()` towards child translator. `writebehind`: if set, indicates that write-behind has done `STACKUNWIND()` towards fuse-bridge. `gotreply`: if set, indicates that write-behind has received reply from child translator for a `writev()` `STACKWIND()`. a request will be destroyed by write-behind only if this flag is set. Currently pending write requests = aggregate size of requests with writebehind = 1 and gotreply = 0. window size limits the aggregate size of currently pending write requests. once the pending requests' size has reached the window size, write-behind blocks writev() calls from fuse-bridge. Blocking is only from application's perspective. Write-behind does `STACK_WIND()` to child translator straight-away, but hold behind the `STACK_UNWIND()` towards fuse-bridge. `STACK_UNWIND()` is done only once write-behind gets enough replies to accommodate for currently blocked request. Flush behind If `option flush-behind on` is specified in volume specification file, then write-behind sends aggregate write requests to child translator, instead of regular per request `STACK_WIND()`s.
Project: Vineyard (Runtime / Cloud Native Storage)
File: README.md

vineyard-ml: Accelerating Data Science Pipelines ================================================ Vineyard has been tightly integrated with the data preprocessing pipelines in widely-adopted machine learning frameworks like PyTorch, TensorFlow, and MXNet. Shared objects in vineyard, e.g., `vineyard::Tensor`, `vineyard::DataFrame`, `vineyard::Table`, etc., can be directly used as the inputs of the training and inference tasks in these frameworks. Examples -- The following examples shows how `DataFrame` in vineyard can be used as the input of Dataset for PyTorch: ```python import os import numpy as np import pandas as pd import torch import vineyard client = vineyard.connect(os.environ['VINEYARDIPCSOCKET']) df = pd.DataFrame({ 'data': vineyard.data.dataframe.NDArrayArray(np.random.rand(1000, 10)), 'label': np.random.rand(1000) }) object_id = client.put(df) from vineyard.contrib.ml.torch import torch_context with torch_context(): ds = client.get(object_id) from vineyard.contrib.ml.torch import datapipe pipe = datapipe(ds) for data, label in pipe: pass ``` The following example shows how to use vineyard to share pytorch modules between processes: ```python import torch import vineyard client = vineyard.connect(os.environ['VINEYARDIPCSOCKET']) class Model(nn.Module): def init(self): super().init() self.conv1 = nn.Conv2d(1, 20, 5) self.conv2 = nn.Conv2d(20, 20, 5) def forward(self, x): x = F.relu(self.conv1(x)) return F.relu(self.conv2(x)) model = Model() from vineyard.contrib.ml.torch import torch_context with torch_context(): object_id = client.put(model) model = Model() with torch_context(): statedict = client.get(objectid) model.loadstatedict(state_dict, assign=True) ``` Reference and Implementation : including PyTorch datasets, torcharrow and torchdata. For more details about vineyard itself, please refer to the project.
Project: Velero (Runtime / Cloud Native Storage)
File: upgrade-to-1.2.md

title: "Upgrading to Velero 1.2" layout: docs Velero or installed. Note: if you're upgrading from v1.0, follow the instructions first. Install the Velero v1.2 command-line interface (CLI) by following the . Verify that you've properly installed it by running: ```bash velero version --client-only ``` You should see the following output: ```bash Client: Version: v1.2.0 Git commit: <git SHA> ``` Scale down the existing Velero deployment: ```bash kubectl scale deployment/velero \ --namespace velero \ --replicas 0 ``` Update the container image used by the Velero deployment and, optionally, the restic daemon set: ```bash kubectl set image deployment/velero \ velero=velero/velero:v1.2.0 \ --namespace velero kubectl set image daemonset/restic \ restic=velero/velero:v1.2.0 \ --namespace velero ``` If using AWS, Azure, or GCP, add the respective plugin to your Velero deployment: For AWS: ```bash velero plugin add velero/velero-plugin-for-aws:v1.0.0 ``` For Azure: ```bash velero plugin add velero/velero-plugin-for-microsoft-azure:v1.0.0 ``` For GCP: ```bash velero plugin add velero/velero-plugin-for-gcp:v1.0.0 ``` Update the Velero custom resource definitions (CRDs) to include the structural schemas: ```bash velero install --crds-only --dry-run -o yaml | kubectl apply -f - ``` Scale back up the existing Velero deployment: ```bash kubectl scale deployment/velero \ --namespace velero \ --replicas 1 ``` Confirm that the deployment is up and running with the correct version by running: ```bash velero version ``` You should see the following output: ```bash Client: Version: v1.2.0 Git commit: <git SHA> Server: Version: v1.2.0 ```
Project: JuiceFS (Runtime / Cloud Native Storage)
File: integrations.md

sidebar_label: Integrations sidebar_position: 2 slug: /integrations team contributed . team participated in the development of JuiceFSRuntime cache engine, please refer to . team has integrated JuiceFS into , please refer to . Build a distributed cluster based on JuiceFS, the Milvus team wrote a and . that is a OLAP engine could deploy with the JuiceFS in dissaggregated storage and compute architecture on every public cloud platform, there is (in Chinese) and for this use case. supports JuiceFS since v0.10.0, you can refer to to learn how to configure JuiceFS. by Toowoxx IT GmbH, an IT service company from Germany
Project: Cilium (Runtime / Cloud Native Network)
File: cilium-dbg_bpf_recorder_list.md

<!-- This file was autogenerated via cilium cmdref, do not edit manually--> List PCAP recorder entries ``` cilium-dbg bpf recorder list [flags] ``` ``` -h, --help help for list -o, --output string json| yaml| jsonpath='{}' ``` ``` --config string Config file (default is $HOME/.cilium.yaml) -D, --debug Enable debug messages -H, --host string URI to server-side API ``` - PCAP recorder