cwielenb/CNCF-Dataset huggingface dataset:
Project: Velero (Runtime / Cloud Native Storage)
File: ark_restore_get.md

title: "ark restore get" layout: docs Get restores Get restores ``` ark restore get [flags] ``` ``` -h, --help help for get --label-columns stringArray a comma-separated list of labels to be displayed as columns -o, --output string Output display format. For create commands, display the object but do not send it to the server. Valid formats are 'table', 'json', and 'yaml'. (default "table") -l, --selector string only show items matching this label selector --show-labels show labels in the last column ``` ``` --alsologtostderr log to standard error as well as files --kubeconfig string Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration --kubecontext string The context to use to talk to the Kubernetes apiserver. If unset defaults to whatever your current-context is (kubectl config current-context) --logbacktraceat traceLocation when logging hits line file:N, emit a stack trace (default :0) --log_dir string If non-empty, write log files in this directory --logtostderr log to standard error instead of files -n, --namespace string The namespace in which Ark should operate (default "heptio-ark") --stderrthreshold severity logs at or above this threshold go to stderr (default 2) -v, --v Level log level for V logs --vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging ``` - Work with restores
Project: Rook (Runtime / Cloud Native Storage)
File: openshift-common-issues.md

title: OpenShift Common Issues OpenShift Console uses OpenShift Prometheus for monitoring and populating data in Storage Dashboard. Additional configuration is required to monitor the Ceph Cluster from the storage dashboard. Change the monitoring namespace to `openshift-monitoring` Change the namespace of the RoleBinding `rook-ceph-metrics` from `rook-ceph` to `openshift-monitoring` for the `prometheus-k8s` ServiceAccount in . ```yaml subjects: kind: ServiceAccount name: prometheus-k8s namespace: openshift-monitoring ``` Enable Ceph Cluster monitoring Follow . Set the required label on the namespace ```console oc label namespace rook-ceph "openshift.io/cluster-monitoring=true" ``` !!! attention Switch to `rook-ceph` namespace using `oc project rook-ceph`. Ensure ceph-mgr pod is Running ```console $ oc get pods -l app=rook-ceph-mgr NAME READY STATUS RESTARTS AGE rook-ceph-mgr 1/1 Running 0 14h ``` Ensure service monitor is present ```console $ oc get servicemonitor rook-ceph-mgr NAME AGE rook-ceph-mgr 14h ``` Ensure the prometheus rules object has been created ```console $ oc get prometheusrules -l prometheus=rook-prometheus NAME AGE prometheus-ceph-rules 14h ```
Project: Alluxio (Runtime / Cloud Native Storage)
File: Software-Requirements.md

layout: global title: Alluxio Software Requirements Listed below are the generic requirements to run Alluxio locally or as a cluster. Cluster nodes should be running one of the following supported operating systems: MacOS 10.10 or later CentOS - 6.8 or 7 RHEL - 7.x Ubuntu - 16.04 Java JDK 11 (Oracle or OpenJDK distributions supported) Alluxio works on IPv4 networks only. Allow the following ports and protocols: Inbound TCP 22 - ssh as a user to install Alluxio components across specified nodes. It is recommended to use hardware based on the x86 architecture. It is verified that Alluxio can run on the ARM architecture, but it may be possible that certain features may not work. Alluxio's overall performance is similar between the two architectures, based on benchmark results when running on (ex. ) versus (ex. ). There are Alluxio-specific requirements for cluster nodes running the master process. Note that these are bare minimum requirements to run the software. Running Alluxio at scale and under high load will increase these requirements. Minimum 1 GB disk space Minimum 1 GB memory (6 GB if using embedded journal) Minimum 2 CPU cores Allow the following ports and protocols: Inbound TCP 19998 - The Alluxio master's default RPC port Inbound TCP 19999 - The Alluxio master's default web UI port: `http://<master-hostname>:19999` Inbound TCP 20001 - The Alluxio job master's default RPC port Inbound TCP 20002 - The Alluxio job master's default web UI port Embedded Journal Requirements Only Inbound TCP 19200 - The Alluxio master's default port for internal leader election Inbound TCP 20003 - The Alluxio job master's default port for internal leader election There are Alluxio-specific requirements for cluster nodes running the worker process: Minimum 1 GB disk space Minimum 1 GB memory Minimum 2 CPU cores Allow the following ports and protocols: Inbound TCP 29999 - The Alluxio worker's default RPC port Inbound TCP 30000 - The Alluxio worker's default web UI port: `http://<worker-hostname>:30000` Inbound TCP 30001 - The Alluxio job worker's default RPC port Inbound TCP 30002 - The Alluxio job worker's default data port Inbound TCP 30003 - The Alluxio job worker's default web UI port: `http://<worker-hostname>:30003` There are Alluxio-specific requirements for nodes running the fuse process. Note that these are bare minimum requirements to run the software. Running Alluxio Fuse under high load will increase these requirements. Minimum 1 CPU core Minimum 1 GB memory FUSE installed libfuse 2.9.3 or newer for Linux, recommend to use libfuse >= 3.0.0 osxfuse 3.7.1 or newer for MacOS
Project: JuiceFS (Runtime / Cloud Native Storage)
File: articles.md

title: JuiceFS Article Collection sidebar_position: 2 slug: /articles description: Explore JuiceFS' collection of technical articles and real-world case studies in AI, machine learning, deep learning, big data, data sharing, backup, and recovery scenarios. JuiceFS is widely applicable to various data storage and sharing scenarios. This page compiles its technical articles and real-world case studies. Explore valuable insights and practical examples to deepen your understanding of JuiceFS and related applications. We encourage all community users to contribute and maintain this list. , 2024-04-09, Sui Su , 2024-04-03, Xin Wang @ Zhihu , 2024-02-29, Xipeng Guan @ BentoML , 2024-01-24, Juchao Song @ coScene , 2024-01-17, Nam Kyung-wan @ NAVER , 2023-12-14, Jichuan Sun @ SmartMore , 2023-11-09 , 2023-10-12, Jiapeng Sun @ Xiaomi , 2023-06-09, Chen Hong @ Zhejiang Lab , 2023-05-06, Sui Su , 2022-09-06, Dongdong Lv @ Unisound , 2024-02-07, Experienced JuiceFS user , 2023-11-01 , 2023-09-28, Ming Li @ DMALL , 2023-09-14, Weihong Ke @ NetEase Games , 2023-08-09, Chang Liu & Yangliang Li @ Yimian , 2023-05-10, Fengyu Cao @ Douban , 2021-10-09, Gaoding SRE Team , 2021-10-09, Teng @ Shopee , 2022-09-19, Miaocheng & Xiaofeng @ Trip.com , 2024-05-16, Jian Zhi , 2024-05-08, Jian Zhi , 2024-04-30, Jiefeng Huang , 2024-04-22, Jian Zhi , 2024-04-18, Herald Yu , 2024-03-27, Jet , 2024-03-07, Feihu Mo , 2024-02-22, Sandy , 2023-12-28, Herald Yu , 2023-12-20, Jian Zhi , 2023-12-07, Yifu Liu , 2023-11-23, Herald Yu , 2023-11-20 , 2023-10-26, Sandy , 2023-08-29, Herald Yu , 2023-07-19, Herald Yu , 2023-06-06, Changjian Gao , 2023-04-25, Changjian Gao , Youtube video, by Education Ecosystem , 2022-10-14, Sandy , 2022-07-22, Changjian Gao on Redis Monthly Live with Davies Liu and Mikhail Volkov, YouTube video , George Liu (eva2000) , Dollarkillerx , 2023-11-16, Herald Yu , 2023-10-20, Changjian Gao , 2023-09-31, Yifu Liu , 2023-09-21, Sandy If you want to add JuiceFS application cases to this list, you can do so through the following methods: Feel free to contribute by creating a branch in this repository on GitHub. Add the title and URL of your case page to the appropriate category, and then submit a pull request for review. Our team will review the submission and merge the branch if approved. You can join the official JuiceFS . There, you can get in touch with any staff member to discuss your contribution.
Project: OpenIO (Runtime / Cloud Native Storage)
File: README.md

(TBD) Adding a new filter entails the following: Writing a module in oio.event.filters, refer to the already defined filters for a template. Adding the filter to the file. Modify the event agent configuration template in to include an entry for the filter at the bottom (a `[filter:mynewfilter]` entry)and to place the filter in the pipelines of needed handlers. _(cascading filters is possible and working)_
Project: Inclavare Containers (Runtime / Container Runtime)
File: CONTRIBUTING.md

Welcome to Kubernetes. We are excited about the prospect of you joining our ! The Kubernetes community abides by the CNCF . Here is an excerpt: As contributors and maintainers of this project, and in the interest of fostering an open and welcoming community, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities. We have full documentation on how to get started contributing here: Kubernetes projects require that you sign a Contributor License Agreement (CLA) before we can accept your pull requests - Main contributor documentation, or you can just jump directly to the - Common resources for existing developers - We have a diverse set of mentorship programs available that are always looking for volunteers!
Project: OpenEBS (Runtime / Cloud Native Storage)
File: 2019152019-jiva-autosnap-deletion.md

oep-number: draft Auto Snapshot deletion 20190715 title: Jiva auto snapshot deletion authors: "@utkarshmani1997" owners: "@vishnuitta" "@payes" "@utkarshmani1997" editor: "@utkarshmani1997" creation-date: 2019-07-15 last-updated: 2019-09-03 status: provisional see-also: NA replaces: NA superseded-by: NA - - - - This proposal is aimed for the users who experience very frequent network glitch at there cluster, due to which replica get restarted and autogenerated snapshots files are created. A max of 512 snapshots can be created. Once 512 snapshots are created, the replica won't be able to connect again automatically and requires manual cleanup of snapshot. The motivation behind this approach is to remove/reduce the efforts of the user for managing the storage layer Ability to remove autogenerated snapshots automatically in background without any manual intervention from the user. Though there is a working flow of deletion of snapshot via REST Api/ jivactl cli but this requires manual intervention from user i.e, user needs to list the snap shots and delete it one by one via given approaches but if number of snapshots have been grown significantly it will take too much time to cleanup those snapshots. The proposed design is to have a background job in jiva controller that keeps autogenerated snapshots in check. Jiva controller will start a goroutine for the cleanup of snapshots. Cleanup goroutine will start the cleanup process as follows. Cleanup goroutine will verify the rebuild, check if the replication factor is met and other prerequisites, and then snapshot deletion will be triggered in the background. Verify that snapshot count is not reached at threshold, if the snapshot count is below the threshold, exit the cleanup job. Deletion of snapshot is time taking process since data needs to be merged to its siblings, so larger the data in snapshot, time taken for this will grow significantly and may impact the current IO's. There are some cases where we may have stale snapshot entries if replica is restarted while snapshot deletion was in progress. There are following approaches which can be used to implement this feature: a) Manual deletion: Delete snapshots via cli tool jivactl `jivactl snapshot ls` to list snapshots `jivactl snapshot rm <snap_name>` from above output cons: User needs to delete the snapshots one by one by specifying snapshot name Requires manual cleanup of snapshots if replicas are restarted while snapshot deletion is in progress by validating the chain and restart is required to rebuild the replica to be on the safer side. b) Cleanup in background by picking the snapshots with smallest size: Run a goroutine which will pick up snapshots based on its size and start cleaning up the snapshots which has size less then any given size (for exp < 2-5G). NOTE: This approach will help in reducing the performance imapact and can be implemented in future.
Project: Velero (Runtime / Cloud Native Storage)
File: ark_describe.md

title: "ark describe" layout: docs Describe ark resources Describe ark resources ``` -h, --help help for describe ``` ``` --alsologtostderr log to standard error as well as files --kubeconfig string Path to the kubeconfig file to use to talk to the Kubernetes apiserver. If unset, try the environment variable KUBECONFIG, as well as in-cluster configuration --kubecontext string The context to use to talk to the Kubernetes apiserver. If unset defaults to whatever your current-context is (kubectl config current-context) --logbacktraceat traceLocation when logging hits line file:N, emit a stack trace (default :0) --log_dir string If non-empty, write log files in this directory --logtostderr log to standard error instead of files -n, --namespace string The namespace in which Ark should operate (default "heptio-ark") --stderrthreshold severity logs at or above this threshold go to stderr (default 2) -v, --v Level log level for V logs --vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging ``` - Back up and restore Kubernetes cluster resources. - Describe backups - Describe restores - Describe schedules
Project: HwameiStor (Runtime / Cloud Native Storage)
File: crd_and_cr.md

sidebar_position: 4 sidebar_label: "CRD and CR" `CRD` is the abbreviation of `Custom Resource Definition`, and is a resource type natively provided by `Kubernetes`. It is the definition of Custom Resource (CR) to describe what a custom resource is. A CRD can register a new resource with the `Kubernetes` cluster to extend the capabilities of the `Kubernetes` cluster. With `CRD`, you can define the abstraction of the underlying infrastructure, customize resource types based on business needs, and use the existing resources and capabilities of `Kubernetes` to define higher-level abstractions through a Lego-like building blocks. `CR` is the abbreviation of `Custom Resource`. In practice, it is an instance of `CRD`, a resource description that matches with the field format in `CRD`. We all know that `Kubernetes` has powerful scalability, but only `CRD` is not useful. It also needs the support of controller (`Custom Controller`) to reflect the value of `CRD`. `Custom Controller` can listen `CRUD` events of `CR` to implement custom business logic. In `Kubernetes`, `CRDs + Controllers = Everything`. See also the official documentation provided by Kubernetes:
Project: Curve (Runtime / Cloud Native Storage)
File: CODE_OF_CONDUCT.md

In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation. Examples of behavior that contributes to creating a positive environment include: Using welcoming and inclusive language Being respectful of differing viewpoints and experiences Gracefully accepting constructive criticism Focusing on what is best for the community Showing empathy towards other community members Examples of unacceptable behavior by participants include: The use of sexualized language or imagery and unwelcome sexual attention or advances Trolling, insulting/derogatory comments, and personal or political attacks Public or private harassment Publishing others' private information, such as a physical or electronic address, without explicit permission Other conduct which could reasonably be considered inappropriate in a professional setting Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior. Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful. This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers. Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at storage_mgm@163.com. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately. Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership. This Code of Conduct is adapted from the , version 1.4, available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html For answers to common questions about this code of conduct, see https://www.contributor-covenant.org/faq
Project: Kube-router (Runtime / Cloud Native Network)
File: pod-toolbox.md

When kube-router is ran as a Pod within your Kubernetes cluster, it also ships with a number of tools automatically configured for your cluster. These can be used to troubleshoot issues and learn more about how cluster networking is performed. Here's a quick way to get going on a random node in your cluster: ```sh KR_POD=$(basename $(kubectl -n kube-system get pods -l k8s-app=kube-router --output name|head -n1)) kubectl -n kube-system exec -it ${KR_POD} bash ``` Use `kubectl -n kube-system get pods -l k8s-app=kube-router -o wide` to see what nodes are running which pods. This will help if you want to investigate a particular node. Once logged in you will see some help on using the tools in the container. For example: ```console Welcome to kube-router on "node1.zbrbdl"! For debugging, the following tools are available: ipvsadm | Gather info about Virtual Services and Real Servers via IPVS. | Examples: | ## Show all options | ipvsadm --help | ## List Services and Endpoints handled by IPVS | ipvsadm -ln | ## Show traffic rate information | ipvsadm -ln --rate | ## Show cumulative traffic | ipvsadm -ln --stats gobgp | Get BGP related information from your nodes. | | Tab-completion is ready to use, just type "gobgp <TAB>" | to see the subcommands available. | | By default gobgp will query the Node this Pod is running | on, i.e. "node1.zbrbdl". To query a different node use | "gobgp --host node02.mydomain" as an example. | | For more examples see: https://github.com/osrg/gobgp/blob/master/docs/sources/cli-command-syntax.md Here's a quick look at what's happening on this Node BGP Server Configuration AS: 64512 Router-ID: 10.10.3.2 Listening Port: 179, Addresses: 0.0.0.0, :: BGP Neighbors Peer AS Up/Down State |#Received Accepted 64512 2d 01:05:07 Establ | 1 1 BGP Route Info Network Next Hop AS_PATH Age Attrs *> 10.2.0.0/24 10.10.3.3 4000 400000 300000 40001 2d 01:05:20 [{Origin: i} {LocalPref: 100}] *> 10.2.1.0/24 10.10.3.2 4000 400000 300000 40001 00:00:36 [{Origin: i}] IPVS Services IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 10.3.0.1:443 rr persistent 10800 mask 0.0.0.0 -> 10.10.3.2:443 Masq 1 0 0 TCP 10.3.0.10:53 rr -> 10.2.0.2:53 Masq 1 0 0 TCP 10.3.0.15:2379 rr -> 10.10.3.3:2379 Masq 1 45 0 TCP 10.3.0.155:2379 rr -> 10.10.3.3:2379 Masq 1 0 0 UDP 10.3.0.10:53 rr -> 10.2.0.2:53 Masq 1 0 0 ```
Project: Carina (Runtime / Cloud Native Storage)
File: device-register.md

Device registration works for carina versions under v0.9.0. When carina manages local disks, it treats them as devices and registed them to kubelet. Whenever there is an new local device or PV, the disk usage will be updated. ```shell $ kubectl get node 10.20.9.154 -o template --template={{.status.capacity}} map[ carina.storage.io/carina-vg-hdd:160 carina.storage.io/carina-vg-ssd:0 cpu:2 ephemeral-storage:208655340Ki hugepages-1Gi:0 hugepages-2Mi:0 memory:3880376Ki pods:110 ] $ kubectl get node 10.20.9.154 -o template --template={{.status.allocatable}} map[ carina.storage.io/carina-vg-hdd:150 carina.storage.io/carina-vg-ssd:0 cpu:2 ephemeral-storage:192296761026 hugepages-1Gi:0 hugepages-2Mi:0 memory:3777976Ki pods:110 ] ``` For each device, carina will records its capacity and allocatable. 10G of disk space is reserved for each device. carina scheduler will do scheduling based on each node's disk usage. Carina also tracks those informaction in an configmap. ```shell $ kubectl get configmap carina-node-storage -n kube-system -o yaml data: node: '[{ "allocatable.carina.storage.io/carina-vg-hdd": "150", "allocatable.carina.storage.io/carina-vg-ssd": "0", "capacity.carina.storage.io/carina-vg-hdd": "160", "capacity.carina.storage.io/carina-vg-ssd": "0", "nodeName": "10.20.9.154" }, { "allocatable.carina.storage.io/carina-vg-hdd": "146", "allocatable.carina.storage.io/carina-vg-ssd": "0", "capacity.carina.storage.io/carina-vg-hdd": "170", "capacity.carina.storage.io/carina-vg-ssd": "0", "nodeName": "10.20.9.153" }]' ```