# Resource Quotas

Resource QuotasWhen several users or teams share a cluster with a fixed number of nodes, there is a concern that one team could use more than its fair share of resources.Resource quotas are a tool for administrators to address this concern.A resource quota, defined by a ResourceQuota object, provides constraints that limit aggregate resource consumption per namespace. A ResourceQuota can also limit the quantity of objects that can be created in a namespace by API kind, as well as the total amount of infrastructure resources that may be consumed by API objects found in that namespace.Caution:Neither contention nor changes to quota will affect already created resources.How Kubernetes ResourceQuotas workResourceQuotas work like this:Different teams work in different namespaces. This separation can be enforced with RBAC or any other authorization mechanism.A cluster administrator creates at least one ResourceQuota for each namespace.To make sure the enforcement stays enforced, the cluster administrator should also restrict access to delete or update that ResourceQuota; for example, by defining a ValidatingAdmissionPolicy.Users create resources (pods, services, etc.) in the namespace, and the quota system tracks usage to ensure it does not exceed hard resource limits defined in a ResourceQuota.You can apply a scope to a ResourceQuota to limit where it applies,If creating or updating a resource violates a quota constraint, the control plane rejects that request with HTTP status code 403 Forbidden. The error includes a message explaining the constraint that would have been violated.If quotas are enabled in a namespace for resource such as cpu and memory, users must specify requests or limits for those values when they define a Pod; otherwise, the quota system may reject pod creation.The resource quota walkthrough shows an example of how to avoid this problem.Note:You can define a LimitRange to force defaults on pods that make no compute resource requirements (so that users don't have to remember to do that).You often do not create Pods directly; for example, you more usually create a workload management object such as a Deployment. If you create a Deployment that tries to use more resources than are available, the creation of the Deployment (or other workload management object) succeeds, but the Deployment may not be able to get all of the Pods it manages to exist. In that case you can check the status of the Deployment, for example with kubectl describe, to see what has happened.For cpu and memory resources, ResourceQuotas enforce that every (new) pod in that namespace sets a limit for that resource. If you enforce a resource quota in a namespace for either cpu or memory, you and other clients, must specify either requests or limits for that resource, for every new Pod you submit. If you don't, the control plane may reject admission for that Pod.For other resources: ResourceQuota works and will ignore pods in the namespace without setting a limit or request for that resource. It means that you can create a new pod without limit/request for ephemeral storage if the resource quota limits the ephemeral storage of this namespace.You can use a LimitRange to automatically set a default request for these resources.The name of a ResourceQuota object must be a valid DNS subdomain name.Examples of policies that could be created using namespaces and quotas are:In a cluster with a capacity of 32 GiB RAM, and 16 cores, let team A use 20 GiB and 10 cores, let B use 10GiB and 4 cores, and hold 2GiB and 2 cores in reserve for future allocation.Limit the "testing" namespace to using 1 core and 1GiB RAM. Let the "production" namespace use any amount.In the case where the total capacity of the cluster is less than the sum of the quotas of the namespaces, there may be contention for resources. This is handled on a first-come-first-served basis.Enabling Resource QuotaResourceQuota support is enabled by default for many Kubernetes distributions. It is enabled when the API server --enable-admission-plugins= flag has ResourceQuota as one of its arguments.A resource quota is enforced in a particular namespace when there is a ResourceQuota in that namespace.Types of resource quotaThe ResourceQuota mechanism lets you enforce different kinds of limits. This section describes the types of limit that you can enforce.Quota for infrastructure resourcesYou can limit the total sum of compute resources that can be requested in a given namespace.The following resource types are supported:Resource NameDescriptionlimits.cpuAcross all pods in a non-terminal state, the sum of CPU limits cannot exceed this value.limits.memoryAcross all pods in a non-terminal state, the sum of memory limits cannot exceed this value.requests.cpuAcross all pods in a non-terminal state, the sum of CPU requests cannot exceed this value.requests.memoryAcross all pods in a non-terminal state, the sum of memory requests cannot exceed this value.hugepages-<size>Across all pods in a non-terminal state, the number of huge page requests of the specified size cannot exceed this value.cpuSame as requests.cpumemorySame as requests.memoryQuota for extended resourcesIn addition to the resources mentioned above, in release 1.10, quota support for extended resources is added.As overcommit is not allowed for extended resources, it makes no sense to specify both requests and limits for the same extended resource in a quota. So for extended resources, only quota items with prefix requests. are allowed.Take the GPU resource as an example, if the resource name is nvidia.com/gpu, and you want to limit the total number of GPUs requested in a namespace to 4, you can define a quota as follows:requests.nvidia.com/gpu: 4See Viewing and Setting Quotas for more details.Quota for storageYou can limit the total sum of storage for volumes that can be requested in a given namespace.In addition, you can limit consumption of storage resources based on associated StorageClass.Resource NameDescriptionrequests.storageAcross all persistent volume claims, the sum of storage requests cannot exceed this value.persistentvolumeclaimsThe total number of PersistentVolumeClaims that can exist in the namespace.<storage-class-name>.storageclass.storage.k8s.io/requests.storageAcross all persistent volume claims associated with the <storage-class-name>, the sum of storage requests cannot exceed this value.<storage-class-name>.storageclass.storage.k8s.io/persistentvolumeclaimsAcross all persistent volume claims associated with the <storage-class-name>, the total number of persistent volume claims that can exist in the namespace.For example, if you want to quota storage with gold StorageClass separate from a bronze StorageClass, you can define a quota as follows:gold.storageclass.storage.k8s.io/requests.storage: 500Gibronze.storageclass.storage.k8s.io/requests.storage: 100GiQuota for local ephemeral storageFEATURE STATE: Kubernetes v1.8 [alpha]Resource NameDescriptionrequests.ephemeral-storageAcross all pods in the namespace, the sum of local ephemeral storage requests cannot exceed this value.limits.ephemeral-storageAcross all pods in the namespace, the sum of local ephemeral storage limits cannot exceed this value.ephemeral-storageSame as requests.ephemeral-storage.Note:When using a CRI container runtime, container logs will count against the ephemeral storage quota. This can result in the unexpected eviction of pods that have exhausted their storage quotas.Refer to Logging Architecture for details.Quota on object countYou can set quota for the total number of one particular resource kind in the Kubernetes API, using the following syntax:count/<resource>.<group> for resources from non-core API groupscount/<resource> for resources from the core API groupFor example, the PodTemplate API is in the core API group and so if you want to limit the number of PodTemplate objects in a namespace, you use count/podtemplates.These types of quotas are useful to protect against exhaustion of control plane storage. For example, you may want to limit the number of Secrets in a server given their large size. Too many Secrets in a cluster can actually prevent servers and controllers from starting. You can set a quota for Jobs to protect against a poorly configured CronJob. CronJobs that create too many Jobs in a namespace can lead to a denial of service.If you define a quota this way, it applies to Kubernetes' APIs that are part of the API server, and to any custom resources backed by a CustomResourceDefinition. For example, to create a quota on a widgets custom resource in the example.com API group, use count/widgets.example.com. If you use API aggregation to add additional, custom APIs that are not defined as CustomResourceDefinitions, the core Kubernetes control plane does not enforce quota for the aggregated API. The extension API server is expected to provide quota enforcement if that's appropriate for the custom API.Generic syntaxThis is a list of common examples of object kinds that you may want to put under object count quota, listed by the configuration string that you would use.count/podscount/persistentvolumeclaimscount/servicescount/secretscount/configmapscount/deployments.appscount/replicasets.appscount/statefulsets.appscount/jobs.batchcount/cronjobs.batchSpecialized syntaxThere is another syntax only to set the same type of quota, that only works for certain API kinds. The following types are supported:Resource NameDescriptionconfigmapsThe total number of ConfigMaps that can exist in the namespace.persistentvolumeclaimsThe total number of PersistentVolumeClaims that can exist in the namespace.podsThe total number of Pods in a non-terminal state that can exist in the namespace. A pod is in a terminal state if .status.phase in (Failed, Succeeded) is true.replicationcontrollersThe total number of ReplicationControllers that can exist in the namespace.resourcequotasThe total number of ResourceQuotas that can exist in the namespace.servicesThe total number of Services that can exist in the namespace.services.loadbalancersThe total number of Services of type LoadBalancer that can exist in the namespace.services.nodeportsThe total number of NodePorts allocated to Services of type NodePort or LoadBalancer that can exist in the namespace.secretsThe total number of Secrets that can exist in the namespace.For example, pods quota counts and enforces a maximum on the number of pods created in a single namespace that are not terminal. You might want to set a pods quota on a namespace to avoid the case where a user creates many small pods and exhausts the cluster's supply of Pod IPs.You can find more examples on Viewing and Setting Quotas.Quota ScopesEach quota can have an associated set of scopes. A quota will only measure usage for a resource if it matches the intersection of enumerated scopes.When a scope is added to the quota, it limits the number of resources it supports to those that pertain to the scope. Resources specified on the quota outside of the allowed set results in a validation error.ScopeDescriptionTerminatingMatch pods where .spec.activeDeadlineSeconds >= 0NotTerminatingMatch pods where .spec.activeDeadlineSeconds is nilBestEffortMatch pods that have best effort quality of service.NotBestEffortMatch pods that do not have best effort quality of service.PriorityClassMatch pods that references the specified priority class.CrossNamespacePodAffinityMatch pods that have cross-namespace pod (anti)affinity terms.VolumeAttributesClassMatch persistentvolumeclaims that references the specified volume attributes class.The BestEffort scope restricts a quota to tracking the following resource:podsThe Terminating, NotTerminating, NotBestEffort and PriorityClass scopes restrict a quota to tracking the following resources:podscpumemoryrequests.cpurequests.memorylimits.cpulimits.memoryNote that you cannot specify both the Terminating and the NotTerminating scopes in the same quota, and you cannot specify both the BestEffort and NotBestEffort scopes in the same quota either.The scopeSelector supports the following values in the operator field:InNotInExistsDoesNotExistWhen using one of the following values as the scopeName when defining the scopeSelector, the operator must be Exists.TerminatingNotTerminatingBestEffortNotBestEffortIf the operator is In or NotIn, the values field must have at least one value. For example: scopeSelector: matchExpressions: - scopeName: PriorityClass operator: In values: - middle If the operator is Exists or DoesNotExist, the values field must NOT be specified.Resource Quota Per PriorityClassFEATURE STATE: Kubernetes v1.17 [stable]Pods can be created at a specific priority. You can control a pod's consumption of system resources based on a pod's priority, by using the scopeSelector field in the quota spec.A quota is matched and consumed only if scopeSelector in the quota spec selects the pod.When quota is scoped for priority class using scopeSelector field, quota object is restricted to track only following resources:podscpumemoryephemeral-storagelimits.cpulimits.memorylimits.ephemeral-storagerequests.cpurequests.memoryrequests.ephemeral-storageThis example creates a quota object and matches it with pods at specific priorities. The example works as follows:Pods in the cluster have one of the three priority classes, "low", "medium", "high".One quota object is created for each priority.Save the following YAML to a file quota.yaml.policy/quota.yaml apiVersion: v1 kind: List items: - apiVersion: v1 kind: ResourceQuota metadata: name: pods-high spec: hard: cpu: "1000" memory: "200Gi" pods: "10" scopeSelector: matchExpressions: - operator: In scopeName: PriorityClass values: ["high"] - apiVersion: v1 kind: ResourceQuota metadata: name: pods-medium spec: hard: cpu: "10" memory: "20Gi" pods: "10" scopeSelector: matchExpressions: - operator: In scopeName: PriorityClass values: ["medium"] - apiVersion: v1 kind: ResourceQuota metadata: name: pods-low spec: hard: cpu: "5" memory: "10Gi" pods: "10" scopeSelector: matchExpressions: - operator: In scopeName: PriorityClass values: ["low"] Apply the YAML using kubectl create.kubectl create -f ./quota.yaml resourcequota/pods-high created resourcequota/pods-medium created resourcequota/pods-low created Verify that Used quota is 0 using kubectl describe quota.kubectl describe quota Name: pods-high Namespace: default Resource Used Hard -------- ---- ---- cpu 0 1k memory 0 200Gi pods 0 10 Name: pods-low Namespace: default Resource Used Hard -------- ---- ---- cpu 0 5 memory 0 10Gi pods 0 10 Name: pods-medium Namespace: default Resource Used Hard -------- ---- ---- cpu 0 10 memory 0 20Gi pods 0 10 Create a pod with priority "high". Save the following YAML to a file high-priority-pod.yaml.policy/high-priority-pod.yaml apiVersion: v1 kind: Pod metadata: name: high-priority spec: containers: - name: high-priority image: ubuntu command: ["/bin/sh"] args: ["-c", "while true; do echo hello; sleep 10;done"] resources: requests: memory: "10Gi" cpu: "500m" limits: memory: "10Gi" cpu: "500m" priorityClassName: high Apply it with kubectl create.kubectl create -f ./high-priority-pod.yaml Verify that "Used" stats for "high" priority quota, pods-high, has changed and that the other two quotas are unchanged.kubectl describe quota Name: pods-high Namespace: default Resource Used Hard -------- ---- ---- cpu 500m 1k memory 10Gi 200Gi pods 1 10 Name: pods-low Namespace: default Resource Used Hard -------- ---- ---- cpu 0 5 memory 0 10Gi pods 0 10 Name: pods-medium Namespace: default Resource Used Hard -------- ---- ---- cpu 0 10 memory 0 20Gi pods 0 10 Cross-namespace Pod Affinity QuotaFEATURE STATE: Kubernetes v1.24 [stable]Operators can use CrossNamespacePodAffinity quota scope to limit which namespaces are allowed to have pods with affinity terms that cross namespaces. Specifically, it controls which pods are allowed to set namespaces or namespaceSelector fields in pod affinity terms.Preventing users from using cross-namespace affinity terms might be desired since a pod with anti-affinity constraints can block pods from all other namespaces from getting scheduled in a failure domain.Using this scope operators can prevent certain namespaces (foo-ns in the example below) from having pods that use cross-namespace pod affinity by creating a resource quota object in that namespace with CrossNamespacePodAffinity scope and hard limit of 0:apiVersion: v1 kind: ResourceQuota metadata: name: disable-cross-namespace-affinity namespace: foo-ns spec: hard: pods: "0" scopeSelector: matchExpressions: - scopeName: CrossNamespacePodAffinity operator: Exists If operators want to disallow using namespaces and namespaceSelector by default, and only allow it for specific namespaces, they could configure CrossNamespacePodAffinity as a limited resource by setting the kube-apiserver flag --admission-control-config-file to the path of the following configuration file:apiVersion: apiserver.config.k8s.io/v1 kind: AdmissionConfiguration plugins: - name: "ResourceQuota" configuration: apiVersion: apiserver.config.k8s.io/v1 kind: ResourceQuotaConfiguration limitedResources: - resource: pods matchScopes: - scopeName: CrossNamespacePodAffinity operator: Exists With the above configuration, pods can use namespaces and namespaceSelector in pod affinity only if the namespace where they are created have a resource quota object with CrossNamespacePodAffinity scope and a hard limit greater than or equal to the number of pods using those fields.Resource Quota Per VolumeAttributesClassFEATURE STATE: Kubernetes v1.34 [stable] (enabled by default: true)PersistentVolumeClaims can be created with a specific volume attributes class, and might be modified after creation. You can control a PVC's consumption of storage resources based on the associated volume attributes classes, by using the scopeSelector field in the quota spec.The PVC references the associated volume attributes class by the following fields:spec.volumeAttributesClassNamestatus.currentVolumeAttributesClassNamestatus.modifyVolumeStatus.targetVolumeAttributesClassNameA quota is matched and consumed only if scopeSelector in the quota spec selects the PVC.When the quota is scoped for the volume attributes class using the scopeSelector field, the quota object is restricted to track only the following resources:persistentvolumeclaimsrequests.storageThis example creates a quota object and matches it with PVC at specific volume attributes classes. The example works as follows:PVCs in the cluster have at least one of the three volume attributes classes, "gold", "silver", "copper".One quota object is created for each volume attributes class.Save the following YAML to a file quota-vac.yaml.policy/quota-vac.yaml apiVersion: v1 kind: List items: - apiVersion: v1 kind: ResourceQuota metadata: name: pvcs-gold spec: hard: requests.storage: "10Gi" persistentvolumeclaims: "10" scopeSelector: matchExpressions: - operator: In scopeName: VolumeAttributesClass values: ["gold"] - apiVersion: v1 kind: ResourceQuota metadata: name: pvcs-silver spec: hard: requests.storage: "20Gi" persistentvolumeclaims: "10" scopeSelector: matchExpressions: - operator: In scopeName: VolumeAttributesClass values: ["silver"] - apiVersion: v1 kind: ResourceQuota metadata: name: pvcs-copper spec: hard: requests.storage: "30Gi" persistentvolumeclaims: "10" scopeSelector: matchExpressions: - operator: In scopeName: VolumeAttributesClass values: ["copper"] Apply the YAML using kubectl create.kubectl create -f ./quota-vac.yaml resourcequota/pvcs-gold created resourcequota/pvcs-silver created resourcequota/pvcs-copper created Verify that Used quota is 0 using kubectl describe quota.kubectl describe quota Name: pvcs-gold Namespace: default Resource Used Hard -------- ---- ---- persistentvolumeclaims 0 10 requests.storage 0 10Gi Name: pvcs-silver Namespace: default Resource Used Hard -------- ---- ---- persistentvolumeclaims 0 10 requests.storage 0 20Gi Name: pvcs-copper Namespace: default Resource Used Hard -------- ---- ---- persistentvolumeclaims 0 10 requests.storage 0 30Gi Create a pvc with volume attributes class "gold". Save the following YAML to a file gold-vac-pvc.yaml.policy/gold-vac-pvc.yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: gold-vac-pvc spec: accessModes: - ReadWriteOnce resources: requests: storage: 2Gi storageClassName: # change this to the name of the storage class you want to use volumeAttributesClassName: gold Apply it with kubectl create.kubectl create -f ./gold-vac-pvc.yaml Verify that "Used" stats for "gold" volume attributes class quota, pvcs-gold has changed and that the other two quotas are unchanged.kubectl describe quota Name: pvcs-gold Namespace: default Resource Used Hard -------- ---- ---- persistentvolumeclaims 1 10 requests.storage 2Gi 10Gi Name: pvcs-silver Namespace: default Resource Used Hard -------- ---- ---- persistentvolumeclaims 0 10 requests.storage 0 20Gi Name: pvcs-copper Namespace: default Resource Used Hard -------- ---- ---- persistentvolumeclaims 0 10 requests.storage 0 30Gi Once the PVC is bound, it is allowed to modify the desired volume attributes class. Let's change it to "silver" with kubectl patch.kubectl patch pvc gold-vac-pvc --type='merge' -p '{"spec":{"volumeAttributesClassName":"silver"}}' Verify that "Used" stats for "silver" volume attributes class quota, pvcs-silver has changed, pvcs-copper is unchanged, and pvcs-gold might be unchanged or released, which depends on the PVC's status.kubectl describe quota Name: pvcs-gold Namespace: default Resource Used Hard -------- ---- ---- persistentvolumeclaims 1 10 requests.storage 2Gi 10Gi Name: pvcs-silver Namespace: default Resource Used Hard -------- ---- ---- persistentvolumeclaims 1 10 requests.storage 2Gi 20Gi Name: pvcs-copper Namespace: default Resource Used Hard -------- ---- ---- persistentvolumeclaims 0 10 requests.storage 0 30Gi Let's change it to "copper" with kubectl patch.kubectl patch pvc gold-vac-pvc --type='merge' -p '{"spec":{"volumeAttributesClassName":"copper"}}' Verify that "Used" stats for "copper" volume attributes class quota, pvcs-copper has changed, pvcs-silver and pvcs-gold might be unchanged or released, which depends on the PVC's status.kubectl describe quota Name: pvcs-gold Namespace: default Resource Used Hard -------- ---- ---- persistentvolumeclaims 1 10 requests.storage 2Gi 10Gi Name: pvcs-silver Namespace: default Resource Used Hard -------- ---- ---- persistentvolumeclaims 1 10 requests.storage 2Gi 20Gi Name: pvcs-copper Namespace: default Resource Used Hard -------- ---- ---- persistentvolumeclaims 1 10 requests.storage 2Gi 30Gi Print the manifest of the PVC using the following command:kubectl get pvc gold-vac-pvc -o yaml It might show the following output:apiVersion: v1 kind: PersistentVolumeClaim metadata: name: gold-vac-pvc spec: accessModes: - ReadWriteOnce resources: requests: storage: 2Gi storageClassName: default volumeAttributesClassName: copper status: accessModes: - ReadWriteOnce capacity: storage: 2Gi currentVolumeAttributesClassName: gold phase: Bound modifyVolumeStatus: status: InProgress targetVolumeAttributesClassName: silver storageClassName: default Wait a moment for the volume modification to complete, then verify the quota again.kubectl describe quota Name: pvcs-gold Namespace: default Resource Used Hard -------- ---- ---- persistentvolumeclaims 0 10 requests.storage 0 10Gi Name: pvcs-silver Namespace: default Resource Used Hard -------- ---- ---- persistentvolumeclaims 0 10 requests.storage 0 20Gi Name: pvcs-copper Namespace: default Resource Used Hard -------- ---- ---- persistentvolumeclaims 1 10 requests.storage 2Gi 30Gi Requests compared to LimitsWhen allocating compute resources, each container may specify a request and a limit value for either CPU or memory. The quota can be configured to quota either value.If the quota has a value specified for requests.cpu or requests.memory, then it requires that every incoming container makes an explicit request for those resources. If the quota has a value specified for limits.cpu or limits.memory, then it requires that every incoming container specifies an explicit limit for those resources.Viewing and Setting Quotaskubectl supports creating, updating, and viewing quotas:kubectl create namespace myspace cat <<EOF > compute-resources.yaml apiVersion: v1 kind: ResourceQuota metadata: name: compute-resources spec: hard: requests.cpu: "1" requests.memory: "1Gi" limits.cpu: "2" limits.memory: "2Gi" requests.nvidia.com/gpu: 4 EOF kubectl create -f ./compute-resources.yaml --namespace=myspace cat <<EOF > object-counts.yaml apiVersion: v1 kind: ResourceQuota metadata: name: object-counts spec: hard: configmaps: "10" persistentvolumeclaims: "4" pods: "4" replicationcontrollers: "20" secrets: "10" services: "10" services.loadbalancers: "2" EOF kubectl create -f ./object-counts.yaml --namespace=myspace kubectl get quota --namespace=myspace NAME AGE compute-resources 30s object-counts 32s kubectl describe quota compute-resources --namespace=myspace Name: compute-resources Namespace: myspace Resource Used Hard -------- ---- ---- limits.cpu 0 2 limits.memory 0 2Gi requests.cpu 0 1 requests.memory 0 1Gi requests.nvidia.com/gpu 0 4 kubectl describe quota object-counts --namespace=myspace Name: object-counts Namespace: myspace Resource Used Hard -------- ---- ---- configmaps 0 10 persistentvolumeclaims 0 4 pods 0 4 replicationcontrollers 0 20 secrets 1 10 services 0 10 services.loadbalancers 0 2 kubectl also supports object count quota for all standard namespaced resources using the syntax count/<resource>.<group>:kubectl create namespace myspace kubectl create quota test --hard=count/deployments.apps=2,count/replicasets.apps=4,count/pods=3,count/secrets=4 --namespace=myspace kubectl create deployment nginx --image=nginx --namespace=myspace --replicas=2 kubectl describe quota --namespace=myspace Name: test Namespace: myspace Resource Used Hard -------- ---- ---- count/deployments.apps 1 2 count/pods 2 3 count/replicasets.apps 1 4 count/secrets 1 4 Quota and Cluster CapacityResourceQuotas are independent of the cluster capacity. They are expressed in absolute units. So, if you add nodes to your cluster, this does not automatically give each namespace the ability to consume more resources.Sometimes more complex policies may be desired, such as:Proportionally divide total cluster resources among several teams.Allow each tenant to grow resource usage as needed, but have a generous limit to prevent accidental resource exhaustion.Detect demand from one namespace, add nodes, and increase quota.Such policies could be implemented using ResourceQuotas as building blocks, by writing a "controller" that watches the quota usage and adjusts the quota hard limits of each namespace according to other signals.Note that resource quota divides up aggregate cluster resources, but it creates no restrictions around nodes: pods from several namespaces may run on the same node.Limit Priority Class consumption by defaultIt may be desired that pods at a particular priority, such as "cluster-services", should be allowed in a namespace, if and only if, a matching quota object exists.With this mechanism, operators are able to restrict usage of certain high priority classes to a limited number of namespaces and not every namespace will be able to consume these priority classes by default.To enforce this, kube-apiserver flag --admission-control-config-file should be used to pass path to the following configuration file:apiVersion: apiserver.config.k8s.io/v1 kind: AdmissionConfiguration plugins: - name: "ResourceQuota" configuration: apiVersion: apiserver.config.k8s.io/v1 kind: ResourceQuotaConfiguration limitedResources: - resource: pods matchScopes: - scopeName: PriorityClass operator: In values: ["cluster-services"] Then, create a resource quota object in the kube-system namespace:policy/priority-class-resourcequota.yaml apiVersion: v1 kind: ResourceQuota metadata: name: pods-cluster-services spec: scopeSelector: matchExpressions: - operator : In scopeName: PriorityClass values: ["cluster-services"]kubectl apply -f https://k8s.io/examples/policy/priority-class-resourcequota.yaml -n kube-system resourcequota/pods-cluster-services created In this case, a pod creation will be allowed if:the Pod's priorityClassName is not specified.the Pod's priorityClassName is specified to a value other than cluster-services.the Pod's priorityClassName is set to cluster-services, it is to be created in the kube-system namespace, and it has passed the resource quota check.A Pod creation request is rejected if its priorityClassName is set to cluster-services and it is to be created in a namespace other than kube-system.What's nextSee a detailed example for how to use resource quota.Read the ResourceQuota API referenceLearn about LimitRangesYou can read the historical ResourceQuota design document for more information.You can also read the Quota support for priority class design document.