Repository: openbao/openbao
Pull request #1161: Add namespace-aware rate-limit quota
Author: login: voigt
Labels: name: core/namespaces
description: Related to namespace support
Created: 2025-03-25T23:11:21Z | Merged: 2025-04-30T20:34:04Z

This PR addresses #1084 and replicates namespace-aware quota behavior.

#### 1. Enables quota configuration per namespace
  ```bash
$ bao namespace create ns1
Key                Value
---                -----
custom_metadata    map[]
id                 g7B5Hn
path               ns1/
uuid               4f4afac3-adbf-baad-6576-6444a9e19a64
$ bao write sys/quotas/rate-limit/foo-limit path="ns1" rate=100 interval=2s
Success! Data written to: sys/quotas/rate-limit/foo-limit
$ bao read sys/quotas/rate-limit/foo-limit
Key               Value
---               -----
block_interval    0
inheritable       false
interval          2
name              foo-limit
path              ns1/
rate              100
role              n/a
type              rate-limit
  ```

The quota api endpoints under `/sys/quotas` remain restricted and can't be called in namespaces. Consequently, writing a quota with namespace header will fail.

```bash
$ bao write -ns ns1 sys/quotas/rate-limit/foo-limit rate=100 interval=2s
Error writing data to sys/quotas/rate-limit/foo-limit: Error making API request.

Namespace: ns1/
URL: PUT http://127.0.0.1:8200/v1/sys/quotas/rate-limit/foo-limit
Code: 400. Errors:

* unavailable operation (PUT /v1/sys/quotas/rate-limit/foo-limit)
```

#### 2. Writes quota configuration to `/sys/quotas`

Global, as well as namespace specific quotas will be stored in the same, global, location.

```bash
$ bao write sys/quotas/rate-limit/global-limit rate=3000 interval=1m
Success! Data written to: sys/quotas/rate-limit/global-limit
$ bao list sys/quotas/rate-limit/
Keys
----
foo-limit
global-limit
```

#### 3. Namespace-aware quota evaluation

#### 4. Adds `inheritable` flag to quotas
  ```bash
# Update `foo-limit` to inherit quota configuration to child namespaces
$ bao write sys/quotas/rate-limit/foo-limit path="ns1" rate=2 interval=2s inheritable=true
Success! Data written to: sys/quotas/rate-limit/foo-limit
$ bao namespace create -ns ns1 ns1.1
Key                Value
---                -----
custom_metadata    map[]
id                 3i7Ufc
path               ns1/ns1.1/
uuid               382c8d0d-f713-413c-c724-36e1f0712045
# Quota of `ns1` now also applies to `ns1/ns1.1`

$ while true; sleep 0.1; bao namespace list -ns=ns1/ns1.1; end
No namespaces found
No namespaces found
No namespaces found
No namespaces found
Error listing namespaces: Error making API request.

Namespace: ns1/ns1.1/
URL: GET http://127.0.0.1:8200/v1/sys/namespaces?list=true
Code: 429. Errors:

* request path "sys/namespaces/": rate limit quota exceeded
  ```


Diff:
diff --git a/helper/namespace/namespace.go b/helper/namespace/namespace.go
index 6f0f01fc5c9..e48103c1514 100644
--- a/helper/namespace/namespace.go
+++ b/helper/namespace/namespace.go
@@ -118,10 +118,16 @@ func (n *Namespace) HasParent(possibleParent *Namespace) bool {
 // ParentPath returns the path of the parent namespace. n.Path must be a
 // canonicalized path.
 func (n *Namespace) ParentPath() (string, bool) {
-	if n.Path == "" {
+	return ParentOf(n.Path)
+}
+
+// ParentOf returns the path of the parent namespace. path must be a
+// canonicalized path.
+func ParentOf(path string) (string, bool) {
+	if path == "" {
 		return "", false
 	}
-	segments := strings.SplitAfter(n.Path, "/")
+	segments := strings.SplitAfter(path, "/")
 	if len(segments) <= 2 {
 		return "", true
 	}
diff --git a/http/util.go b/http/util.go
index a4699455615..eb98cb05af3 100644
--- a/http/util.go
+++ b/http/util.go
@@ -98,6 +98,11 @@ var (
 			}
 		}
 
+		namespacedQuotaRequest := (strings.Contains(fullURL, "sys/quotas") && ns.Path != "")
+		if namespacedQuotaRequest {
+			return http.StatusBadRequest
+		}
+
 		for _, api := range containsAPIs {
 			if strings.Contains(fullURL, api) && !strings.HasPrefix(fullURL, api) {
 				return http.StatusBadRequest
@@ -143,12 +148,6 @@ func wrapMaxRequestSizeHandler(handler http.Handler, props *vault.HandlerPropert
 
 func rateLimitQuotaWrapping(handler http.Handler, core *vault.Core) http.Handler {
 	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
-		ns, err := namespace.FromContext(r.Context())
-		if err != nil {
-			respondError(w, http.StatusInternalServerError, err)
-			return
-		}
-
 		// We don't want to do buildLogicalRequestNoAuth here because, if the
 		// request gets allowed by the quota, the same function will get called
 		// again, which is not desired.
@@ -157,6 +156,7 @@ func rateLimitQuotaWrapping(handler http.Handler, core *vault.Core) http.Handler
 			respondError(w, status, err)
 			return
 		}
+		ns, _ := core.NamespaceByPath(r.Context(), path)
 		mountPath := strings.TrimPrefix(core.MatchingMount(r.Context(), path), ns.Path)
 
 		quotaReq := &quotas.Request{
diff --git a/vault/external_tests/quotas/quotas_test.go b/vault/external_tests/quotas/quotas_test.go
index d1f824c754b..b72a0c2acb2 100644
--- a/vault/external_tests/quotas/quotas_test.go
+++ b/vault/external_tests/quotas/quotas_test.go
@@ -108,6 +108,19 @@ func testRPS(reqFunc func(numSuccess, numFail *atomic.Int32), d time.Duration) (
 	return numSuccess.Load(), numFail.Load(), time.Since(start)
 }
 
+func testRPSWithNS(reqFunc func(numSuccess, numFail *atomic.Int32, ns string), d time.Duration, ns string) (int32, int32, time.Duration) {
+	numSuccess := atomic.NewInt32(0)
+	numFail := atomic.NewInt32(0)
+
+	start := time.Now()
+	end := start.Add(d)
+	for time.Now().Before(end) {
+		reqFunc(numSuccess, numFail, ns)
+	}
+
+	return numSuccess.Load(), numFail.Load(), time.Since(start)
+}
+
 func waitForRemovalOrTimeout(c *api.Client, path string, tick, to time.Duration) error {
 	ticker := time.Tick(tick)
 	timeout := time.After(to)
@@ -300,32 +313,7 @@ func TestQuotas_RateLimitQuota_Mount(t *testing.T) {
 	client := cluster.Cores[0].Client
 	vault.TestWaitActive(t, core)
 
-	err := client.Sys().Mount("pki", &api.MountInput{
-		Type: "pki",
-	})
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	_, err = client.Logical().Write("pki/root/generate/internal", map[string]interface{}{
-		"common_name": "testvault.com",
-		"ttl":         "200h",
-		"ip_sans":     "127.0.0.1",
-	})
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	_, err = client.Logical().Write("pki/roles/test", map[string]interface{}{
-		"require_cn":       false,
-		"allowed_domains":  "testvault.com",
-		"allow_subdomains": true,
-		"max_ttl":          "2h",
-		"generate_lease":   true,
-	})
-	if err != nil {
-		t.Fatal(err)
-	}
+	setupMounts(t, client)
 
 	reqFunc := func(numSuccess, numFail *atomic.Int32) {
 		_, err := client.Logical().Read("pki/cert/ca_chain")
@@ -340,7 +328,7 @@ func TestQuotas_RateLimitQuota_Mount(t *testing.T) {
 	// Create a rate limit quota with a low RPS of 7.7, which means we can process
 	// ⌈7.7⌉*2 requests in the span of roughly a second -- 8 initially, followed
 	// by a refill rate of 7.7 per-second.
-	_, err = client.Logical().Write("sys/quotas/rate-limit/rlq", map[string]interface{}{
+	_, err := client.Logical().Write("sys/quotas/rate-limit/rlq", map[string]interface{}{
 		"rate": 7.7,
 		"path": "pki/",
 	})
@@ -376,6 +364,8 @@ func TestQuotas_RateLimitQuota_Mount(t *testing.T) {
 	if numFail > 0 {
 		t.Fatalf("unexpected number of failed requests: %d", numFail)
 	}
+
+	teardownMounts(t, client)
 }
 
 func TestQuotas_RateLimitQuota_MountPrecedence(t *testing.T) {
@@ -391,35 +381,10 @@ func TestQuotas_RateLimitQuota_MountPrecedence(t *testing.T) {
 	vault.TestWaitActive(t, core)
 
 	// create PKI mount
-	err := client.Sys().Mount("pki", &api.MountInput{
-		Type: "pki",
-	})
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	_, err = client.Logical().Write("pki/root/generate/internal", map[string]interface{}{
-		"common_name": "testvault.com",
-		"ttl":         "200h",
-		"ip_sans":     "127.0.0.1",
-	})
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	_, err = client.Logical().Write("pki/roles/test", map[string]interface{}{
-		"require_cn":       false,
-		"allowed_domains":  "testvault.com",
-		"allow_subdomains": true,
-		"max_ttl":          "2h",
-		"generate_lease":   true,
-	})
-	if err != nil {
-		t.Fatal(err)
-	}
+	setupMounts(t, client)
 
 	// create a root rate limit quota
-	_, err = client.Logical().Write("sys/quotas/rate-limit/root-rlq", map[string]interface{}{
+	_, err := client.Logical().Write("sys/quotas/rate-limit/root-rlq", map[string]interface{}{
 		"name": "root-rlq",
 		"rate": 14.7,
 	})
@@ -463,6 +428,8 @@ func TestQuotas_RateLimitQuota_MountPrecedence(t *testing.T) {
 	if want := int32(ideal + 1); numSuccess > want {
 		t.Fatalf("too many successful requests; want: %d, numSuccess: %d, numFail: %d, elapsed: %d", want, numSuccess, numFail, elapsed)
 	}
+
+	teardownMounts(t, client)
 }
 
 func TestQuotas_RateLimitQuota(t *testing.T) {
@@ -477,41 +444,213 @@ func TestQuotas_RateLimitQuota(t *testing.T) {
 
 	vault.TestWaitActive(t, core)
 
-	err := client.Sys().EnableAuthWithOptions("userpass", &api.EnableAuthOptions{
-		Type: "userpass",
+	// Create a rate limit quota with a low RPS of 7.7, which means we can process
+	// ⌈7.7⌉*2 requests in the span of roughly a second -- 8 initially, followed
+	// by a refill rate of 7.7 per-second.
+	_, err := client.Logical().Write("sys/quotas/rate-limit/rlq", map[string]interface{}{
+		"rate": 7.7,
 	})
 	if err != nil {
 		t.Fatal(err)
 	}
 
-	_, err = client.Logical().Write("auth/userpass/users/foo", map[string]interface{}{
-		"password": "bar",
+	reqFunc := func(numSuccess, numFail *atomic.Int32) {
+		_, err := client.Logical().Read("sys/quotas/rate-limit/rlq")
+
+		if err != nil {
+			numFail.Add(1)
+		} else {
+			numSuccess.Add(1)
+		}
+	}
+
+	numSuccess, numFail, elapsed := testRPS(reqFunc, 5*time.Second)
+
+	// evaluate the ideal RPS as (ceil(RPS) + (RPS * totalSeconds))
+	ideal := 8 + (7.7 * float64(elapsed) / float64(time.Second))
+
+	// ensure there were some failed requests
+	if numFail == 0 {
+		t.Fatalf("expected some requests to fail; numSuccess: %d, numFail: %d, elapsed: %d", numSuccess, numFail, elapsed)
+	}
+
+	// ensure that we should never get more requests than allowed
+	if want := int32(ideal + 1); numSuccess > want {
+		t.Fatalf("too many successful requests; want: %d, numSuccess: %d, numFail: %d, elapsed: %d", want, numSuccess, numFail, elapsed)
+	}
+
+	// allow time (1s) for rate limit to refill before updating the quota
+	time.Sleep(time.Second)
+
+	// update the rate limit quota with a high RPS such that no requests should fail
+	_, err = client.Logical().Write("sys/quotas/rate-limit/rlq", map[string]interface{}{
+		"rate": 10000.0,
 	})
 	if err != nil {
 		t.Fatal(err)
 	}
 
-	// Create a rate limit quota with a low RPS of 7.7, which means we can process
+	_, numFail, _ = testRPS(reqFunc, 5*time.Second)
+	if numFail > 0 {
+		t.Fatalf("unexpected number of failed requests: %d", numFail)
+	}
+}
+
+func TestQuotas_RateLimitQuotaNS(t *testing.T) {
+	conf, opts := teststorage.ClusterSetup(coreConfig, nil, nil)
+	opts.NoDefaultQuotas = true
+	cluster := vault.NewTestCluster(t, conf, opts)
+	cluster.Start()
+	defer cluster.Cleanup()
+
+	core := cluster.Cores[0].Core
+	client := cluster.Cores[0].Client
+
+	vault.TestWaitActive(t, core)
+
+	// Create a global rate limit with a low RPS of 7.7, which means we can process
 	// ⌈7.7⌉*2 requests in the span of roughly a second -- 8 initially, followed
 	// by a refill rate of 7.7 per-second.
-	_, err = client.Logical().Write("sys/quotas/rate-limit/rlq", map[string]interface{}{
+	// As inheritable is set to true, the quota will be inherited by child namespaces
+	_, err := client.Logical().Write("sys/quotas/rate-limit/global-rlq", map[string]interface{}{
 		"rate": 7.7,
 	})
 	if err != nil {
 		t.Fatal(err)
 	}
 
-	reqFunc := func(numSuccess, numFail *atomic.Int32) {
-		_, err := client.Logical().Read("sys/quotas/rate-limit/rlq")
+	// Create Parent Namespace ns1
+	// ns1 intentionaly does not have a quota, so it should be able to do more requests than root
+	_, err = client.Logical().Write("sys/namespaces/ns1", map[string]interface{}{})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// Create Childnamespace, so we get a hierarchy of ns1/ns1.1
+	client.SetNamespace("ns1")
+	_, err = client.Logical().Write("sys/namespaces/ns1.1", map[string]interface{}{})
+	if err != nil {
+		t.Fatal(err)
+	}
+	client.ClearNamespace()
+
+	// Create a rate limit for namespace ns1/ns1.1 with a higher RPS than the global quota
+	_, err = client.Logical().Write("sys/quotas/rate-limit/ns1.1-rlq", map[string]interface{}{
+		"rate": 9.9,
+		"path": "ns1/ns1.1",
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	reqFunc := func(numSuccess, numFail *atomic.Int32, ns string) {
+		// list all namespaces in ns1/ns1.1
+		// the quota of ns1 should apply
+		client.SetNamespace(ns)
+		_, err := client.Logical().List("sys/namespaces")
 
 		if err != nil {
 			numFail.Add(1)
 		} else {
 			numSuccess.Add(1)
 		}
+		client.ClearNamespace()
 	}
 
-	numSuccess, numFail, elapsed := testRPS(reqFunc, 5*time.Second)
+	// Test global rate limit quota
+	numSuccess, numFail, elapsed := testRPSWithNS(reqFunc, 5*time.Second, "")
+	// evaluate the ideal RPS as (ceil(RPS) + (RPS * totalSeconds))
+	ideal := 8 + (7.7 * float64(elapsed) / float64(time.Second))
+	// ensure there were some failed requests
+	if numFail == 0 {
+		t.Fatalf("expected some requests to fail; numSuccess: %d, numFail: %d, elapsed: %d", numSuccess, numFail, elapsed)
+	}
+	// ensure that we should never get more requests than allowed
+	if want := int32(ideal + 1); numSuccess > want {
+		t.Fatalf("too many successful requests; want: %d, numSuccess: %d, numFail: %d, elapsed: %d", want, numSuccess, numFail, elapsed)
+	}
+
+	// Test ns1 quota
+	// Global quota should apply!
+	_, numFail, _ = testRPSWithNS(reqFunc, 5*time.Second, "ns1")
+	// ensure there were some failed requests
+	if numFail == 0 {
+		t.Fatalf("expected some requests to fail; numSuccess: %d, numFail: %d, elapsed: %d", numSuccess, numFail, elapsed)
+	}
+	// ensure that we should never get more requests than allowed
+	if want := int32(ideal + 1); numSuccess > want {
+		t.Fatalf("too many successful requests; want: %d, numSuccess: %d, numFail: %d, elapsed: %d", want, numSuccess, numFail, elapsed)
+	}
+
+	// Test ns1/ns1.1 rate limit quota
+	// Should allow more requests than the global quota, as the rate limit for ns1/ns1.1 is higher
+	numSuccess, numFail, elapsed = testRPSWithNS(reqFunc, 5*time.Second, "ns1/ns1.1")
+	// evaluate the ideal RPS as (ceil(RPS) + (RPS * totalSeconds))
+	newIdeal := 10 + (9.9 * float64(elapsed) / float64(time.Second))
+	// ensure there were some failed requests
+	if numFail == 0 {
+		t.Fatalf("expected some requests to fail; numSuccess: %d, numFail: %d, elapsed: %d", numSuccess, numFail, elapsed)
+	}
+	// ensure that we should never get more requests than allowed
+	if want := int32(newIdeal + 1); numSuccess > want {
+		t.Fatalf("too many successful requests; want: %d, numSuccess: %d, numFail: %d, elapsed: %d", want, numSuccess, numFail, elapsed)
+	}
+}
+
+func TestQuotas_RateLimitQuotaInheritableNS(t *testing.T) {
+	conf, opts := teststorage.ClusterSetup(coreConfig, nil, nil)
+	opts.NoDefaultQuotas = true
+	cluster := vault.NewTestCluster(t, conf, opts)
+	cluster.Start()
+	defer cluster.Cleanup()
+
+	core := cluster.Cores[0].Core
+	client := cluster.Cores[0].Client
+
+	vault.TestWaitActive(t, core)
+
+	// Create Parent Namespace
+	_, err := client.Logical().Write("sys/namespaces/ns1", map[string]interface{}{})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// Create a rate limit quota for parent namespace with a low RPS of 7.7, which means we can process
+	// ⌈7.7⌉*2 requests in the span of roughly a second -- 8 initially, followed
+	// by a refill rate of 7.7 per-second.
+	// As inheritable is set to true, the quota will be inherited by child namespaces
+	_, err = client.Logical().Write("sys/quotas/rate-limit/rlq", map[string]interface{}{
+		"rate":        7.7,
+		"path":        "ns1",
+		"inheritable": true,
+	})
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// Create Childnamespace, so we get a hierarchy of ns1/ns1.1
+	client.SetNamespace("ns1")
+	_, err = client.Logical().Write("sys/namespaces/ns1.1", map[string]interface{}{})
+	if err != nil {
+		t.Fatal(err)
+	}
+	client.ClearNamespace()
+
+	reqFunc := func(numSuccess, numFail *atomic.Int32, ns string) {
+		// list all namespaces in ns1/ns1.1
+		// the quota of ns1 should apply
+		client.SetNamespace(ns)
+		_, err := client.Logical().List("sys/namespaces")
+
+		if err != nil {
+			numFail.Add(1)
+		} else {
+			numSuccess.Add(1)
+		}
+		client.ClearNamespace()
+	}
+
+	numSuccess, numFail, elapsed := testRPSWithNS(reqFunc, 5*time.Second, "ns1/ns1.1")
 
 	// evaluate the ideal RPS as (ceil(RPS) + (RPS * totalSeconds))
 	ideal := 8 + (7.7 * float64(elapsed) / float64(time.Second))
@@ -529,16 +668,31 @@ func TestQuotas_RateLimitQuota(t *testing.T) {
 	// allow time (1s) for rate limit to refill before updating the quota
 	time.Sleep(time.Second)
 
-	// update the rate limit quota with a high RPS such that no requests should fail
+	// update the rate limit quota to inheritable false
+	// as a result the quota should not anymore apply to ns1.1, but only ns1
 	_, err = client.Logical().Write("sys/quotas/rate-limit/rlq", map[string]interface{}{
-		"rate": 10000.0,
+		"rate":        7.7,
+		"path":        "ns1",
+		"inheritable": false,
 	})
 	if err != nil {
 		t.Fatal(err)
 	}
 
-	_, numFail, _ = testRPS(reqFunc, 5*time.Second)
+	// as there is no quota for ns1/ns1.1, there should be no fails
+	_, numFail, _ = testRPSWithNS(reqFunc, 5*time.Second, "ns1/ns1.1")
 	if numFail > 0 {
 		t.Fatalf("unexpected number of failed requests: %d", numFail)
 	}
+
+	// as there the quota applies to ns1, there should be some fail
+	numSuccess, numFail, elapsed = testRPSWithNS(reqFunc, 5*time.Second, "ns1")
+	// ensure there were some failed requests
+	if numFail == 0 {
+		t.Fatalf("expected some requests to fail; numSuccess: %d, numFail: %d, elapsed: %d", numSuccess, numFail, elapsed)
+	}
+	// ensure that we should never get more requests than allowed
+	if want := int32(ideal + 1); numSuccess > want {
+		t.Fatalf("too many successful requests; want: %d, numSuccess: %d, numFail: %d, elapsed: %d", want, numSuccess, numFail, elapsed)
+	}
 }
diff --git a/vault/logical_system_quotas.go b/vault/logical_system_quotas.go
index e754d0de48a..8c4e8b33c63 100644
--- a/vault/logical_system_quotas.go
+++ b/vault/logical_system_quotas.go
@@ -149,6 +149,11 @@ The 'rate' must be positive.`,
 					Description: `If set, when a client reaches a rate limit threshold, the client will be prohibited
 from any further requests until after the 'block_interval' has elapsed.`,
 				},
+				"inheritable": {
+					Type: framework.TypeBool,
+					Description: `If set to true, child namespaces will use this quota, unless another more specific
+quota exists. Can only be set on namespace quotas. A quota on the root namespace will by default be inheritable.`,
+				},
 			},
 			Operations: map[logical.Operation]framework.OperationHandler{
 				logical.UpdateOperation: &framework.PathOperation{
@@ -199,6 +204,10 @@ from any further requests until after the 'block_interval' has elapsed.`,
 									Type:     framework.TypeInt,
 									Required: true,
 								},
+								"inheritable": {
+									Type:     framework.TypeBool,
+									Required: true,
+								},
 							},
 						}},
 					},
@@ -301,10 +310,7 @@ func (b *SystemBackend) handleRateLimitQuotasUpdate() framework.OperationFunc {
 		}
 
 		mountPath := sanitizePath(d.Get("path").(string))
-		ns := namespace.RootNamespace
-		if ns.ID != namespace.RootNamespaceID {
-			mountPath = strings.TrimPrefix(mountPath, ns.Path)
-		}
+		ns, mountPath := b.Core.NamespaceByPath(ctx, mountPath)
 
 		var pathSuffix string
 		if mountPath != "" {
@@ -354,9 +360,17 @@ func (b *SystemBackend) handleRateLimitQuotasUpdate() framework.OperationFunc {
 			return nil, err
 		}
 
+		inheritable := false
+		inheritableI, ok := d.GetOk("inheritable")
+		if ok {
+			inheritable = inheritableI.(bool)
+		} else if ns.Path == "" && mountPath == "" {
+			inheritable = true
+		}
+
 		switch {
 		case quota == nil:
-			quota = quotas.NewRateLimitQuota(name, ns.Path, mountPath, pathSuffix, role, rate, interval, blockInterval)
+			quota = quotas.NewRateLimitQuota(name, ns.Path, mountPath, pathSuffix, role, rate, interval, blockInterval, inheritable)
 		default:
 			// Re-inserting the already indexed object in memdb might cause problems.
 			// So, clone the object. See https://github.com/hashicorp/go-memdb/issues/76.
@@ -368,6 +382,7 @@ func (b *SystemBackend) handleRateLimitQuotasUpdate() framework.OperationFunc {
 			rlq.Rate = rate
 			rlq.Interval = interval
 			rlq.BlockInterval = blockInterval
+			rlq.Inheritable = inheritable
 			quota = rlq
 		}
 
@@ -416,6 +431,7 @@ func (b *SystemBackend) handleRateLimitQuotasRead() framework.OperationFunc {
 			"rate":           rlq.Rate,
 			"interval":       int(rlq.Interval.Seconds()),
 			"block_interval": int(rlq.BlockInterval.Seconds()),
+			"inheritable":    rlq.Inheritable,
 		}
 
 		return &logical.Response{
diff --git a/vault/mount.go b/vault/mount.go
index f9c6b71cbb5..6da0c4be13e 100644
--- a/vault/mount.go
+++ b/vault/mount.go
@@ -2206,7 +2206,10 @@ func (c *Core) mountEntryView(me *MountEntry) (BarrierView, error) {
 	}
 
 	switch me.Type {
-	case mountTypeSystem:
+	case mountTypeSystem, mountTypeNSSystem:
+		if me.Namespace() != nil && me.NamespaceID != namespace.RootNamespaceID {
+			return c.namespaceMountEntryView(me.Namespace(), systemBarrierPrefix), nil
+		}
 		return NewBarrierView(c.barrier, systemBarrierPrefix), nil
 	case mountTypeToken:
 		return NewBarrierView(c.barrier, systemBarrierPrefix+tokenSubPath), nil
diff --git a/vault/namespaces_oss.go b/vault/namespaces_oss.go
index facef49b94c..f3382119fb7 100644
--- a/vault/namespaces_oss.go
+++ b/vault/namespaces_oss.go
@@ -31,3 +31,9 @@ func NamespaceView(barrier logical.Storage, ns *namespace.Namespace) BarrierView
 
 	return NewBarrierView(barrier, path.Join(namespaceBarrierPrefix, ns.UUID)+"/")
 }
+
+// NamespaceByPath returns the namespace and the path prefix for the given path.
+// Note, that it is on the caller to ensure that the namespace is resolved, as NamespaceByPath otherwise resolves to root.
+func (c *Core) NamespaceByPath(ctx context.Context, path string) (*namespace.Namespace, string) {
+	return c.namespaceStore.GetNamespaceByLongestPrefix(ctx, path)
+}
diff --git a/vault/namespaces_store.go b/vault/namespaces_store.go
index d7a8acf979b..522bbf09924 100644
--- a/vault/namespaces_store.go
+++ b/vault/namespaces_store.go
@@ -9,6 +9,7 @@ import (
 	"fmt"
 	"net/http"
 	"path"
+	"strings"
 	"sync"
 	"sync/atomic"
 	"time"
@@ -444,6 +445,24 @@ func (ns *NamespaceStore) GetNamespaceByAccessor(ctx context.Context, id string)
 	return item.Clone(), nil
 }
 
+func (ns *NamespaceStore) GetNamespaceByLongestPrefix(ctx context.Context, path string) (*namespace.Namespace, string) {
+	ctxNs, err := namespace.FromContext(ctx)
+	if err != nil {
+		ctxNs = namespace.RootNamespace
+	}
+
+	// e.g. needed for /sys/unseal, where the namespace store is not yet initialized
+	if ns == nil {
+		return ctxNs, path
+	}
+
+	combinedPath := ctxNs.Path + path
+	ns.lock.RLock()
+	prefix, entry, _ := ns.namespacesByPath.LongestPrefix(combinedPath)
+	ns.lock.RUnlock()
+	return entry, strings.TrimPrefix(combinedPath, prefix)
+}
+
 // GetNamespaceByPath is used to fetch the namespace with the given full path.
 func (ns *NamespaceStore) GetNamespaceByPath(ctx context.Context, path string) (*namespace.Namespace, error) {
 	defer metrics.MeasureSince([]string{"namespace", "get_namespace_by_path"}, time.Now())
diff --git a/vault/quotas/quotas.go b/vault/quotas/quotas.go
index 41dd3cfb4d8..9de2f8df12f 100644
--- a/vault/quotas/quotas.go
+++ b/vault/quotas/quotas.go
@@ -200,6 +200,9 @@ type Quota interface {
 	// QuotaName is the name of the quota rule
 	QuotaName() string
 
+	// IsInheritable returns whether the quota is inheritable
+	IsInheritable() bool
+
 	// initialize sets up the fields in the quota type to begin operating
 	initialize(log.Logger, *metricsutil.ClusterMetricSink) error
 
@@ -576,18 +579,26 @@ func (m *Manager) queryQuota(txn *memdb.Txn, req *Request) (Quota, error) {
 	// If the request belongs to "root" namespace, then we have already looked at
 	// global quotas when fetching namespace specific quota rule. When the request
 	// belongs to a non-root namespace, and when there are no namespace specific
-	// quota rules present, we fallback on the global quotas.
+	// quota rules present, we fallback on the inheritable quotas.
 	if req.NamespacePath == "root" {
 		return nil, nil
 	}
 
-	// Fetch global quota
-	quota, err = quotaFetchFunc(indexNamespace, "root", false, false, false)
-	if err != nil {
-		return nil, err
-	}
-	if quota != nil {
-		return quota, nil
+	for path, isParent := namespace.ParentOf(req.NamespacePath); isParent; path, isParent = namespace.ParentOf(path) {
+		if path == "" {
+			path = "root"
+		}
+		quota, err = quotaFetchFunc(indexNamespace, path, false, false, false)
+		if err != nil {
+			return nil, err
+		}
+		if quota != nil && quota.IsInheritable() {
+			return quota, nil
+		}
+
+		if path == "root" {
+			break
+		}
 	}
 
 	return nil, nil
diff --git a/vault/quotas/quotas_rate_limit.go b/vault/quotas/quotas_rate_limit.go
index 2c46eb28b87..772de4e2c0d 100644
--- a/vault/quotas/quotas_rate_limit.go
+++ b/vault/quotas/quotas_rate_limit.go
@@ -76,6 +76,10 @@ type RateLimitQuota struct {
 	// reaches the rate limit.
 	BlockInterval time.Duration `json:"block_interval"`
 
+	// Inheritable defines a quota on a namespace to be inheritable by child
+	// namespaces.
+	Inheritable bool `json:"inheritable"`
+
 	lock                *sync.RWMutex
 	store               limiter.Store
 	logger              log.Logger
@@ -92,7 +96,7 @@ type RateLimitQuota struct {
 // provided, which will default to 1s when initialized. An optional block
 // duration may be provided, where if set, when a client reaches the rate limit,
 // subsequent requests will fail until the block duration has passed.
-func NewRateLimitQuota(name, nsPath, mountPath, pathSuffix, role string, rate float64, interval, block time.Duration) *RateLimitQuota {
+func NewRateLimitQuota(name, nsPath, mountPath, pathSuffix, role string, rate float64, interval, block time.Duration, inheritable bool) *RateLimitQuota {
 	id, err := uuid.GenerateUUID()
 	if err != nil {
 		// Fall back to generating with a hash of the name, later in initialize
@@ -109,6 +113,7 @@ func NewRateLimitQuota(name, nsPath, mountPath, pathSuffix, role string, rate fl
 		Rate:          rate,
 		Interval:      interval,
 		BlockInterval: block,
+		Inheritable:   inheritable,
 		purgeInterval: DefaultRateLimitPurgeInterval,
 		staleAge:      DefaultRateLimitStaleAge,
 	}
@@ -126,6 +131,7 @@ func (q *RateLimitQuota) Clone() Quota {
 		BlockInterval: q.BlockInterval,
 		Rate:          q.Rate,
 		Interval:      q.Interval,
+		Inheritable:   q.Inheritable,
 	}
 	return rlq
 }
@@ -276,6 +282,10 @@ func (rlq *RateLimitQuota) QuotaName() string {
 	return rlq.Name
 }
 
+func (rlq *RateLimitQuota) IsInheritable() bool {
+	return rlq.Inheritable
+}
+
 // allow decides if the request is allowed by the quota. An error will be
 // returned if the request ID or address is empty. If the path is exempt, the
 // quota will not be evaluated. Otherwise, the client rate limiter is retrieved
diff --git a/vault/quotas/quotas_rate_limit_test.go b/vault/quotas/quotas_rate_limit_test.go
index 2a487725064..7fb4ea0d375 100644
--- a/vault/quotas/quotas_rate_limit_test.go
+++ b/vault/quotas/quotas_rate_limit_test.go
@@ -30,7 +30,7 @@ func TestNewRateLimitQuota(t *testing.T) {
 		rlq       *RateLimitQuota
 		expectErr bool
 	}{
-		{"valid rate", NewRateLimitQuota("test-rate-limiter", "qa", "/foo/bar", "", "", 16.7, time.Second, 0), false},
+		{"valid rate", NewRateLimitQuota("test-rate-limiter", "qa", "/foo/bar", "", "", 16.7, time.Second, 0, false), false},
 	}
 
 	for _, tc := range testCases {
@@ -47,7 +47,7 @@ func TestNewRateLimitQuota(t *testing.T) {
 }
 
 func TestRateLimitQuota_Close(t *testing.T) {
-	rlq := NewRateLimitQuota("test-rate-limiter", "qa", "/foo/bar", "", "", 16.7, time.Second, time.Minute)
+	rlq := NewRateLimitQuota("test-rate-limiter", "qa", "/foo/bar", "", "", 16.7, time.Second, time.Minute, false)
 	require.NoError(t, rlq.initialize(logging.NewVaultLogger(log.Trace), metricsutil.BlackholeSink()))
 	require.NoError(t, rlq.close(context.Background()))
 
@@ -221,7 +221,7 @@ func TestRateLimitQuota_Update(t *testing.T) {
 	qm, err := NewManager(logging.NewVaultLogger(log.Trace), metricsutil.BlackholeSink(), true)
 	require.NoError(t, err)
 
-	quota := NewRateLimitQuota("quota1", "", "", "", "", 10, time.Second, 0)
+	quota := NewRateLimitQuota("quota1", "", "", "", "", 10, time.Second, 0, false)
 	require.NoError(t, qm.SetQuota(context.Background(), TypeRateLimit.String(), quota, true))
 	require.NoError(t, qm.SetQuota(context.Background(), TypeRateLimit.String(), quota, true))
 
diff --git a/vault/quotas/quotas_test.go b/vault/quotas/quotas_test.go
index 0121a3679e9..f7ab248f2e7 100644
--- a/vault/quotas/quotas_test.go
+++ b/vault/quotas/quotas_test.go
@@ -19,7 +19,7 @@ func TestQuotas_MountPathOverwrite(t *testing.T) {
 	qm, err := NewManager(logging.NewVaultLogger(log.Trace), metricsutil.BlackholeSink(), true)
 	require.NoError(t, err)
 
-	quota := NewRateLimitQuota("tq", "", "kv1/", "", "", 10, time.Second, 0)
+	quota := NewRateLimitQuota("tq", "", "kv1/", "", "", 10, time.Second, 0, false)
 	require.NoError(t, qm.SetQuota(context.Background(), TypeRateLimit.String(), quota, false))
 	quota = quota.Clone().(*RateLimitQuota)
 	quota.MountPath = "kv2/"
@@ -48,7 +48,7 @@ func TestQuotas_Precedence(t *testing.T) {
 
 	setQuotaFunc := func(t *testing.T, name, nsPath, mountPath, pathSuffix, role string) Quota {
 		t.Helper()
-		quota := NewRateLimitQuota(name, nsPath, mountPath, pathSuffix, role, 10, time.Second, 0)
+		quota := NewRateLimitQuota(name, nsPath, mountPath, pathSuffix, role, 10, time.Second, 0, false)
 		require.NoError(t, qm.SetQuota(context.Background(), TypeRateLimit.String(), quota, true))
 		return quota
 	}
@@ -137,7 +137,7 @@ func TestQuotas_QueryResolveRole_RateLimitQuotas(t *testing.T) {
 	require.False(t, required)
 
 	// Create a non-role-based RLQ on mount1/ and make sure it doesn't require role resolution
-	rlq := NewRateLimitQuota("tq", rlqReq.NamespacePath, rlqReq.MountPath, rlqReq.Path, rlqReq.Role, 10, 1*time.Minute, 10*time.Second)
+	rlq := NewRateLimitQuota("tq", rlqReq.NamespacePath, rlqReq.MountPath, rlqReq.Path, rlqReq.Role, 10, 1*time.Minute, 10*time.Second, false)
 	require.NoError(t, qm.SetQuota(context.Background(), TypeRateLimit.String(), rlq, false))
 
 	required, err = qm.QueryResolveRoleQuotas(rlqReq)
@@ -146,7 +146,7 @@ func TestQuotas_QueryResolveRole_RateLimitQuotas(t *testing.T) {
 
 	// Create a role-based RLQ on mount1/ and make sure it requires role resolution
 	rlqReq.Role = "test"
-	rlq = NewRateLimitQuota("tq", rlqReq.NamespacePath, rlqReq.MountPath, rlqReq.Path, rlqReq.Role, 10, 1*time.Minute, 10*time.Second)
+	rlq = NewRateLimitQuota("tq", rlqReq.NamespacePath, rlqReq.MountPath, rlqReq.Path, rlqReq.Role, 10, 1*time.Minute, 10*time.Second, false)
 	require.NoError(t, qm.SetQuota(context.Background(), TypeRateLimit.String(), rlq, false))
 
 	required, err = qm.QueryResolveRoleQuotas(rlqReq)


Reviews:
user: login: wslabosz-reply
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-27T17:05:57Z
user: login: driif
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-02T08:52:13Z
user: login: driif
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-02T08:55:00Z
user: login: driif
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-02T14:09:49Z
user: login: driif
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-02T14:29:30Z
user: login: satoqz
body: LGTM, let's get this merged!

Edit: Just discussed with @voigt that we'll include the `req.Storage`/`mountEntryView` changes from #1282 in here first.
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-30T08:15:01Z
user: login: wslabosz-reply
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T08:29:59Z
user: login: wslabosz-reply
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T08:31:14Z
user: login: wslabosz-reply
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T08:41:06Z
user: login: voigt
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-30T09:04:16Z
user: login: voigt
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-30T09:07:33Z
user: login: satoqz
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-30T09:08:27Z
user: login: wslabosz-reply
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T09:09:05Z
user: login: wslabosz-reply
body: Looks good to me, the one path that let's you create a quota for a namespace (mainly when you write the quota with ns path specified and not the header) will be accounted for in #1279, so LGTM
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-30T09:12:30Z
user: login: voigt
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-30T09:12:34Z
user: login: cipherboy
body: I think this looks good as well. I'm surprised but mildly happy that these changes were so small. 
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-30T20:32:39Z
user: login: satoqz
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-01T08:03:17Z

Review comments:
diff_hunk: @@ -301,7 +301,10 @@ func (b *SystemBackend) handleRateLimitQuotasUpdate() framework.OperationFunc {
 		}
 
 		mountPath := sanitizePath(d.Get("path").(string))
-		ns := namespace.RootNamespace
+		ns, err := namespace.FromContext(ctx)
+		if err != nil {
+			return logical.ErrorResponse("invalid namespace"), nil
path: vault/logical_system_quotas.go
user: login: wslabosz-reply
body: By doing an exact string matching against the repo, there are 6 examples of following error format when we cannot retrieve the namespace from the context, so I'd suggest using something like that
```suggestion
			return logical.ErrorResponse(fmt.Errorf("failed to find namespace in context: %w", err).Error()), nil
```

created_at: 2025-03-27T17:05:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -301,7 +302,10 @@ func (b *SystemBackend) handleRateLimitQuotasUpdate() framework.OperationFunc {
 		}
 
 		mountPath := sanitizePath(d.Get("path").(string))
-		ns := namespace.RootNamespace
+		ns, err := namespace.FromContext(ctx)
+		if err != nil {
+			return logical.ErrorResponse(fmt.Errorf("failed to find namespace in context: %w", err).Error()), nil
+		}
path: vault/logical_system_quotas.go
user: login: driif
body: ```suggestion
		if err != nil {
			return logical.ErrorResponse("failed to find namespace in context: %v", err), nil
		}
```
created_at: 2025-04-02T08:52:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -301,7 +302,10 @@ func (b *SystemBackend) handleRateLimitQuotasUpdate() framework.OperationFunc {
 		}
 
 		mountPath := sanitizePath(d.Get("path").(string))
-		ns := namespace.RootNamespace
+		ns, err := namespace.FromContext(ctx)
+		if err != nil {
+			return logical.ErrorResponse(fmt.Errorf("failed to find namespace in context: %w", err).Error()), nil
+		}
path: vault/logical_system_quotas.go
user: login: driif
body: We could simplify it here. Also Using %v instead of %w since we're immediately converting to string via Error().
created_at: 2025-04-02T08:55:00Z
author_association: CONTRIBUTOR
diff_hunk: @@ -44,9 +44,6 @@ var (
 		"sys/loggers",
 		"sys/metrics",
 		"sys/monitor",
-		"sys/quotas/config",
-		"sys/quotas/lease-count",
-		"sys/quotas/rate-limit",
path: http/util.go
user: login: driif
body: I think this might explain why the Vault API uses paths instead of namespaces.

https://developer.hashicorp.com/vault/tutorials/operations/resource-quotas#vault-enterprise-namespaces

Historically, sys/quotas/... might have been in restrictedAPIs. This meant you couldn't even route a request like POST /v1/ns1/sys/quotas/rate-limit/some-quota to the handler. The request would be rejected upfront by the HTTP layer based purely on the path structure. Removing it from the list allows the request to reach the SystemBackend handler, even when the request has a namespace context.

Seems like Removing sys/quotas/config, sys/quotas/lease-count, and sys/quotas/rate-limit could cause some issues. 
In this case the Safeguard should be ACLs, but if they are not properly configured, than a user within ns1 could have a possibility to change global quotas. 

@cipherboy @voigt @klaus-sap @wslabosz-reply 
created_at: 2025-04-02T14:09:49Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1152,6 +1152,28 @@ func TestCore_MountEntryView(t *testing.T) {
 
 			wantError: true,
 		},
+		{
+			name: "entry of 'ns_system' mount type",
+			mountEntry: &MountEntry{
path: vault/mount_test.go
user: login: driif
body: I suggest adding tests to check quota rate limits in namespaces. These could be placed in `vault/logical_system_quotas_test.go` or `vault/external_tests_quotas/quotas_test.go`.

Currently, we have tests for mounting, but we aren't testing rate limits in namespaces yet. It would also be valuable to add tests for checking Global Quota Modification from namespaces.
created_at: 2025-04-02T14:29:29Z
author_association: CONTRIBUTOR
diff_hunk: @@ -31,3 +31,7 @@ func NamespaceView(barrier logical.Storage, ns *namespace.Namespace) BarrierView
 
 	return NewBarrierView(barrier, path.Join(namespaceBarrierPrefix, ns.UUID)+"/")
 }
+
+func (c *Core) NamespaceByPath(ctx context.Context, path string) (*namespace.Namespace, string) {
path: vault/namespaces_oss.go
user: login: wslabosz-reply
body: I'd add a comment that this won't necessarily returns the desired namespace, but the one that got matched as the longest prefix of the path, and it's on the caller to ensure that the namespace is resolved before calling this, otherwise this could resolve to the `root`
created_at: 2025-04-30T08:29:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -428,8 +445,7 @@ func (b *SystemBackend) handleRateLimitQuotasDelete() framework.OperationFunc {
 	return func(ctx context.Context, req *logical.Request, d *framework.FieldData) (*logical.Response, error) {
 		name := d.Get("name").(string)
 		qType := quotas.TypeRateLimit.String()
-
-		if err := req.Storage.Delete(ctx, quotas.QuotaStoragePath(qType, name)); err != nil {
+		if err := b.Core.systemBarrierView.Delete(ctx, quotas.QuotaStoragePath(qType, name)); err != nil {
path: vault/logical_system_quotas.go
user: login: wslabosz-reply
body: just an open question, as #1206 was merged, are quotas also deleted when deleting a namespace?
created_at: 2025-04-30T08:31:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -576,18 +579,27 @@ func (m *Manager) queryQuota(txn *memdb.Txn, req *Request) (Quota, error) {
 	// If the request belongs to "root" namespace, then we have already looked at
 	// global quotas when fetching namespace specific quota rule. When the request
 	// belongs to a non-root namespace, and when there are no namespace specific
-	// quota rules present, we fallback on the global quotas.
+	// quota rules present, we fallback on the inheritable quotas.
 	if req.NamespacePath == "root" {
 		return nil, nil
 	}
 
-	// Fetch global quota
-	quota, err = quotaFetchFunc(indexNamespace, "root", false, false, false)
-	if err != nil {
-		return nil, err
-	}
-	if quota != nil {
-		return quota, nil
+	for path, isParent := namespace.ParentOf(req.NamespacePath); isParent; path, isParent = namespace.ParentOf(path) {
+		if path == "" {
+			path = "root"
+		}
+		quota, err = quotaFetchFunc(indexNamespace, path, false, false, false)
+		if err != nil {
+			return nil, err
+		}
+		if quota != nil && quota.IsInheritable() {
+			return quota, nil
+		} else if quota != nil {
+		}
path: vault/quotas/quotas.go
user: login: wslabosz-reply
body: is this empty `else if` block required?
created_at: 2025-04-30T08:41:06Z
author_association: CONTRIBUTOR
diff_hunk: @@ -31,3 +31,7 @@ func NamespaceView(barrier logical.Storage, ns *namespace.Namespace) BarrierView
 
 	return NewBarrierView(barrier, path.Join(namespaceBarrierPrefix, ns.UUID)+"/")
 }
+
+func (c *Core) NamespaceByPath(ctx context.Context, path string) (*namespace.Namespace, string) {
path: vault/namespaces_oss.go
user: login: voigt
body: I'll add a comment.
created_at: 2025-04-30T09:04:15Z
author_association: MEMBER
diff_hunk: @@ -576,18 +579,27 @@ func (m *Manager) queryQuota(txn *memdb.Txn, req *Request) (Quota, error) {
 	// If the request belongs to "root" namespace, then we have already looked at
 	// global quotas when fetching namespace specific quota rule. When the request
 	// belongs to a non-root namespace, and when there are no namespace specific
-	// quota rules present, we fallback on the global quotas.
+	// quota rules present, we fallback on the inheritable quotas.
 	if req.NamespacePath == "root" {
 		return nil, nil
 	}
 
-	// Fetch global quota
-	quota, err = quotaFetchFunc(indexNamespace, "root", false, false, false)
-	if err != nil {
-		return nil, err
-	}
-	if quota != nil {
-		return quota, nil
+	for path, isParent := namespace.ParentOf(req.NamespacePath); isParent; path, isParent = namespace.ParentOf(path) {
+		if path == "" {
+			path = "root"
+		}
+		quota, err = quotaFetchFunc(indexNamespace, path, false, false, false)
+		if err != nil {
+			return nil, err
+		}
+		if quota != nil && quota.IsInheritable() {
+			return quota, nil
+		} else if quota != nil {
+		}
path: vault/quotas/quotas.go
user: login: voigt
body: Its totally not. Removing it!
created_at: 2025-04-30T09:07:33Z
author_association: MEMBER
diff_hunk: @@ -428,8 +445,7 @@ func (b *SystemBackend) handleRateLimitQuotasDelete() framework.OperationFunc {
 	return func(ctx context.Context, req *logical.Request, d *framework.FieldData) (*logical.Response, error) {
 		name := d.Get("name").(string)
 		qType := quotas.TypeRateLimit.String()
-
-		if err := req.Storage.Delete(ctx, quotas.QuotaStoragePath(qType, name)); err != nil {
+		if err := b.Core.systemBarrierView.Delete(ctx, quotas.QuotaStoragePath(qType, name)); err != nil {
path: vault/logical_system_quotas.go
user: login: satoqz
body: In short, yes
created_at: 2025-04-30T09:08:27Z
author_association: MEMBER
diff_hunk: @@ -428,8 +445,7 @@ func (b *SystemBackend) handleRateLimitQuotasDelete() framework.OperationFunc {
 	return func(ctx context.Context, req *logical.Request, d *framework.FieldData) (*logical.Response, error) {
 		name := d.Get("name").(string)
 		qType := quotas.TypeRateLimit.String()
-
-		if err := req.Storage.Delete(ctx, quotas.QuotaStoragePath(qType, name)); err != nil {
+		if err := b.Core.systemBarrierView.Delete(ctx, quotas.QuotaStoragePath(qType, name)); err != nil {
path: vault/logical_system_quotas.go
user: login: wslabosz-reply
body: @voigt 
investigated with @satoqz and turns out the `(Core).unmountInternal` handles the quota deletion for the ns (by `ns.Path`) deleting it both from storage and in-mem so we are fine on this front!
created_at: 2025-04-30T09:09:05Z
author_association: CONTRIBUTOR
diff_hunk: @@ -428,8 +445,7 @@ func (b *SystemBackend) handleRateLimitQuotasDelete() framework.OperationFunc {
 	return func(ctx context.Context, req *logical.Request, d *framework.FieldData) (*logical.Response, error) {
 		name := d.Get("name").(string)
 		qType := quotas.TypeRateLimit.String()
-
-		if err := req.Storage.Delete(ctx, quotas.QuotaStoragePath(qType, name)); err != nil {
+		if err := b.Core.systemBarrierView.Delete(ctx, quotas.QuotaStoragePath(qType, name)); err != nil {
path: vault/logical_system_quotas.go
user: login: voigt
body: Just double checked: quotas are not deleted.
Also I'm not quite sure about the expected behavior here tbh. Should quotas be deleted as well?

---

Edit: fooled myself; was testing it wrong. Deleting a namespace also deletes quotas.
created_at: 2025-04-30T09:12:34Z
author_association: MEMBER
diff_hunk: @@ -354,9 +360,17 @@ func (b *SystemBackend) handleRateLimitQuotasUpdate() framework.OperationFunc {
 			return nil, err
 		}
 
+		inheritable := false
+		inheritableI, ok := d.GetOk("inheritable")
+		if ok {
+			inheritable = inheritableI.(bool)
path: vault/logical_system_quotas.go
user: login: cipherboy
body: (nit) For consistency:

```suggestion
		inheritableRaw, ok := d.GetOk("inheritable")
		if ok {
			inheritable = inheritableRaw.(bool)
```
created_at: 2025-04-30T20:23:31Z
author_association: MEMBER
diff_hunk: @@ -444,6 +445,24 @@ func (ns *NamespaceStore) GetNamespaceByAccessor(ctx context.Context, id string)
 	return item.Clone(), nil
 }
 
+func (ns *NamespaceStore) GetNamespaceByLongestPrefix(ctx context.Context, path string) (*namespace.Namespace, string) {
+	ctxNs, err := namespace.FromContext(ctx)
+	if err != nil {
+		ctxNs = namespace.RootNamespace
+	}
+
+	// e.g. needed for /sys/unseal, where the namespace store is not yet initialized
path: vault/namespaces_store.go
user: login: cipherboy
body: While necessary, this is less than ideal. I wish we could ensure we either have primitive paths (which never exist within a namespace; those routed by `http` directly without calling `core.RouteRequest(...)`) or a fully-initialized store so that the response is always correct (+/- unloaded namespaces). 

Then again, if this request were to a missing namespace, the subsequent route should fail so that's fine :thinking:... 

My 2c. 
created_at: 2025-04-30T20:27:05Z
author_association: MEMBER
diff_hunk: @@ -444,6 +445,24 @@ func (ns *NamespaceStore) GetNamespaceByAccessor(ctx context.Context, id string)
 	return item.Clone(), nil
 }
 
+func (ns *NamespaceStore) GetNamespaceByLongestPrefix(ctx context.Context, path string) (*namespace.Namespace, string) {
+	ctxNs, err := namespace.FromContext(ctx)
+	if err != nil {
+		ctxNs = namespace.RootNamespace
+	}
+
+	// e.g. needed for /sys/unseal, where the namespace store is not yet initialized
path: vault/namespaces_store.go
user: login: satoqz
body: I have plans in that direction :D
created_at: 2025-05-01T08:03:17Z
author_association: MEMBER

PR comments:
user: login: klaus-sap
created_at: 2025-03-27T12:55:51Z
body: When defining a quota on a namespace, should we check that this quota does **not** exceed the quota of the parent namespace?
author_association: CONTRIBUTOR
user: login: cipherboy
created_at: 2025-03-27T18:02:53Z
body: @klaus-sap Hmm, interesting question:

> When defining a quota on a namespace, should we check that this quota does not exceed the quota of the parent namespace?

I think quota enforcement will ensure that, independent of what we do at quota creation time.

In the context [of potential future work on horizontal scaling](https://openbao.org/blog/vision-for-namespaces/#offline-recovery-mode) and in particular, allowing loading of Namespaces across different clusters, we have the option to have parent quotas applying to children from different clusters. Thus the namespace itself could set up some higher rate limit X and the specific cluster could (in the parent namespace) set up a rate limit Y < X (before even loading the child potentially, if it is a global rate limit for instance). Thus enforcing this would require online verification from all clusters loading that namespace, some of which might be offline.

Less distantly, it breaks the veil of the parent namespace being opaque to a child, as much as necessary. 

Thoughts?
author_association: MEMBER
user: login: klaus-sap
created_at: 2025-03-28T13:19:02Z
body: @cipherboy @voigt I wonder if we shouldn't combine horizontal scaling with multiple storage trees. IMHO, one tree should not be distributed over two or more clusters. In that case, it would still be consistent to check the quota of a namespace with its parent namespace. And as the quota is non-secret configuration data, I would not see an issue with the veil of the parent namespace being opaque to a child.
author_association: CONTRIBUTOR
user: login: cipherboy
created_at: 2025-03-28T13:53:29Z
body: > IMHO, one tree should not be distributed over two or more clusters.

Note that only one active node (in one cluster) would have (write) ownership of it at a time and would be predicated on having the same storage backend shared by both. Mostly it decouples the plugin logic from the storage/clustering layer a bit. It doesn't change the strict write safety guarantees we have currently.

This is a rather powerful concept, IMHO. Right now we need symmetric scaling: the entire cluster needs to ramp up to support the needs of all users. However, if there are latency-sensitive workloads, there's no way to scale memory usage or compute for particular areas. Creating fully separate clusters complicates the DR recovery plan, so having specific namespaces with specific resource requirements (e.g., a collection of PKI and Transit engines could be very compute heavy with low memory requirements) would allow primary leadership of that namespace be owned by high-CPU nodes (with high quota) and secondary ownership for DR purposes on the main cluster, with very low quota in the event of failure.

Similarly, if one wanted to run a K/V store with a larger storage LRU cache (localized to the workload), this'd require all nodes to have high memory to hit latency requirements when combined with other workloads. Instead, pushing this to a memory-heavy cluster within dedicated NSes and replicating back to the primary cluster for DR purposes would be useful. Performance would take a hit in the event of failure. 

This could be solved with multiple clusters, but would require separating workloads by type, not by tenant. By placing these resource-namespaces within the tenant's base NS, you still get logical tenant grouping in storage/path while allowing the namespace itself to float over the relevant namespace cluster. If the thin parent namespace is mounted in both clusters as well, you get the added benefit of single authentication rather than having separate tokens for separate clusters.

---

But, I don't think this discussion is quite suited for a discussion on quotas outside of some examples where having disparate quotas at some parent level would be useful.

My 2c. 
author_association: MEMBER
user: login: klaus-sap
created_at: 2025-03-31T09:24:36Z
body: @cipherboy We should take care to keep OpenBao simple, because in general complicated concepts come with high operational costs.

In our operation model for multi-tenancy, we want to assign first level namespaces to customers, which then in turn can create child namespaces within their parent namespace. In this model, it would be illogical, if customers could assign themselves higher quotas than they got granted on first level.

In case of disparate quota requirements, I would vote for using separate parent namespaces. IMHO, multiple authentication to separate namespaces does not really hurt, does it? 
author_association: CONTRIBUTOR
user: login: phyrog
created_at: 2025-04-14T15:14:55Z
body: There are a bunch of ideas here that require a bigger refactoring of the quota manager. As discussed in the WG call today, we'll go with feature parity with Vault for now and create a separate issue or RFC for the enhancements of quotas. That means, for the first namespaces release, quotas will be implemented in the following way:

- the quota api endpoints unter `/sys/quotas` remain restricted and can't be called in namespaces
- quotas are located in storage under `/sys/quotas`
- quotas are applied as per https://github.com/openbao/openbao/blob/279271da7371719c7939fe493aed172f28562f72/vault/quotas/quotas.go#L481-L490
- there is no enforcement or requirement that quotas can only get smaller the more specific the path, which also means child namespaces can get a bigger quota than parent namespaces

Anything that goes beyond this should be part of a new issue or RFC to properly discuss implications of changes.
author_association: CONTRIBUTOR
user: login: cipherboy
created_at: 2025-04-29T13:03:39Z
body: Ahh, it looks like our startup function has some unfortunate ordering issues too, related to resolving quotas in `http` layer:

```
2025-04-29T12:24:56.3643354Z WARNING: DATA RACE
2025-04-29T12:24:56.3643738Z Write at 0x00c001b7d068 by goroutine 5891:
2025-04-29T12:24:56.3644628Z   github.com/openbao/openbao/vault.(*Core).setupNamespaceStore()
2025-04-29T12:24:56.3645690Z       /home/runner/work/openbao/openbao/vault/namespaces_store.go:173 +0xbb
2025-04-29T12:24:56.3646597Z   github.com/openbao/openbao/vault.standardUnsealStrategy.unseal()
2025-04-29T12:24:56.3647576Z       /home/runner/work/openbao/openbao/vault/core.go:2269 +0x42b
2025-04-29T12:24:56.3648414Z   github.com/openbao/openbao/vault.(*standardUnsealStrategy).unseal()
2025-04-29T12:24:56.3649135Z       <autogenerated>:1 +0x6e
2025-04-29T12:24:56.3649711Z   github.com/openbao/openbao/vault.(*Core).postUnseal()
2025-04-29T12:24:56.3650587Z       /home/runner/work/openbao/openbao/vault/core.go:2389 +0x4b9
2025-04-29T12:24:56.3651312Z   github.com/openbao/openbao/vault.(*Core).postUnseal()
2025-04-29T12:24:56.3652164Z       /home/runner/work/openbao/openbao/vault/core.go:2385 +0x464
2025-04-29T12:24:56.3652936Z   github.com/openbao/openbao/vault.(*Core).waitForLeadership()
2025-04-29T12:24:56.3654004Z       /home/runner/work/openbao/openbao/vault/ha.go:634 +0x1285
2025-04-29T12:24:56.3654828Z   github.com/openbao/openbao/vault.(*Core).performKeyUpgrades()
2025-04-29T12:24:56.3655775Z       /home/runner/work/openbao/openbao/vault/ha.go:995 +0x21b
2025-04-29T12:24:56.3656576Z   github.com/openbao/openbao/vault.(*Core).performKeyUpgrades()
2025-04-29T12:24:56.3657497Z       /home/runner/work/openbao/openbao/vault/ha.go:991 +0x129
2025-04-29T12:24:56.3658266Z   github.com/openbao/openbao/vault.(*Core).performKeyUpgrades()
2025-04-29T12:24:56.3659157Z       /home/runner/work/openbao/openbao/vault/ha.go:987 +0x5b
2025-04-29T12:24:56.3659919Z   github.com/openbao/openbao/vault.(*Core).waitForLeadership()
2025-04-29T12:24:56.3660801Z       /home/runner/work/openbao/openbao/vault/ha.go:578 +0xc75
2025-04-29T12:24:56.3661556Z   github.com/openbao/openbao/vault.(*Core).runStandby.func9()
2025-04-29T12:24:56.3662439Z       /home/runner/work/openbao/openbao/vault/ha.go:448 +0x54
2025-04-29T12:24:56.3663221Z   github.com/oklog/run.(*Group).Run.func1()
2025-04-29T12:24:56.3664309Z       /home/runner/go/pkg/mod/github.com/oklog/run@v1.1.0/group.go:38 +0x39
2025-04-29T12:24:56.3665045Z   github.com/oklog/run.(*Group).Run.gowrap1()
2025-04-29T12:24:56.3665943Z       /home/runner/go/pkg/mod/github.com/oklog/run@v1.1.0/group.go:39 +0x4f
2025-04-29T12:24:56.3666547Z 
2025-04-29T12:24:56.3666794Z Previous read at 0x00c001b7d068 by goroutine 5899:
2025-04-29T12:24:56.3667515Z   github.com/openbao/openbao/vault.(*Core).NamespaceByPath()
2025-04-29T12:24:56.3668493Z       /home/runner/work/openbao/openbao/vault/namespaces_oss.go:36 +0x46
2025-04-29T12:24:56.3669440Z   github.com/openbao/openbao/http.handler.rateLimitQuotaWrapping.func49()
2025-04-29T12:24:56.3670606Z       /home/runner/work/openbao/openbao/http/util.go:159 +0xd8
2025-04-29T12:24:56.3671216Z   net/http.HandlerFunc.ServeHTTP()
2025-04-29T12:24:56.3672185Z       /opt/hostedtoolcache/go/1.24.2/x64/src/net/http/server.go:2294 +0x47
2025-04-29T12:24:56.3673023Z   github.com/openbao/openbao/http.wrapGenericHandler.func1()
2025-04-29T12:24:56.3674156Z       /home/runner/work/openbao/openbao/http/handler.go:437 +0x1999
2025-04-29T12:24:56.3674800Z   net/http.HandlerFunc.ServeHTTP()
2025-04-29T12:24:56.3675644Z       /opt/hostedtoolcache/go/1.24.2/x64/src/net/http/server.go:2294 +0x47
2025-04-29T12:24:56.3676607Z   github.com/openbao/openbao/http.handler.wrapMaxRequestSizeHandler.func50()
2025-04-29T12:24:56.3677634Z       /home/runner/work/openbao/openbao/http/util.go:145 +0x1fa
2025-04-29T12:24:56.3678266Z   net/http.HandlerFunc.ServeHTTP()
2025-04-29T12:24:56.3679136Z       /opt/hostedtoolcache/go/1.24.2/x64/src/net/http/server.go:2294 +0x47
2025-04-29T12:24:56.3680096Z   github.com/openbao/openbao/http.handler.PrintablePathCheckHandler.func51()
2025-04-29T12:24:56.3681355Z       /home/runner/go/pkg/mod/github.com/hashicorp/go-cleanhttp@v0.5.2/handlers.go:42 +0xb4
2025-04-29T12:24:56.3682156Z   net/http.HandlerFunc.ServeHTTP()
2025-04-29T12:24:56.3683010Z       /opt/hostedtoolcache/go/1.24.2/x64/src/net/http/server.go:2294 +0x47
2025-04-29T12:24:56.3683695Z   net/http.serverHandler.ServeHTTP()
2025-04-29T12:24:56.3684723Z       /opt/hostedtoolcache/go/1.24.2/x64/src/net/http/server.go:3301 +0x2a1
2025-04-29T12:24:56.3685420Z   net/http.initALPNRequest.ServeHTTP()
2025-04-29T12:24:56.3686298Z       /opt/hostedtoolcache/go/1.24.2/x64/src/net/http/server.go:3974 +0x358
2025-04-29T12:24:56.3686988Z   net/http.(*initALPNRequest).ServeHTTP()
2025-04-29T12:24:56.3687522Z       <autogenerated>:1 +0x7b
2025-04-29T12:24:56.3687980Z   net/http.Handler.ServeHTTP-fm()
2025-04-29T12:24:56.3688486Z       <autogenerated>:1 +0x67
2025-04-29T12:24:56.3688959Z   net/http.(*http2serverConn).runHandler()
2025-04-29T12:24:56.3689865Z       /opt/hostedtoolcache/go/1.24.2/x64/src/net/http/h2_bundle.go:6529 +0x253
2025-04-29T12:24:56.3690654Z   net/http.(*http2serverConn).scheduleHandler.gowrap1()
2025-04-29T12:24:56.3691616Z       /opt/hostedtoolcache/go/1.24.2/x64/src/net/http/h2_bundle.go:6463 +0x5d
```

@driif @satoqz it looks like this would be another vote in favor of moving quota enforcement and namespace resolution into core, once unseal has finished. 
author_association: MEMBER
user: login: satoqz
created_at: 2025-04-29T14:36:54Z
body: @cipherboy ah, that race shouldn't occur in either of #1262 or #1279. The call from `http.rateLimitQuotaWrapping()` into `core.NamespaceByPath()` does not read-lock core's state, so accessing `c.namespaceStore` is bad.

I wonder what the best Git-strategy is here? Optimally we would merge #1279 first and then rebase this on top, if we want to merge this before #1279 however then the below should suffice:

```go
func (c *Core) NamespaceByPath(ctx context.Context, path string) (*namespace.Namespace, string) {
        c.stateLock.RLock()
        defer c.stateLock.RUnlock()
	return c.namespaceStore.GetNamespaceByLongestPrefix(ctx, path)
}
```
author_association: MEMBER
user: login: cipherboy
created_at: 2025-04-29T15:28:56Z
body: @satoqz If we're happy with this PR, I'm fine merging it as-is and then fixing the race test in #1279 or a later PR. My 2c.

But would like another review on this before I look at it. :-) 
author_association: MEMBER
user: login: cipherboy
created_at: 2025-04-30T20:33:54Z
body: Given my nit is small, I'll go ahead and merge and we can improve in a subsequent follow-up when we review #1165. 
author_association: MEMBER
user: login: cipherboy
created_at: 2025-04-30T20:34:16Z
body: Thank you @voigt and @phyrog! 
author_association: MEMBER