Repository: kubernetes
Issue: Don't use namespace index when btree is enabled
Labels: sig/api-machinery,kind/feature,triage/accepted

### What would you like to be added?

There are two competing improvements to perfomance of namespaced LIST: 
* StorageNamespaceIndex - introduced to Beta in 1.30, configures pod resource to index by namespace.
* BtreeWatchCache - introduced to Beta in 1.31, allows prefix based listing which includes namespace. Removes need to filter and sort results from cache.

By default apiserver watch cache will try to use index if available, so `StorageNamespaceIndex` will be preferred. However, with introduction of btree it is faster to list from btree instead of index.

Benchmarks results, for current scalability envelope:
```
$ benchstat -col /Store,/Indexed -filter '/Paginate:0  -/RV:Exact' ./benchmark/master2.txt
goos: linux
goarch: amd64
pkg: k8s.io/apiserver/pkg/storage/cacher
cpu: AMD Ryzen Threadripper PRO 3945WX 12-Cores     
                                                                                                │                        Btree                        │                                       Map                                       │
                                                                                                │     true     │                false                 │                true                 │                   false                   │
                                                                                                │    sec/op    │    sec/op     vs base                │   sec/op     vs base                │     sec/op      vs base                   │
StoreList/Namespaces=10000/Pods=150000/Nodes=5000/RV=/Scope=Namespace/Paginate=0-24               20.77µ ± 10%   20.14µ ± 13%        ~ (p=1.000 n=10)   19.73µ ± 6%        ~ (p=0.105 n=10)   1067.34µ ± 10%   +5037.73% (p=0.000 n=10)
StoreList/Namespaces=10000/Pods=150000/Nodes=5000/RV=NotOlderThan/Scope=Namespace/Paginate=0-24   3.943µ ±  6%   3.928µ ±  6%        ~ (p=0.218 n=10)   3.665µ ± 3%   -7.05% (p=0.000 n=10)   944.641µ ±  1%  +23857.41% (p=0.000 n=10)
StoreList/Namespaces=50/Pods=150000/Nodes=5000/RV=/Scope=Namespace/Paginate=0-24                  303.3µ ±  2%   258.2µ ±  2%  -14.85% (p=0.000 n=10)   340.1µ ± 3%  +12.15% (p=0.000 n=10)    1668.6µ ±  4%    +450.23% (p=0.000 n=10)
StoreList/Namespaces=50/Pods=150000/Nodes=5000/RV=NotOlderThan/Scope=Namespace/Paginate=0-24      286.2µ ±  3%   234.7µ ±  1%  -17.99% (p=0.000 n=10)   326.9µ ± 2%  +14.22% (p=0.000 n=10)    1347.7µ ±  4%    +370.91% (p=0.000 n=10)
StoreList/Namespaces=100/Pods=110000/Nodes=1000/RV=/Scope=Namespace/Paginate=0-24                 125.3µ ±  2%   112.3µ ±  5%  -10.38% (p=0.000 n=10)   137.5µ ± 2%   +9.81% (p=0.000 n=10)    1395.1µ ±  8%   +1013.78% (p=0.000 n=10)
StoreList/Namespaces=100/Pods=110000/Nodes=1000/RV=NotOlderThan/Scope=Namespace/Paginate=0-24     120.6µ ±  2%   113.2µ ±  1%   -6.13% (p=0.000 n=10)   133.8µ ± 1%  +10.92% (p=0.000 n=10)    1719.1µ ±  5%   +1325.35% (p=0.000 n=10)
geomean                                                                                           68.94µ         62.73µ         -9.02%                  72.72µ        +5.48%                    1.326m         +1823.40%
                                                                                                │                       Btree                        │                                       Map                                       │
                                                                                                │     true     │                false                │                true                 │                   false                   │
                                                                                                │     B/op     │     B/op      vs base               │     B/op      vs base               │      B/op       vs base                   │
StoreList/Namespaces=10000/Pods=150000/Nodes=5000/RV=/Scope=Namespace/Paginate=0-24               28.43Ki ± 3%   27.91Ki ± 0%  -1.82% (p=0.000 n=10)   28.41Ki ± 0%       ~ (p=0.671 n=10)   2371.96Ki ± 0%   +8243.54% (p=0.000 n=10)
StoreList/Namespaces=10000/Pods=150000/Nodes=5000/RV=NotOlderThan/Scope=Namespace/Paginate=0-24   11.69Ki ± 0%   11.16Ki ± 0%  -4.56% (p=0.000 n=10)   11.69Ki ± 0%       ~ (p=0.065 n=10)   2355.11Ki ± 0%  +20047.27% (p=0.000 n=10)
StoreList/Namespaces=50/Pods=150000/Nodes=5000/RV=/Scope=Namespace/Paginate=0-24                  1.790Mi ± 0%   1.742Mi ± 0%  -2.64% (p=0.000 n=10)   1.790Mi ± 0%       ~ (p=0.579 n=10)     4.031Mi ± 0%    +125.24% (p=0.000 n=10)
StoreList/Namespaces=50/Pods=150000/Nodes=5000/RV=NotOlderThan/Scope=Namespace/Paginate=0-24      1.773Mi ± 0%   1.726Mi ± 0%  -2.66% (p=0.000 n=10)   1.773Mi ± 0%       ~ (p=0.424 n=10)     4.015Mi ± 0%    +126.46% (p=0.000 n=10)
StoreList/Namespaces=100/Pods=110000/Nodes=1000/RV=/Scope=Namespace/Paginate=0-24                 690.1Ki ± 0%   671.9Ki ± 0%  -2.64% (p=0.000 n=10)   690.3Ki ± 0%       ~ (p=0.089 n=10)    2391.7Ki ± 0%    +246.56% (p=0.000 n=10)
StoreList/Namespaces=100/Pods=110000/Nodes=1000/RV=NotOlderThan/Scope=Namespace/Paginate=0-24     673.3Ki ± 0%   655.0Ki ± 0%  -2.72% (p=0.000 n=10)   673.3Ki ± 0%       ~ (p=0.516 n=10)    2374.9Ki ± 0%    +252.74% (p=0.000 n=10)
geomean                                                                                           283.0Ki        274.9Ki       -2.84%                  283.0Ki       +0.00%                    2.785Mi         +907.87%

                                                                                                │                     Btree                      │                                  Map                                  │
                                                                                                │    true    │               false               │               true                │               false               │
                                                                                                │ allocs/op  │ allocs/op   vs base               │ allocs/op   vs base               │ allocs/op   vs base               │
StoreList/Namespaces=10000/Pods=150000/Nodes=5000/RV=/Scope=Namespace/Paginate=0-24               325.0 ± 0%   319.0 ± 0%  -1.85% (p=0.000 n=10)   325.0 ± 0%       ~ (p=0.736 n=10)   322.0 ± 1%  -0.92% (p=0.016 n=10)
StoreList/Namespaces=10000/Pods=150000/Nodes=5000/RV=NotOlderThan/Scope=Namespace/Paginate=0-24   68.00 ± 0%   62.00 ± 0%  -8.82% (p=0.000 n=10)   68.00 ± 0%       ~ (p=1.000 n=10)   63.00 ± 0%  -7.35% (p=0.000 n=10)
StoreList/Namespaces=50/Pods=150000/Nodes=5000/RV=/Scope=Namespace/Paginate=0-24                  357.0 ± 3%   348.0 ± 2%  -2.52% (p=0.003 n=10)   358.5 ± 2%       ~ (p=0.669 n=10)   343.0 ± 1%  -3.92% (p=0.000 n=10)
StoreList/Namespaces=50/Pods=150000/Nodes=5000/RV=NotOlderThan/Scope=Namespace/Paginate=0-24      88.00 ± 1%   83.00 ± 1%  -5.68% (p=0.000 n=10)   88.00 ± 1%       ~ (p=1.000 n=10)   83.00 ± 0%  -5.68% (p=0.000 n=10)
StoreList/Namespaces=100/Pods=110000/Nodes=1000/RV=/Scope=Namespace/Paginate=0-24                 344.0 ± 2%   340.0 ± 2%  -1.16% (p=0.038 n=10)   347.0 ± 3%       ~ (p=0.117 n=10)   338.0 ± 0%  -1.74% (p=0.001 n=10)
StoreList/Namespaces=100/Pods=110000/Nodes=1000/RV=NotOlderThan/Scope=Namespace/Paginate=0-24     85.00 ± 1%   79.00 ± 0%  -7.06% (p=0.000 n=10)   85.00 ± 0%       ~ (p=0.087 n=10)   79.00 ± 0%  -7.06% (p=0.000 n=10)
geomean                                                                                           165.2        157.6       -4.56%                  165.5       +0.21%                  157.8       -4.48%



```

Those result show:
* Namespace index is hugely beneficial when using map
* With btree disabling namespace index brings 4-17% performance improvement 

/cc @jpbetz @deads2k @wojtek-t 
/sig api-machinery

### Why is this needed?

Simplify and improve performance of storage layer. With btree, namespace indexing is not needed.