Repository: jupyterhub/zero-to-jupyterhub-k8s

Issue:
Make 'pack' schedulerStrategy work properly
When running jupytehub on k8s, we ideally want the scheduler to *pack* pods together onto nodes as much as possible. This helps with autoscaling a lot.

We currently *sortof* enable this by setting `schedulerStrategy: pack`. It uses podAffinity to do its thing. But podAffinity does is not weighted - if you have two nodes with 2 and 80 pods each, a new pod can get scheduled on either one. This limits its effectiveness a fair bit!

After digging around and talking to more people in the kubernetes community, I believe a real solution is:

1. Run another copy of the `kube-scheduler` in our cluster, with a `--scheduler-name` set to something custom (so this will only schedule our pods)
2. Put the name of our kube-scheduler in `schedulerName` for all our pods
3. Use a custom policy.json file to configure kube-scheduler to use https://github.com/kubernetes/kubernetes/blob/master/pkg/scheduler/algorithm/priorities/most_requested.go as a priority function. This should do what we need it to do!

https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/ has more general info on this approach.

Defaults of the current algorithms in https://github.com/kubernetes/kubernetes/blob/master/pkg/scheduler/algorithmprovider/defaults/defaults_test.go


Discussion:
/cc @consideRatio who has been looking into this.

also /cc @minrk and @betatim - this will also help mybinder.org a lot if we can make this happen!
A lot of thanks to @msau42 and @bsalamat from the Kubernetes Slack for helping me out and steering me away from a more complex setup involving Scheduler Extenders!
https://github.com/kubernetes/kubernetes/pull/59401/files has info on the current policy.json defaults, so we can just take 'em and modify.
@yuvipanda great investigation and I really appreciate that you keep me updated with your findings!

> We currently sortof enable this by setting schedulerStrategy: pack. It uses podAffinity to do its thing. But podAffinity does is not weighted - if you have two nodes with 2 and 80 pods each, a new pod can get scheduled on either one. This limits its effectiveness a fair bit!

If you have two nodes: one has only a few pods while the other node has more pods, the scheduling is still random? It will not add together the weights of each singleuser pod it finds or something similar?

![image](https://user-images.githubusercontent.com/3837114/36713701-2cf2a916-1b8f-11e8-8623-6d29e1d0154d.png)
I believe you're correct - user pods have an affinity for nodes that have > 0 pods on them, but no concept of "I should compare how _many_ pods are on each node and go to the one with the most pods". Which is super annoying :-P
@consideRatio the default does not, but if you can tweak the policy.json file you can make it! https://github.com/kubernetes/kubernetes/tree/master/pkg/scheduler/algorithm/priorities has list of priorities that can be tweaked.

Resolving patch:
diff --git a/images/hub/jupyterhub_config.py b/images/hub/jupyterhub_config.py
--- a/images/hub/jupyterhub_config.py
+++ b/images/hub/jupyterhub_config.py
@@ -72,6 +72,7 @@
 service_account_name = get_config('singleuser.service-account-name', None)
 if service_account_name:
     c.KubeSpawner.service_account = service_account_name
+c.KubeSpawner.scheduler_name = get_config('singleuser.scheduler-name', "")
 
 c.KubeSpawner.node_selector = get_config('singleuser.node-selector')
 # Configure dynamically provisioning pvc
@@ -336,41 +337,6 @@ def generate_user_name(spawner):
 
     c.KubeSpawner.init_containers.append(ip_block_container)
 
-scheduler_strategy = get_config('singleuser.scheduler-strategy', 'spread')
-
-if scheduler_strategy == 'pack':
-    # FIXME: Support setting affinity directly in KubeSpawner
-    c.KubeSpawner.extra_pod_config = {
-        'affinity': {
-            'podAffinity': {
-                'preferredDuringSchedulingIgnoredDuringExecution': [{
-                    'weight': 50,
-                    'podAffinityTerm': {
-                        'labelSelector': {
-                            'matchExpressions': [{
-                                'key': 'component',
-                                'operator': 'In',
-                                'values': ['hub']
-                            }]
-                        },
-                        'topologyKey': 'kubernetes.io/hostname'
-                    }
-                }, {
-                    'weight': 5,
-                    'podAffinityTerm': {
-                        'labelSelector': {
-                            'matchExpressions': [{
-                                'key': 'component',
-                                'operator': 'In',
-                                'values': ['singleuser-server']
-                            }]
-                        },
-                        'topologyKey': 'kubernetes.io/hostname'
-                    }
-                }],
-            }
-        }
-    }
 
 if get_config('debug.enabled', False):
     c.JupyterHub.log_level = 'DEBUG'


Test patch: