internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: jupyterhub/zero-to-jupyterhub-k8s

Issue:
Breaking change: hub.extraConfig no longer works as a simple string
In v0.7, the following works:

hub:
  extraConfig: |
    print("hi", flush=True)

In 0.8, this fails with:

```
      File "/srv/jupyterhub_config.py", line 424, in <module>
       for key, config_py in sorted(get_config('hub.extraConfig', {}).items()):
   AttributeError: 'str' object has no attribute 'items'
```

We *should* be backwards compatible, since it's a small check.


Discussion:
@yuvipanda, is there a workaround for this? What is the new required structure of extraConfig?
The new structure is:

```
hub:
  extraConfig:
    myConfigName: |
      print("hi", flush=True)
```
Interesting. Relatively easy fix. And myConfigName (the key) can be anything I want? Does this imply that we can have more than one?
the key can be anything, you can indeed have multiple ones.

this allows helm to merge multiple extraConfigs, while previous didnt, for example if you wanted to have a file with secret values and one with non-secret values this would be useful.

Resolving patch:
diff --git a/images/hub/jupyterhub_config.py b/images/hub/jupyterhub_config.py
--- a/images/hub/jupyterhub_config.py
+++ b/images/hub/jupyterhub_config.py
@@ -1,5 +1,6 @@
 import os
 import re
+import sys
 
 from tornado.httpclient import AsyncHTTPClient
 from kubernetes import client
@@ -421,5 +422,33 @@ def camelCaseify(s):
     c.Spawner.debug = True
 
 
-for key, config_py in sorted(get_config('hub.extraConfig', {}).items()):
+extra_config = get_config('hub.extraConfig', {})
+if isinstance(extra_config, str):
+    from textwrap import indent, dedent
+    msg = dedent(
+    """
+    hub.extraConfig should be a dict of strings,
+    but found a single string instead.
+
+    The keys can be anything identifying the
+    block of extra configuration.
+
+    Try this instead:
+
+        hub:
+          extraConfig:
+            myConfig: |
+              {}
+    """
+    )
+    print(
+        msg.format(
+            indent(extra_config, ' ' * 10).lstrip()
+        ),
+        file=sys.stderr
+    )
+    extra_config = {'deprecated string': extra_config}
+
+for key, config_py in sorted(extra_config.items()):
+    print("Loading extra config: %s" % key)
     exec(config_py)


Test patch:
Repository: jupyterhub/zero-to-jupyterhub-k8s

Issue:
PR Discussion - mounting jupyterhub_config.py
# Mounting jupyterhub_config.py into the hub container
We have a configuration file that is built into the image. This makes for a more complicated and slower local development, CI builds, and provides no real benefit over writing it to a ConfigMap that we mount as far as I can tell.

## Relevant understanding
- ConfigMap and Secrets are limited to 1 MB in file size as that is what ETCD which is used by k8s is limited to. Our current configmap and secret for the hub is currently less than 30 kB together, and the jupyterhub_config.py is 18 kB, so we have a large margin here.

## PR work to be done (#1407)
- [x] Create or reuse a helm template .yaml file for a k8s ConfigMap resource, that reads the content of the jupyterhub_config.py and puts it into the `data.jupyterhub_config.py` key's value.
- [x] Mount it on the hub pod.
  - Don't override the full folder where you mount it by mounting the full ConfigMap and all its key's under `data` to a folder. Instead mount the specific key directly to `/etc/jupyterhub/jupyterhub_config.py`. To accomplish this, use subPath to select a specific key to mount and set the mountPath to a filename instead of a directory.
    ```yaml
              volumeMounts:
                - mountPath: /etc/jupyterhub/jupyterhub_config.py
                  name: <config-volume-name-here>
                  subPath: jupyterhub_config.py
    ```
  - [x] Make sure to verify this works and the file can be read as it should, or if we need to also mount it as a specific user or similar.
- [x] Repeat the idea for z2jh.py and cull_idle_servers.py


Discussion:

Resolving patch:
diff --git a/images/hub/cull_idle_servers.py b/jupyterhub/files/hub/cull_idle_servers.py
similarity index 100%
rename from images/hub/cull_idle_servers.py
rename to jupyterhub/files/hub/cull_idle_servers.py
diff --git a/images/hub/jupyterhub_config.py b/jupyterhub/files/hub/jupyterhub_config.py
similarity index 99%
rename from images/hub/jupyterhub_config.py
rename to jupyterhub/files/hub/jupyterhub_config.py
--- a/images/hub/jupyterhub_config.py
+++ b/jupyterhub/files/hub/jupyterhub_config.py
@@ -359,7 +359,8 @@ def camelCaseify(s):
 
 if get_config('cull.enabled', False):
     cull_cmd = [
-        '/usr/local/bin/cull_idle_servers.py',
+        'python3',
+        '/etc/jupyterhub/cull_idle_servers.py',
     ]
     base_url = c.JupyterHub.get('base_url', '/')
     cull_cmd.append(
diff --git a/images/hub/z2jh.py b/jupyterhub/files/hub/z2jh.py
similarity index 100%
rename from images/hub/z2jh.py
rename to jupyterhub/files/hub/z2jh.py


Test patch:
Repository: jupyterhub/zero-to-jupyterhub-k8s

Issue:
Document the fact that access to cloud metadata is disabled by default
It's not immediately obvious that `iptables` are overridden with an init container, blocking access to EC2 metadata.

This should be documented somewhere. Perhaps in https://zero-to-jupyterhub.readthedocs.io/en/latest/amazon/step-zero-aws.html.


Discussion:
I'd be happy to write something up, can one of the maintainers assign the ticket to me?
@cam72cam are you working on this issue?
@yuvipanda could I pick this issue, if no one else is working on it?

Thanks!
@metonymic-smokey as it has passed long time, you can certainly work on this! :heart:!

Note that I think this is a Kubernetes issue in general, at least on GKE as well. I don't understand all this so well either other than we block network access from our user pods to avoid an issue.
@consideRatio @wookasz [this page](https://zero-to-jupyterhub.readthedocs.io/en/v0.5-doc/security.html) in the Security docs touches upon something similar in the Cloud Metadata server security section. It has a brief on how to run `iptables` in an init container, which could potentially set specific firewall rules for the user pod.

For a start, I've added a brief note about init containers overriding `iptables` and linked to the page referred above. Would love more pointers on what I could add!

Another issue with the page: The [page](https://zero-to-jupyterhub.readthedocs.io/en/latest/amazon/step-zero-aws.html) referred above has some commands misaligned with their code blocks and some minor errors. I've tried to fix those here too. 

Thanks!

Resolving patch:
diff --git a/doc/source/conf.py b/doc/source/conf.py
--- a/doc/source/conf.py
+++ b/doc/source/conf.py
@@ -138,7 +138,19 @@ def parse_schema(d, md=[], depth=0, pre=''):
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
+
 html_theme = 'pydata_sphinx_theme'
+html_theme_options = {
+    "github_url": "https://github.com/jupyterhub/zero-to-jupyterhub-k8s/",
+    "use_edit_page_button": True,
+}
+html_context = {
+    "github_user": "jupyterhub",
+    "github_repo": "zero-to-jupyterhub-k8s",
+    "github_version": "master",
+    "doc_path":"doc",
+}
+
 html_favicon = '_static/images/logo/favicon.ico'
 html_logo = '_static/images/logo/logo.png'
 


Test patch:
Repository: jupyterhub/zero-to-jupyterhub-k8s

Issue:
CI: linkcheck of our docs fails
### Bug description
Our test fails [like this](https://travis-ci.org/github/jupyterhub/zero-to-jupyterhub-k8s/jobs/699287339), which is initialized as described in `.travis.yml` under `docs:linkcheck`.

```
/home/travis/build/jupyterhub/zero-to-jupyterhub-k8s/doc/source/customizing/user-environment.rst:4: WARNING: Duplicate explicit target name: "jupyterlab".
```

### How to reproduce

I think something like this, but check `.travis.yml` for details on how it is done there.

- Clone repo, enter the `doc/` folder.
- Install doc-requirements.txt with pip.
- `make linkcheck`


Discussion:

Resolving patch:
diff --git a/doc/source/conf.py b/doc/source/conf.py
--- a/doc/source/conf.py
+++ b/doc/source/conf.py
@@ -77,9 +77,6 @@ def setup(app):
 # The suffix(es) of source filenames.
 source_suffix = ['.rst', '.md']
 
-# Add any paths that contain templates here, relative to this directory.
-templates_path = ['_templates']
-
 
 # -- Generate the Helm chart configuration reference from a schema file ------
 


Test patch:
Repository: jupyterhub/zero-to-jupyterhub-k8s

Issue:
/usr/local/etc/jupyterhub/jupyterhub_config.d subdirectory made jupyterhub fail to start
I mounted extra configuration to the /usr/local/etc/jupyterhub/jupyterhub_config.d folder. Using a k8s configmap without `subPath` in the volumeMount, it meant that I not only added a my_config.py file, but k8s also created a hidden folder as it does.

This led to z2jh's jupyterhub_config.py crashed while loading files from the jupyterhub_config.d folder, as it detected a folder and got upset.

```
      File "/usr/local/etc/jupyterhub/jupyterhub_config.py", line 415, in <module>
        with open(f"{config_dir}/{file_name}") as f:
    IsADirectoryError: [Errno 21] Is a directory: '/usr/local/etc/jupyterhub/jupyterhub_config.d/..2021_02_02_10_04_22.654959900'
```

@minrk @yuvipanda do you think we should ignore or include content within subdirectories of the jupyterhub_config.d folder?

## Action point
- [x] Decide on how to act regarding directories under the jupyterhub_config.d folder - ignore and warn or load files recursively?
- [x] Implement decision


Discussion:
I don't think we should iterate recursively. This should be fixed pretty straightforwardly by changing from `os.listdir(config_dir)` to list all files to `glob.glob(f"{config_dir}/*.py")`. We shouldn't ever be `exec`ing any files that aren't .py.
Probably `sorted(glob.glob(...))` for good measure, to allow consistent loading in lexicographical order.
Agree with @minrk 

Resolving patch:
diff --git a/jupyterhub/files/hub/jupyterhub_config.py b/jupyterhub/files/hub/jupyterhub_config.py
--- a/jupyterhub/files/hub/jupyterhub_config.py
+++ b/jupyterhub/files/hub/jupyterhub_config.py
@@ -1,3 +1,4 @@
+import glob
 import os
 import re
 import sys
@@ -410,9 +411,10 @@ def camelCaseify(s):
 # load /usr/local/etc/jupyterhub/jupyterhub_config.d config files
 config_dir = "/usr/local/etc/jupyterhub/jupyterhub_config.d"
 if os.path.isdir(config_dir):
-    for file_name in sorted(os.listdir(config_dir)):
+    for file_path in sorted(glob.glob(f"{config_dir}/*.py")):
+        file_name = os.path.basename(file_path)
         print(f"Loading {config_dir} config: {file_name}")
-        with open(f"{config_dir}/{file_name}") as f:
+        with open(file_path) as f:
             file_content = f.read()
         # compiling makes debugging easier: https://stackoverflow.com/a/437857
         exec(compile(source=file_content, filename=file_name, mode="exec"))


Test patch:
Repository: jupyterhub/zero-to-jupyterhub-k8s

Issue:
hub.extraConfig as a string is not supported - update docs and remove potential related logic
As reported by @danielballan in https://github.com/jupyterhub/zero-to-jupyterhub-k8s/pull/2062#issue-578712676 (point 1), it may not be supported for us to set hub.extraConfig to a string.

> ```yaml
> hub:
>   extraConfig: |
>     import time
>     c.Spawner.environment += {
>        "CURRENT_TIME": str(time.time())
>     }
> ```
> 
> The documentation claims that extraConfig accepts either a dict of strings or a single string (of Python code), but it actually only accepts the first one. If you give it a single string, following the documented example, it warns
> coalesce.go:196: warning: cannot overwrite table with non table for extraConfig (map[])
> 
> and it does not take effect. See #653 which seems to observe the same issue.

## Action point
1. Reproduce and conclude that helm 3 and 0.11.1 that this doesn't work.

   ```yaml
   hub:
     extraConfig: |
       print("THIS SHOULD BE SEEN IN LOGS ON STARTUP, IS IT?")
   ```



Discussion:

Resolving patch:
diff --git a/jupyterhub/files/hub/jupyterhub_config.py b/jupyterhub/files/hub/jupyterhub_config.py
--- a/jupyterhub/files/hub/jupyterhub_config.py
+++ b/jupyterhub/files/hub/jupyterhub_config.py
@@ -442,37 +442,7 @@ def camelCaseify(s):
         cfg.pop("keys", None)
     c[app].update(cfg)
 
-# execute hub.extraConfig string
-extra_config = get_config("hub.extraConfig", {})
-if isinstance(extra_config, str):
-    from textwrap import indent, dedent
-
-    msg = dedent(
-        """
-    hub.extraConfig should be a dict of strings,
-    but found a single string instead.
-
-    extraConfig as a single string is deprecated
-    as of the jupyterhub chart version 0.6.
-
-    The keys can be anything identifying the
-    block of extra configuration.
-
-    Try this instead:
-
-        hub:
-          extraConfig:
-            myConfig: |
-              {}
-
-    This configuration will still be loaded,
-    but you are encouraged to adopt the nested form
-    which enables easier merging of multiple extra configurations.
-    """
-    )
-    print(msg.format(indent(extra_config, " " * 10).lstrip()), file=sys.stderr)
-    extra_config = {"deprecated string": extra_config}
-
-for key, config_py in sorted(extra_config.items()):
+# execute hub.extraConfig entries
+for key, config_py in sorted(get_config("hub.extraConfig", {}).items()):
     print("Loading extra config: %s" % key)
     exec(config_py)


Test patch:
Repository: jupyterhub/zero-to-jupyterhub-k8s

Issue:
resourceQuotas plus blockWithIptables prevents user pods from spawning
<!-- Thank you for contributing. These HTML comments will not render in the issue, but you can delete them once you've read them if you prefer! -->

### Bug description
If you are using user namespaces with resourceQuotas set, then if you are also running with singleuser.cloudMetadata.blockWithIptables on, then the initContainer spawned to do the iptables magic won't spawn, because it lacks resource limits, and therefore the user pod will never spawn.

#### Expected behaviour
I expected to get a user pod...

#### Actual behaviour
...but I didn't.

### Suggested solution
Set resource limits on the initContainer.  It doesn't have much to do, so "way smaller than any reasonable quota" should work fine.


Discussion:
Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs:
<br>If you haven't done so already, check out [Jupyter's Code of Conduct](https://github.com/jupyter/governance/blob/master/conduct/code_of_conduct.md). Also, please try to follow the issue template as it helps other other community members to contribute more effectively.
![welcome](https://raw.githubusercontent.com/jupyterhub/.github/master/images/welcome.jpg)
You can meet the other [Jovyans](https://jupyter.readthedocs.io/en/latest/community/content-community.html?highlight=jovyan#what-is-a-jovyan) by joining our [Discourse forum](http://discourse.jupyter.org/). There is also an intro thread there where you can stop by and say Hi! :wave:
<br>Welcome to the Jupyter community! :tada:


Resolving patch:
diff --git a/jupyterhub/files/hub/jupyterhub_config.py b/jupyterhub/files/hub/jupyterhub_config.py
--- a/jupyterhub/files/hub/jupyterhub_config.py
+++ b/jupyterhub/files/hub/jupyterhub_config.py
@@ -406,6 +406,7 @@ def camelCaseify(s):
     # Use iptables to block access to cloud metadata by default
     network_tools_image_name = get_config("singleuser.networkTools.image.name")
     network_tools_image_tag = get_config("singleuser.networkTools.image.tag")
+    network_tools_resources = get_config("singleuser.networkTools.resources")
     ip_block_container = client.V1Container(
         name="block-cloud-metadata",
         image=f"{network_tools_image_name}:{network_tools_image_tag}",
@@ -423,6 +424,7 @@ def camelCaseify(s):
             run_as_user=0,
             capabilities=client.V1Capabilities(add=["NET_ADMIN"]),
         ),
+        resources=network_tools_resources,
     )
 
     c.KubeSpawner.init_containers.append(ip_block_container)


Test patch:
Repository: jupyterhub/zero-to-jupyterhub-k8s

Issue:
Adopt kubernetes_asyncio client for KubeSpawner 3.x
KubeSpawner 3.x will use tomplus/kubernetes_asyncio instead of the official synchronous K8s client.  This change reflects that difference in the underlying implementation.


Discussion:
Thanks for submitting your first pull request! You are awesome! :hugs:
<br>If you haven't done so already, check out [Jupyter's Code of Conduct](https://github.com/jupyter/governance/blob/master/conduct/code_of_conduct.md). Also, please make sure you followed the pull request template, as this will help us review your contribution more quickly.
![welcome](https://raw.githubusercontent.com/jupyterhub/.github/master/images/welcome.jpg)
You can meet the other [Jovyans](https://jupyter.readthedocs.io/en/latest/community/content-community.html?highlight=jovyan#what-is-a-jovyan) by joining our [Discourse forum](http://discourse.jupyter.org/). There is also a intro thread there where you can stop by and say Hi! :wave:
<br>Welcome to the Jupyter community! :tada:

(not going to pass tests until KubeSpawner 3.x is adopted)
@athornton can you document in this PRs original post a copy-pasted example of an error message supressed by this, and describe that this PR patches a library to supress those messages stemming from the upcoming switch to kubespawner using a different kubernetes api-server client libary that relies on async functions.

I'm thinking this kind of patch is the last resort solution. I want to dig into the details and come to the conclusion it really seem to be the last resort we have before approving a workaround like this in a downstream project.

Practically, I'm thinking of this as a kubespawner issue to be resolved before wanting to consider a z2jh workaround.
I opened https://github.com/jupyterhub/kubespawner/issues/578, let's switch to discuss things there!
https://github.com/jupyterhub/kubespawner/pull/579 is a much better way to do this, so I'm withdrawing the monkeypatch.

The remaining change should be uncontroversial.
@athornton I'd like PR titles and descriptions to be clear and easy to understand for a) people looking into active PRs in general to contribute with review effort, b) anyone reading a changelog.

We can merge this change along with a kubespawner version bump at a later time (I don't want to put our main branch in a broken state), but please update your PR title/description!

Resolving patch:
diff --git a/jupyterhub/files/hub/jupyterhub_config.py b/jupyterhub/files/hub/jupyterhub_config.py
--- a/jupyterhub/files/hub/jupyterhub_config.py
+++ b/jupyterhub/files/hub/jupyterhub_config.py
@@ -6,7 +6,7 @@
 from binascii import a2b_hex
 
 from tornado.httpclient import AsyncHTTPClient
-from kubernetes import client
+from kubernetes_asyncio import client
 from jupyterhub.utils import url_path_join
 
 # Make sure that modules placed in the same directory as the jupyterhub config are added to the pythonpath


Test patch:
Repository: jupyterhub/zero-to-jupyterhub-k8s

Issue:
kubespawner: pass "allow_privilege_escalation"
In order to address https://github.com/jupyterhub/zero-to-jupyterhub-k8s/issues/2429#issuecomment-1096060016 and permit `sudo` on z2jh installations without jupyterhub_config level changes, I believe we'll need to be able to override the `allow_privilege_escalation` from values.yaml


Discussion:

Resolving patch:
diff --git a/jupyterhub/files/hub/jupyterhub_config.py b/jupyterhub/files/hub/jupyterhub_config.py
--- a/jupyterhub/files/hub/jupyterhub_config.py
+++ b/jupyterhub/files/hub/jupyterhub_config.py
@@ -142,6 +142,7 @@ def camelCaseify(s):
     ("events_enabled", "events"),
     ("extra_labels", None),
     ("extra_annotations", None),
+    # ("allow_privilege_escalation", None), # Managed manually below
     ("uid", None),
     ("fs_gid", None),
     ("service_account", "serviceAccountName"),
@@ -179,6 +180,15 @@ def camelCaseify(s):
 
     c.KubeSpawner.image = image
 
+# allow_privilege_escalation defaults to False in KubeSpawner 2+. Since its a
+# property where None, False, and True all are valid values that users of the
+# Helm chart may want to set, we can't use the set_config_if_not_none helper
+# function as someone may want to override the default False value to None.
+#
+c.KubeSpawner.allow_privilege_escalation = get_config(
+    "singleuser.allowPrivilegeEscalation"
+)
+
 # Combine imagePullSecret.create (single), imagePullSecrets (list), and
 # singleuser.image.pullSecrets (list).
 image_pull_secrets = []


Test patch:
Repository: jupyterhub/zero-to-jupyterhub-k8s

Issue:
Move changelog to docs website
I suggest we make the changelog become part of the rendered documentation instead of a dedicated markdown file to be read from GitHub, and that we let CHANGELOG.md point to this rendered view.

Note for reference that jupyterhub/jupyterhub repo only has rendered website documentation currently, [see it here](https://jupyterhub.readthedocs.io/en/stable/changelog.html).

![image](https://user-images.githubusercontent.com/3837114/119242003-c4e93280-bb5a-11eb-9fdf-c951204624bc.png)

---

I think there are two key benefits of this:
1. We can get navigation assistance out of the box
2. We can make use of myst formatting, such as note/warning callouts


Discussion:
I think this is a good idea.

Resolving patch:
diff --git a/docs/source/conf.py b/docs/source/conf.py
--- a/docs/source/conf.py
+++ b/docs/source/conf.py
@@ -228,6 +228,9 @@ def parse_schema(d, md=[], depth=0, pre=""):
 linkcheck_ignore = [
     r"(.*)github\.com(.*)#",  # javascript based anchors
     r"(.*)/#%21(.*)/(.*)",  # /#!forum/jupyter - encoded anchor edge case
+    r"https://github.com/[^/]*$",  # too many github usernames / searches in changelog
+    "https://github.com/jupyterhub/zero-to-jupyterhub-k8s/pull/",  # too many PRs in changelog
+    "https://github.com/jupyterhub/zero-to-jupyterhub-k8s/compare/",  # too many comparisons in changelog
     "https://your-domain.com",  # example
     "https://your-domain-name.com",  # example
     "https://kubernetes.io/docs/tutorials/kubernetes-basics/",  # works


Test patch: