Repository: jupyterhub/jupyterhub

Issue:
4.0.0b1's introduced singleuser/templates/page.html but doesn't bundle it with the wheel
I'm testing a lot of combinations on how a user server can be started and such, and found attempting to start jupyterhub 4.0.0b1 together with jupyter_server 1 broke like this, while jupyterhub 3.1.1 + jupyter_server 1 worked.

Ideally, this would not be a breaking change to help users working with misc images upgrade ahead of time.

### Logs from user server

```
kubectl logs jupyter-erik--jh4-2djs1-2dlab
[I 2023-03-02 15:31:11.050 SingleUserLabApp mixins:547] Starting jupyterhub single-user server version 4.0.0b1
[I 2023-03-02 15:31:11.050 SingleUserLabApp mixins:561] Extending jupyterlab.labhubapp.SingleUserLabApp from jupyterlab 3.6.1
[I 2023-03-02 15:31:11.050 SingleUserLabApp mixins:561] Extending jupyter_server.serverapp.ServerApp from jupyter_server 1.23.6
[D 2023-03-02 15:31:11.056 SingleUserLabApp application:190] Searching ['/home/jovyan/.jupyter', '/home/jovyan/.local/etc/jupyter', '/usr/local/etc/jupyter', '/etc/jupyter'] for config files
[D 2023-03-02 15:31:11.056 SingleUserLabApp application:902] Looking for jupyter_config in /etc/jupyter
[D 2023-03-02 15:31:11.056 SingleUserLabApp application:902] Looking for jupyter_config in /usr/local/etc/jupyter
[D 2023-03-02 15:31:11.056 SingleUserLabApp application:902] Looking for jupyter_config in /home/jovyan/.local/etc/jupyter
[D 2023-03-02 15:31:11.056 SingleUserLabApp application:902] Looking for jupyter_config in /home/jovyan/.jupyter
[D 2023-03-02 15:31:11.057 SingleUserLabApp application:902] Looking for jupyter_server_config in /etc/jupyter
[D 2023-03-02 15:31:11.057 SingleUserLabApp application:902] Looking for jupyter_server_config in /usr/local/etc/jupyter
[D 2023-03-02 15:31:11.057 SingleUserLabApp application:902] Looking for jupyter_server_config in /home/jovyan/.local/etc/jupyter
[D 2023-03-02 15:31:11.057 SingleUserLabApp application:902] Looking for jupyter_server_config in /home/jovyan/.jupyter
[D 2023-03-02 15:31:11.058 SingleUserLabApp config_manager:93] Paths used for configuration of jupyter_server_config: 
    	/etc/jupyter/jupyter_server_config.json
[D 2023-03-02 15:31:11.058 SingleUserLabApp config_manager:93] Paths used for configuration of jupyter_server_config: 
    	/usr/local/etc/jupyter/jupyter_server_config.d/jupyter_server_fileid.json
    	/usr/local/etc/jupyter/jupyter_server_config.d/jupyter_server_ydoc.json
    	/usr/local/etc/jupyter/jupyter_server_config.d/jupyterlab.json
    	/usr/local/etc/jupyter/jupyter_server_config.d/nbclassic.json
    	/usr/local/etc/jupyter/jupyter_server_config.d/nbgitpuller.json
    	/usr/local/etc/jupyter/jupyter_server_config.d/notebook_shim.json
    	/usr/local/etc/jupyter/jupyter_server_config.json
[D 2023-03-02 15:31:11.059 SingleUserLabApp config_manager:93] Paths used for configuration of jupyter_server_config: 
    	/home/jovyan/.local/etc/jupyter/jupyter_server_config.json
[D 2023-03-02 15:31:11.059 SingleUserLabApp config_manager:93] Paths used for configuration of jupyter_server_config: 
    	/home/jovyan/.jupyter/jupyter_server_config.json
[D 2023-03-02 15:31:11.089 SingleUserLabApp application:457] Config changed: {'FileContentsManager': {'delete_to_trash': False}, 'SingleUserLabApp': {'default_url': '/lab'}, 'ServerApp': {'jpserver_extensions': <LazyConfigValue value={'jupyter_server_fileid': True, 'jupyter_server_ydoc': True, 'jupyterlab': True, 'nbclassic': True, 'nbgitpuller': True, 'notebook_shim': True}>}}
[I 2023-03-02 15:31:11.090 SingleUserLabApp manager:344] jupyter_server_fileid | extension was successfully linked.
[D 2023-03-02 15:31:11.091 SingleUserLabApp application:457] Config changed: {'FileContentsManager': {'delete_to_trash': False}, 'SingleUserLabApp': {'default_url': '/lab'}, 'ServerApp': {'jpserver_extensions': <LazyConfigValue value={'jupyter_server_fileid': True, 'jupyter_server_ydoc': True, 'jupyterlab': True, 'nbclassic': True, 'nbgitpuller': True, 'notebook_shim': True}>}}
[I 2023-03-02 15:31:11.092 SingleUserLabApp manager:344] jupyter_server_ydoc | extension was successfully linked.
[I 2023-03-02 15:31:11.094 SingleUserLabApp manager:344] jupyterlab | extension was successfully linked.
[I 2023-03-02 15:31:11.096 SingleUserLabApp manager:344] nbclassic | extension was successfully linked.
[I 2023-03-02 15:31:11.096 SingleUserLabApp manager:344] nbgitpuller | extension was successfully linked.
[D 2023-03-02 15:31:11.293 SingleUserLabApp config_manager:93] Paths used for configuration of jupyter_notebook_config: 
    	/home/jovyan/.jupyter/jupyter_notebook_config.json
[D 2023-03-02 15:31:11.294 SingleUserLabApp config_manager:93] Paths used for configuration of jupyter_notebook_config: 
    	/etc/jupyter/jupyter_notebook_config.json
[D 2023-03-02 15:31:11.294 SingleUserLabApp config_manager:93] Paths used for configuration of jupyter_notebook_config: 
    	/usr/local/etc/jupyter/jupyter_notebook_config.d/jupyter_server_ydoc.json
    	/usr/local/etc/jupyter/jupyter_notebook_config.d/jupyterlab.json
    	/usr/local/etc/jupyter/jupyter_notebook_config.d/nbgitpuller.json
    	/usr/local/etc/jupyter/jupyter_notebook_config.json
[D 2023-03-02 15:31:11.294 SingleUserLabApp config_manager:93] Paths used for configuration of jupyter_notebook_config: 
    	/home/jovyan/.local/etc/jupyter/jupyter_notebook_config.json
[D 2023-03-02 15:31:11.294 SingleUserLabApp config_manager:93] Paths used for configuration of jupyter_notebook_config: 
    	/home/jovyan/.jupyter/jupyter_notebook_config.json
[I 2023-03-02 15:31:11.294 SingleUserLabApp manager:344] notebook_shim | extension was successfully linked.
[I 2023-03-02 15:31:11.304 SingleUserLabApp manager:362] notebook_shim | extension was successfully loaded.
[I 2023-03-02 15:31:11.304 FileIdExtension] Configured File ID manager: ArbitraryFileIdManager
[I 2023-03-02 15:31:11.304 FileIdExtension] ArbitraryFileIdManager : Configured root dir: /home/jovyan
[I 2023-03-02 15:31:11.304 FileIdExtension] ArbitraryFileIdManager : Configured database path: /home/jovyan/.local/share/jupyter/file_id_manager.db
[I 2023-03-02 15:31:11.304 FileIdExtension] ArbitraryFileIdManager : Successfully connected to database file.
[I 2023-03-02 15:31:11.304 FileIdExtension] ArbitraryFileIdManager : Creating File ID tables and indices with journal_mode = DELETE
[I 2023-03-02 15:31:11.315 SingleUserLabApp manager:362] jupyter_server_fileid | extension was successfully loaded.
[I 2023-03-02 15:31:11.316 SingleUserLabApp manager:362] jupyter_server_ydoc | extension was successfully loaded.
[I 2023-03-02 15:31:11.316 LabApp] JupyterLab extension loaded from /usr/local/lib/python3.11/site-packages/jupyterlab
[I 2023-03-02 15:31:11.316 LabApp] JupyterLab application directory is /usr/local/share/jupyter/lab
[I 2023-03-02 15:31:11.320 SingleUserLabApp manager:362] jupyterlab | extension was successfully loaded.
[I 2023-03-02 15:31:11.324 SingleUserLabApp manager:362] nbclassic | extension was successfully loaded.
[I 2023-03-02 15:31:11.324 SingleUserLabApp manager:362] nbgitpuller | extension was successfully loaded.
Traceback (most recent call last):
  File "/usr/local/bin/jupyterhub-singleuser", line 8, in <module>

  _   _          _      _
 | | | |_ __  __| |__ _| |_ ___
 | |_| | '_ \/ _` / _` |  _/ -_)
  \___/| .__/\__,_\__,_|\__\___|
       |_|
                                                                           
    sys.exit(main())
Read the migration plan to Notebook 7 to learn about the new features and the actions to take if you are using extensions.

https://jupyter-notebook.readthedocs.io/en/latest/migrate_to_notebook7.html

Please note that updating to Notebook 7 might break some of your extensions.

             ^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jupyterhub/singleuser/app.py", line 76, in main
    return SingleUserLabApp.launch_instance()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jupyter_core/application.py", line 277, in launch_instance
    return super().launch_instance(argv=argv, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/traitlets/config/application.py", line 1042, in launch_instance
    app.initialize(argv)
  File "/usr/local/lib/python3.11/site-packages/jupyterhub/singleuser/mixins.py", line 951, in initialize
    result = super().initialize(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jupyterhub/singleuser/mixins.py", line 608, in initialize
    self.patch_templates()
  File "/usr/local/lib/python3.11/site-packages/jupyterhub/singleuser/mixins.py", line 755, in patch_templates
    with template_dir.joinpath("page.html").open() as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/pathlib.py", line 1044, in open
    return io.open(self, mode, buffering, encoding, errors, newline)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.11/site-packages/jupyterhub/singleuser/templates/page.html'
```


Discussion:
I could not reproduce this in jupyterhub's own test suite by pinning jupyter_server=1 systematically. Is servers tested to spawn successfully become ready with `jupyterlab` + `jupyter_server==1.*`?

This can be reproduced just with a dummy startup of `jupyterhub-singleuser` like this using a container I built based on [z2jh's singleuser-sample](https://github.com/jupyterhub/zero-to-jupyterhub-k8s/blob/580ae7961aa8e11221e3e1d7d2ef59e811276528/images/singleuser-sample/Dockerfile) but first pinning `jupyter_server==1.*` and `jupyterhub==4.0.0b1` and refreezing the requirements.txt file.

```shell
docker run -it --rm -e JUPYTERHUB_API_TOKEN=dummy -e JUPYTERHUB_CLIENT_ID=test docker.io/consideratio/k8s-singleuser-sample:jh4-js1 jupyterhub-singleuser
```

Note that one can replace `jupyterhub-singleuser` with `bash` to debug this inside the container by modifying things first etc.

### Possibly related code

**singeluser/templates/page.html**

https://github.com/jupyterhub/jupyterhub/blob/850f430ad6e0f5f730599d6a4db79fa4eb56d075/jupyterhub/singleuser/templates/page.html#L1-L9

**patch_templates function raising "No such file or directory"**

https://github.com/jupyterhub/jupyterhub/blob/850f430ad6e0f5f730599d6a4db79fa4eb56d075/jupyterhub/singleuser/mixins.py#L753-L765

**SingleUserLabApp from jupyterlab** - [_source code_](https://github.com/jupyterlab/jupyterlab/blob/35611c47e8a44db3786a4bec50475fc077fc0045/jupyterlab/labhubapp.py#L13-L23)

```python
if not os.environ.get("JUPYTERHUB_SINGLEUSER_APP"):
    # setting this env prior to import of jupyterhub.singleuser avoids unnecessary import of notebook
    os.environ["JUPYTERHUB_SINGLEUSER_APP"] = "jupyter_server.serverapp.ServerApp"

try:
    from jupyterhub.singleuser.mixins import make_singleuser_app
except ImportError:
    # backward-compat with jupyterhub < 1.3
    from jupyterhub.singleuser import SingleUserNotebookApp as SingleUserServerApp
else:
    SingleUserServerApp = make_singleuser_app(ServerApp)
```
https://github.com/jupyterhub/jupyterhub/blob/850f430ad6e0f5f730599d6a4db79fa4eb56d075/jupyterhub/singleuser/mixins.py#L753-L756

I added a small `print(template_dir)` statement before erroring, and then concluded that folder didn't exist. It was `/usr/local/lib/python3.11/site-packages/jupyterhub/singleuser/templates` and the parent folder had the following content:

```shell
ls -alh /usr/local/lib/python3.11/site-packages/jupyterhub/singleuser
total 96K
drwxr-xr-x 1 root root 4.0K Mar  6 21:10 .
drwxr-xr-x 1 root root 4.0K Mar  2 15:07 ..
-rw-r--r-- 1 root root 2.6K Mar  2 15:07 __init__.py
-rw-r--r-- 1 root root   58 Mar  2 15:07 __main__.py
drwxr-xr-x 2 root root 4.0K Mar  2 15:07 __pycache__
-rw-r--r-- 1 root root 3.9K Mar  2 15:07 _disable_user_config.py
-rw-r--r-- 1 root root 2.4K Mar  2 15:07 app.py
-rw-r--r-- 1 root root  23K Mar  2 15:07 extension.py
-rw-r--r-- 1 root root  33K Mar  6 21:10 mixins.py
```
### Conclusion

Got it! jupyterhub/singleuser/templates/page.html isn't included with the wheel, only the source distribution!

![image](https://user-images.githubusercontent.com/3837114/223233026-c78def28-c066-414f-abce-35136110aba3.png)


Resolving patch:
diff --git a/ci/check_installed_data.py b/ci/check_installed_data.py
--- a/ci/check_installed_data.py
+++ b/ci/check_installed_data.py
@@ -12,7 +12,7 @@
 DATA_FILES_PATH = Path(DATA_FILES_PATH)
 assert DATA_FILES_PATH.is_dir(), DATA_FILES_PATH
 for subpath in (
-    "templates/page.html",
+    "templates/spawn.html",
     "static/css/style.min.css",
     "static/components/jquery/dist/jquery.js",
     "static/js/admin-react.js",
@@ -28,6 +28,7 @@
     "alembic.ini",
     "alembic/versions/833da8570507_rbac.py",
     "event-schemas/server-actions/v1.yaml",
+    "singleuser/templates/page.html",
 ):
     path = jupyterhub_path / subpath
     assert path.is_file(), path
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -55,7 +55,7 @@ def get_package_data():
         'alembic/*',
         'alembic/versions/*',
         'event-schemas/*/*.yaml',
-        'jupyterhub/singleuser/templates/*.html',
+        'singleuser/templates/*.html',
     ]
     return package_data
 


Test patch: