ENH: Add JupyterLite-powered shell for the website (reprise of #47428)
State: closed

- [x] closes #60747
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

This pull request aims to reinstate the machinery for adding a JupyterLite shell for the `pandas` website, based on discussions in #60747 and on Slack, please see them for a rationale on this change (short answer: it's been a while in 2025 and things might be smoother now).

An interactive shell was first discussed in #46682 and later removed in #49807. There was a request to reinstate this in https://github.com/pandas-dev/pandas/pull/49807#issuecomment-1361430698 which might not have been noticed, so I hope #60747 and this PR bring more visibility to the proposal.

In particular, this PR adds back the previous changes from #47428, with only minor differences in the configuration. I've used (and pinned) the `jupyterlite-pyodide-kernel` for the REPL, which in-turn provides Pyodide version 0.5.2, which comes with `pandas` version 2.2.3.

The differences in the changes from the previous PR are as follows:
- newer versions of JupyterLite and `jupyterlite-pyodide-kernel` have been incorporated, i.e., 0.5.0 and 0.5.2 respectively
- source maps have been disabled when building JuptyerLite, which should reduce the size of static assets significantly and allow for faster load times
- the word "Experimental" has been added to the REPL's Markdown heading to indicate it as such

As mentioned in #60747, it is also possible to use the https://jupyterlite.github.io/demo/ REPL in the iframe to remove dependencies, however, it would come at the cost of disabled optimisations and a general lack of control for how the REPL is built.

cc: @jtpio, please feel free to add any suggestions!

Labels:
name: Enhancement
name: Web
description: pandas website

Comments:
Well, looking at the website from the downloaded artifact, having it at 100% width definitely doesn't look great:

<img width="1512" alt="pandas website displayed from the documentation build artifact from the build from gh-60758. This image is that of the Getting Started section, where a heading 'Experimental, try pandas in your browser' is displayed, with an interactive shell to run pandas code in the browser. It occupies the entire width of the website area from the left margin to the right" src="https://github.com/user-attachments/assets/d686b0f0-e429-4e95-b647-87594e01ee65" />

<br>
<br>

and reducing the width to 50% would affect mobile devices and/or narrower screen sizes. I'll try to fix this by adding some inline CSS.
I hope it looks better now:

<img width="1512" alt="pandas website displayed from the documentation build artifact from the build from gh-60758. This image is that of the Getting Started section, where a heading 'Experimental, try pandas in your browser' is displayed, with an interactive shell to run pandas code in the browser. It now occupies a smaller portion of the screen and stays true to its given width, with a maximum width of 650px. This retains the aspect ratio and makes the REPL look cleaner. There is also a dark blue border around the REPL, as it is " src="https://github.com/user-attachments/assets/a628374a-1540-481a-ad3c-9c242fe9ebf9" />

I also fixed a scaling issue for narrower screens for the YouTube video a few sections above this, which also affected how the REPL looked on mobile devices.

cc @datapythonista if you're interested in taking a look, I plan to review in the next few days.
Many thanks for the review, @rhshadrach! I'm sorry for not having responded earlier than I could; I was on vacation recently. This should be ready for another look whenever you get a chance.
Also, re-requesting a review from @jtpio, just in case I missed something.
> Well, looking at the website from the downloaded artifact, having it at 100% width definitely doesn't look great:

Maybe this could be taking 100% of the width in a future iteration, when JupyterLite 0.6.0 final is released, which will let users configure where the code console prompt cell should be placed:

![image](https://github.com/user-attachments/assets/9f43ec01-4bf0-4ca5-9fbe-92bab56ef7d5)

Thanks for the review, @jtpio – it would be nice to customise the location of the prompts (through URL parameters, I assume – I'm waiting for https://github.com/jupyterlite/jupyterlite/issues/148 and https://github.com/jupyterlite/jupyterlite/pull/1573 to land)!
@pandas-dev/pandas-core - Is anyone concerned about the increased load on the server? This was tried previous so I think we should be okay, but wanted to make sure before merging.
> @pandas-dev/pandas-core - Is anyone concerned about the increased load on the server? This was tried previous so I think we should be okay, but wanted to make sure before merging.

If I understand how this would work, every time you opened up the getting started page, it would load an `iframe` with the interactive shell, and there is a lot of bandwidth used to do that (70 MB?).  If that's the case, I think this should have its own page, i.e., people who are just looking at the Getting Started page shouldn't have to load the JupyterLite `iframe` .  Then the Getting Started page could point to the page with the shell, and we can measure how often people are using it.
> > @pandas-dev/pandas-core - Is anyone concerned about the increased load on the server? This was tried previous so I think we should be okay, but wanted to make sure before merging.
> 
> If I understand how this would work, every time you opened up the getting started page, it would load an `iframe` with the interactive shell, and there is a lot of bandwidth used to do that (70 MB?). If that's the case, I think this should have its own page, i.e., people who are just looking at the Getting Started page shouldn't have to load the JupyterLite `iframe` . Then the Getting Started page could point to the page with the shell, and we can measure how often people are using it.

Actually, the bandwidth usage will be in two steps. The first iteration will use up to ~7 MiB of bandwidth, and it is only when one presses <kbd>Enter</kbd> in the `import pandas as pd` code prompt that it will start loading `pandas` and its dependencies.

I would be happy to move to a dedicated page, though, based on the consensus that will be achieved here – that also helps Plausible do its job better.
@Dr-Irv - it doesn't load with the page. See https://github.com/pandas-dev/pandas/pull/60758#discussion_r1938498238
> The first iteration will use up to ~7 MiB of bandwidth

That's still a fairly big load for a web page (or at least it seems to me to be  a big load)

> that also helps Plausible do its job better.

Can you explain what you mean here by "Plausible do its job better" ?
> > The first iteration will use up to ~7 MiB of bandwidth
> 
> That's still a fairly big load for a web page (or at least it seems to me to be a big load)

I agree – I can't deny it :) Though, from a quick test just now, I noticed that https://pandas.pydata.org/ without the shell accumulates 4.35 MiB of bandwidth, so it might not be _too_ bad.

> > that also helps Plausible do its job better.
> 
> Can you explain what you mean here by "Plausible do its job better" ?

Oh, I meant that a dedicated page would show up separately in the dashboard on https://views.scientific-python.org/pandas.pydata.org/, so the pandas team would be able to see better metrics; viz. entry/exit links, the time spent on the page, and so on, to see how much the shell would be utilised in the wild. Having it on the Getting Started page makes gaining said metrics a bit more complex, as [pageview goals](https://plausible.io/docs/pageview-goals) need to be set up.

I'd probably rephrase my statement from "helps Plausible do its job better" to "helps the pandas website team that makes use of Plausible do their job better" – this seems to better convey what I wanted to say.
I agree with @Dr-Irv. The terminal is very cool, but at the same time tricky. It didn't work very well when we already tried in the past. And even in a best case scenario, it can be a problem for example to mobile users with a limited data plan. Users should be able to access our getting started page without exposure to the terminal in my opinion. And personally, I'd also prefer if the terminal is hosted elsewhere and we just link to it. No strong objection to having it in this repo amd our hosting, but I think it's better if it has it's own space (I think another repo in our organization and github pages for example). It allows to iterate quickly on it, and puts less pressure on having to deliver a fast and reliable experience to the users, if it's perceived as a external resource. Just my opinion.
Thanks for your thoughts, @datapythonista. As two people have now suggested moving it to a separate page, I'm happy to do so. I agree with moving the shell to a separate repository in `pandas` wired up with GitHub Pages, and I can help with the maintenance of such a repository if one could be set up in the organisation from the template at https://github.com/jupyterlite/demo.

One option is to use an external deployment from that repository at https://jupyterlite.github.io/demo, but I'll have to note that it comes with source maps enabled (https://github.com/jupyterlite/demo/issues/151), which means that the JupyterLite assets are slightly bigger in size. I could further switch to that if you would like, as [NumPy uses it as well](https://github.com/numpy/numpy.org/blob/4d22f788d7dfa6f0624bede75d85a8c6e7f672bd/layouts/partials/shell.html#L20)  for https://numpy.org/, but I would suggest not doing so at the moment as these extra infrastructure changes where we set up a shell of our own will help me with a follow-up PR after this is merged that will enable interactivity for the ["API reference" examples for `pandas`](https://pandas.pydata.org/docs/reference/index.html), which relies on the same tooling. For example, here's a parallel attempt for NumPy from a while ago: https://github.com/numpy/numpy/pull/26745.

Like it is here, I have plans to change the NumPy website's shell to a deployment that doesn't enable source maps in the short term.
@agriyakhetarpal thanks for separating the page.  
No further objections, looks like! 😄
Thanks @agriyakhetarpal @jtpio!