Package: notebook-restified
Introductionnotebook_restifiedis a library that thinks about Jupyter Notebooks as stand-alone functions withinput parametersand areturnvalue suitable to be used as REST endpoints or as callback functions within other Notebooks.  Put another way,notebook_restifiedtries to implement aModel-View-Controllercontext in the Jupyter framework.  It leveragespapermillfor reading and parameterizing Notebook files, and is inspired by some of the same goals behindipython_blocking.CaveatThis library and the ideas inside it are still being developed.  Consider this an alpha (or at most a beta) offering, syntax may change!  I believe the ideas here are solid and will stand the test of time, but the implementation is still in flux.Notebook life cycleIn my own experience, many workflow implementations follow a similar pattern.  They start in a Notebook then migrate to something that could be considered "production", which may or may not have a Jupyter/Notebook connection.  Hopefully this section resonates with your own experience using Jupyter in Enterprise organizations.First, the Notebook author explores a problem space in a "scratchpad" style Notebook: query APIs for data, clean and combine data, answer questions about the data.  User input variables are defined in code cells in this stage.  The application logic is linear, synchronous, and broken into small input/output cells to better understand each step of the process. The "result" of the Notebook is not strictly defined.Second, the Notebook author documents the workflow using Markdown cells and explanatory tools liketoc2.  User input, application logic, and the result of a Notebook haven't changed from step one.  The Notebook is ready for public consumption now and the sections (what is input, what is the conclusion, what are the steps in the workflow) are more explicitly laid out.Third, the Notebook author refactors the Notebook to a "Dashboard" mode (e.g.voila), which means users should not see or interact with the code.  This mode is attractive for customers who are intimidated by code, or for Notebooks that need to undergo peer review/compliance review and then become "immutable".In step three, user input comes fromipywidgetsforms instead of being defined in code cells.  The application logic is refactored into callback functions instead of being linear/synchronous/introspective.  The result of the Notebook is interactive in nature even though the code that generated the results is not.Optionally, step Three could be to refactor the application logic of the original Notebook into a REST endpoint.  The user input is either url parameters or the body of a POST.  The "users" (often other programs/services) certainly don't see the code.  The result of the Notebook (web app?) is very strictly defined and is not interactive.Finally, when something goes wrong in step Three or the Notebook author wants to add new features, they "un-refactor" out of the Dashboard/REST API implementation back to a more interactive/introspective mode to figure out their problem before then refactoring back again.notebook_restifiedtries to thread the needle between the three general ways that we use Notebooks: interactive mode, widgetized Dashboards, and as a REST API.Binder demosMore demo's to come.Installnotebook_restifiedis on Pypi, install it with pip:pipinstallnotebook_restifiedTo enable the/restified/endpoint on your Jupyter server and execute Notebooks as REST endpoints, enable the server extension using:jupyterserverextensionenablenotebook_restified--sys-prefixModelThe core ofnotebook_restifiedis treating a Notebook as aModelin the MVC sense of the word.  In an interactive mode, theModelNotebook is no different than any other Jupyter Notebook: introspective, exploratory, explanatory.  TheModelNotebook shouldn't have any widget/gui/user-interaction code in it -- that is saved for theViewNotebooks which can be run in Dashboard mode.  Alternatively, theModelNotebooks can be executed as REST endpoints.ModelNotebooks should have cell tags (see below) that parameterize the Notebook, define cells that shouldn't be executed in a REST/callback-function mode, and what the "return" value of the Notebook should be.  So far, there are two implementations forModelexecution:PythonModelandKernelModel.PythonModelexecutes a Notebook usingexecandevalon the code cells.  The benefit of usingPythonModelis that it runs in the same Kernel as theViewNotebook that is using it, so globals from theViewNotebook can be passed to theModelNotebook.KernelModelexecutes a Notebook by spinning up a new Jupyter Kernel and running the code there.  The benefit of usingKernelModelis that it is language agnostic.See the Binder examples for hands-on demo, but generally the syntax for aModelis:importnotebook_restifiedmodel=notebook_restified.KernelModel('model.ipynb')params={'x':42,'string':'foo'}result=model.execute(params)Cell TagsThere are three tags that are used innotebook_restifiedto control how a Notebook is executed.parametersis used to parameterize a Notebook with the papermill library.  Include this tag on any cells where you define variables that are going to be overriden inexecute_notebook(nb_path, params)or with url arguments in the/restified/endpoint.skipis used to denote cells that shouldn't be run when the Notebook is run RESTfully or withexecute_notebook-- informational plots, introspective details,printstatements, etc.returnis what is ultimately returned fromexecute_notebookor a RESTful execution (json encoded) via anevalcall, so it should be a very simple cell with no if statements, assignment, etc.restified endpointWhen the server extension is enabled, then you can call any Notebook as a REST endpoint by replacing/notebooks/in the Notebook url with/restified/.  Pass in anyparametersas url parameters with a GET or as a json-encoded body with a POST.Thoughts on Notebooks calling NotebooksI generally agree with Joel Grus's views (video,image) onimporting one notebook into another.  If a function or class is written in one Notebook, imported into another Notebook, and used there, then that sets off a red flag for me.  Python has a concept of packages and package structure, there is no need to work around that concept by putting re-usable code in a Notebook instead of a package (.py files).notebook_restifieddoesn't import code from one Notebook to another, but rather executes a Notebook as if it were a function.Thoughts on refactoringThere is certainly a grey area between refactoring code into a package and executing that in a callback function or web app versus using a Notebook withnotebook_restifiedin a callback function or web app.  In my view, the calculus to use is readability and comprehendibility.  In some ways, a package is better (better version control, unit tests, README/readthedocs).  In other ways, a Notebook can be better (table of contents extension, interactivity).Workflows that are task-specific and benefit from rich documentation about the tradecraft are perfect candidates fornotebook_restifiedinstead of refactored code.