Package: imjoy-jupyter-extension
ImJoy Jupyter ExtensionTry the demo here:Using ImJoy with Jupyter Notebook and JupyterLabThis extension utilizes theimjoy-rpclibrary to enable bidirectional RPC calls between the ImJoy core and plugins.The library has an abstract transport interface that can support different types of transport. For now, we supportJupyter comms messagewhich is a custom message protocol used in Jupyter notebooks (for example powers jupyter widgets).To use it, you need to install theimjoy-jupyter-extensionin Python which will also include the imjoy-rpc as a dependency (see below).Install the extension for Jupyter Notebook or JupyterLabFor Jupyter Notebookspipinstall-Uimjoy-jupyter-extensionYou can verify the installation by running:jupyter nbextension listAnd you should see something likeimjoy_jupyter_extension/index  enabledin the output.
If you still have issue, you can try to install it manually.gitclonehttps://github.com/imjoy-team/imjoy-jupyter-extension.gitcdimjoy-jupyter-extension
jupyternbextensioninstallimjoy_jupyter_extension--py# add --sys-prefix if using virtualenv or condajupyternbextensionenableimjoy_jupyter_extension--py# add --sys-prefix if using virtualenv or condajupyternbextensionlistFor Jupyter Labpipinstall-Uimjoy-jupyter-extension

jupyterlabextensioninstallimjoy-jupyter-extensionUse ImJoy plugins inside Jupyter notebooksNow you can start a jupyter notebook via for examplejupyter notebookcommand, you should be able to see an ImJoy icon in the toolbar if everything goes well.For JupyterLab, start it viajupyter labcommand, then create a notebook and you should see an ImJoy button in the toolbar:Now run ImJoy plugins in a cell, see an example that uses itk-vtk-viewer to visualize images:importimageioimportnumpyasnpfromimjoy_rpcimportapiclassImJoyPlugin():defsetup(self):api.log('plugin initialized')asyncdefrun(self,ctx):viewer=awaitapi.showDialog(src="https://kitware.github.io/itk-vtk-viewer/app/")# show a 3D volumeimage_array=np.random.randint(0,255,[10,10,10],dtype='uint8')# show a 2D image# image_array = imageio.imread('imageio:chelsea.png')awaitviewer.setImage(image_array)api.export(ImJoyPlugin())With the above code, you created an ImJoy plugin. To run it, click the Run button with the ImJoy icon. It will then call the run function of your plugin.GIF Demo:Visualizing 2D imageGIF Demo:Visualizing 3D volumeGIF Demo:Load ImageAnnotatorYou can also do the reverse by running a notebook inside ImJoy, to do that, please first create an empty jupyter notebook. Then copy and paste the url into the "+ PLUGINS" dialog, press enter and install the plugin. Click the newly installed plugin and you will get a notebook page open in ImJoy.Try to copy and paste the itk-vtk-viewer example to a cell and execute it. Similarily, if you now click the run ImJoy button in the toolbar, you will get the viewer open insided ImJoy.Load plugin dynamically in a notebookThere are different ways to load a plugin in a notebook, you can load from URL for the source code of the plugin (e.g. gist or github file url) or a hosted web app (e.g. the itk-vtk-viewer app used above). Besides that, you can also write a window plugin directly in a notebook. This is practical especially for debugging window plugin for your GUI.You can directly pass a string with the plugin source code toapi.getPlugin,api.createWindow(src=...)orapi.showDialog(src=...), however, that doesn't give you syntax highliting in a Jupyter notebook cell. A workaround solution (which also works for Google Colab) is to write plugin code in a seperate cell startting with## ImJoy Pluginand wrap the source code withIPython.display.HTML(). For example:## ImJoy Plugin
from IPython.display import HTML
my_plugin_source = HTML('''<docslang="markdown">[TODO: write documentation for this plugin.]</docs><configlang="json">{
  "name": "Untitled Plugin",
  "type": "window",
  "tags": [],
  "ui": "",
  "version": "0.1.0",
  "cover": "",
  "description": "[TODO: describe this plugin with one sentence.]",
  "icon": "extension",
  "inputs": null,
  "outputs": null,
  "api_version": "0.1.8",
  "env": "",
  "permissions": [],
  "requirements": [],
  "dependencies": [],
  "defaults": {"w": 20, "h": 10}
}</config><scriptlang="javascript">classImJoyPlugin{asyncsetup(){api.log('initialized')}asyncrun(ctx){}}api.export(newImJoyPlugin())</script><windowlang="html"><div><p>Hello World</p></div></window><stylelang="css"></style>''')Then in another cell, you can passing the plugin source code toapi.getPlugin,api.createWindow(src=...)orapi.showDialog(src=...)to make an actually plugin:fromimjoy_rpcimportapiclassImJoyPlugin():asyncdefsetup(self):passasyncdefrun(self,ctx):# for regular plugin# p = await api.getPlugin(my_plugin_source)# or for window plugin# await api.createWindow(src=my_plugin_source)awaitapi.showDialog(src=my_plugin_source)api.export(ImJoyPlugin())DevelopmentTo develop the JupyterLab extension, open a terminal and run:cdimjoy_jupyterlab_extension
npmrunwatchIn another terminal window run:cd imjoy_jupyterlab_extension
npm run build
jupyter labextension link .
jupyter lab