Repository: getnikola/nikola

Issue:
ipynb listings
It would be nice if I could throw an `.ipynb` file in the listings directory and have it render.

Issues:
- would anyone ever want to show off the raw (json) file?
- strange behavior, compared to other formats (including html)



Discussion:
Honestly, it would stinkk of special-casing. We could have a task plugin that took /notebooks and rendered it somewhere in the proper way tho

> It would be nice if I could throw an .ipynb file in the listings directory and have it render.

In fact I think it would be nice to have something like:

`listing: foo.ipynb cells[2:5] python`

I mean, some way to read the content from cell 2 to 5 and show that but rendered correctly (as nbconverted html snippet... 

> would anyone ever want to show off the raw (json) file?

Not to much frequently...

> strange behavior, compared to other formats (including html)

What do you mean with strange?

@Kwpolska On second thought ... how does that differ from throwing the notebook in stories/ ?

I would have to take care of directory listings myself, create ~~.py~~ .rst code
block stories out of non-ipynb related files, and it won't display in
listings/.

So... currently we just use pygments to highlight stuff in listings. So, to support this usage, we'd need to have a mechanism to support different processors for different extensions. With that, it would not be horribly hard :-)

If anyone can describe how this feature would work, I'm happy to take a shot at it.

One problem is that the listings **directive** has no way to support this, so it will make things more complicated.

Another is that it should be, if it is to be implemented, a reasonably complete thing that provides support for arbitrary renderers, which means it has huge feature overlap with stories. In fact, it looks like "stories organized by folder hierarchy" which is sort of easy?

I actually looked into this now and an implementation in the “full” listings is really simple, will have a PR ready in a moment.

As for the directive, would that be useful? If yes, I think it could be implemented with a simple `get_code_from_file` modification that would return raw HTML nodes from that ipynb plugin.

Implementing “listings = stories with folders” could be problematic. You see, the current idea takes ipynb only, because it’s very unlikely someone would actually want to display the raw JSON files (and if they do, they can just change the extension and also get JSON code highlighting), and displaying notebooks sounds like something a scientist might want to do. But, we could get a web developer who wants to showcase some `.html`, `.md`, `.rst`, and this could be a problem, because we would need some setting/magic to tell raw and rendered modes apart.


Resolving patch:
diff --git a/nikola/plugins/compile/ipynb.py b/nikola/plugins/compile/ipynb.py
--- a/nikola/plugins/compile/ipynb.py
+++ b/nikola/plugins/compile/ipynb.py
@@ -66,19 +66,23 @@ def set_site(self, site):
         self.logger = get_logger('compile_ipynb', STDERR_HANDLER)
         super(CompileIPynb, self).set_site(site)
 
-    def compile_html(self, source, dest, is_two_file=True):
-        """Compile source file into HTML and save as dest."""
+    def compile_html_string(self, source, is_two_file=True):
+        """Export notebooks as HTML strings."""
         if flag is None:
             req_missing(['ipython[notebook]>=2.0.0'], 'build this site (compile ipynb)')
-        makedirs(os.path.dirname(dest))
         HTMLExporter.default_template = 'basic'
         c = Config(self.site.config['IPYNB_CONFIG'])
         exportHtml = HTMLExporter(config=c)
+        with io.open(source, "r", encoding="utf8") as in_file:
+            nb_json = nbformat.read(in_file, current_nbformat)
+        (body, resources) = exportHtml.from_notebook_node(nb_json)
+        return body
+
+    def compile_html(self, source, dest, is_two_file=True):
+        """Compile source file into HTML and save as dest."""
+        makedirs(os.path.dirname(dest))
         with io.open(dest, "w+", encoding="utf8") as out_file:
-            with io.open(source, "r", encoding="utf8") as in_file:
-                nb_json = nbformat.read(in_file, current_nbformat)
-            (body, resources) = exportHtml.from_notebook_node(nb_json)
-            out_file.write(body)
+            out_file.write(self.compile_html_string(source, is_two_file))
 
     def read_metadata(self, post, file_metadata_regexp=None, unslugify_titles=False, lang=None):
         """Read metadata directly from ipynb file.
diff --git a/nikola/plugins/task/listings.py b/nikola/plugins/task/listings.py
--- a/nikola/plugins/task/listings.py
+++ b/nikola/plugins/task/listings.py
@@ -30,6 +30,7 @@
 
 import sys
 import os
+import lxml.html
 
 from pygments import highlight
 from pygments.lexers import get_lexer_for_filename, TextLexer
@@ -113,7 +114,17 @@ def gen_tasks(self):
         ignored_extensions = (".pyc", ".pyo")
 
         def render_listing(in_name, out_name, input_folder, output_folder, folders=[], files=[]):
-            if in_name:
+            needs_ipython_css = False
+            if in_name and in_name.endswith('.ipynb'):
+                # Special handling: render ipynbs in listings (Issue #1900)
+                ipynb_compiler = self.site.plugin_manager.getPluginByName("ipynb", "PageCompiler").plugin_object
+                ipynb_raw = ipynb_compiler.compile_html_string(in_name, True)
+                ipynb_html = lxml.html.fromstring(ipynb_raw)
+                # The raw HTML contains garbage (scripts and styles), we can’t leave it in
+                code = lxml.html.tostring(ipynb_html.xpath('//*[@id="notebook"]')[0], encoding='unicode')
+                title = os.path.basename(in_name)
+                needs_ipython_css = True
+            elif in_name:
                 with open(in_name, 'r') as fd:
                     try:
                         lexer = get_lexer_for_filename(in_name)
@@ -154,6 +165,10 @@ def render_listing(in_name, out_name, input_folder, output_folder, folders=[], f
                 'source_link': source_link,
                 'pagekind': ['listing'],
             }
+            if needs_ipython_css:
+                # If someone does not have ipynb posts and only listings, we
+                # need to enable ipynb CSS for ipynb listings.
+                context['needs_ipython_css'] = True
             self.site.render_template('listing.tmpl', out_name, context)
 
         yield self.group_task()


Test patch: