Repository: ipython/ipython

Issue:
nbconvert does not respect line breaks in markdown cells
I can see how the conversion of ipynb files to markdown and/or latex (using nbconvert) would be extremely useful for gathering comments stored in many ipython notebooks to begin writing a paper. Unfortunately, the utility does not respect the newline characters stored in .ipynb file. 

When composing a latex document (especially when using git to track changes), it is extremely useful to adopt a one sentence per line convention. This renders diffs with alternate versions easily readable and the document easily searchable, since there are no unexpected line breaks.

While the ipynb format does properly store linebreaks as entered, nbconvert clobbers this information by inserting linebreaks wherever it determines is best to maintain uniform line length in the output file. These changes are of course invisible in the final view of the TeX'ed up document or rendered markdown document, but they destroy the ability to usefully track changes on the markdown or latex source files. 

Is there a flag to disable this behavior? If not, is adding such a flag or replacing the default behavior a possiblity? Thanks!



Discussion:
IPython relies on pandoc for md→latex, so if there is such an option, it would be on pandoc.

That is good to know. Unfortunatlye, the problem exists for both latex AND
markdown output. So I would still need to know how to get nbconvert to
output markdown text that respects user entered line breaks. Is there such
an option? Is it hard to fix?

On Tue, Sep 16, 2014 at 2:30 PM, Min RK notifications@github.com wrote:

> IPython relies on pandoc for md→latex, so if there is such an option, it
> would be on pandoc.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/ipython/ipython/issues/6484#issuecomment-55790520.


Resolving patch:
diff --git a/IPython/nbconvert/filters/markdown.py b/IPython/nbconvert/filters/markdown.py
--- a/IPython/nbconvert/filters/markdown.py
+++ b/IPython/nbconvert/filters/markdown.py
@@ -45,7 +45,7 @@ class NodeJSMissing(ConversionException):
     """Exception raised when node.js is missing."""
     pass
 
-def markdown2latex(source):
+def markdown2latex(source, extra_args=None):
     """Convert a markdown string to LaTeX via pandoc.
 
     This function will raise an error if pandoc is not installed.
@@ -61,7 +61,7 @@ def markdown2latex(source):
     out : string
       Output as returned by pandoc.
     """
-    return pandoc(source, 'markdown', 'latex')
+    return pandoc(source, 'markdown', 'latex', extra_args=extra_args)
 
 
 @undoc
@@ -155,9 +155,10 @@ def markdown2html_mistune(source):
     """Convert a markdown string to HTML using mistune"""
     return MarkdownWithMath(renderer=IPythonRenderer()).render(source)
 
-def markdown2html_pandoc(source):
+def markdown2html_pandoc(source, extra_args=None):
     """Convert a markdown string to HTML via pandoc"""
-    return pandoc(source, 'markdown', 'html', extra_args=['--mathjax'])
+    extra_args = extra_args or ['--mathjax']
+    return pandoc(source, 'markdown', 'html', extra_args=extra_args)
 
 def _find_nodejs():
     global _node
@@ -188,7 +189,7 @@ def markdown2html_marked(source, encoding='utf-8'):
 # The mistune renderer is the default, because it's simple to depend on it
 markdown2html = markdown2html_mistune
 
-def markdown2rst(source):
+def markdown2rst(source, extra_args=None):
     """Convert a markdown string to ReST via pandoc.
 
     This function will raise an error if pandoc is not installed.
@@ -204,7 +205,7 @@ def markdown2rst(source):
     out : string
       Output as returned by pandoc.
     """
-    return pandoc(source, 'markdown', 'rst')
+    return pandoc(source, 'markdown', 'rst', extra_args=extra_args)
 
 def _verify_node(cmd):
     """Verify that the node command exists and is at least the minimum supported


Test patch:
diff --git a/IPython/nbconvert/filters/tests/test_markdown.py b/IPython/nbconvert/filters/tests/test_markdown.py
--- a/IPython/nbconvert/filters/tests/test_markdown.py
+++ b/IPython/nbconvert/filters/tests/test_markdown.py
@@ -11,6 +11,7 @@
 from ...tests.base import TestsBase
 from ..markdown import markdown2latex, markdown2html, markdown2rst
 
+from jinja2 import Environment
 
 class TestMarkdown(TestsBase):
 
@@ -47,6 +48,26 @@ def test_markdown2latex(self):
         for index, test in enumerate(self.tests):
             self._try_markdown(markdown2latex, test, self.tokens[index])
 
+    @dec.onlyif_cmds_exist('pandoc')
+    def test_pandoc_extra_args(self):
+        # pass --no-wrap
+        s = '\n'.join([
+            "#latex {{long_line | md2l(['--no-wrap'])}}",
+            "#rst {{long_line | md2r(['--columns', '5'])}}",
+        ])
+        long_line = ' '.join(['long'] * 30)
+        env = Environment()
+        env.filters.update({
+            'md2l': markdown2latex,
+            'md2r': markdown2rst,
+        })
+        tpl = env.from_string(s)
+        rendered = tpl.render(long_line=long_line)
+        _, latex, rst = rendered.split('#')
+        
+        self.assertEqual(latex.strip(), 'latex %s' % long_line)
+        self.assertEqual(rst.strip(), 'rst %s' % long_line.replace(' ', '\n'))
+
     def test_markdown2html(self):
         """markdown2html test"""
         for index, test in enumerate(self.tests):