Repository: mwouts/jupytext
Pull request #725 (2021-01-24 02:54:03)

Check nbformat version
In this PR we add checks to make sure that Jupytext is used on notebook in versions of nbformat with which it has been tested. See for instance #681 and #715.

codecov[bot]: # [Codecov](https://codecov.io/gh/mwouts/jupytext/pull/725?src=pr&el=h1) Report
> Merging [#725](https://codecov.io/gh/mwouts/jupytext/pull/725?src=pr&el=desc) (ad77ada) into [master](https://codecov.io/gh/mwouts/jupytext/commit/41b7e9c93b50da1d6feb486aceb2c4d534374090?el=desc) (41b7e9c) will **decrease** coverage by `0.17%`.
> The diff coverage is `99.60%`.

[![Impacted file tree graph](https://codecov.io/gh/mwouts/jupytext/pull/725/graphs/tree.svg?width=650&height=150&src=pr&token=a1pNjsiILi)](https://codecov.io/gh/mwouts/jupytext/pull/725?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #725      +/-   ##
==========================================
- Coverage   99.12%   98.95%   -0.18%     
==========================================
  Files          95       96       +1     
  Lines        9502     9547      +45     
==========================================
+ Hits         9419     9447      +28     
- Misses         83      100      +17     
```


| [Impacted Files](https://codecov.io/gh/mwouts/jupytext/pull/725?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [jupytext/metadata\_filter.py](https://codecov.io/gh/mwouts/jupytext/pull/725/diff?src=pr&el=tree#diff-anVweXRleHQvbWV0YWRhdGFfZmlsdGVyLnB5) | `99.13% <ø> (ø)` | |
| [tests/test\_auto\_ext.py](https://codecov.io/gh/mwouts/jupytext/pull/725/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9hdXRvX2V4dC5weQ==) | `100.00% <ø> (ø)` | |
| [tests/test\_cells.py](https://codecov.io/gh/mwouts/jupytext/pull/725/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9jZWxscy5weQ==) | `100.00% <ø> (ø)` | |
| [tests/test\_config.py](https://codecov.io/gh/mwouts/jupytext/pull/725/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9jb25maWcucHk=) | `100.00% <ø> (ø)` | |
| [tests/test\_hide\_remove\_input\_outputs\_rmarkdown.py](https://codecov.io/gh/mwouts/jupytext/pull/725/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9oaWRlX3JlbW92ZV9pbnB1dF9vdXRwdXRzX3JtYXJrZG93bi5weQ==) | `100.00% <ø> (ø)` | |
| [tests/test\_ipynb\_to\_py.py](https://codecov.io/gh/mwouts/jupytext/pull/725/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9pcHluYl90b19weS5weQ==) | `100.00% <ø> (ø)` | |
| [tests/test\_ipynb\_to\_rmd.py](https://codecov.io/gh/mwouts/jupytext/pull/725/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9pcHluYl90b19ybWQucHk=) | `100.00% <ø> (ø)` | |
| [tests/test\_jupytext\_errors.py](https://codecov.io/gh/mwouts/jupytext/pull/725/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9qdXB5dGV4dF9lcnJvcnMucHk=) | `100.00% <ø> (ø)` | |
| [tests/test\_jupytext\_nbconvert\_round\_trip.py](https://codecov.io/gh/mwouts/jupytext/pull/725/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9qdXB5dGV4dF9uYmNvbnZlcnRfcm91bmRfdHJpcC5weQ==) | `100.00% <ø> (ø)` | |
| [tests/test\_jupytext\_read.py](https://codecov.io/gh/mwouts/jupytext/pull/725/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9qdXB5dGV4dF9yZWFkLnB5) | `100.00% <ø> (ø)` | |
| ... and [80 more](https://codecov.io/gh/mwouts/jupytext/pull/725/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/mwouts/jupytext/pull/725?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/mwouts/jupytext/pull/725?src=pr&el=footer). Last update [b909549...ad77ada](https://codecov.io/gh/mwouts/jupytext/pull/725?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md
index 0895d4be..8163c176 100644
--- a/docs/CHANGELOG.md
+++ b/docs/CHANGELOG.md
@@ -10,6 +10,7 @@ Jupytext ChangeLog
 
 **Fixed**
 - Indented magic commands are supported ([#694](https://github.com/mwouts/jupytext/issues/694))
+- Jupytext will issue an informative error or warning on notebooks in a version of nbformat that is not known to be supported ([#681](https://github.com/mwouts/jupytext/issues/681), [#715](https://github.com/mwouts/jupytext/issues/715))
 
 **Added**
 - We made sure that `py:percent` scripts end with exactly one blank line ([#682](https://github.com/mwouts/jupytext/issues/682))
diff --git a/jupytext/jupytext.py b/jupytext/jupytext.py
index 25da61d5..97654d5e 100644
--- a/jupytext/jupytext.py
+++ b/jupytext/jupytext.py
@@ -4,6 +4,7 @@ import io
 import logging
 import os
 import sys
+import warnings
 from copy import copy, deepcopy
 
 import nbformat
@@ -37,6 +38,11 @@ from .metadata_filter import filter_metadata, update_metadata_filters
 from .myst import MYST_FORMAT_NAME, myst_extensions, myst_to_notebook, notebook_to_myst
 from .pandoc import md_to_notebook, notebook_to_md
 from .pep8 import pep8_lines_between_cells
+from .version import __version__
+
+
+class NotSupportedNBFormatVersion(NotImplementedError):
+    pass
 
 
 class TextNotebookConverter(NotebookReader, NotebookWriter):
@@ -343,7 +349,15 @@ def reads(text, fmt, as_version=nbformat.NO_CONVERT, **kwargs):
     ext = fmt["extension"]
 
     if ext == ".ipynb":
-        return nbformat.reads(text, as_version, **kwargs)
+        nb = nbformat.reads(text, as_version, **kwargs)
+        (version, version_minor) = nbformat.reader.get_version(nb)
+        if version != 4:
+            warnings.warn(
+                f"Notebooks in nbformat version {version} are not supported by Jupytext. "
+                f"Please consider converting them to nbformat version 4 with "
+                f"'jupyter nbconvert --to notebook --inplace'"
+            )
+        return nb
 
     format_name = read_format_from_metadata(text, ext) or fmt.get("format_name")
 
@@ -408,7 +422,7 @@ def read(fp, as_version=nbformat.NO_CONVERT, fmt=None, **kwargs):
 
 
 def writes(notebook, fmt, version=nbformat.NO_CONVERT, **kwargs):
-    """Return text representation of the notebook
+    """Return the text representation of the notebook
 
     :param notebook: the notebook
     :param fmt: the jupytext format like `md`, `py:percent`, ...
@@ -416,6 +430,25 @@ def writes(notebook, fmt, version=nbformat.NO_CONVERT, **kwargs):
     :param kwargs: (not used) additional parameters for nbformat.writes
     :return: the text representation of the notebook
     """
+    if version is not nbformat.NO_CONVERT:
+        if not isinstance(version, int):
+            raise TypeError(
+                "The argument 'version' should be either nbformat.NO_CONVERT, or an integer."
+            )
+        notebook = nbformat.convert(notebook, version)
+    (version, version_minor) = nbformat.reader.get_version(notebook)
+    if version < 4:
+        raise NotSupportedNBFormatVersion(
+            f"Notebooks in nbformat version {version}.{version_minor} are not supported by Jupytext. "
+            f"Please convert your notebooks to nbformat version 4 with "
+            f"'jupyter nbconvert --to notebook --inplace', or call this function with 'version=4'."
+        )
+    if version > 4 or (version == 4 and version_minor > 4):
+        warnings.warn(
+            f"Notebooks in nbformat version {version}.{version_minor} "
+            f"have not been tested with Jupytext version {__version__}."
+        )
+
     metadata = deepcopy(notebook.metadata)
     rearrange_jupytext_metadata(metadata)
     fmt = copy(fmt)
@@ -438,7 +471,7 @@ def writes(notebook, fmt, version=nbformat.NO_CONVERT, **kwargs):
                 cells=notebook.cells,
             ),
             version,
-            **kwargs
+            **kwargs,
         )
 
     if not format_name:
@@ -491,8 +524,8 @@ def write(nb, fp, version=nbformat.NO_CONVERT, fmt=None, **kwargs):
     if isinstance(content, bytes):
         content = content.decode("utf8")
     fp.write(content)
-    if not content.endswith(u"\n"):
-        fp.write(u"\n")
+    if not content.endswith("\n"):
+        fp.write("\n")
 
 
 def create_prefix_dir(nb_file, fmt):


Test patch:
diff --git a/tests/test_cli.py b/tests/test_cli.py
index 6e81b820..f00d3b09 100644
--- a/tests/test_cli.py
+++ b/tests/test_cli.py
@@ -509,13 +509,13 @@ def test_sync(nb_file, tmpdir, capsys):
     assert os.path.isfile(tmp_rmd)
     compare_notebooks(read(tmp_rmd), nb, "Rmd")
 
-    write(nb, tmp_rmd, "Rmd")
+    write(nb, tmp_rmd, fmt="Rmd")
     jupytext(["--sync", tmp_ipynb])
 
     nb2 = read(tmp_ipynb)
     compare_notebooks(nb2, nb, "Rmd", compare_outputs=True)
 
-    write(nb, tmp_py, "py")
+    write(nb, tmp_py, fmt="py")
     jupytext(["--sync", tmp_ipynb])
 
     nb2 = read(tmp_ipynb)
@@ -581,7 +581,7 @@ def test_cli_can_infer_jupytext_format(nb_file, ext, tmpdir):
     compare_notebooks(nb2, nb)
 
     # Percent format to Jupyter notebook
-    write(nb, tmp_text, ext + ":percent")
+    write(nb, tmp_text, fmt=ext + ":percent")
     jupytext(["--to", "notebook", tmp_text])
     nb2 = read(tmp_ipynb)
     compare_notebooks(nb2, nb)
@@ -758,7 +758,7 @@ def test_cli_sync_file_with_suffix(tmpdir):
         metadata={"jupytext": {"formats": "ipynb,.pct.py:percent,.lgt.py:light,Rmd"}},
     )
 
-    write(nb, tmp_pct_py, ".pct.py:percent")
+    write(nb, tmp_pct_py, fmt=".pct.py:percent")
     jupytext(["--sync", tmp_pct_py])
     assert os.path.isfile(tmp_lgt_py)
     assert os.path.isfile(tmp_rmd)
diff --git a/tests/test_nbformat_version.py b/tests/test_nbformat_version.py
new file mode 100644
index 00000000..881cc691
--- /dev/null
+++ b/tests/test_nbformat_version.py
@@ -0,0 +1,67 @@
+import pytest
+from nbformat import writes as nbformat_writes
+from nbformat.v3.nbbase import new_code_cell, new_notebook, new_text_cell, new_worksheet
+from nbformat.v4.nbbase import new_markdown_cell
+from nbformat.v4.nbbase import new_notebook as new_notebook_v4
+
+from jupytext import reads, writes
+from jupytext.jupytext import NotSupportedNBFormatVersion
+
+
+@pytest.fixture()
+def sample_notebook_v3():
+    return new_notebook(
+        worksheets=[
+            new_worksheet(
+                cells=[new_code_cell("1 + 1"), new_text_cell("markdown", "Hi")]
+            )
+        ]
+    )
+
+
+@pytest.fixture()
+def sample_notebook_v3_json(sample_notebook_v3):
+    return nbformat_writes(sample_notebook_v3)
+
+
+@pytest.fixture()
+def sample_notebook_v4_5():
+    nb = new_notebook_v4(cells=[new_markdown_cell("Hi")])
+    nb["nbformat_minor"] = 5
+    nb["cells"][0]["id"] = "unique-id"
+    return nb
+
+
+def test_jupytext_can_read_nbformat_3(
+    sample_notebook_v3_json,
+):
+    with pytest.warns(Warning, match="jupyter nbconvert --to notebook --inplace"):
+        nb = reads(sample_notebook_v3_json, fmt="ipynb")
+        assert nb["nbformat"] == 3
+
+    nb = reads(sample_notebook_v3_json, as_version=4, fmt="ipynb")
+    assert nb["nbformat"] == 4
+
+
+@pytest.mark.parametrize("fmt", ["py:light", "py:percent", "md"])
+def test_jupytext_gives_a_meaningful_error_when_writing_nbformat_3(
+    sample_notebook_v3, fmt
+):
+    with pytest.raises(
+        NotSupportedNBFormatVersion,
+        match="Notebooks in nbformat version 3.0 are not supported by Jupytext",
+    ):
+        writes(sample_notebook_v3, fmt=fmt)
+
+    writes(sample_notebook_v3, version=4, fmt=fmt)
+
+
+@pytest.mark.parametrize("fmt", ["py:light", "py:percent", "md"])
+def test_jupytext_gives_a_meaningful_error_when_writing_nbformat_4_5(
+    sample_notebook_v4_5, fmt
+):
+    with pytest.warns(
+        Warning,
+        match="Notebooks in nbformat version 4.5 have not been tested",
+    ):
+        writes(sample_notebook_v4_5, fmt=fmt)