Repository: mwouts/jupytext
Pull request #300 (2019-07-20 06:40:41)

Improve usability of read and write functions
And document how to use Jupytext with the pre-commit package #292 

codecov[bot]: # [Codecov](https://codecov.io/gh/mwouts/jupytext/pull/300?src=pr&el=h1) Report
> Merging [#300](https://codecov.io/gh/mwouts/jupytext/pull/300?src=pr&el=desc) into [master](https://codecov.io/gh/mwouts/jupytext/commit/69c9a6809841c8637b797773b500f1bc39b6dd90?src=pr&el=desc) will **decrease** coverage by `0.07%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/mwouts/jupytext/pull/300/graphs/tree.svg?width=650&token=a1pNjsiILi&height=150&src=pr)](https://codecov.io/gh/mwouts/jupytext/pull/300?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master     #300      +/-   ##
==========================================
- Coverage    99.2%   99.12%   -0.08%     
==========================================
  Files          69       70       +1     
  Lines        6655     6531     -124     
==========================================
- Hits         6602     6474     -128     
- Misses         53       57       +4
```


| [Impacted Files](https://codecov.io/gh/mwouts/jupytext/pull/300?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [tests/test\_read\_write\_functions.py](https://codecov.io/gh/mwouts/jupytext/pull/300/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9yZWFkX3dyaXRlX2Z1bmN0aW9ucy5weQ==) | `100% <100%> (ø)` | |
| [jupytext/jupytext.py](https://codecov.io/gh/mwouts/jupytext/pull/300/diff?src=pr&el=tree#diff-anVweXRleHQvanVweXRleHQucHk=) | `100% <100%> (ø)` | :arrow_up: |
| [jupytext/pep8.py](https://codecov.io/gh/mwouts/jupytext/pull/300/diff?src=pr&el=tree#diff-anVweXRleHQvcGVwOC5weQ==) | `95.23% <0%> (-4.77%)` | :arrow_down: |
| [jupytext/cell\_reader.py](https://codecov.io/gh/mwouts/jupytext/pull/300/diff?src=pr&el=tree#diff-anVweXRleHQvY2VsbF9yZWFkZXIucHk=) | `96.76% <0%> (-0.21%)` | :arrow_down: |
| [tests/test\_pep8.py](https://codecov.io/gh/mwouts/jupytext/pull/300/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9wZXA4LnB5) | `98.36% <0%> (-0.03%)` | :arrow_down: |
| [tests/test\_cli.py](https://codecov.io/gh/mwouts/jupytext/pull/300/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9jbGkucHk=) | `100% <0%> (ø)` | :arrow_up: |
| [tests/test\_black.py](https://codecov.io/gh/mwouts/jupytext/pull/300/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9ibGFjay5weQ==) | `100% <0%> (ø)` | :arrow_up: |
| [tests/test\_active\_cells.py](https://codecov.io/gh/mwouts/jupytext/pull/300/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9hY3RpdmVfY2VsbHMucHk=) | `100% <0%> (ø)` | :arrow_up: |
| [tests/test\_knitr\_spin.py](https://codecov.io/gh/mwouts/jupytext/pull/300/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9rbml0cl9zcGluLnB5) | `100% <0%> (ø)` | :arrow_up: |
| [tests/test\_compare.py](https://codecov.io/gh/mwouts/jupytext/pull/300/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9jb21wYXJlLnB5) | `100% <0%> (ø)` | :arrow_up: |
| ... and [20 more](https://codecov.io/gh/mwouts/jupytext/pull/300/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/mwouts/jupytext/pull/300?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/mwouts/jupytext/pull/300?src=pr&el=footer). Last update [69c9a68...24d468d](https://codecov.io/gh/mwouts/jupytext/pull/300?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index ed38edf0..5856dc5e 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -38,7 +38,7 @@ Most of Jupytext's code is written in Python. To develop the Python part of Jupy
 cd jupytext
 conda create -n jupytext-dev jupyterlab pyyaml
 conda activate jupytext-dev
-pip install -r requirements.txt -r requirements-dev.txt
+pip install -r requirements-dev.txt
 ```
 
 Tests are executed with `pytest`. Install `pytest-xdist` to allow running the tests in parallel:
diff --git a/HISTORY.rst b/HISTORY.rst
index edb5cec0..1f65aa6e 100644
--- a/HISTORY.rst
+++ b/HISTORY.rst
@@ -9,6 +9,12 @@ Release History
 **Improvements**
 
 - Documented how to use Jupytext with JupyterLab 0.35 (#299)
+- Improved the usability of `read` and `write` functions (#292)
+
+**BugFixes**
+
+- Jupytext now ships the ``jupyterlab-jupytext`` extension in version 1.0.2. The version 1.0.1 erroneously introduces a `target_formats` metadata in the jupytext section, instead of `formats`, and works only after two clicks.
+
 
 1.2.0 (2019-07-18)
 ++++++++++++++++++++++
diff --git a/docs/using-cli.md b/docs/using-cli.md
index 28631b72..b62e3c07 100644
--- a/docs/using-cli.md
+++ b/docs/using-cli.md
@@ -70,6 +70,34 @@ git reset HEAD **/*.ipynb
 ```
 Note that these hooks do not update the `.ipynb` notebook when you pull. Make sure to either run `jupytext` in the other direction, or to use our paired notebook and our contents manager for Jupyter. Also, Jupytext does not offer a merge driver. If a conflict occurs, solve it on the text representation and then update or recreate the `.ipynb` notebook. Or give a try to nbdime and its [merge driver](https://nbdime.readthedocs.io/en/stable/vcs.html#merge-driver).
 
+## Using Jupytext with the pre-commit package manager
+
+Using Jupytext with the [pre-commit package manager](https://pre-commit.com/) is another option. You could add the following to your `.pre-commit-config.yaml` file:
+```
+repos:
+-   repo: local
+    hooks:
+    - id: jupytext
+      name: jupytext
+      entry: jupytext --to md
+      files: .ipynb
+      language: python
+```
+
+Alternatively, you could also create your own pre-commit hook. A sample `jupytext_hook.py` script could be
+
+```python
+import sys
+import pathlib
+import jupytext
+
+nbfile = pathlib.Path(sys.argv[1])
+mdfile = nbfile.with_suffix(".md")
+
+nb = jupytext.read(nbfile)
+jupytext.write(nb, mdfile)
+```
+
 ## Testing the round-trip conversion
 
 Representing Jupyter notebooks as scripts requires a solid round trip conversion. You don't want your notebooks (nor your scripts) to be modified because you are converting them to the other form. Our test suite includes a few hundred tests to ensure that round trip conversion is safe.
diff --git a/jupytext/jupytext.py b/jupytext/jupytext.py
index 08d951cd..34aa1373 100644
--- a/jupytext/jupytext.py
+++ b/jupytext/jupytext.py
@@ -6,7 +6,6 @@ import sys
 import logging
 import warnings
 from copy import copy, deepcopy
-from ipython_genutils import py3compat
 from nbformat.v4.rwbase import NotebookReader, NotebookWriter
 from nbformat.v4.nbbase import new_notebook, new_code_cell, NotebookNode
 import nbformat
@@ -190,8 +189,16 @@ class TextNotebookConverter(NotebookReader, NotebookWriter):
 
 
 def reads(text, fmt, as_version=4, **kwargs):
-    """Read a notebook from a string"""
-    fmt = copy(fmt)
+    """
+    Read a notebook from a string
+
+    :param text: the text representation of the notebook
+    :param fmt: (optional) the jupytext format like `md`, `py:percent`, ...
+    :param as_version: see nbformat.reads
+    :param kwargs: (not used) additional parameters for nbformat.reads
+    :return: the notebook
+    """
+    fmt = copy(fmt) if fmt else divine_format(text)
     fmt = long_form_one_format(fmt)
     ext = fmt['extension']
 
@@ -221,30 +228,48 @@ def reads(text, fmt, as_version=4, **kwargs):
 
 
 def read(fp, as_version=4, fmt=None, **kwargs):
-    """Read a notebook from a file"""
+    """"
+    Read a notebook from a file name or a file object
+
+    :param fp: a file name or a file object
+    :param as_version: see nbformat.read
+    :param fmt: (optional) the jupytext format like `md`, `py:percent`, ...
+    :param kwargs: (not used) additional parameters for nbformat.read
+    :return: the notebook
+    """
     if fp == '-':
         text = sys.stdin.read()
-        fmt = fmt or divine_format(text)
         return reads(text, fmt)
 
-    if isinstance(fp, (py3compat.unicode_type, bytes)):
+    if not hasattr(fp, 'read'):
+        # Treat fp as a file name
+        fp = str(fp)
         _, ext = os.path.splitext(fp)
         fmt = copy(fmt or {})
         fmt.update({'extension': ext})
         with io.open(fp, encoding='utf-8') as stream:
             return read(stream, as_version=as_version, fmt=fmt, **kwargs)
 
-    fmt = long_form_one_format(fmt)
-    if fmt['extension'] == '.ipynb':
-        notebook = nbformat.read(fp, as_version, **kwargs)
-        rearrange_jupytext_metadata(notebook.metadata)
-        return notebook
+    if fmt is not None:
+        fmt = long_form_one_format(fmt)
+        if fmt['extension'] == '.ipynb':
+            notebook = nbformat.read(fp, as_version, **kwargs)
+            rearrange_jupytext_metadata(notebook.metadata)
+            return notebook
 
     return reads(fp.read(), fmt, **kwargs)
 
 
 def writes(notebook, fmt, version=nbformat.NO_CONVERT, **kwargs):
-    """Write a notebook to a string"""
+    """"
+    Write a notebook to a file name or a file object
+
+    :param notebook: the notebook
+    :param fmt: the jupytext format like `md`, `py:percent`, ...
+    :param version: see nbformat.writes
+    :param kwargs: (not used) additional parameters for nbformat.writes
+    :return: the text representation of the notebook
+    """
     metadata = deepcopy(notebook.metadata)
     rearrange_jupytext_metadata(metadata)
     fmt = copy(fmt)
@@ -278,14 +303,24 @@ def writes(notebook, fmt, version=nbformat.NO_CONVERT, **kwargs):
 
 
 def write(nb, fp, version=nbformat.NO_CONVERT, fmt=None, **kwargs):
-    """Write a notebook to a file"""
+    """"
+    Write a notebook to a file name or a file object
+
+    :param nb: the notebook
+    :param fp: a file name or a file object
+    :param version: see nbformat.write
+    :param fmt: (optional if fp is a file name) the jupytext format like `md`, `py:percent`, ...
+    :param kwargs: (not used) additional parameters for nbformat.write
+    """
     if fp == '-':
         write(nb, sys.stdout, version=version, fmt=fmt, **kwargs)
         return
 
-    if isinstance(fp, (py3compat.unicode_type, bytes)):
-        fmt = copy(fmt or {})
+    if not hasattr(fp, 'write'):
+        # Treat fp as a file name
+        fp = str(fp)
         _, ext = os.path.splitext(fp)
+        fmt = copy(fmt or {})
         fmt = long_form_one_format(fmt, update={'extension': ext})
         create_prefix_dir(fp, fmt)
 
diff --git a/requirements-dev.txt b/requirements-dev.txt
index 4955baaf..c1b8f940 100644
--- a/requirements-dev.txt
+++ b/requirements-dev.txt
@@ -5,3 +5,6 @@ codecov
 notebook
 jupyter_client
 nbconvert
+
+# Python 2
+pathlib
\ No newline at end of file


Test patch:
diff --git a/tests/test_read_write_functions.py b/tests/test_read_write_functions.py
new file mode 100644
index 00000000..06ff8f13
--- /dev/null
+++ b/tests/test_read_write_functions.py
@@ -0,0 +1,78 @@
+from io import StringIO
+from pathlib import Path
+import nbformat
+from nbformat.v4.nbbase import new_notebook, new_markdown_cell
+import jupytext
+from jupytext.compare import compare
+
+
+def test_simple_hook(tmpdir):
+    nb_file = str(tmpdir.join('notebook.ipynb'))
+    md_file = str(tmpdir.join('notebook.md'))
+    nbformat.write(new_notebook(cells=[new_markdown_cell('Some text')]), nb_file)
+
+    nb = jupytext.read(nb_file)
+    jupytext.write(nb, md_file)
+
+    with open(md_file) as fp:
+        text = fp.read()
+
+    assert 'Some text' in text.splitlines()
+
+
+def test_simple_hook_with_explicit_format(tmpdir):
+    nb_file = str(tmpdir.join('notebook.ipynb'))
+    py_file = str(tmpdir.join('notebook.py'))
+    nbformat.write(new_notebook(cells=[new_markdown_cell('Some text')]), nb_file)
+
+    nb = jupytext.read(nb_file)
+    jupytext.write(nb, py_file, fmt='py:percent')
+
+    with open(py_file) as fp:
+        text = fp.read()
+
+    assert '# %% [markdown]' in text.splitlines()
+    assert '# Some text' in text.splitlines()
+
+
+def test_no_error_on_path_object(tmpdir):
+    nb_file = Path(str(tmpdir.join('notebook.ipynb')))
+    md_file = nb_file.with_suffix('.md')
+    nbformat.write(new_notebook(cells=[new_markdown_cell('Some text')]), str(nb_file))
+
+    nb = jupytext.read(nb_file)
+    jupytext.write(nb, md_file)
+
+
+def test_read_ipynb_from_stream():
+    def stream():
+        return StringIO(u"""{
+ "cells": [
+  {
+   "cell_type": "code",
+   "metadata": {},
+   "source": [
+    "1 + 1"
+   ]
+  }
+ ],
+ "metadata": {},
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
+""")
+
+    nb = jupytext.read(stream())
+    nb2 = jupytext.read(stream(), fmt='ipynb')
+    compare(nb, nb2)
+
+
+def test_read_py_percent_from_stream():
+    def stream():
+        return StringIO(u"""# %%
+1 + 1
+""")
+
+    nb = jupytext.read(stream())
+    nb2 = jupytext.read(stream(), fmt='py:percent')
+    compare(nb, nb2)