Repository: mwouts/jupytext
Pull request #666 (2020-10-15 17:48:03)

Filter out the execution cell metadata by default
And fix the issue documented at #656 - the global configuration was uneffective on cell metadata present in the text file.

codecov[bot]: # [Codecov](https://codecov.io/gh/mwouts/jupytext/pull/666?src=pr&el=h1) Report
> Merging [#666](https://codecov.io/gh/mwouts/jupytext/pull/666?src=pr&el=desc) into [master](https://codecov.io/gh/mwouts/jupytext/commit/d629246bc9a1c4b52a36ed85d99c6c7422955844?el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/mwouts/jupytext/pull/666/graphs/tree.svg?width=650&height=150&src=pr&token=a1pNjsiILi)](https://codecov.io/gh/mwouts/jupytext/pull/666?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master     #666   +/-   ##
=======================================
  Coverage   99.17%   99.17%           
=======================================
  Files          94       94           
  Lines        9371     9382   +11     
=======================================
+ Hits         9294     9305   +11     
  Misses         77       77           
```


| [Impacted Files](https://codecov.io/gh/mwouts/jupytext/pull/666?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [jupytext/cell\_metadata.py](https://codecov.io/gh/mwouts/jupytext/pull/666/diff?src=pr&el=tree#diff-anVweXRleHQvY2VsbF9tZXRhZGF0YS5weQ==) | `98.98% <ø> (ø)` | |
| [jupytext/metadata\_filter.py](https://codecov.io/gh/mwouts/jupytext/pull/666/diff?src=pr&el=tree#diff-anVweXRleHQvbWV0YWRhdGFfZmlsdGVyLnB5) | `99.13% <100.00%> (-0.87%)` | :arrow_down: |
| [tests/test\_metadata\_filter.py](https://codecov.io/gh/mwouts/jupytext/pull/666/diff?src=pr&el=tree#diff-dGVzdHMvdGVzdF9tZXRhZGF0YV9maWx0ZXIucHk=) | `100.00% <100.00%> (ø)` | |
| [jupytext/formats.py](https://codecov.io/gh/mwouts/jupytext/pull/666/diff?src=pr&el=tree#diff-anVweXRleHQvZm9ybWF0cy5weQ==) | `98.74% <0.00%> (+0.31%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/mwouts/jupytext/pull/666?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/mwouts/jupytext/pull/666?src=pr&el=footer). Last update [c693dc3...eef6a00](https://codecov.io/gh/mwouts/jupytext/pull/666?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index de3098b7..82cd0d87 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -15,11 +15,13 @@
 - The GitHub actions run on both push events and pull requests, and duplicate jobs are skipped (#605)
 - Jupytext has a `tox.ini` file, thanks to Chris Sewell (#605)
 - Jupytext is tested against Python 3.9
+- The `execution` cell metadata is now filtered by default (#656)
 
 **Fixed**
 - Optional dependency on `sphinx-gallery` frozen to version `~=0.7.0` (#614)
 - Codecov/patch reports should be OK now (#639)
 - Jupytext tests work on non-English locales (#636)
+- Cell metadata that are already present in text notebook can be filtered out using a config file (#656)
 
 
 1.6.0 (2020-09-01)
diff --git a/environment-ci.yml b/environment-ci.yml
index bbf8985a..d41b8d57 100644
--- a/environment-ci.yml
+++ b/environment-ci.yml
@@ -16,6 +16,6 @@ dependencies:
   - pytest-cov
   - coverage
   - flake8
-  - pandoc>=2.9
+  - pandoc>=2.11
   # pathlib is required on Python 2
   - pathlib
diff --git a/environment.yml b/environment.yml
index f4c729af..c2033e67 100644
--- a/environment.yml
+++ b/environment.yml
@@ -26,7 +26,7 @@ dependencies:
   - pip
   - setuptools
   - twine
-  - pandoc>=2.9
+  - pandoc>=2.11
   # make html in docs folder
   - sphinx
   - tox-conda
diff --git a/jupytext/cell_metadata.py b/jupytext/cell_metadata.py
index 0df74a90..7dbb1ed3 100644
--- a/jupytext/cell_metadata.py
+++ b/jupytext/cell_metadata.py
@@ -56,6 +56,7 @@ _IGNORE_CELL_METADATA = ",".join(
         "collapsed",
         "scrolled",
         "trusted",
+        "execution",
         "ExecuteTime",
     ]
     + _JUPYTEXT_CELL_METADATA
diff --git a/jupytext/metadata_filter.py b/jupytext/metadata_filter.py
index 75bb634d..b9633c63 100644
--- a/jupytext/metadata_filter.py
+++ b/jupytext/metadata_filter.py
@@ -81,7 +81,15 @@ def metadata_filter_as_string(metadata_filter):
 def update_metadata_filters(metadata, jupyter_md, cell_metadata):
     """Update or set the notebook and cell metadata filters"""
 
-    if "cell_metadata_filter" in metadata.get("jupytext", {}):
+    if not jupyter_md:
+        # Set a metadata filter equal to the current metadata in script
+        metadata.setdefault("jupytext", {})["notebook_metadata_filter"] = "-all"
+        metadata["jupytext"].setdefault(
+            "cell_metadata_filter",
+            metadata_filter_as_string({"additional": cell_metadata, "excluded": "all"}),
+        )
+    elif "cell_metadata_filter" in metadata.get("jupytext", {}):
+        # Update the existing metadata filter
         metadata_filter = metadata_filter_as_dict(
             metadata.get("jupytext", {})["cell_metadata_filter"]
         )
@@ -97,14 +105,6 @@ def update_metadata_filters(metadata, jupyter_md, cell_metadata):
         metadata.setdefault("jupytext", {})[
             "cell_metadata_filter"
         ] = metadata_filter_as_string(metadata_filter)
-
-    if not jupyter_md:
-        # Set a metadata filter equal to the current metadata in script
-        cell_metadata = {"additional": cell_metadata, "excluded": "all"}
-        metadata.setdefault("jupytext", {})["notebook_metadata_filter"] = "-all"
-        metadata.setdefault("jupytext", {})[
-            "cell_metadata_filter"
-        ] = metadata_filter_as_string(cell_metadata)
     else:
         # Update the notebook metadata filter to include existing entries 376
         nb_md_filter = (


Test patch:
diff --git a/tests/notebooks/mirror/ipynb_to_pandoc/julia_benchmark_plotly_barchart.md b/tests/notebooks/mirror/ipynb_to_pandoc/julia_benchmark_plotly_barchart.md
index 4d06e8ae..e1d22186 100644
--- a/tests/notebooks/mirror/ipynb_to_pandoc/julia_benchmark_plotly_barchart.md
+++ b/tests/notebooks/mirror/ipynb_to_pandoc/julia_benchmark_plotly_barchart.md
@@ -3,7 +3,7 @@ jupyter:
   kernelspec:
     display_name: Julia 1.1.1
     language: julia
-    name: 'julia-1.1'
+    name: julia-1.1
   nbformat: 4
   nbformat_minor: 1
 ---
diff --git a/tests/test_metadata_filter.py b/tests/test_metadata_filter.py
index 95dc759d..377174e8 100644
--- a/tests/test_metadata_filter.py
+++ b/tests/test_metadata_filter.py
@@ -1,6 +1,7 @@
 import pytest
-from nbformat.v4.nbbase import new_notebook
+from nbformat.v4.nbbase import new_notebook, new_code_cell
 from jupytext import reads, writes
+from jupytext.cli import jupytext as jupytext_cli
 from jupytext.metadata_filter import filter_metadata, metadata_filter_as_dict
 
 
@@ -120,3 +121,50 @@ def test_filter_nested_metadata():
 
     # That one is not supported yet
     # assert filter_metadata(metadata, 'I.1.a', '-I') == {'I': {'1': {'a': 1}}}
+
+
+def test_filter_out_execution_metadata():
+    nb = new_notebook(
+        cells=[
+            new_code_cell(
+                "1 + 1",
+                metadata={
+                    "execution": {
+                        "iopub.execute_input": "2020-10-12T19:13:45.306603Z",
+                        "iopub.status.busy": "2020-10-12T19:13:45.306233Z",
+                        "iopub.status.idle": "2020-10-12T19:13:45.316103Z",
+                        "shell.execute_reply": "2020-10-12T19:13:45.315429Z",
+                        "shell.execute_reply.started": "2020-10-12T19:13:45.306577Z",
+                    }
+                },
+            )
+        ]
+    )
+
+    text = writes(nb, fmt="py:percent")
+    assert "execution" not in text
+
+
+def test_default_config_has_priority_over_current_metadata(
+    tmpdir,
+    text="""# %% some_metadata_key=5
+1 + 1
+""",
+):
+    py_file = tmpdir.join("notebook.py")
+    py_file.write(text)
+
+    cfg_file = tmpdir.join("jupytext.toml")
+    cfg_file.write(
+        """default_jupytext_formats = "ipynb,py:percent"
+default_cell_metadata_filter = "-some_metadata_key"
+"""
+    )
+
+    jupytext_cli([str(py_file), "--sync"])
+    assert (
+        py_file.read()
+        == """# %%
+1 + 1
+"""
+    )