Repository: mwouts/jupytext (language: python)

Issue:
File modification times and symbolic link ".ipynb seems more recent than .py" 
I'm a very newbie with jupytext and git, so I set up an odd pattern of working which seems to have exposed an issue.
I've installed jupytext and created a notebook for trying it out in the source tree I usually use for my jupyter projects.  
I successfully paired the notebook with a .py file (I use jupyterlab so the command palette 'pair notebook with light script')
So I have
```
 jupyter/notebook.ipynb
 jupyter/notebook.py
```
All works fine.

Now I went off piste as follows:
- move the notebook.py file to a project in my github environment
- put a symbolic link to that github/notebook.py file in my jupyter directory, so now I have:
```
jupyter/notebook.ipynb
jupyter/notebook.py -> github/notebook.py
```
Again, all works fine with git seeing just the input cells of my notebook and I get nice succinct diffs (great, thank you).  

However, when I come to reload the notebook.ipynb or notebook.py,  I get the file sync error.  

It would appear that the modification timestamp of the notebook.py symbolic link is being compared, rather than the modification timestamp of the file it points at.

_(my reasoning for using the symbolic link to the .py file was that I wanted to version control the text and code portions of the notebook using git & github, while hiding from git the .ipynb file, .ipynb_checkpoints and any other files containing private data that I might have in my jupyter project tree.  I can work around by adding entries to .gitignore to explicitly hide, which doesn't feel quite so safe as using links to explicitly expose)_


Discussion / hints:
Hi @siddalp-actual , thanks for sharing this!

Well if I am correct we are getting the timestamp from the original context manager, through
`notebook.services.contents.filemanager.FileContentsManager._base_model`. That function seems to get the timestamp with this code:
```
info = os.lstat(os_path)
last_modified = tz.utcfromtimestamp(info.st_mtime)
```
Do you know how to get the timestamp of the file pointed by the symbolic link? Maybe then we could make a PR on `jupyter/notebook` that would solve your issue.

Alternatively, you could
- set `outdated_text_notebook_margin` to a large number (infinite), but you lose the corresponding security - if you modify the `.ipynb` file without having Jupytext activated, those changes will be lost the next time you open the notebook with Jupytext on (so use this carefully) 
- or, and maybe that is a better option, you could pair your py and ipynb files to different subfolders like in the third example [here](https://jupytext.readthedocs.io/en/latest/config.html#configuring-paired-notebooks-globally), i.e. pair your `.ipynb` notebooks in the folder `notebooks` with text files under another folder `scripts`, by adding this to your `jupytext.toml` file:
```
# Pair notebooks in subfolders of 'notebooks' to scripts in subfolders of 'scripts'
default_jupytext_formats = "notebooks///ipynb,scripts///py:light"
```


Thanks Marc,
I have created a notebook with a recreation of the issue and the gist of a fix [here](https://github.com/siddalp-actual/versionWithJupytext/blob/main/Symbolic%20Link%20Mtime.py).  A link can point to a link... so really ought to have a WHILE construct with some max depth protection against infinite loops.

I will also look into your suggestion of using the triple / root  in default_jupytext_formats
Thanks @siddalp-actual .

I've added this to the next milestone - my objective here is to add a test that uses the `jupytext.toml` file documented above, and in which `scripts` is a symbolic link to another folder.

Fix patch:
diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md
index 9928ecc2f..0895d4bef 100644
--- a/docs/CHANGELOG.md
+++ b/docs/CHANGELOG.md
@@ -12,7 +12,8 @@ Jupytext ChangeLog
 - Indented magic commands are supported ([#694](https://github.com/mwouts/jupytext/issues/694))
 
 **Added**
-- Added a test that ensures that `py:percent` scripts end with exactly one blank line ([#682](https://github.com/mwouts/jupytext/issues/682))
+- We made sure that `py:percent` scripts end with exactly one blank line ([#682](https://github.com/mwouts/jupytext/issues/682))
+- We checked that Jupytext works well with symbolic links to folders (not files!) ([#696](https://github.com/mwouts/jupytext/issues/696))
 - We have added `isort` and `autoflake8` to the `pre-commit` configuration file used for developing the Jupytext project ([#709](https://github.com/mwouts/jupytext/issues/709))
 
 
diff --git a/docs/config.md b/docs/config.md
index 204a7e807..dcac5a521 100644
--- a/docs/config.md
+++ b/docs/config.md
@@ -68,14 +68,10 @@ You can pair notebooks in trees with a `root_prefix` separated with three slashe
 default_jupytext_formats = "notebooks///ipynb,scripts///py:percent"
 ```
 The `root_prefix` is matched with the top-most parent folder of the matching name, not above the Jupytext configuration file.
-A document is not considered a paired notebook when its path does not match the paired path specifications.
 
+For instance, with the pairing above, a notebook with path `/home/user/jupyter/notebooks/project1/example.ipynb` will be paired with the Python file `/home/user/jupyter/scripts/project1/example.py`.
 
-With only two slashes, notebooks are paired in directories but not in subfolders:
-```
-# Pair notebooks in folders named 'notebooks' to folders named 'scripts'
-default_jupytext_formats = "notebooks//ipynb,scripts//py:percent"
-```
+In addition to the `root_prefix`, you can use symbolic links if you wish to distribute your notebook folders at different places. Be sure to use symbolic links on folders, not files ([#696](https://github.com/mwouts/jupytext/issues/696)).
 
 To disable the default pairing for an individual notebook, set formats to a single format, with e.g.:
 ```bash


Test patch:
diff --git a/tests/test_cm_config.py b/tests/test_cm_config.py
index 5dcc5c228..76a63b526 100644
--- a/tests/test_cm_config.py
+++ b/tests/test_cm_config.py
@@ -1,4 +1,5 @@
 import os
+import sys
 import unittest.mock as mock
 
 import pytest
@@ -123,3 +124,50 @@ def fake_global_config_directory():
             "notebook.ipynb",
             "notebook.Rmd",
         }
+
+
+@pytest.mark.skipif(
+    sys.platform.startswith("win"),
+    reason="AttributeError: 'LocalPath' object has no attribute 'mksymlinkto'",
+)
+def test_paired_files_and_symbolic_links(tmpdir):
+    """We test that we don't get issues when pairing files into folders
+    that are symbolic links"""
+
+    actual = tmpdir.mkdir("actual_files")
+    actual_notebooks = actual.mkdir("notebooks")
+    actual_scripts = actual.mkdir("scripts")
+
+    # Open the contents manager in another dir
+    jupyter_dir = tmpdir.mkdir("jupyter_dir")
+    cm = jupytext.TextFileContentsManager()
+    cm.root_dir = str(jupyter_dir)
+
+    # Create sym links to the notebook/script folders
+    jupyter_dir.join("link_to_notebooks").mksymlinkto(actual_notebooks)
+    jupyter_dir.join("link_to_scripts").mksymlinkto(actual_scripts)
+
+    # Pair the notebooks in the linked folders
+    jupyter_dir.join("jupytext.toml").write(
+        'default_jupytext_formats = "link_to_notebooks///ipynb,link_to_scripts///py:percent"'
+    )
+
+    # Save a notebook
+    cm.save(notebook_model(SAMPLE_NOTEBOOK), "link_to_notebooks/notebook.ipynb")
+
+    # This creates two files in the destinations folders
+    assert actual_notebooks.join("notebook.ipynb").isfile()
+    assert actual_scripts.join("notebook.py").isfile()
+
+    # Re-open the notebook (here, the text version)
+    cm.get("link_to_scripts/notebook.py")
+
+    # Update the text version
+    jupyter_dir.join("link_to_scripts").join("notebook.py").write_text(
+        "# %%\n3 + 3\n", encoding="utf-8"
+    )
+
+    # Reload and make sure that we get the updated notebook
+    model = cm.get("link_to_notebooks/notebook.ipynb")
+    nb = model["content"]
+    assert nb.cells == [new_code_cell("3 + 3")]