Repository: jupyter/nbconvert

Issue:
[PDF] Unable to recognise local paths
TLDR: nbconvert appears to be unable to recognise local path. I tried to change a notebook to pdf and it says that an image does not exist.  It says ../images/errorcorrectionmodel.png does not exist when it does. VSCode was showing a link to the file in the error message.

Here is my directory structure:

.qecft
├── build
│   ├── cec
│   │   ├── cec.ipynb
│   │   ├── ....
│   ├──images
│   │   ├── errorcorrectionalmodel.png
│   │   ├── ...
│   ├── ....
|──images
│   ├── errorcorrectionalmodel.png
│   ├── ....

OS Name:                   Microsoft Windows 10 Home
OS Version:                10.0.19044 N/A Build 19044
MiKTeX-XeTeX 4.10 (MiKTeX 23.5)
jupyter nbconvert --version: 7.4.0

Here is the command and error log:
(envqecft) C:\Users\khanm\OneDrive\Desktop\Projects\qecft\build\cec>jupyter nbconvert --to pdf cec.ipynb
[NbConvertApp] Converting notebook cec.ipynb to pdf
C:\Users\khanm\OneDrive\Desktop\Projects\qecft\envqecft\lib\site-packages\nbformat\__init__.py:93: MissingIDFieldWarning: Code cell is missing an id field, this will become a hard error in future nbformat versions. You may want to use `normalize()` on your notebooks before validations (available since nbformat 5.1.4). Previous versions of nbformat are fixing this issue transparently, and will stop doing so in the future.
  validate(nb)
[NbConvertApp] Writing 61774 bytes to notebook.tex
[NbConvertApp] Building PDF
[NbConvertApp] Running xelatex 3 times: ['xelatex', 'notebook.tex', '-quiet']
[NbConvertApp] CRITICAL | x failed: xelatex notebook.tex -quiet
b'\r\ndvipdfmx:fatal: Image inclusion failed. Could not find file: ../images/errorcorrectionmodel.png\r\n\r\nNo output PDF file written.\r\n\r\nSorry, but xelatex did not succeed.\r\n\r\nThe log file hopefully contains the information to get MiKTeX going again:\r\n\r\n  C:\\Users\\khanm\\AppData\\Local\\MiKTeX\\miktex\\log\\xelatex.log\r\n'
Traceback (most recent call last):
  File "C:\Users\khanm\miniconda3\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\khanm\miniconda3\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\khanm\OneDrive\Desktop\Projects\qecft\envqecft\Scripts\jupyter-nbconvert.EXE\__main__.py", line 7, in <module>
  File "C:\Users\khanm\OneDrive\Desktop\Projects\qecft\envqecft\lib\site-packages\jupyter_core\application.py", line 277, in launch_instance
    return super().launch_instance(argv=argv, **kwargs)
  File "C:\Users\khanm\OneDrive\Desktop\Projects\qecft\envqecft\lib\site-packages\traitlets\config\application.py", line 1043, in launch_instance
    app.start()
  File "C:\Users\khanm\OneDrive\Desktop\Projects\qecft\envqecft\lib\site-packages\nbconvert\nbconvertapp.py", line 414, in start
    self.convert_notebooks()
  File "C:\Users\khanm\OneDrive\Desktop\Projects\qecft\envqecft\lib\site-packages\nbconvert\nbconvertapp.py", line 589, in convert_notebooks
    self.convert_single_notebook(notebook_filename)
  File "C:\Users\khanm\OneDrive\Desktop\Projects\qecft\envqecft\lib\site-packages\nbconvert\nbconvertapp.py", line 555, in convert_single_notebook
    output, resources = self.export_single_notebook(
  File "C:\Users\khanm\OneDrive\Desktop\Projects\qecft\envqecft\lib\site-packages\nbconvert\nbconvertapp.py", line 481, in export_single_notebook
    output, resources = self.exporter.from_filename(
  File "C:\Users\khanm\OneDrive\Desktop\Projects\qecft\envqecft\lib\site-packages\nbconvert\exporters\templateexporter.py", line 388, in from_filename
    return super().from_filename(filename, resources, **kw)  # type:ignore
  File "C:\Users\khanm\OneDrive\Desktop\Projects\qecft\envqecft\lib\site-packages\nbconvert\exporters\exporter.py", line 201, in from_filename
    return self.from_file(f, resources=resources, **kw)
  File "C:\Users\khanm\OneDrive\Desktop\Projects\qecft\envqecft\lib\site-packages\nbconvert\exporters\templateexporter.py", line 394, in from_file
    return super().from_file(file_stream, resources, **kw)  # type:ignore
  File "C:\Users\khanm\OneDrive\Desktop\Projects\qecft\envqecft\lib\site-packages\nbconvert\exporters\exporter.py", line 220, in from_file
    return self.from_notebook_node(
  File "C:\Users\khanm\OneDrive\Desktop\Projects\qecft\envqecft\lib\site-packages\nbconvert\exporters\pdf.py", line 199, in from_notebook_node
    self.run_latex(tex_file)
  File "C:\Users\khanm\OneDrive\Desktop\Projects\qecft\envqecft\lib\site-packages\nbconvert\exporters\pdf.py", line 168, in run_latex
    return self.run_command(
  File "C:\Users\khanm\OneDrive\Desktop\Projects\qecft\envqecft\lib\site-packages\nbconvert\exporters\pdf.py", line 158, in run_command
    raise raise_on_failure(msg)
nbconvert.exporters.pdf.LatexFailed: PDF creating failed, captured latex output:
Failed to run "xelatex notebook.tex -quiet" command:

dvipdfmx:fatal: Image inclusion failed. Could not find file: ../images/errorcorrectionmodel.png

No output PDF file written.

Sorry, but xelatex did not succeed.

The log file hopefully contains the information to get MiKTeX going again:

  C:\Users\khanm\AppData\Local\MiKTeX\miktex\log\xelatex.log

-------------------------------------------

MiKTeX logs:
2023-05-31 23:12:00,183+0300 INFO  xelatex - this process (6244) started by pandoc in directory C:\Users\khanm\OneDrive\Desktop\Projects\qecft with command line: xelatex -halt-on-error -interaction nonstopmode -output-directory C:/Users/khanm/AppData/Local/Temp/tex2pdf.-5bc1ca47d967cf83 C:/Users/khanm/AppData/Local/Temp/tex2pdf.-5bc1ca47d967cf83/input.tex
2023-05-31 23:12:00,184+0300 INFO  xelatex - running on Windows 10.0.19044

This is nowhere near the current time. 

Any ideas on how to fix this?

Discussion:

Patch:
diff --git a/nbconvert/exporters/latex.py b/nbconvert/exporters/latex.py
index d3ae76ef..a23013cc 100644
--- a/nbconvert/exporters/latex.py
+++ b/nbconvert/exporters/latex.py
@@ -3,12 +3,14 @@
 # Copyright (c) Jupyter Development Team.
 # Distributed under the terms of the Modified BSD License.
 
+import os
 
 from traitlets import default
 from traitlets.config import Config
 
 from nbconvert.filters.filter_links import resolve_references
 from nbconvert.filters.highlight import Highlight2Latex
+from nbconvert.filters.pandoc import ConvertExplicitlyRelativePaths
 
 from .templateexporter import TemplateExporter
 
@@ -77,6 +79,16 @@ class LatexExporter(TemplateExporter):
         )
         self.register_filter("highlight_code", highlight_code)
 
+        # Need to make sure explicit relative paths are visible to latex for pdf conversion
+        # https://github.com/jupyter/nbconvert/issues/1998
+        nb_path = resources.get("metadata", {}).get("path") if resources else None
+        texinputs = os.path.abspath(nb_path) if nb_path else os.getcwd()
+        convert_explicitly_relative_paths = self.filters.get(
+            "convert_explicitly_relative_paths",
+            ConvertExplicitlyRelativePaths(texinputs=texinputs, parent=self),
+        )
+        self.register_filter("convert_explicitly_relative_paths", convert_explicitly_relative_paths)
+
         return super().from_notebook_node(nb, resources, **kw)
 
     def _create_environment(self):
diff --git a/nbconvert/filters/pandoc.py b/nbconvert/filters/pandoc.py
index 09eecfab..165e98ba 100644
--- a/nbconvert/filters/pandoc.py
+++ b/nbconvert/filters/pandoc.py
@@ -1,4 +1,12 @@
-"""Convert between any two formats using pandoc."""
+"""
+Convert between any two formats using pandoc,
+and related filters
+"""
+import os
+
+from pandocfilters import Image, applyJSONFilters  # type:ignore
+
+from nbconvert.utils.base import NbConvertBase
 from nbconvert.utils.pandoc import pandoc
 
 
@@ -23,3 +31,48 @@ def convert_pandoc(source, from_format, to_format, extra_args=None):
         Output as returned by pandoc.
     """
     return pandoc(source, from_format, to_format, extra_args=extra_args)
+
+
+# When converting to pdf, explicitly relative references
+# like "./" and "../" doesn't work with TEXINPUTS.
+# So we need to convert them to absolute paths.
+# See https://github.com/jupyter/nbconvert/issues/1998
+class ConvertExplicitlyRelativePaths(NbConvertBase):
+    """A converter that handles relative path references."""
+
+    def __init__(self, texinputs=None, **kwargs):
+        """Initialize the converter."""
+        # texinputs should be the directory of the notebook file
+        self.nb_dir = os.path.abspath(texinputs) if texinputs else ""
+        self.ancestor_dirs = self.nb_dir.split("/")
+        super().__init__(**kwargs)
+
+    def __call__(self, source):
+        """Invoke the converter."""
+        # If this is not set for some reason, we can't do anything,
+        if self.nb_dir:
+            return applyJSONFilters([self.action], source)
+        return source
+
+    def action(self, key, value, frmt, meta):
+        """Perform the action."""
+        # Convert explicitly relative paths:
+        # ./path -> path  (This should be visible to the latex engine since TEXINPUTS already has .)
+        # ../path -> /abs_path
+        # assuming all relative references are at the start of a given path
+        if key == "Image":
+            # Image seems to have this composition, according to https://github.com/jgm/pandoc-types
+            attr, caption, [filename, typedef] = value
+
+            if filename[:2] == "./":
+                filename = filename[2:]
+            elif filename[:3] == "../":
+                n_up = 0
+                while filename[:3] == "../":
+                    n_up += 1
+                    filename = filename[3:]
+                ancestors = "/".join(self.ancestor_dirs[:-n_up]) + "/"
+                filename = ancestors + filename
+            return Image(attr, caption, [filename, typedef])
+        # If not image, return "no change"
+        return None
diff --git a/share/templates/latex/document_contents.tex.j2 b/share/templates/latex/document_contents.tex.j2
index 4dc172b1..61935c69 100644
--- a/share/templates/latex/document_contents.tex.j2
+++ b/share/templates/latex/document_contents.tex.j2
@@ -65,7 +65,7 @@
 
 % Render markdown
 ((* block markdowncell scoped *))
-    ((( cell.source | citation2latex | strip_files_prefix | convert_pandoc('markdown+tex_math_double_backslash', 'json',extra_args=[]) | resolve_references | convert_pandoc('json','latex'))))
+    ((( cell.source | citation2latex | strip_files_prefix | convert_pandoc('markdown+tex_math_double_backslash', 'json',extra_args=[]) | resolve_references | convert_explicitly_relative_paths | convert_pandoc('json','latex'))))
 ((* endblock markdowncell *))
 
 % Don't display unknown types


Test patch:
diff --git a/nbconvert/filters/tests/test_pandoc.py b/nbconvert/filters/tests/test_pandoc.py
new file mode 100644
index 00000000..21d9bb9c
--- /dev/null
+++ b/nbconvert/filters/tests/test_pandoc.py
@@ -0,0 +1,59 @@
+"""
+Module with tests for Pandoc filters
+"""
+
+# Copyright (c) Jupyter Development Team.
+# Distributed under the terms of the Modified BSD License.
+
+import json
+
+from ...tests.base import TestsBase
+from ...tests.utils import onlyif_cmds_exist
+from ..pandoc import ConvertExplicitlyRelativePaths, convert_pandoc
+
+
+class TestPandocFilters(TestsBase):
+    @onlyif_cmds_exist("pandoc")
+    def test_convert_explicitly_relative_paths(self):
+        """
+        Do the image links in a markdown file located in dir get processed correctly?
+        """
+        inp_dir = "/home/user/src"
+        fltr = ConvertExplicitlyRelativePaths(texinputs=inp_dir)
+
+        # pairs of input, expected
+        tests = {
+            # TEXINPUTS is enough, abs_path not needed
+            "im.png": "im.png",
+            "./im.png": "im.png",
+            "./images/im.png": "images/im.png",
+            # TEXINPUTS is not enough, abs_path needed
+            "../im.png": "/home/user/im.png",
+            "../images/im.png": "/home/user/images/im.png",
+            "../../images/im.png": "/home/images/im.png",
+        }
+
+        # this shouldn't be modified by the filter
+        # since it is a code block inside markdown,
+        # not an image link itself
+        fake = """
+        ```
+        \\includegraphics{../fake.png}
+        ```
+        """
+
+        # convert to markdown image
+        def foo(filename):
+            return f"![]({filename})"
+
+        # create input markdown and convert to pandoc json
+        inp = convert_pandoc(fake + "\n".join(map(foo, tests.keys())), "markdown", "json")
+        expected = convert_pandoc(fake + "\n".join(map(foo, tests.values())), "markdown", "json")
+        # Do this to fix string formatting
+        expected = json.dumps(json.loads(expected))
+        self.assertEqual(expected, fltr(inp))
+
+    def test_convert_explicitly_relative_paths_no_texinputs(self):
+        # no texinputs should lead to just returning
+        fltr = ConvertExplicitlyRelativePaths(texinputs="")
+        self.assertEqual("test", fltr("test"))