Repository: HEPData/hepdata_lib

Issue:
Add flag to disable features requiring ROOT
There are use cases where the tool is used but no ROOT installation is available. We should add an option that disables all features that use ROOT to make the other features work independently (e.g. reading in a text file). For ROOT TTrees, we could fall back to uproot.

Discussion:
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/HEPData/hepdata_lib/pull/259?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPData) Report
Attention: Patch coverage is `86.66667%` with `2 lines` in your changes are missing coverage. Please review.
> Project coverage is 88.05%. Comparing base [(`eb392fe`)](https://app.codecov.io/gh/HEPData/hepdata_lib/commit/eb392fe6caf8a29c3401401678758349527b6f60?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPData) to head [(`f842b3a`)](https://app.codecov.io/gh/HEPData/hepdata_lib/pull/259?dropdown=coverage&src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPData).

| [Files](https://app.codecov.io/gh/HEPData/hepdata_lib/pull/259?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPData) | Patch % | Lines |
|---|---|---|
| [hepdata\_lib/root\_utils.py](https://app.codecov.io/gh/HEPData/hepdata_lib/pull/259?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPData#diff-aGVwZGF0YV9saWIvcm9vdF91dGlscy5weQ==) | 84.61% | [0 Missing and 2 partials :warning: ](https://app.codecov.io/gh/HEPData/hepdata_lib/pull/259?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPData) |

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##             main     #259      +/-   ##
==========================================
+ Coverage   87.99%   88.05%   +0.05%     
==========================================
  Files           5        5              
  Lines        1083     1088       +5     
  Branches      231      224       -7     
==========================================
+ Hits          953      958       +5     
  Misses         91       91              
  Partials       39       39              
```

| [Flag](https://app.codecov.io/gh/HEPData/hepdata_lib/pull/259/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPData) | Coverage Δ | |
|---|---|---|
| [unittests-3.10](https://app.codecov.io/gh/HEPData/hepdata_lib/pull/259/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPData) | `88.05% <86.66%> (+0.05%)` | :arrow_up: |
| [unittests-3.6](https://app.codecov.io/gh/HEPData/hepdata_lib/pull/259/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPData) | `?` | |
| [unittests-3.7](https://app.codecov.io/gh/HEPData/hepdata_lib/pull/259/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPData) | `?` | |
| [unittests-3.8](https://app.codecov.io/gh/HEPData/hepdata_lib/pull/259/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPData) | `87.86% <86.66%> (+0.05%)` | :arrow_up: |
| [unittests-3.9](https://app.codecov.io/gh/HEPData/hepdata_lib/pull/259/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPData) | `87.86% <86.66%> (+0.05%)` | :arrow_up: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPData#carryforward-flags-in-the-pull-request-comment) to find out more.


</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/HEPData/hepdata_lib/pull/259?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPData).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=HEPData).

GraemeWatt: @clelange @alexander-held : I made a crude but simple attempt in this PR to address #99 by only importing ROOT functions in the `hepdata_lib` functions where they are actually needed.  This means that `import hepdata_lib` now works without a ROOT installation as does the basic functionality (e.g. demonstrated in the [Getting_started.ipynb](https://github.com/HEPData/hepdata_lib/blob/main/examples/Getting_started.ipynb) notebook).
alexander-held: This is very nice to see, thanks for taking care of this!
clelange: Hi @GraemeWatt -- thanks for this! I'm wondering, however, if for `hepdata_lib/root_utils.py` (and probably also `hepdata_lib/c_file_reader.py`) it's not OK to actually keep the top-level ROOT import in a `try`-`catch` statement and rather print out a warning if the ROOT import failed.

My main issue with these changes is that while they move some of the ROOT import failures to later and therefore `import hepdata_lib` will not fail immediately, we currently don't have visibility how this would fail. It would be nice to actually have tests without a ROOT installation and then label in the docs which features work without a ROOT installation.

What do you think?
GraemeWatt: @clelange : Yes, I agree that keeping the top-level ROOT import and adding a `try`/`except` condition is a better (and simpler) solution than my first attempt at only importing ROOT functions in the `hepdata_lib` functions where they are actually needed.  I've added a new commit on top of the `upgrade-pylint` branch and force-pushed to the `change-root-import` branch, so you should merge PR #260 before this one.  I've added an explanation at the end of the `README.md` file.  I don't see an easy way to modify the CI to work without a ROOT installation, but I tested locally (in a Python 3.11 virtual environment with an incompatible ROOT built with Python 3.12) that the code behaves as expected.  Feel free to expand the tests run by the CI if you think it is important and you can work out how to do it.
clelange: > Feel free to expand the tests run by the CI if you think it is important and you can work out how to do it.

I think we can just run some of the tests without installing ROOT, i.e. a separate set of tests. However, we can do that in a later PR.

There's a conflict for `hepdata_lib/c_file_reader.py`. Could you please check if you can fix it? You could also rebase on top of the current `main` branch now that #260 is merged, whichever is easier for you.
GraemeWatt: @clelange : I used a Pytest [custom marker](https://docs.pytest.org/en/7.1.x/example/markers.html) to add a marker `needs_root` to the tests that require a ROOT installation.  I then introduced an additional element to the CI matrix without a ROOT installation `root-version = ""` and ran the tests using `pytest -m 'not needs_root'`.  I also resolved the conflict with the `main` branch.

Patch:
diff --git a/README.md b/README.md
index 6f9ba74..4b716ce 100644
--- a/README.md
+++ b/README.md
@@ -72,8 +72,8 @@ There are a few more examples available that can directly be run using the [bind
 - [Reading TGraph and TGraphError from '.C' files](https://github.com/HEPData/hepdata_lib/blob/main/examples/read_c_file.ipynb)
 [![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/HEPData/hepdata_lib/main?filepath=examples/read_c_file.ipynb)
 <br/><br/>
-- [Preparing scikit-hep histograms](https://github.com/HEPData/hepdata_lib/blob/main/examples/reading_scikithep_histogram.ipynb)
-[![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/HEPData/hepdata_lib/main?filepath=examples/reading_scikihep_histogram.ipynb)
+- [Preparing scikit-hep histograms](https://github.com/HEPData/hepdata_lib/blob/main/examples/reading_scikithep_histograms.ipynb)
+[![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/HEPData/hepdata_lib/main?filepath=examples/reading_scikihep_histograms.ipynb)
 <br/><br/>
 
 ## External dependencies
@@ -82,3 +82,8 @@ There are a few more examples available that can directly be run using the [bind
 - [ImageMagick](https://www.imagemagick.org)
 
 Make sure that you have `ROOT` in your `$PYTHONPATH` and that the `convert` command is available by adding its location to your `$PATH` if needed.
+
+A ROOT installation is not strictly required if your input data is not in a ROOT format, for example, if
+your input data is provided as text files or `scikit-hep/hist` histograms.  Most of the `hepdata_lib`
+functionality can be used without a ROOT installation, other than the `RootFileReader` and `CFileReader` classes,
+and other functions of the `hepdata_lib.root_utils` module.
\ No newline at end of file
diff --git a/docs/dev.rst b/docs/dev.rst
index 86e1f5f..e3a5e1b 100644
--- a/docs/dev.rst
+++ b/docs/dev.rst
@@ -15,6 +15,12 @@ To run the tests, move into the ``hepdata_lib`` directory while in your virtual
 
 It is a good idea to run the tests manually to ensure that your changes do not cause any issues.
 
+If you don't have a working ROOT installation, a subset of the tests can still be run without ROOT:
+
+::
+
+    pytest tests -m "not needs_root"
+
 Definition of test cases
 ++++++++++++++++++++++++
 
diff --git a/hepdata_lib/c_file_reader.py b/hepdata_lib/c_file_reader.py
index a07c714..2c957e3 100644
--- a/hepdata_lib/c_file_reader.py
+++ b/hepdata_lib/c_file_reader.py
@@ -2,7 +2,10 @@
 import io
 from array import array
 from future.utils import raise_from
-from ROOT import TGraph, TGraphErrors  # pylint: disable=no-name-in-module
+try:
+    from ROOT import TGraph, TGraphErrors  # pylint: disable=no-name-in-module
+except ImportError as e:  # pragma: no cover
+    print(f'Cannot import ROOT: {str(e)}')
 import hepdata_lib.root_utils as ru
 from hepdata_lib.helpers import check_file_existence
 
diff --git a/hepdata_lib/root_utils.py b/hepdata_lib/root_utils.py
index c6d0269..ff1dd28 100644
--- a/hepdata_lib/root_utils.py
+++ b/hepdata_lib/root_utils.py
@@ -3,7 +3,10 @@ from collections import defaultdict
 import ctypes
 from future.utils import raise_from
 import numpy as np
-import ROOT as r
+try:
+    import ROOT as r
+except ImportError as e:  # pragma: no cover
+    print(f'Cannot import ROOT: {str(e)}')
 from hepdata_lib.helpers import check_file_existence
 
 class RootFileReader:
diff --git a/pyproject.toml b/pyproject.toml
index 456e710..afbb950 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -14,3 +14,6 @@ addopts = [
 ]
 log_cli_level = "info"
 testpaths = "tests"
+markers = [
+    "needs_root: requires a ROOT installation"
+]
\ No newline at end of file
diff --git a/setup.py b/setup.py
index 088f89d..7091159 100644
--- a/setup.py
+++ b/setup.py
@@ -34,6 +34,7 @@ setup(
         'Programming Language :: Python :: 3.9',
         'Programming Language :: Python :: 3.10',
         'Programming Language :: Python :: 3.11',
+        'Programming Language :: Python :: 3.12',
     ],
     keywords='HEPData physics OpenData',
     packages=find_packages(exclude=['contrib', 'docs', 'tests']),


Test patch:
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml
index 251d276..b560594 100644
--- a/.github/workflows/tests.yml
+++ b/.github/workflows/tests.yml
@@ -25,7 +25,7 @@ jobs:
     strategy:
       matrix:
         os: [ubuntu-latest]
-        root-version: ["6.24", "6.26", "6.28", "6.30"]
+        root-version: ["", "6.24", "6.26", "6.28", "6.30"]
         python-version: ["3.6", "3.7", "3.8", "3.9", "3.10", "3.11"]
         exclude:
           - root-version: "6.24"
@@ -55,6 +55,7 @@ jobs:
     - uses: actions/checkout@v4
 
     - name: Setup Micromamba environment
+      if: ${{ matrix.root-version }}
       uses: mamba-org/setup-micromamba@v1
       with:
         environment-name: ci
@@ -68,7 +69,22 @@ jobs:
           channels:
             - conda-forge
 
+    - name: Setup Micromamba environment without ROOT
+      if: ${{ !matrix.root-version }}
+      uses: mamba-org/setup-micromamba@v1
+      with:
+        environment-name: ci
+        create-args: >-
+          python=${{ matrix.python-version }}
+          imagemagick
+          ghostscript
+          pip
+        condarc: |
+          channels:
+            - conda-forge
+
     - name: ROOT info
+      if: ${{ matrix.root-version }}
       run: |
         root-config --version
         root-config --python-version
@@ -94,23 +110,30 @@ jobs:
 
     # Use python -m pytest to add current working dir as src/ dir layout not used
     - name: Run pytest
+      if: ${{ matrix.root-version }}
       run: |
         python -m pytest tests
 
+    - name: Run pytest without ROOT
+      if: ${{ !matrix.root-version }}
+      run: |
+        python -m pytest tests -m 'not needs_root'
+
     - name: Report coverage with Codecov
+      if: ${{ matrix.root-version }}
       uses: codecov/codecov-action@v3
       with:
         files: ./coverage.xml
         flags: unittests-${{ matrix.python-version }}
 
     - name: Save notebooks
-      if: ${{ always() }}
+      if: ${{ always() && matrix.root-version }}
       run: |
         python -m jupyter kernelspec list
         python -m jupyter nbconvert --ExecutePreprocessor.timeout=600 --ExecutePreprocessor.allow_errors=True --to html --execute examples/*.ipynb
 
     - name: Upload notebooks
-      if: ${{ always() }}
+      if: ${{ always() && matrix.root-version }}
       uses: actions/upload-artifact@v4
       with:
         name: notebooks-${{ matrix.root-version }}-${{ matrix.python-version }}-${{ matrix.os }} py3-${{ matrix.root-version }}-${{ matrix.python-version }}-${{ matrix.os }}
diff --git a/tests/test_cfilereader.py b/tests/test_cfilereader.py
index b77511f..d19c47a 100644
--- a/tests/test_cfilereader.py
+++ b/tests/test_cfilereader.py
@@ -3,8 +3,10 @@
 from unittest import TestCase
 import os
 import numpy as np
+import pytest
 from hepdata_lib.c_file_reader import CFileReader
 
+@pytest.mark.needs_root
 class TestCFileReader(TestCase):
     """Test the CFileReader class."""
 
diff --git a/tests/test_notebooks.py b/tests/test_notebooks.py
index 7783579..b74b0de 100644
--- a/tests/test_notebooks.py
+++ b/tests/test_notebooks.py
@@ -13,6 +13,7 @@ def common_kwargs(tmpdir):
         'cwd' : 'examples'
     }
 
+@pytest.mark.needs_root
 def test_correlation(common_kwargs):# pylint: disable=redefined-outer-name
     """Tests examples/correlation.ipynb"""
     pm.execute_notebook('examples/correlation.ipynb', **common_kwargs)
@@ -21,14 +22,21 @@ def test_getting_started(common_kwargs):# pylint: disable=redefined-outer-name
     """Tests examples/Getting_started.ipynb"""
     pm.execute_notebook('examples/Getting_started.ipynb', **common_kwargs)
 
+@pytest.mark.needs_root
 def test_reading_histograms(common_kwargs):# pylint: disable=redefined-outer-name
     """Tests examples/reading_histograms.ipynb"""
     pm.execute_notebook('examples/reading_histograms.ipynb', **common_kwargs)
 
+@pytest.mark.needs_root
 def test_combine_limits(common_kwargs):# pylint: disable=redefined-outer-name
     """Tests examples/combine_limits.ipynb"""
     pm.execute_notebook('examples/combine_limits.ipynb', **common_kwargs)
 
+@pytest.mark.needs_root
 def test_c_file(common_kwargs):# pylint: disable=redefined-outer-name
     """Tests examples/read_c_file.ipynb"""
     pm.execute_notebook('examples/read_c_file.ipynb', **common_kwargs)
+
+def test_scikithep_histograms(common_kwargs):# pylint: disable=redefined-outer-name
+    """Tests examples/reading_scikithep_histograms.ipynb"""
+    pm.execute_notebook('examples/reading_scikithep_histograms.ipynb', **common_kwargs)
diff --git a/tests/test_rootfilereader.py b/tests/test_rootfilereader.py
index 4cd0c52..23d1d94 100644
--- a/tests/test_rootfilereader.py
+++ b/tests/test_rootfilereader.py
@@ -5,12 +5,17 @@ from array import array
 import os
 import ctypes
 import numpy as np
-import ROOT
+import pytest
+try:
+    import ROOT
+except ImportError as e:
+    print(f'Cannot import ROOT: {str(e)}')
 from hepdata_lib.root_utils import (RootFileReader, get_graph_points,
                                     get_hist_1d_points, get_hist_2d_points)
 from .test_utilities import float_compare, tuple_compare, histogram_compare_1d, make_tmp_root_file
 
 
+@pytest.mark.needs_root
 class TestRootFileReader(TestCase):
     """Test the RootFileReader class."""
 
diff --git a/tests/test_utilities.py b/tests/test_utilities.py
index eee8d9d..21ebbb0 100644
--- a/tests/test_utilities.py
+++ b/tests/test_utilities.py
@@ -4,8 +4,10 @@
 import os
 import random
 import string
-
-import ROOT
+try:
+    import ROOT
+except ImportError as e:
+    print(f'Cannot import ROOT: {str(e)}')
 from future.utils import raise_from