Repository: bids-standard/pybids

Issue:
should pybids more lenient or at least get a dedicated exception like NonCompliantBIDSError
ATM pybids immediately throws a generic ValueError if found contained derivatives dataset is not fully compliant:

```shell
/tmp > datalad install https://github.com/OpenNeuroDatasets/ds001868 && cd ds001868
[INFO   ] Cloning https://github.com/OpenNeuroDatasets/ds001868 [1 other candidates] into '/tmp/ds001868' 
[INFO   ]   Remote origin not usable by git-annex; setting annex-ignore                                         
[INFO   ] access to 1 dataset sibling s3-PRIVATE not auto-enabled, enable with:
| 		datalad siblings -d "/tmp/ds001868" enable -s s3-PRIVATE 
install(ok): /tmp/ds001868 (dataset)

/tmp/ds001868 > python -c 'from bids import BIDSLayout; b=BIDSLayout(".", derivatives=True)' 
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/yoh/proj/bids/pybids/bids/layout/layout.py", line 227, in __init__
    index_metadata=index_metadata)
  File "/home/yoh/proj/bids/pybids/bids/layout/layout.py", line 555, in add_derivatives
    raise ValueError("Every valid BIDS-derivatives dataset must "
ValueError: Every valid BIDS-derivatives dataset must have a PipelineDescription.Name field set inside dataset_description.json.
```
But I wonder
- if this should be a warning and not error (optionally or not)
- there should be a dedicated exception (e.g. `NonCompliantBIDSError(ValueError)` and `NonCompliantDerivativesBIDSError(NonCompliantBIDSError)`) which would be thrown here instead of a generic ValueError which makes it fragile to handle such cases programmatically 


Discussion:
I'm :+1: for a sensible set of exceptions.

```
BIDSError
 - BIDSValidationError
   - DerivativeValidationError
   - ... (other optional BEPs?)
```

That said, I'm pretty sure you can use `strict=False` to avoid seeing that error.
Also +1 for a hierarchy of exceptions—feel free to open a PR!
My thinking on this has changed some. I spent some time a while back looking into it, and it seems like people are split pretty much 50-50 on whether best practice is to implement custom exceptions for every package, or to avoid it like the plague unless there's some critical reason for it. I don't have a strong feeling one way or the other, but the tiebreaker is that it would involve a fair amount of work to decide on a hierarchy, implement it, and then sweep through the codebase and replace the generic exceptions. Unless someone wants to volunteer to take this on, I propose to close this.
Actually, I'm just going to go ahead and close it, to feed my issue-decluttering roll. If someone wants to volunteer to take this on, they can re-open it.
> . I spent some time a while back looking into it, and it seems like people are split pretty much 50-50 on whether best practice is to implement custom exceptions for every package

could you please show me some argumentation against "implementing custom exceptions"? note that here it was not really about "implementing" but rather "defining" as subclasses of base exceptions.  I will now prep a basic PR to kick start the effort ;)

Resolving patch:
diff --git a/bids/exceptions.py b/bids/exceptions.py
new file mode 100644
--- /dev/null
+++ b/bids/exceptions.py
@@ -0,0 +1,37 @@
+""" Exceptions.
+
+Exceptions relating to problems with BIDS itself, should carry BIDS in their
+name.  All exceptions should subclass from PyBIDSError
+"""
+
+
+class PyBIDSError(Exception):
+    """ Base class.  Typically for mix-in."""
+
+
+class ConfigError(ValueError, PyBIDSError):
+    """ Problems with config file. """
+
+
+class NoMatchError(ValueError, PyBIDSError):
+    """ No match found where it is required. """
+
+
+class BIDSEntityError(AttributeError, PyBIDSError):
+    """ An unknown entity. """
+
+
+class TargetError(ValueError, PyBIDSError):
+    """ An unknown target. """
+
+
+class BIDSValidationError(ValueError, PyBIDSError):
+    """ An invalid BIDS dataset. """
+
+
+class BIDSDerivativesValidationError(BIDSValidationError):
+    """ An invalid BIDS derivative dataset. """
+
+
+class BIDSConflictingValuesError(BIDSValidationError):
+    """ A value conflict (e.g. in filename and side-car .json) """
diff --git a/bids/layout/index.py b/bids/layout/index.py
--- a/bids/layout/index.py
+++ b/bids/layout/index.py
@@ -6,6 +6,7 @@
 from bids_validator import BIDSValidator
 from .models import Config, Entity, Tag, FileAssociation
 from ..utils import listify, make_bidsfile
+from ..exceptions import BIDSConflictingValuesError
 
 
 def _extract_entities(bidsfile, entities):
@@ -367,8 +368,9 @@ def create_association_pair(src, dst, kind, kind2=None):
                             "filename {} (value='{}') versus its JSON sidecar "
                             "(value='{}'). Please reconcile this discrepancy."
                         )
-                        raise ValueError(msg.format(md_key, bf.path, file_val,
-                                                    md_val))
+                        raise BIDSConflictingValuesError(
+                            msg.format(md_key, bf.path, file_val,
+                            md_val))
                     continue
                 if md_key not in all_entities:
                     all_entities[md_key] = Entity(md_key, is_metadata=True)
diff --git a/bids/layout/layout.py b/bids/layout/layout.py
--- a/bids/layout/layout.py
+++ b/bids/layout/layout.py
@@ -24,6 +24,14 @@
 from .index import BIDSLayoutIndexer
 from .utils import BIDSMetadata
 from .. import config as cf
+from ..exceptions import (
+    BIDSDerivativesValidationError,
+    BIDSEntityError,
+    BIDSValidationError,
+    ConfigError,
+    NoMatchError,
+    TargetError,
+)
 
 try:
     from os.path import commonpath
@@ -122,10 +130,10 @@ def add_config_paths(**kwargs):
     """
     for k, path in kwargs.items():
         if not os.path.exists(path):
-            raise ValueError(
+            raise ConfigError(
                 'Configuration file "{}" does not exist'.format(k))
         if k in cf.get_option('config_paths'):
-            raise ValueError('Configuration {!r} already exists'.format(k))
+            raise ConfigError('Configuration {!r} already exists'.format(k))
 
     kwargs.update(**cf.get_option('config_paths'))
     cf.set_option('config_paths', kwargs)
@@ -307,7 +315,7 @@ def __getattr__(self, key):
                 if sing in entities:
                     ent_name = sing
                 else:
-                    raise AttributeError(
+                    raise BIDSEntityError(
                         "'get_{}' can't be called because '{}' isn't a "
                         "recognized entity name.".format(ent_name, ent_name))
             return partial(self.get, return_type='id', target=ent_name)
@@ -480,7 +488,7 @@ def _validate_root(self):
         target = os.path.join(self.root, 'dataset_description.json')
         if not os.path.exists(target):
             if self.validate:
-                raise ValueError(
+                raise BIDSValidationError(
                     "'dataset_description.json' is missing from project root."
                     " Every valid BIDS dataset must have this file."
                     "\nExample contents of 'dataset_description.json': \n%s" %
@@ -494,9 +502,9 @@ def _validate_root(self):
             if self.validate:
                 for k in MANDATORY_BIDS_FIELDS:
                     if k not in self.description:
-                        raise ValueError("Mandatory %r field missing from "
-                                         "'dataset_description.json'."
-                                         "\nExample: %s" % (k, MANDATORY_BIDS_FIELDS[k])
+                        raise BIDSValidationError("Mandatory %r field missing from "
+                                                  "'dataset_description.json'."
+                                                  "\nExample: %s" % (k, MANDATORY_BIDS_FIELDS[k])
                         )
 
     def _validate_force_index(self):
@@ -805,13 +813,15 @@ def check_for_description(bids_dir):
             pipeline_name = description.get(
                 'PipelineDescription', {}).get('Name')
             if pipeline_name is None:
-                raise ValueError("Every valid BIDS-derivatives dataset must "
+                raise BIDSDerivativesValidationError(
+                                 "Every valid BIDS-derivatives dataset must "
                                  "have a PipelineDescription.Name field set "
                                  "inside 'dataset_description.json'. "
                                  "\nExample: %s" %
                                  MANDATORY_DERIVATIVES_FIELDS['PipelineDescription.Name'])
             if pipeline_name in self.derivatives:
-                raise ValueError("Pipeline name '%s' has already been added "
+                raise BIDSDerivativesValidationError(
+                                 "Pipeline name '%s' has already been added "
                                  "to this BIDSLayout. Every added pipeline "
                                  "must have a unique name!")
             # Default config and sources values
@@ -968,7 +978,7 @@ def get(self, return_type='object', target=None, scope='all',
                 message = "Did you mean one of: {}?".format(suggestions)
             else:
                 message = "Valid targets are: {}".format(potential)
-            raise ValueError(("Unknown target '{}'. " + message)
+            raise TargetError(("Unknown target '{}'. " + message)
                              .format(target))
 
         results = []
@@ -992,7 +1002,7 @@ def get(self, return_type='object', target=None, scope='all',
 
         elif return_type in ['id', 'dir']:
             if target is None:
-                raise ValueError('If return_type is "id" or "dir", a valid '
+                raise TargetError('If return_type is "id" or "dir", a valid '
                                  'target entity must also be specified.')
 
             results = [x for x in results if target in x.entities]
@@ -1254,7 +1264,7 @@ def get_nearest(self, path, return_type='filename', strict=True,
         if not filters.get('suffix'):
             f = self.get_file(path)
             if 'suffix' not in f.entities:
-                raise ValueError(
+                raise BIDSValidationError(
                     "File '%s' does not have a valid suffix, most "
                     "likely because it is not a valid BIDS file." % path
                 )
@@ -1424,7 +1434,7 @@ def get_tr(self, derivatives=False, **filters):
         images = self.get(extension=['nii', 'nii.gz'], scope=scope,
                           **filters)
         if not images:
-            raise ValueError("No functional images that match criteria found.")
+            raise NoMatchError("No functional images that match criteria found.")
 
         all_trs = set()
         for img in images:
@@ -1432,7 +1442,7 @@ def get_tr(self, derivatives=False, **filters):
             all_trs.add(round(float(md['RepetitionTime']), 5))
 
         if len(all_trs) > 1:
-            raise ValueError("Unique TR cannot be found given filters {!r}"
+            raise NoMatchError("Unique TR cannot be found given filters {!r}"
                              .format(filters))
         return all_trs.pop()
 
@@ -1515,7 +1525,8 @@ def build_path(self, source, path_patterns=None, strict=False,
         to_check = os.path.join(os.path.sep, built)
 
         if validate and not BIDSValidator().is_bids(to_check):
-            raise ValueError("Built path {} is not a valid BIDS filename. "
+            raise BIDSValidationError(
+                             "Built path {} is not a valid BIDS filename. "
                              "Please make sure all provided entity values are "
                              "spec-compliant.".format(built))
 


Test patch:
diff --git a/bids/layout/tests/test_layout.py b/bids/layout/tests/test_layout.py
--- a/bids/layout/tests/test_layout.py
+++ b/bids/layout/tests/test_layout.py
@@ -18,6 +18,13 @@
 from bids.tests import get_test_data_path
 from bids.utils import natural_sort
 
+from bids.exceptions import (
+    BIDSValidationError,
+    ConfigError,
+    NoMatchError,
+    TargetError,
+)
+
 
 def test_layout_init(layout_7t_trt):
     assert isinstance(layout_7t_trt.files, dict)
@@ -194,11 +201,11 @@ def test_get_metadata_error(layout_7t_trt):
 
 
 def test_get_with_bad_target(layout_7t_trt):
-    with pytest.raises(ValueError) as exc:
+    with pytest.raises(TargetError) as exc:
         layout_7t_trt.get(target='unicorn')
         msg = exc.value.message
         assert 'subject' in msg and 'reconstruction' in msg and 'proc' in msg
-    with pytest.raises(ValueError) as exc:
+    with pytest.raises(TargetError) as exc:
         layout_7t_trt.get(target='sub')
         msg = exc.value.message
         assert 'subject' in msg and 'reconstruction' not in msg
@@ -440,11 +447,11 @@ def test_derivative_getters():
 
 def test_get_tr(layout_7t_trt):
     # Bad subject, should fail
-    with pytest.raises(ValueError) as exc:
+    with pytest.raises(NoMatchError) as exc:
         layout_7t_trt.get_tr(subject="zzz")
         assert exc.value.message.startswith("No functional images")
     # There are multiple tasks with different TRs, so this should fail
-    with pytest.raises(ValueError) as exc:
+    with pytest.raises(NoMatchError) as exc:
         layout_7t_trt.get_tr(subject=['01', '02'])
         assert exc.value.message.startswith("Unique TR")
     # This should work
@@ -543,10 +550,10 @@ def test_deriv_indexing():
 def test_add_config_paths():
     bids_dir = dirname(bids.__file__)
     bids_json = os.path.join(bids_dir, 'layout', 'config', 'bids.json')
-    with pytest.raises(ValueError) as exc:
+    with pytest.raises(ConfigError) as exc:
         add_config_paths(test_config1='nonexistentpath.json')
     assert str(exc.value).startswith('Configuration file')
-    with pytest.raises(ValueError) as exc:
+    with pytest.raises(ConfigError) as exc:
         add_config_paths(bids=bids_json)
     assert str(exc.value).startswith("Configuration 'bids' already")
     add_config_paths(dummy=bids_json)
@@ -625,7 +632,7 @@ def test_layout_save(tmp_path, layout_7t_trt):
 
 def test_indexing_tag_conflict():
     data_dir = join(get_test_data_path(), 'ds005_conflict')
-    with pytest.raises(ValueError) as exc:
+    with pytest.raises(BIDSValidationError) as exc:
         layout = BIDSLayout(data_dir)
         print(exc.value.message)
         assert exc.value.message.startswith("Conflicting values found")