Repository: beetbox/beets

Issue:
NumericQuery is useless when there's a typo in the value
Say I do `beets ls original_year:196*`. There's a mistake: to trigger a regex match I need `::`. This will be a simple `beets.dbcore.query.NumericQuery`. However instead of failing it will match _everything_, equivalent to `beets ls`.

I think that this should raise an error (e.g. `InvalidQuery` that later turns into an `UserError`).



Discussion:
Seems like a reasonable failure mode for parse errors, yes. We should think very carefully, though, about where these exceptions are bounded—for example, I think that you're right that we need a special InvalidQuery exception raised from the various parsing functions, but I'm not sure where to handle that and transform it into a UserError.

I have a patch ready but I didn't submit it yet for that exact reason.
I define and raise `InvalidQuery` in `beets.dbcore.query`. It could then be caught in `beets.library` but that module cannot raise an `ui.UserError`. Otherwise the role is delegated to the module calling e.g `Library.items()`, so `beets.ui.commands` in our case: it will catch `InvalidQuery` and raise a `UserError`.

Downside: every command which can use a query (and therefore most commands...) will have to this. Which isn't nice. Therefore I see the 3 options:
- let the InvalidQuery bubble up to the user and crash beets
- let every command catch & raise
- add another error in `beets.library`, and _there_ catch InvalidQuery and raise that error.

Third point: we could add a `QueryError` which extends `HumanReadableError` and is raised when an `InvalidQuery` is caught. The `reason` would be the part that's wrong (e.g. "196\* is not an int or a float") and `verb` would be the query itself (e.g. "The Beatles original_year:196*") (yeah that's not really a verb but it's akin to an action which is what's "semantically" expected).

The resulting message would be something like (copied from my clipboard) `Error: In query u'Beatles All Together Now original_year:196*': u'196*' is not an int or a float`

Yeah, this is tricky.

I think the path of least resistance is to just catch InvalidQueryError at the top level—i.e., let it bubble all the way up to main by default. Commands can still catch it themselves if they want to provide special handling.

But then in most cases you'd get the stacktrace which IMHO is a bit ugly. Unless it gets caught by `beets.ui.main` (like numerous other exceptions), and a simple "Invalid query: u'196\* is not an int or a float" would be printed. Providing more context might be nice (though I'd personally think this is sufficient)

Yes, we'd add another top-level handler. With a little work the exception could even include more detailed context, such as the full query component.


Resolving patch:
diff --git a/beets/dbcore/query.py b/beets/dbcore/query.py
--- a/beets/dbcore/query.py
+++ b/beets/dbcore/query.py
@@ -20,6 +20,14 @@
 from datetime import datetime, timedelta
 
 
+class InvalidQueryError(ValueError):
+    def __init__(self, what, expected, detail=None):
+        message = "{0!r} is not {1}".format(what, expected)
+        if detail:
+            message = "{0}: {1}".format(message, detail)
+        super(InvalidQueryError, self).__init__(message)
+
+
 class Query(object):
     """An abstract class representing a query into the item database.
     """
@@ -139,15 +147,22 @@ def string_match(cls, pattern, value):
 class RegexpQuery(StringFieldQuery):
     """A query that matches a regular expression in a specific item
     field.
+
+    Raises InvalidQueryError when the pattern is not a valid regular
+    expression.
     """
-    @classmethod
-    def string_match(cls, pattern, value):
+    def __init__(self, field, pattern, false=True):
+        super(RegexpQuery, self).__init__(field, pattern, false)
         try:
-            res = re.search(pattern, value)
-        except re.error:
+            self.pattern = re.compile(self.pattern)
+        except re.error as exc:
             # Invalid regular expression.
-            return False
-        return res is not None
+            raise InvalidQueryError(pattern, "a regular expression",
+                                    format(exc))
+
+    @classmethod
+    def string_match(cls, pattern, value):
+        return pattern.search(value) is not None
 
 
 class BooleanQuery(MatchQuery):
@@ -191,6 +206,9 @@ class NumericQuery(FieldQuery):
     """Matches numeric fields. A syntax using Ruby-style range ellipses
     (``..``) lets users specify one- or two-sided ranges. For example,
     ``year:2001..`` finds music released since the turn of the century.
+
+    Raises InvalidQueryError when the pattern does not represent an int or
+    a float.
     """
     def _convert(self, s):
         """Convert a string to a numeric type (float or int). If the
@@ -203,7 +221,7 @@ def _convert(self, s):
             try:
                 return float(s)
             except ValueError:
-                return None
+                raise InvalidQueryError(s, "an int or a float")
 
     def __init__(self, field, pattern, fast=True):
         super(NumericQuery, self).__init__(field, pattern, fast)
diff --git a/beets/ui/__init__.py b/beets/ui/__init__.py
--- a/beets/ui/__init__.py
+++ b/beets/ui/__init__.py
@@ -38,6 +38,7 @@
 from beets import config
 from beets.util import confit
 from beets.autotag import mb
+from beets.dbcore import query as db_query
 
 # On Windows platforms, use colorama to support "ANSI" terminal colors.
 if sys.platform == 'win32':
@@ -960,6 +961,9 @@ def main(args=None):
     except confit.ConfigError as exc:
         log.error(u'configuration error: {0}', exc)
         sys.exit(1)
+    except db_query.InvalidQueryError as exc:
+        log.error(u'invalid query: {0}', exc)
+        sys.exit(1)
     except IOError as exc:
         if exc.errno == errno.EPIPE:
             # "Broken pipe". End silently.


Test patch:
diff --git a/test/test_query.py b/test/test_query.py
--- a/test/test_query.py
+++ b/test/test_query.py
@@ -21,7 +21,7 @@
 import beets.library
 from beets import dbcore
 from beets.dbcore import types
-from beets.dbcore.query import NoneQuery
+from beets.dbcore.query import NoneQuery, InvalidQueryError
 from beets.library import Library, Item
 
 
@@ -218,11 +218,6 @@ def test_year_range(self):
             'baz qux',
         ])
 
-    def test_bad_year(self):
-        q = 'year:delete from items'
-        results = self.lib.items(q)
-        self.assert_matched(results, [])
-
     def test_singleton_true(self):
         q = 'singleton:true'
         results = self.lib.items(q)
@@ -280,10 +275,15 @@ def test_numeric_search_negative(self):
         results = self.lib.items(q)
         self.assertFalse(results)
 
-    def test_numeric_empty(self):
-        q = dbcore.query.NumericQuery('year', '')
-        results = self.lib.items(q)
-        self.assertTrue(results)
+    def test_invalid_query(self):
+        with self.assertRaises(InvalidQueryError) as raised:
+            dbcore.query.NumericQuery('year', '199a')
+        self.assertIn('not an int', str(raised.exception))
+
+        with self.assertRaises(InvalidQueryError) as raised:
+            dbcore.query.RegexpQuery('year', '199(')
+        self.assertIn('not a regular expression', str(raised.exception))
+        self.assertIn('unbalanced parenthesis', str(raised.exception))
 
 
 class MatchTest(_common.TestCase):