Repository: qutebrowser/qutebrowser

Issue:
word hints: "ValueError: Got invalid match type Match.ambiguous"
Report on 42a4c1c.

@NoctuaNivalis any chance you could take a look at this? :smile:
## Report
- typing a hint, m-a-l-l letters into  "malloc" hint on news.ycombinator.com
- There was probably another hint, "mall", to cause this crash, but I am not sure (did not notice).
- "malloc" was a "smart" hint (read something from url to generate hint). Are smart hints properly uniq'd against generated word hints?
- Have used the new word-hints feature since its release and have never had an ambiguous match crash before
## Config

``` ini
[hints]
border = 1px solid #ff4500
opacity = 0.9
mode = word
scatter = false
uppercase = true
```
## Log

```
14:38:36 DEBUG    commands   command:run:490 command called: hint ['all', 'tab']
14:38:36 DEBUG    commands   command:run:504 Calling qutebrowser.browser.hints.HintManager.start(<qutebrowser.browser.hints.HintManager object at 0x7f5529b2da68>, False, <Group.all: 1>, <Target.tab: 2>, win_id=0)
14:38:36 DEBUG    hints      hints:_init_elements:680 hints: axis, changes, ella, gold, minutes, gory, able, freedom, peep, sawn, hours, amid, gags, gene, seed, andris, thin, warn, sulk, nick, jeer, beam, bmw, loth, serf, pm, puke, lube, lacy, discuss, lion, loll, wimp, crow, lupe, ruiz, flub, malloc, dyer, ages, grid, gunk, lire, state, kadira, fri, buff, leos, tues, craw, feud, lady, loyd, date, suzy, lr, lax, mold, else, toes, pres, googles, flag, olga, rn, lola, ohsa, killing, aces, cast, lois, sale, hips, swop, eetimes, razer, hype, help, fest, turkey, ibtimes, ponyous, hies, mugs, nous, release, wololo, scow, outs, any, slur, duds, love, gelt, olav, kurt, prut, proto, github, jive, ajax, disc, colo, writ, brig, gyms, bart, mack, yawl, basic, jogs, aloe, hutu, tass, free, andy, itch, tshtf, lien, glut, hump, murk, sick, canistr, seam, knob, apr, drones, furs, rezist, pugs, pare, mire, nigh, pour, prim, duly, joys, beet, hams, burg, elorant, sake, real, hews, ultra, fins, gray, coax, kyle, puny, swig, peck, jonbaer, whiz, comment, lace, nook, burp, nkurz, yawn, deed, feta, said, wired, anus, eula, ass, ariz, toot, wrapapi, arciini, yowl, mall
14:38:36 DEBUG    message    message:set_text:323 Follow hint in new tab...
14:38:36 DEBUG    statusbar  text:set_text:59 Setting normal text to 'Follow hint in new tab...'.
14:38:36 DEBUG    hints      hints:_connect_frame_signals:642 Connecting frame <PyQt5.QtWebKitWidgets.QWebFrame object at 0x7f5529e4fe58>
14:38:36 DEBUG    modes      modeman:enter:242 Entering mode KeyMode.hint (reason: HintManager.start)
14:38:36 DEBUG    modes      modeman:_eventFilter_keypress:197 handled: True, forward-unbound-keys: auto, passthrough: False, is_non_alnum: False, is_tab False --> filter: True (focused: <qutebrowser.browser.webview.WebView tab_id=36 url='https://news.ycombinator.com/'>)
14:38:36 DEBUG    modes      modeman:_eventFilter_keyrelease:217 filter: False
14:38:36 DEBUG    modes      modeman:_eventFilter_keyrelease:217 filter: True
14:38:37 DEBUG    modes      modeman:_eventFilter_keypress:168 got keypress in mode KeyMode.hint - delegating to <qutebrowser.keyinput.modeparsers.HintKeyParser supports_chains=True supports_count=False>
14:38:37 DEBUG    keyboard   basekeyparser:_debug_log:107 Got key: 0x4d / text: 'm'
14:38:37 DEBUG    keyboard   basekeyparser:_debug_log:107 No match for 'm' (added m)
14:38:37 DEBUG    hints      hints:handle_partial_key:829 Handling new keystring: 'm'
14:38:37 DEBUG    modes      modeman:_eventFilter_keypress:197 handled: True, forward-unbound-keys: auto, passthrough: False, is_non_alnum: False, is_tab False --> filter: True (focused: <qutebrowser.browser.webview.WebView tab_id=36 url='https://news.ycombinator.com/'>)
14:38:37 DEBUG    modes      modeman:_eventFilter_keypress:168 got keypress in mode KeyMode.hint - delegating to <qutebrowser.keyinput.modeparsers.HintKeyParser supports_chains=True supports_count=False>
14:38:37 DEBUG    keyboard   basekeyparser:_debug_log:107 Got key: 0x41 / text: 'a'
14:38:37 DEBUG    keyboard   basekeyparser:_debug_log:107 No match for 'ma' (added a)
14:38:37 DEBUG    hints      hints:handle_partial_key:829 Handling new keystring: 'ma'
14:38:37 DEBUG    modes      modeman:_eventFilter_keypress:197 handled: True, forward-unbound-keys: auto, passthrough: False, is_non_alnum: False, is_tab False --> filter: True (focused: <qutebrowser.browser.webview.WebView tab_id=36 url='https://news.ycombinator.com/'>)
14:38:37 DEBUG    modes      modeman:_eventFilter_keyrelease:217 filter: True
14:38:37 DEBUG    modes      modeman:_eventFilter_keyrelease:217 filter: True
14:38:37 DEBUG    modes      modeman:_eventFilter_keypress:168 got keypress in mode KeyMode.hint - delegating to <qutebrowser.keyinput.modeparsers.HintKeyParser supports_chains=True supports_count=False>
14:38:37 DEBUG    keyboard   basekeyparser:_debug_log:107 Got key: 0x4c / text: 'l'
14:38:37 DEBUG    keyboard   basekeyparser:_debug_log:107 No match for 'mal' (added l)
14:38:37 DEBUG    hints      hints:handle_partial_key:829 Handling new keystring: 'mal'
14:38:37 DEBUG    modes      modeman:_eventFilter_keypress:197 handled: True, forward-unbound-keys: auto, passthrough: False, is_non_alnum: False, is_tab False --> filter: True (focused: <qutebrowser.browser.webview.WebView tab_id=36 url='https://news.ycombinator.com/'>)
14:38:37 DEBUG    modes      modeman:_eventFilter_keyrelease:217 filter: True
14:38:37 DEBUG    modes      modeman:_eventFilter_keypress:168 got keypress in mode KeyMode.hint - delegating to <qutebrowser.keyinput.modeparsers.HintKeyParser supports_chains=True supports_count=False>
14:38:37 DEBUG    keyboard   basekeyparser:_debug_log:107 Got key: 0x4c / text: 'l'
14:38:37 DEBUG    keyboard   basekeyparser:_debug_log:107 Ambiguous match for 'mall'.
14:38:37 DEBUG    keyboard   basekeyparser:_debug_log:107 Ambiguous match for 'mall'
14:38:37 DEBUG    keyboard   basekeyparser:_debug_log:107 Scheduling execution of mall in 500ms
14:38:37 ERROR    misc       crashsignal:exception_hook:211 Uncaught exception
Traceback (most recent call last):
  File "/home/username/git/qutebrowser/qutebrowser/app.py", line 862, in eventFilter
    return handler(event)
  File "/home/username/git/qutebrowser/qutebrowser/app.py", line 822, in _handle_key_event
    return man.eventFilter(event)
  File "/home/username/git/qutebrowser/qutebrowser/keyinput/modeman.py", line 319, in eventFilter
    return self._eventFilter_keypress(event)
  File "/home/username/git/qutebrowser/qutebrowser/keyinput/modeman.py", line 169, in _eventFilter_keypress
    handled = parser.handle(event)
  File "/home/username/git/qutebrowser/qutebrowser/keyinput/modeparsers.py", line 192, in handle
    raise ValueError("Got invalid match type {}!".format(match))
ValueError: Got invalid match type Match.ambiguous!
```



Discussion:

Resolving patch:
diff --git a/qutebrowser/browser/hints.py b/qutebrowser/browser/hints.py
--- a/qutebrowser/browser/hints.py
+++ b/qutebrowser/browser/hints.py
@@ -1004,11 +1004,14 @@ class WordHinter:
     def __init__(self):
         # will be initialized on first use.
         self.words = set()
+        self.dictionary = None
 
     def ensure_initialized(self):
         """Generate the used words if yet uninialized."""
-        if not self.words:
-            dictionary = config.get("hints", "dictionary")
+        dictionary = config.get("hints", "dictionary")
+        if not self.words or self.dictionary != dictionary:
+            self.words.clear()
+            self.dictionary = dictionary
             try:
                 with open(dictionary, encoding="UTF-8") as wordfile:
                     alphabet = set(string.ascii_lowercase)
@@ -1068,12 +1071,17 @@ def any_prefix(self, hint, existing):
         return any(hint.startswith(e) or e.startswith(hint)
                    for e in existing)
 
-    def new_hint_for(self, elem, existing):
+    def filter_prefixes(self, hints, existing):
+        return (h for h in hints if not self.any_prefix(h, existing))
+
+    def new_hint_for(self, elem, existing, fallback):
         """Return a hint for elem, not conflicting with the existing."""
         new = self.tag_words_to_hints(self.extract_tag_words(elem))
-        no_prefixes = (h for h in new if not self.any_prefix(h, existing))
+        new_no_prefixes = self.filter_prefixes(new, existing)
+        fallback_no_prefixes = self.filter_prefixes(fallback, existing)
         # either the first good, or None
-        return next(no_prefixes, None)
+        return (next(new_no_prefixes, None) or
+                next(fallback_no_prefixes, None))
 
     def hint(self, elems):
         """Produce hint labels based on the html tags.
@@ -1093,7 +1101,9 @@ def hint(self, elems):
         used_hints = set()
         words = iter(self.words)
         for elem in elems:
-            hint = self.new_hint_for(elem, used_hints) or next(words)
+            hint = self.new_hint_for(elem, used_hints, words)
+            if not hint:
+                raise WordHintingError("Not enough words in the dictionary.")
             used_hints.add(hint)
             hints.append(hint)
         return hints


Test patch:
diff --git a/tests/integration/data/hinting.txt b/tests/integration/data/hinting.txt
new file mode 100644
--- /dev/null
+++ b/tests/integration/data/hinting.txt
@@ -0,0 +1 @@
+hinting
diff --git a/tests/integration/data/hints/issue1393.html b/tests/integration/data/hints/issue1393.html
new file mode 100644
--- /dev/null
+++ b/tests/integration/data/hints/issue1393.html
@@ -0,0 +1,32 @@
+<!DOCTYPE html>
+<html>
+    <head>
+        <meta charset="utf-8">
+        <title>Let's Hint some words</title>
+    </head>
+    <body>
+        <h1>Word hints</h1>
+
+        <h2>Smart hints</h2>
+        <p>In qutebrowser, urls can not only be hinted with letters and
+        numbers, but also with <a href="../words.txt">words</a>. When there is
+        a sensible url text available, qutebrowser will even use that
+        text to create a <a href="../smart.txt">smart</a> hint.</p>
+
+        <h2>Filled hints</h2>
+        <p>When no smart hints are available, because the hint text is
+        <a href="../l33t.txt">too</a> short or <a href="../l33t.txt">l33t</a> to
+        use, words from a dictionary will be used.</p>
+
+        <h2>Hint conflicts</h2>
+        <p>Of course, hints have to be unique. For instance, all hints
+        below should get a different hint, whether they're smart or
+        not:</p>
+        <ul>
+            <li><a href="../hinting.txt">hinting</a> should be a smart hint</li>
+            <li><a href="../l33t.txt">word</a> is a prefix of words</li>
+            <li><a href="../l33t.txt">3</a> is too 1337</li>
+            <li><a href="../l33t.txt">4</a> is too 1337</li>
+        </ul>
+    </body>
+</html>
diff --git a/tests/integration/data/l33t.txt b/tests/integration/data/l33t.txt
new file mode 100644
--- /dev/null
+++ b/tests/integration/data/l33t.txt
@@ -0,0 +1 @@
+l33t
diff --git a/tests/integration/data/smart.txt b/tests/integration/data/smart.txt
new file mode 100644
--- /dev/null
+++ b/tests/integration/data/smart.txt
@@ -0,0 +1 @@
+smart
diff --git a/tests/integration/data/words.txt b/tests/integration/data/words.txt
new file mode 100644
--- /dev/null
+++ b/tests/integration/data/words.txt
@@ -0,0 +1 @@
+words
diff --git a/tests/integration/test_hints_html.py b/tests/integration/test_hints_html.py
--- a/tests/integration/test_hints_html.py
+++ b/tests/integration/test_hints_html.py
@@ -1,3 +1,4 @@
+
 # vim: ft=python fileencoding=utf-8 sts=4 sw=4 et:
 
 # Copyright 2016 Florian Bruhin (The Compiler) <mail@qutebrowser.org>
@@ -25,6 +26,7 @@
 import yaml
 import pytest
 import bs4
+import textwrap
 
 
 def collect_tests():
@@ -54,3 +56,35 @@ def test_hints(test_name, quteproc):
     quteproc.wait_for(message='hints: a', category='hints')
     quteproc.send_cmd(':follow-hint a')
     quteproc.wait_for_load_finished('data/' + parsed['target'])
+
+
+def test_word_hints_issue1393(quteproc, tmpdir):
+    dict_file = tmpdir / 'dict'
+    dict_file.write(textwrap.dedent("""
+        alph
+        beta
+        gamm
+        delt
+        epsi
+    """))
+    targets = [
+        ('words', 'words.txt'),
+        ('smart', 'smart.txt'),
+        ('hinting', 'hinting.txt'),
+        ('alph', 'l33t.txt'),
+        ('beta', 'l33t.txt'),
+        ('gamm', 'l33t.txt'),
+        ('delt', 'l33t.txt'),
+        ('epsi', 'l33t.txt'),
+    ]
+
+    quteproc.set_setting('hints', 'mode', 'word')
+    quteproc.set_setting('hints', 'dictionary', str(dict_file))
+
+    for hint, target in targets:
+        quteproc.open_path('data/hints/issue1393.html')
+        quteproc.wait_for_load_finished('data/hints/issue1393.html')
+        quteproc.send_cmd(':hint')
+        quteproc.wait_for(message='hints: *', category='hints')
+        quteproc.send_cmd(':follow-hint {}'.format(hint))
+        quteproc.wait_for_load_finished('data/{}'.format(target))