Repository: qutebrowser/qutebrowser

Issue:
Sometimes, pressing the first key of a two key hint follows it prematurely
I've noticed this bug for a while, but I didn't put the time to reproduce it properly.

Copy this into a file (test.html):

```
<body>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
<a href="url">abc</a>
</body>
```

Then open the file in a `--temp-basedir` and hint the links:

![2017-11-04-161855_1920x1080_scrot](https://user-images.githubusercontent.com/4349709/32409112-e0e99fec-c17b-11e7-99bd-1414b6bac3ab.png)

pressing `l` follows the hint at this point. Another interesting thing to note is that there's only one hint starting with `l` (`la`) (shouldn't the hint display `l` in this case)?

I'm happy to take a look at this later on.

```
qutebrowser v1.0.3
Git commit: af93c0fbc (2017-11-04 17:15:20 +0100)
Backend: QtWebEngine (Chromium 49.0.2623.111)

CPython: 3.5.3
Qt: 5.7.1
PyQt: 5.7

sip: 4.18.1
colorama: no
pypeg2: 2.15
jinja2: 2.8
pygments: 2.2.0
yaml: 3.12
cssutils: noqutebrowser v1.0.3
Git commit: af93c0fbc (2017-11-04 17:15:20 +0100)
Backend: QtWebEngine (Chromium 49.0.2623.111)

CPython: 3.5.3
Qt: 5.7.1
PyQt: 5.7

sip: 4.18.1
colorama: no
pypeg2: 2.15
jinja2: 2.8
pygments: 2.2.0
yaml: 3.12
cssutils: no
attr: 16.3.0
PyQt5.QtWebEngineWidgets: yes
PyQt5.QtWebKitWidgets: yes
pdf.js: no
sqlite: 3.16.2
QtNetwork SSL: OpenSSL 1.0.2l  25 May 2017

Style: QFusionStyle
Platform: Linux-4.9.0-4-amd64-x86_64-with-debian-9.2, 64bit
Linux distribution: Debian GNU/Linux 9 (stretch) (debian)
Frozen: False
Imported from /home/jay/Code/qutebrowser/qutebrowser
Qt library executable path: /usr/lib/x86_64-linux-gnu/qt5/libexec, data path: /usr/share/qt5

Paths:
cache: /tmp/qutebrowser-basedir-jd9b2psn/cache
config: /tmp/qutebrowser-basedir-jd9b2psn/config
data: /tmp/qutebrowser-basedir-jd9b2psn/data
runtime: /tmp/qutebrowser-basedir-jd9b2psn/runtime
attr: 16.3.0
PyQt5.QtWebEngineWidgets: yes
PyQt5.QtWebKitWidgets: yes
pdf.js: no
sqlite: 3.16.2
QtNetwork SSL: OpenSSL 1.0.2l  25 May 2017

Style: QFusionStyle
Platform: Linux-4.9.0-4-amd64-x86_64-with-debian-9.2, 64bit
Linux distribution: Debian GNU/Linux 9 (stretch) (debian)
Frozen: False
Imported from /home/jay/Code/qutebrowser/qutebrowser
Qt library executable path: /usr/lib/x86_64-linux-gnu/qt5/libexec, data path: /usr/share/qt5

Paths:
cache: /tmp/qutebrowser-basedir-jd9b2psn/cache
config: /tmp/qutebrowser-basedir-jd9b2psn/config
data: /tmp/qutebrowser-basedir-jd9b2psn/data
runtime: /tmp/qutebrowser-basedir-jd9b2psn/runtime
```


Discussion:
Also reproduced with v0.11.0.
Logs:

```
23:30:29 DEBUG    commands   command:run:502 Calling qutebrowser.browser.hints.HintManager.start(<qutebrowser.browser.hints.HintManager object at 0x7f76a4990828>, False, <Group.all: 1>, <Target.normal: 1>, win_id=0, mode=None, add_history=False)
23:30:29 DEBUG    modes      modeman:_eventFilter_keypress:188 handled: True, forward_unbound_keys: auto, passthrough: False, is_non_alnum: False --> filter: True (focused: <PyQt5.QtWidgets.QWidget object at 0x7f76a4931438>)
23:30:29 DEBUG    hints      hints:_start_cb:590 hints: a, jf, kf, s, jg, kg, d, jh, kh, f, jj, kj, g, jk, kk, h, jl, kl, ja, ka, la, js, ks, jd, kd
23:30:29 DEBUG    message    message:set_text:261 Follow hint...
23:30:29 DEBUG    statusbar  text:set_text:57 Setting normal text to 'Follow hint...'.
23:30:29 DEBUG    modes      modeman:enter:234 Entering mode KeyMode.hint (reason: HintManager.start)
23:30:29 DEBUG    modes      modeman:_eventFilter_keyrelease:208 filter: True
23:30:30 DEBUG    modes      modeman:_eventFilter_keypress:161 got keypress in mode KeyMode.hint - delegating to <qutebrowser.keyinput.modeparsers.HintKeyParser supports_chains=True supports_count=False>
23:30:30 DEBUG    keyboard   basekeyparser:_debug_log:108 Got key: 0x4c / text: 'l'
23:30:30 DEBUG    keyboard   basekeyparser:_debug_log:108 No match for 'l' (added l)
23:30:30 DEBUG    hints      hints:handle_partial_key:765 Handling new keystring: 'l'
23:30:30 DEBUG    modes      modeman:leave:296 Leaving mode KeyMode.hint (reason: followed)
23:30:30 DEBUG    keyboard   basekeyparser:_debug_log:108 discarding keystring 'l'.
23:30:30 DEBUG    hints      hints:handle_partial_key:765 Handling new keystring: ''
23:30:30 DEBUG    statusbar  text:maybe_reset_text:70 Resetting: 'Follow hint...'
23:30:30 DEBUG    statusbar  text:set_text:57 Setting normal text to ''.
23:30:30 DEBUG    webelem    webelem:click:392 Clicking <qutebrowser.browser.webengine.webengineelem.WebEngineElement html='<a href="url">abc</a>'> with click_target ClickTarget.normal, force_event False
```
This is caused by an incorrect calculation here: https://github.com/qutebrowser/qutebrowser/blob/master/qutebrowser/browser/hints.py#L449-L450.

The line is 
```
short_count = math.floor((len(chars) ** needed - len(elems)) /
                                     len(chars))
```

Walking through this, `len(chars) ** needed` calculates the total space we have available to use, then we subtract the number of elements we need to match. Then we divide by the number of character, as every short hint will need `len(chars)` of the space in order to be allocated.

However, for every short hint being added, we need one less added to `len(elems)`, as it's counted for by the short hints rather than the long ones, so this line seems to be overcounting a bit. 

I tried playing around with a more accurate formula, but wolframalpha started giving me differential equations as answers.... I suspect there's a more proper way to be doing this, but I don't know enough combonatorics to figure it out... 

Resolving patch:
diff --git a/qutebrowser/browser/hints.py b/qutebrowser/browser/hints.py
--- a/qutebrowser/browser/hints.py
+++ b/qutebrowser/browser/hints.py
@@ -446,8 +446,17 @@ def _hint_scattered(self, min_chars, chars, elems):
         # Short hints are the number of hints we can possibly show which are
         # (needed - 1) digits in length.
         if needed > min_chars:
-            short_count = math.floor((len(chars) ** needed - len(elems)) /
-                                     len(chars))
+            total_space = len(chars) ** needed
+            # Calculate short_count naively, by finding the avaiable space and
+            # dividing by the number of spots we would loose by adding a
+            # short element
+            short_count = math.floor((total_space - len(elems)) /
+                                     (len(chars)))
+            # Check if we double counted above to warrant another short_count
+            # https://github.com/qutebrowser/qutebrowser/issues/3242
+            if total_space - (short_count * len(chars) +
+                              (len(elems) - short_count)) >= len(chars) - 1:
+                short_count += 1
         else:
             short_count = 0
 


Test patch:
diff --git a/tests/end2end/test_hints_html.py b/tests/end2end/test_hints_html.py
--- a/tests/end2end/test_hints_html.py
+++ b/tests/end2end/test_hints_html.py
@@ -22,12 +22,17 @@
 import os
 import os.path
 import textwrap
+import string
+import functools
+import operator
 
 import attr
 import yaml
 import pytest
 import bs4
 
+import qutebrowser.browser.hints
+
 
 def collect_tests():
     basedir = os.path.dirname(__file__)
@@ -146,3 +151,51 @@ def test_word_hints_issue1393(quteproc, tmpdir):
         quteproc.wait_for(message='hints: *', category='hints')
         quteproc.send_cmd(':follow-hint {}'.format(hint))
         quteproc.wait_for_load_finished('data/{}'.format(target))
+
+
+@pytest.mark.parametrize('min_len', [0, 3])
+@pytest.mark.parametrize('num_chars', [9])
+@pytest.mark.parametrize('num_elements', range(1, 26))
+def test_scattered_hints_count(win_registry, mode_manager, min_len,
+                               num_chars, num_elements):
+    """Test scattered hints function.
+
+    Tests many properties from an invocation of _hint_scattered, including
+
+    1. Hints must be unique
+    2. There can only be two hint lengths, only 1 apart
+    3. There are no unique prefixes for long hints, such as 'la' with no 'l<x>'
+    """
+    manager = qutebrowser.browser.hints.HintManager(0, 0)
+    chars = string.ascii_lowercase[:num_chars]
+
+    hints = manager._hint_scattered(min_len, chars,
+                                    list(range(num_elements)))
+
+    # Check if hints are unique
+    assert len(hints) == len(set(hints))
+
+    # Check if any hints are shorter than min_len
+    assert not any(x for x in hints if len(x) < min_len)
+
+    # Check we don't have more than 2 link lengths
+    # Eg: 'a' 'bc' and 'def' cannot be in the same hint string
+    hint_lens = {len(h) for h in hints}
+    assert len(hint_lens) <= 2
+
+    if len(hint_lens) == 2:
+        # Check if hint_lens are more than 1 apart
+        # Eg: 'abc' and 'd' cannot be in the same hint sequence, but
+        # 'ab' and 'c' can
+        assert abs(functools.reduce(operator.sub, hint_lens)) <= 1
+
+    longest_hints = [x for x in hints if len(x) == max(hint_lens)]
+
+    if min_len < max(hint_lens) - 1:
+        # Check if we have any unique prefixes. For example, 'la'
+        # alone, with no other 'l<x>'
+        count_map = {}
+        for x in longest_hints:
+            prefix = x[:-1]
+            count_map[prefix] = count_map.get(prefix, 0) + 1
+        assert all(e != 1 for e in count_map.values())