Repository: qutebrowser/qutebrowser

Issue:
Open/Close tabs is slow when many tabs are open
The more tabs are open the slower it gets to open/close new tabs, especially at the beginning of the tablist.

Steps to reproduce:
1. Open many tabs: `$ qutebrowser -T $(seq 50)`
2. Go to tab 1: `:buffer 1`
3. Delete it: `d`

The same goes when opening a new tab at that position with `tabs.new_position = next`.

Opening and closing tabs at the end of the list remains fast, however many tabs are open. I suppose the tablist is saved in a contiguous sequence container that needs to move all subsequent elements when one is added or removed in the middle.

http://paste.the-compiler.org/view/869eccb3

<!-- If this is a bug report, please remember to mention your version info from
`:open qute:version` or `qutebrowser --version` -->

Add a cache for pinned statistics
Combined with config caching, I think this should raise the amount of usable tabs to about ~100.

This is a little more complicated than @The-Compiler s [initial patch](http://paste.the-compiler.org/view/3a2c5ae6), but I think it should be a little better this way (as it only clears the pinned statistic cache when needed). A cleaner way of doing this is keeping track of the pinned width as the tabs change (by adding and removing sizes) rather than iterating over them every time. However, I think this is more than enough to fix it from being a bottleneck with lots of tabs.

This helps out with #4069, but only for extreme amounts of tabs (~100), wheras config caching would help everything. I think this patch is the one that helps the most with deleting the first tab (which is the described issue in #4069) as well.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/qutebrowser/qutebrowser/4070)
<!-- Reviewable:end -->



Discussion:
Hi,

I'm actually semi-actively working on this even though I don't think there was
an issue for it, I'm going to submit a PR that helps with this with extreme
amounts of tabs (~200).

Your situation might be different though, so could you submit a profile?

You should be able to get one by running the "./scripts/dev/run_profile.py"
script, and submitting the "callgraph file" which is placed in the temporary
directory. Ideally, you would submit the callgraph after repeating slow
actions, to help us find what parts of the code are slow.

Right now, I think that the config system is the slowest part of the path for
that amount of tabs, and I have a couple of ways of fixing it.

1. Add a cached_val object to the config variable, which automatically caches
it: https://github.com/jgkamat/qutebrowser/tree/jay/configcache-perf
2. Optimize the config system itself. I don't think this is actually worth it,
this is the only situation I think it's going to matter in.
3. Create a new object which is just a wrapper around a simple dict, which
highly used code paths can use to query cached versions. This isn't part of
the config system at all, and so it can be very simple and has little chances
of causing other problems.
4. Cache config values "manually" by storing them as variables, and updating
those variables on config change.

I personally really like #3, even though I have something done for #1 already.
@The-Compiler, could you pick one of the solutions above (or propose a
different one)? Once that's done, I can start writing polished patch to fix
this.

Also see https://github.com/qutebrowser/qutebrowser/pull/3582 for the previous
attempt of this.

Ram-Z writes:

> The more tabs are open the slower it gets to open/close new tabs, especially at the beginning of the tablist.
>
> Steps to reproduce:
> 1. Open many tabs: `$ qutebrowser -T $(seq 50)`
> 2. Go to tab 1: `:buffer 1`
> 3. Delete it: `d`
>
> The same goes when opening a new tab at that position with `tabs.new_position = next`.
>
> Opening and closing tabs at the end of the list remains fast, however many tabs are open. I suppose the tablist is saved in a contiguous sequence container that needs to move all subsequent elements when one is added or removed in the middle.
>
> http://paste.the-compiler.org/view/869eccb3
>
> <!-- If this is a bug report, please remember to mention your version info from
> `:open qute:version` or `qutebrowser --version` -->

Here's a link to the callgraph profile: https://rmz.io/i/qutebrowser_callgraph.profile

I opened a new instance of qutebrowser with 50 tabs through the profiler, then went to tab 1 and closed every tab.
For you, it seems like config is the main fixable bottleneck to me, so I'll work on a way to cache config values once we make a decsion from the above.

![2018-07-14-094637_916x773_scrot](https://user-images.githubusercontent.com/4349709/42724999-6f5b89aa-876c-11e8-9c0a-008dd3da2c93.png)

I have this issue with more like 20-30+ tabs; IIRC some other functionality slows down, and qutebrowser becomes basically unusable. I'm guessing the issue is the same for me, but I can trying profiling things at some point as well.
A profile would be very helpful, just to see if your issue is different than the one I'm seeing. If that's too much work for some reason, you could try out the [half-finished branch I made](https://github.com/jgkamat/qutebrowser/tree/jay/configcache-perf), which should have similar performance to implementing config caching properly, and see if that "solves" the issue.
Thanks for the pull request! Note that reviews currently can take a bit longer, as @The-Compiler is busy with exams until September.

(Powered by [GitMate.io](https://gitmate.io))

<!-- Timestamp signature `Thanks for the pull request! Note that reviews currently can take a bit longer, as @The-Compiler is busy with exams until September.:1feMER:MRNgwFDWAF63U_vEwx_SJNGWz4g` -->

Resolving patch:
diff --git a/qutebrowser/mainwindow/tabwidget.py b/qutebrowser/mainwindow/tabwidget.py
--- a/qutebrowser/mainwindow/tabwidget.py
+++ b/qutebrowser/mainwindow/tabwidget.py
@@ -568,15 +568,6 @@ def _text_to_width(text):
             width = max(min_width, width)
         return QSize(width, height)
 
-    def _pinned_statistics(self) -> (int, int):
-        """Get the number of pinned tabs and the total width of pinned tabs."""
-        pinned_list = [idx for idx in range(self.count())
-                       if self._tab_pinned(idx)]
-        pinned_count = len(pinned_list)
-        pinned_width = sum(self.minimumTabSizeHint(idx, ellipsis=False).width()
-                           for idx in pinned_list)
-        return (pinned_count, pinned_width)
-
     def _tab_pinned(self, index: int) -> bool:
         """Return True if tab is pinned."""
         try:
@@ -614,32 +605,17 @@ def tabSizeHint(self, index: int):
                 width = int(confwidth)
             size = QSize(max(minimum_size.width(), width), height)
         else:
-            if config.cache['tabs.pinned.shrink']:
-                pinned = self._tab_pinned(index)
-                pinned_count, pinned_width = self._pinned_statistics()
-            else:
-                pinned = False
-                pinned_count, pinned_width = 0, 0
-            no_pinned_count = self.count() - pinned_count
-            no_pinned_width = self.width() - pinned_width
-
-            if pinned:
+            if config.cache['tabs.pinned.shrink'] and self._tab_pinned(index):
                 # Give pinned tabs the minimum size they need to display their
                 # titles, let Qt handle scaling it down if we get too small.
                 width = self.minimumTabSizeHint(index, ellipsis=False).width()
             else:
-                width = no_pinned_width / no_pinned_count
+                # Request as much space as possible so we fill the tabbar, let
+                # Qt shrink us down
+                width = self.width()
 
-            # If no_pinned_width is not divisible by no_pinned_count, add a
-            # pixel to some tabs so that there is no ugly leftover space.
-            if (no_pinned_count > 0 and
-                    index < no_pinned_width % no_pinned_count):
-                width += 1
-
-            # If we don't have enough space, we return the minimum size so we
-            # get scroll buttons as soon as needed.
+            # If we don't have enough space, we return the minimum size
             width = max(width, minimum_size.width())
-
             size = QSize(width, height)
         qtutils.ensure_valid(size)
         return size


Test patch: