view: don't hit tracker every time the user scrolls

Cache the number of items in the fully loaded view the first time
it's needed, and use that value for all future scrolling.

Note: if we ever support watching the library for changes, we need
to invalidate the cache.

https://bugzilla.gnome.org/show_bug.cgi?id=706025