Remove unused code for partial background rendering

Once upon a time we had an 'optimisation' to render the background
only when needed.

There was still some cruft left, which is removed in this commit.

Fixes #1135