Fixes cleaning up of the rendered pages

Fixes #3942

@timvandermeij can you review this?