Fix performance regression caused by transparent backgrounds on lines and gutter

Fixes #3075