Fixes #115 / Only set background to white when not already set

Since we're not reusing it we can pretty much use it directly in the if statement instead.