More aggressively clear line heights in refresh()

Closes #2157