Fix issue where letters are not updating properly when an overlay is removed