Fix clearing authorship colors which was broken by #1609

Fixes #1620