Fix wrong foreground color choosing logic

This commit is needed for #1 and #42 to be fixed.