Fix async coloring misbehaviour

It's a bit hacky, but seems to work. Any ideas how to improve this are welcome.