Flags everywhere

This will allow further API cleanups. As usual there's more details in the commits.