Fix colours disappearing when permissions are removed