Don't try to load users for private messages