Limiting access to user conversations