Question: is layperson a derogatory term?
Answer: So in short, it's non-offensive if you know your audience is not an expert (or a clergyman if you're using that meaning), but can be very offensive if you're referring to a specific person, since it calls their expertise (or faith) into question.