Clue: One banished from historic mansion leaving quietly? (5)
Answer: EXILE
Explanation: EX (former, "historic") + P-ILE="mansion" minus P (piano, "quietly")