[libcommhistory] Handle contact removal in RecentContactsModel