Remove test cases and add notification that they've been removed