Only delete declared queue if destination is /exchange/*

This way we can be sure the queue was declared by us.