Allow removal of relations, if relations were originally created from 'the other end'.