Do not add a ip port if it was previously added