Fix crash in dealing with IP addresses.

Use the proper destructor, which not only frees, but also sets the
pointer to null so it is not freed again.