Backport of https://review.openstack.org/#/c/108272/3 from Juno tip.

It has yet to be fully accepted, but I don't forsee any major code changes
and this should address our issue with the dhcpnak replies after a restart.

Partial-rally-bug: DE494