Add ARP entry for gateway in the container