After fighting a Brexit campaign driven in part by claims the UK would be able to control its borders, anti-EU Conservatives have seen net migration soar since the 2016 referendum.