Victor is basically Mr. House's eyes and ears, not to mention your guide to the New Vegas Strip. So he will follow you from Goodsprings onwards and can be spotted in various locations along the structured path to New Vegas. You can even confront him or kill him, but like a stray cat you fed once, he always comes back.