As with all seaside destinations, summer is the ideal time to visit Victor Harbor. That said, during the colder winter months, whale spotting is quite popular in the region.