I would suggest spending 2 or 3 days to fully explore everything that Phillip Island has to offer, although a one-day tour around the island is also possible.