Born in Johannesburg, Viljoen was signed from Johannesburg Rangers' colts team by Ipswich in 1966. He spent 12 years at Portman Road, playing 305 games and won two caps for England, both coming in a four-day spell in May 1975. A goalless draw with Northern Ireland in Belfast was followed by a 2–2 draw with Wales at Wembley, both games in the Home International Championship. Viljoen contributed to Ipswich's victorious 1977–78 FA Cup campaign, making four appearances and scoring twice during the run. However he was not part of the squad for the final itself. In 1978, he moved on to Manchester City, before signing for Chelsea for £60,000 in 1980. He was released by the club at the end of the 1981–82 season, and signed for non-League Southall.