Kipyego's first international highlight came as a junior at the 2005 IAAF World Cross Country Championships, where he took the silver medal in the junior race to form a Kenyan sweep of the medals, alongside Augustine Choge and Barnabas Kosgei. At the 2007 World Cross Country Championships he won the bronze medal in the senior race, while the Kenyan team of which he was a part won the team competition. In 2008 he finished tenth individually, and won another team gold medal. He also competed at the 2008 IAAF World Athletics Final that year, taking eighth place in the 5000 metres on the track.