Voldemort was on the edge of death so he needed to wait for the right opportunity. It's stated in the fourth book that voldemort was stuck as a psuedo-spirit and lost in the woods of Albania because any means to restore him required the use of a wand. No followers returned to him for 10 years.