Born in Remouchamps in the municipality of Aywaille, Gilbert turned professional in 2003 by joining after riding as stagiaire for the team in late 2000. During this season he recorded his first victory by winning a stage in the Tour de l'Avenir. In 2004 he began by winning a stage in the Tour Down Under as well as the young rider classification. He participated in the Cycling at the 2004 Summer Olympics – Men's individual road race at the 2004 Summer Olympics where he finished 49th. He also won the Paris–Corrèze. In 2005 he won several races in France, which allowed him to win the Coupe de France de cyclisme sur route. These victories included the Trophée des Grimpeurs, the Tour du Haut Var and the Polynormande. He also took stages in the Four Days of Dunkirk and the Tour Méditerranéen.