Born in Lausanne, Celestini started playing professionally with hometown's FC Lausanne-Sport, then had a four-season spell in France, spending two years each with Troyes AC and Olympique de Marseille. Whilst with the latter he played the 2004 UEFA Cup final, coming on as a substitute in the second half of the 0–2 loss to Valencia CF.