Born in Santa Elena, Entre Ríos, Caballero started his playing career with Boca Juniors in 2001, where he won three major titles in 2003. On 14 December of that year he witnessed – from the bench – his team beat A.C. Milan 3–1 on penalties, in the 2003 Intercontinental Cup.