It depends: if you add cold milk to ginger, the milk may curdle. But when you prepare a hot ginger tea and then add a splash of milk, it won’t curdle. At least, it hasn’t happened to me yet, even when I add cold milk to the hot tea (I usually use about 1/3 cup of milk for 2/3 cup of ginger tea). However, if you want to be sure your ginger latte doesn’t curdle, warming your milk may help (and anyway, if you want to froth your milk, it’s better to heat it a little).