Stale bread gets a little bit hard and it soaks up the custard mixture better. If the bread is not a little stale or a little hard, it will disintegrate into the milk mixture.