A mixture of both is the best option. Or using natural yogurt instead of milk as we do for this recipe. Using only milk will produce a filling without enough structure and might take too long to set. All cream will render a heavy quiche, dense, and too rich.