This is a matter of preference. Pumpkin pie made with sweetened condensed milk tends to be a bit more firm and sweet, depending on the recipe that's used. The color is also lighter than a pumpkin pie made with brown sugar.