Yes, this is okay, but as with almost everything in life, in moderation and adequately adjusted. Extremes are usually never good. Very high or very cold temperatures are neither comfortable nor healthy.
In winter it is not necessary to put the heating on much higher than 18 to 21 degrees Celsius. In summer it is fine at about 24 or 25 degrees, but it is not advisable to lower the temperature below 20 degrees Celsius. What you can and should do is prevent the car’s air vents from directly blowing cold air on the baby, to avoid colds.