The air inside the balloon is at a higher pressure than the air outside, and is therefore more dense. We know the air inside is at a higher pressure because it is pushing outward on the balloon in order for the balloon to hold its shape. Another way to put it. The elastic balloon is exerting a force toward the center of the balloon. Air pressure inside is pushing outward against this force.
Air pressure exerts a force from high toward lower pressure.
Also, if you puncture the balloon, air comes out. This would happen only if the inside pressure were higher than the outside pressure. The weight of the balloon plus the slightly higher density inside an air balloon is why air-filled balloons slowly sink toward the floor or ground when released. On the Purdue University weather map a high will have a pressure of 1012 one day. The next day a low might have a pressure of 1013.