Remove your finger from the lower section and blow again. Compare this sound to the first sound. The sound is higher when the end of the lower section of the straw is open. Each of the straw sections is a tube filled with a column of air. When you blow through the straw, the air inside moves forward, causing the air in the lower section to vibrate.
Air vibrates faster in an open tube than in a closed tube.
Since a faster vibration produces higher pitch, open tubes have a higher pitch than do closed tubes. In wind instruments, such as the trumpet, clarinet, French horn, and pipe organ, vibrations in columns of air produce sounds with different pitches. This experiment and many more can be found on pg. 107 in 203 Icy, Freezing, Frosty, Cool, and Wild Experiments , by Janice VanCleave.