That is the entire sound is captured as a wave form. Files can get quite large. In the pre-web days it was common to need to translate among the three formats. This often involved invoking Unix commands with all kinds of arcane text switches. Thankfully things are much easier now.
Most audio players can read more than one format.
MIDI is different that recording the actual sound. It is more like digital sheet music. These files are much smaller, than wave forms. The downside to MIDI is that it can sound pretty 'tinny'. Most synthesizers on soundcards are not up to producing high quality music.