Might be time-consuming, but it shouldn't be too hard to find enough material to fill up an MP3 player, assuming you don't need the music to be top-40/name-brand/etc.

Some ideas:

* Amazon has free MP3 samplers (and a few free albums even) and gives away free MP3 songs -- this link should get most of them:

http://www.amazon.com/s?ie=UTF8&page=1&rh=n%3A318774...

* There are bands that give away free albums (and songs) -- a couple of good ones that I like (indie rock):

-- The Jane Anchor's "The Second Wave" (and an EP also) is available here:

http://larklanerecords.bandcamp.com/

-- Canterbury's "Thank You" is available here:

http://www.canterburythankyou.com/

-- Harvey Danger (whom you may remember from their hit, "Flagpole Sitta") released their 3rd album for free--it's well done:

http://harveydanger.com/downloads/

(Sadly, Coldplay stopped giving away the concert album they'd been giving away for 3 years...but perhaps there are other popular bands who have things like that.)

* There are lists curated by others:

-- Forbes had a list of Best Free Albums last year, might be worth getting some of those:

http://www.forbes.com/sites/leorgalil/2011/12/13/the-best-fr...

-- And there are things like this:

http://goodbadunknown.blogspot.com/2009/02/11-best-free-albu...

* There's also the Internet Archive which has all sorts of stuff, from a truckload of Grateful Dead concert recordings to stuff in many, many genres, which can be searched for.

http://archive.org

For example, this indie rock is in French--I barely understand a word of it--but it's such ear candy:

http://archive.org/details/HappyHouse_Archives2002_2004

Or -- here -- have a 400 minute compilation of House music:

http://archive.org/details/Frosty2007Housebesuch

* There are loads of "net labels" that give away their catalogs -- lots of electronic music and other stuff--this seems like a good place to check into that:

http://netlabels.org/

* There's Jamendo, which has 55K free albums in a variety of genres:

http://www.jamendo.com/

* There used to be a great blog tracking stuff like this called Free Albums Galore which disappeared suddenly.

Thankfully it's available in pieces on archive.org.

First of all, the list of albums from A-Z is available, which isn't super helpful if you don't know what you're looking for, but the classical and jazz stuff will stand out and some of the artists, genres etc. may be identifiable:

http://web.archive.org/web/20110724125302/http://freealbums....

(Also helpful for finding stuff I once found previously but forgot the name of).

But the commentary is on the albums is what made it -- so it might be worth checking out the blog itself
(in monthly pieces) in archive.org.

Bit of a pain to have to go after each month individually (and then to translate the album links into non-archive.org links, and not all the links will prove to be active, but whatever).

Not that everything will be to one's taste or great, but there are some gems in there if one is willing to look (like that Happy House stuff listed above):

http://web.archive.org/web/20110724124826/http://freealbums....

Tack on YYYY/MM to see each month going back to 2005/07, e.g.

http://web.archive.org/web/20110724124826/http://freealbums....

* Still active is the Free Music Archive:

http://freemusicarchive.org

* What's that? You want some classical too? Sure:

- http://pianosociety.com/cms/index.php?section=1

- http://www.blockmrecords.org/bach/ -- this guy's recording all of Bach's organ works and making them available for free.

And, as always, the Internet Archive is your friend -- search by composer, piece, type of piece, etc.:
e.g.

http://archive.org/details/Mozart_Symphony_40

And, you know, it wouldn't kill the kids to actually learn something, would it? Lots of podcasts out there, free for the taking, like this stuff -- just load up Firefox w/DownloadThemAll and go to town (or scrape the MP3's w/cURL or whatever):

-- http://12byzantinerulers.com/

-- http://thehistoryofrome.typepad.com/

Probably helpful in this regard:

-- http://www.podcastdirectory.com/

-- http://www.openculture.com/freelanguagelessons

That should be enough, though, of course, no doubt there's loads more out there...