Do this a lot. Do this all the time. Ask them nicely and compliment their work, but be clear that you want them to link you as well. I don't recommend those free banner exchanges, like Link Exchange, even though I'm using them. I place them on my link pages.
People expect links to other site there already.
I do, however, recommend exchanging banners as links with other sites that have the same theme as your own site. This can generate more traffic than those free generic exchanges. Create a signature file with your URL and a short description of your site. This is your internet business card. Then, go to these Newsgroups and read some of the conversations that are going on.