Thermal paste, also known as thermal grease, is usually made up of zinc oxide and a silicon grease. It's not as good as a perfectly tight fit, but it's much better than an imperfect fit stifled with downright tiny pockets of air. Of course, many heatsinks, however, don't ship with thermal grease. The quality of the stuff they do ship with varies widely. The problem is that it usually doesn't fill the gaps as well, being it has to be a more solid form to travel well.
Many enthusiasts scrape the conductive tape off the second they get their hands on a heatsink.
If you choose to do so, some rubbing alcohol may be in order to get the last bits of the conductive material off of there. The quality of thermal paste varies as well. While, as mentioned above, most thermal greases are zinc oxide based, but not all. Some use a copper powder or aluminum, and some even use silver as the conducting material. Fortunately for the enthusiast, tubes of paste often come with a rating that lets you immediately compare different pastes.