Get rid of the right-hand column compare block. Open the reports folder in the app/design/frontend/base/default/layout/reports directory.
Remove the product page's Compare link.
For grid and list mode, remove the Compare link from the product list page in two locations.