Many new machines come with software preinstalled, take out the ones that you're not going to use. This type of computer, while they have all the bells and whistles, are coming out of the box with a performance hit. Above all, make sure that you uninstall programs instead of deleting them off the hard drive directly. I don't see why they will in the near future anyway since this basically entices users to upgrade their machines under the false pretense that it is the only way to increase the speed. We need to put pressure on Microsoft to force them to clean up this aspect of Windows.
Sloppy code is no reason to buy a new computer.
Reference For more information, I suggest getting the Windows 95 Resource Kit. It has a ton of info regarding registry and other aspects of the operating system.