I wouldn't pay a dime more. I especially recommend buying used equipment if you like to tinker, and are comfortable with opening the case and poking around the innards. You can practice installing and configuring components, and you learn what features are really important to you. An old computer will most likely not run new software well, so buy or borrow old software. Also, see the Computer Bits classifieds.
Old software is just fine, and in many cases it's easier to learn.
Usually you can buy something brand-new for not much more. If you're more comfortable with simply stepping on the gas and going, and don't care what it looks like under the hood, buy a new system from a local vendor. You can get some very good buys through mail-order houses, but if you want to return something, you pay shipping charges, and sometimes a restocking fee. They will even accept returns on opened software. Some of the local computer stores also offer classes, and it's nice to be able to bring your computer troubles to a real human.