Operating without good records is like flying in a dense fog with no instruments. For your checking account, keep monthly bank statements and canceled checks, if the bank returns them to you. Keep deposit slips showing the source of the bank deposits as well. For instance, loans and personal funds deposited in your business account are not income. Those pieces of paper from transactions, usually expenses, are the second type of records to be saved.
Some folks use an accordion file or a bunch of manila envelopes to keep receipts.
Invoices and purchases orders for equipment should be kept separately from other expenses, since asset purchases are taxed differently than other expenses. Three years is the normal time limit for an IRS audit. So, the very minimum holding period for records is three years after the date you file your tax return. However, don't rush to throw out those old boxes just yet. Six years is a wiser rule of thumb.