There is no penalty if you pay it back. In fact, with a 401k loan, you are simply taking out a loan against yourself. Each month your salary is reduced by an additional amount, over and above your normal contribution, until the loan is paid off. The best part is that you are forced to pay yourself interest during the payback period. That way, you are still earning on your money.
Loans have to be repaid in five years with after tax money.
Most plans are set up whereby the employer will match a certain percentage of the employee's contribution. They actually give you money. Go to your broker and try getting that deal. Do this year after year and you will find yourself sitting pretty in the long run. In fact, you'll probably find yourself sitting pretty on a sandy beach in sunny Florida, after enjoying 18 holes on the golf course day after day.