So if you plan on staying in the home for at least another 3-4 years and the closing costs are minimal and can be recuperated in say 12 months and your bottom line monthly payment is reduced comfortably, then it must be a good time to refinance. Usually no lender will lend on property if it is for sale because the loan will not be on the books long enough for them to make any money on the deal. It is also not beneficial for you since you will have to pay closing costs, go through all the time and running around to close a loan and it does not do the buyer any good since they cannot assume your loan. So already this is a plus. However, there are several factors to this decision.
Shorter term means higher payment.
Also compare would you be better off by making 1-2 additional principal reduction payments per year vs. refinancing. If the rates have gone down enough to make it worth your while, then anytime. Remember all those closing costs you paid will have to be paid again so most people don't want to use up their equity by refinancing too soon after a purchase. Really sit down and compare what you have just paid with what you will have to pay again and look at what the savings will be to decide if it is really worth it. I purchased my home last June.