This depends on the costs involved in refinancing in your specific situation. You may be able to get a second mortgage and avoid prepayment penalties, but overall you will need to calculate the costs in your situation.