It depends. Your refi rate should be a deciding factor. Your credit score could drop, and the life of your loan may extend. If your new rate counters those effects, it may not be worth refinancing at all. Speak to a financial consultant if you need advice about refinancing your mortgage.