This could be a better idea than replacing a low-rate primary mortgage. A HELOC does not affect your first mortgage, so you keep your low rate. However, a HELOC comes with a variable rate, potentially meaning higher payments in the future. Consider a home equity loan, which comes with a fixed rate.