Although you will be charged interest on your loan (far less than tax liability), you do not have to repay it. Any outstanding loans will be deducted from the death benefit to your beneficiary(s) when you die.