Yes, you can terminate the contract at any time, but in that case the accrued interest is not paid out, you only get back the initial deposit.