From what you write, you seem to have the ability to accelerate the payments and take legal action to recover the entire outstanding amount: a promissory note is, in this way, essentially a contract, and like any other contract, the parties are bound by its terms and have whatever rights and remedies it gives to them. If the terms of the note allow acceleration under this circumstance, you may accelerate.