The terms of the annuity contract determine what happens to the remaining balance when the annuity holder passes away. The remaining balance may be paid out to beneficiaries, or it may be forfeited to the insurance company.