You may, however, be able to do an end run around this system. Go back to the bank and politely ask to close the joint account. Then transfer the money to another account in your name only. The only catch is that your husband may still claim rights in the money during the divorce proceedings. Be prepared to show that it was really yours and yours alone.
Annulment is more quaint than divorce.
The concept of annulment is an old-fashioned one adopted by state legislatures many years ago at the behest of certain religions which publicly opposed divorce, but privately recognized that certain marriages were simply not meant to be. For example, in many states, if a prospective husband professed that he wanted to have children, but knew this was impossible because he had an irreversible vasectomy, this might be grounds for an annulment based on fraud. When it comes to dividing property, determining custody or child support or other practical matters, there is little real difference between an annulment and a divorce. If your ex forced you to keep his stuff for two years, a judge isn't going to be too sympathetic if he now claims he can't find his prized high school debate trophy. Your idea that he sign a release is a good one, particularly if it includes an itemized list of all the property.