Product: Mortgage

My husband received the house in our divorce settlement. I was instructed by the court to sign an Inter Spousal Transfer Deed, transferring all my ownership rights over to my ex-husband. He, on the other hand was not instructed to sign anything, but was told he shall assume, indemnify, pay and hold me harmless for the property. That was XXXX. 
Since XXXX my ex-husband has paid the mortgage late and once went as far as foreclosure proceedings in XXXX. 
To my surprise and horror I sit here today with my name on that mortgage and it appears I pay that mortgage late every month on a house I have n't lived in since XXXX. 
In XXXX my neighbor bought the house I 've rented since XXXX for me, and I pay him 8 % interest for an unconventional loan because no one will finance me due to the other property for one reason or another. 
They all say they can acknowledge what the situation is and how it has affected me and my credit, but can not make exceptions for it. 
I have a three year agreement that has reached its end with my neighbor that I would refinance the home by now or he would take it back, as he needs to access the equity, and I can remain in the home as a renter for two more years while I continue to improve my credit. 
There is nothing more I can do to improve what used to be a score in the mid to high XXXX dropped to a horrifying low of XXXX at one point during the four years it took to get divorced. It was XXXX before I started shopping around for refinancing over a month ago and I am still looking for someone willing to lend me money. 
I have done nothing to warrant the way I look on paper and can not legally be held responsible in any monetary way because the mortgage on a house that is not mine is paid late. 
What makes me think anything will be different in two more years? It 's already been five years and instead of refinancing in two years, I 'll be looking for a home purchase loan that requires 20 % down and if I ca n't find a loan the house will go on the market. 
I already had to move out of that house once for six weeks in order for XXXX XXXX to sell it back to me after being my landlord for two years. I have over {$100000.00} invested between rent and mortgage interest and I 'm not a spring chicken anymore to be buying a house to save up the equity when I already have equity here. 
It makes me sick when I think that in two years I will be XXXX and the mortgage I took out in XXXX should have been paid off. The extra money was to go to retirement, retirement was to grow and I would n't be a poor old woman depending on my kids ( I only had XXXX instead of XXXX ) or the government in my old age. 
I already suffered the Universal Default on credit cards because he paid late. 0 % - 2.99 % soared to 24.99 % - 32.68 % and I never paid one day late on anything and to top it off they were not my charges, but mine anyway through marriage. 
What good is a court order if creditors do n't acknowledge them? 
How do you get your name off a house by refinancing when no one is willing to refinance for him. Besides why would anyone want to refinance a 2.875 % loan? 
How does one end up this way after having been through a court of law? What fair or just law takes away ownership rights and leaves the liability? 
I tried calling Wells Fargo for refinancing since they have serviced the loan I 'm on for more than 20 years now, and should be familiar with how the payments were made when I was the one making them, and maybe they could help me. XXXX different times I explained my situation and was told I was being referred to the assumption department which told me to hang up the call could n't be connected which I assume was the nice way to be done talking with me.