Speed controller with positive and negative reference frequency, based on voltage frequency estimation, has been testing and is working.
The error that caused the nans on the flux-linkage estimation was the limit of the negative angle. The positive angles was upper bounded to 360° but the negative wasn't.
Therefore, as the motor spinned the negative angles grew bigger and then it was not possible to read angles from the sine and atan table.
As a consecuence the voltage got struck in a random angle and the frequency simply drops to 0. The solution was just to limit the negatives angle to 0 and add them +360°.