Yes, I think we agree on the fact the we could use mad(resid, center=0). I changed this into the PR code.
Alternatively, we can give the choice to the user by using the 'init' keyword argument. I think it was partly created for this kind of purpose, so why not accepting "init='ols'/None/False" for an initialization with OLS scale, and "init='mad'" for an initialization with MAD?

Interestingly, changing the OLS scale for a MAD did not change the results of the tests I performed...