Cython binding for subtract_mask and test with a rectangular mask