app: port the mask border operation from u8 to float