Remove the handle::number method

With the help of the unit tests, it is very confident to refactor the
code without worry.