cmd/5g: fix bit mask for div/mod routines clobbering R12

This patch is based only on reading the code.  I have not
tried to construct a test case.

Fixes issue 9077.

LGTM=minux
R=minux
CC=golang-codereviews
https://codereview.appspot.com/172110043