time: fix race

Fixes issue 4622

R=golang-dev, dave, dvyukov
CC=golang-dev
https://codereview.appspot.com/7103046