accept that a slow vm might take awhile to fire remove helper

This will hopefully allow the test to pass in jenkins.

Signed-off-by: Serge Hallyn <serge.hallyn@ubuntu.com>