This fixes #7434.  It now only tries once per virtual poll attempt if the attempt to poll they physical device fails.