Do not remove imports that have inline comments

This generalizes the "noqa" check.