driver clean up fixes.

This set of patches prevents crashes occurring during driver removing.