Cleaned up detach kernel driver code.

A tidier version of the changes made in the last couple of commits.