Revert "Try to ensure that interactive backends are loaded properly."

This reverts commit 74d3f078fb7f4c2e0c0bf12fffe4699e813cd649.
Apparently this breaks on some versions of PyGTK.