Create a cache for class loading that evilly caches both teh class and the exception.
Rework loading infrastructure to not throw/catch class cast exceptions.
Migrate ClassLoaderCache back into internal so we can make it package friendly (Since we're exposing API for testing), and implement a bunch of tests.