Make Loader an abstract class, and have loaders constructed with a ClassLoader from which they load ALL the things.  Since there is a loader (or set of) in each graph, each graph gets the classloader of the first module used to configure the graph.