Further refactoring of the assembly loader, centralizing the place we actually load a requested reference if we find a match.