manager: remove unneded private InstanceHolder class

we realy need this so simplify this code a bit.

Also fixes #516

NOTE: this is a candidate for the stable branch