Found the slowdown

We're doing invokeinterface, whereas specialization does invokevirtual.
But we'll keep our way of doing things, as we do want to have sane
mixing without the double fields in specialization.