Fix explicit handling of duplicate plugins

I really don't know how the default behavior is.. so we emulate something here
that I can rely on.