Do not crash when source plugins conflict.

Fixes #3179.

This is one approach to solve the problem by keeping the `pkg` information around until we have finished loading all the source handlers. This information is discarded afterwards.
Let's just use allHandlersWithPkgs later in the function instead of having two similar data structures