@schmittjoh the problem is that your bundle creates a cyclic dependency between Doctrine <-> Serializer. One could not live without the other and vice versa. While we can try to reduce this coupling to a suggest level, there's still a soft dependency between them.
My idea would be linear to what @Ocramius suggests. We could create a simplified version of your Serializer that does not have a dependency of ORM or DBAL and consume it inside them.
It's not an advocate about owners of library. It's about resolving dependencies between packages.