Passing the Entity manager as constructor argument to the factory will create circular references as the factory needs to be set in the configuration. You will be unable to write a working service definition for it in the bundle

The factory should receive the entity manager as a method argument instead (it would be even better if low level classes would not need to depend on the high level abstraction at all, but this would require a major refactoring of Doctrine, which cannot be done in 2.x)