Refactor all factories to use a common templated base class.

This reduces code duplication between them.