Make the manager not track instance of backend.

- This loosens the coupling between three different classes, including
  the key manager.  Just leverage ZCA to acquire the right things as
  required.