Add check that proper logger is passed to modifier builder