Internally handle all transformers as HLSTransformer