mosaicml

/

mpt-7b-instruct

like 461

I am trying to use the "mosaicml/mpt-7b-instruct" model for my project. However, when I instantiate the model, I encounter a TypeError with the message "init() got an unexpected keyword argument 'approximate'". This error seems to be related to the nn.GELU activation function.

Here is the relevant code snippet:
generate = InstructionTextGenerationPipeline( "mosaicml/mpt-7b-instruct", torch_dtype=torch.bfloat16, trust_remote_code=True, ) stop_token_ids = generate.tokenizer.convert_tokens_to_ids([""])

I am using the latest version of the transformers library and the model checkpoint. I have also ensured that all the dependencies are up to date.

Can anyone help me understand why I am getting this error and how I can resolve it? Is there a specific version of the "mosaicml/mpt-7b-instruct" model that supports the 'approximate' argument in nn.GELU?

Any assistance would be greatly appreciated. Thank you!

Note: Make sure to include relevant details about your environment, such as the versions of transformers and PyTorch you are using.

My System -
GPU - NVIDIA GeForce RTX40490
PyTorch - Cuda 11.1