oleksandrfluxon
/

mpt-7b-instruct-evaluate

Text Generation

text-generation-inference

Model card Files Files and versions Community

oleksandrfluxon commited on Jul 22, 2023

Commit

641be9d

•

1 Parent(s): 594d449

Update pipeline.py

Files changed (1) hide show

pipeline.py +2 -2

pipeline.py CHANGED Viewed

@@ -14,8 +14,8 @@ class PreTrainedPipeline():
                 path,
                 trust_remote_code=True
             )
-            config.attn_config['attn_impl'] = 'triton'
-            config.init_device = 'cuda:0' # For fast initialization directly on GPU!
             config.max_seq_len = 4096 # (input + output) tokens can now be up to 4096
             print("===> loading model")

                 path,
                 trust_remote_code=True
             )
+            # config.attn_config['attn_impl'] = 'triton'
+            # config.init_device = 'cuda:0' # For fast initialization directly on GPU!
             config.max_seq_len = 4096 # (input + output) tokens can now be up to 4096
             print("===> loading model")