Hosted Inference
#15
by
synthetisoft
- opened
Does the model architecture mean it can't run on a hosted inference endpoint?
You can, but you need ampere architecture (GPU). Otherwise, the output is gibberish.
@synthetisoft We (MosaicML) have a competing inference product: https://www.mosaicml.com/inference
However, I have told our contacts at HuggingFace that there is community interest in inference endpoint examples of MPT and they are working on adding it to the docs
sam-mosaic
changed discussion status to
closed