nm-testing/granite-7b-lab-pruned50-quant-ds

Install:

pip install deepsparse[llm]==1.7

Usage:

>>> from deepsparse import TextGeneration
>>> model = TextGeneration("nm-testing/granite-7b-lab-pruned50-quant-ds")
>>> model("Hello my name is")
TextGenerationOutput(
  created=datetime.datetime(2024, 5, 1, 17, 41, 13, 176274),
  prompts='Hello my name is',
  generations=[GeneratedText(text='Alex and I am a senior in high school. I am taking a dual credit biology class this semester and I really need help with the lab. I am having trouble with the lab on dissociation and I was hoping that you could help me. The lab requires that we set up a reaction between a strong acid and a strong base and measure the pH of the solution at various points in the reaction.\n\nHere is a summary of the lab that I was given:\n\n1', score=None, finished=True, finished_reason='max_new_tokens')],
  input_tokens=None
)