metadata

license: apache-2.0

Model Card for Model ID

slim-sentiment is part of the SLIM ("Structured Language Instruction Model") model series, providing a set of small, specialized decoder-based LLMs, fine-tuned for function-calling.

slim-sentiment has been fine-tuned for sentiment analysis function calls, generating output consisting of JSON dictionary corresponding to specified keys.

Each slim model has a corresponding 'tool' in a separate repository, e.g.,

'slim-sentiment-tool', which a 4-bit quantized gguf version of the model that is intended to be used for inference.

Inference speed and loading time is much faster with the 'tool' versions of the model.

Model Description

Developed by: llmware
Model type: Small, specialized LLM
Language(s) (NLP): English
License: Apache 2.0
Finetuned from model: Tiny Llama 1B

Uses

The intended use of SLIM models is to re-imagine traditional 'hard-coded' classifiers through the use of function calls.

Example:

text = "The stock market declined yesterday as investors worried increasingly about the slowing economy."   

model generation - {"sentiment": ["negative"]}

keys = "sentiment"

All of the SLIM models use a novel prompt instruction structured as follows:

"<human> " + text + "<classify> " + keys + "</classify>" + "/n<bot>: "

How to Get Started with the Model

The fastest way to get started with BLING is through direct import in transformers:

import ast
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("llmware/slim-sentiment")
tokenizer = AutoTokenizer.from_pretrained("llmware/slim-sentiment")

text = "The markets declined for a second straight days on news of disappointing earnings."

keys = "sentiment"

prompt = "<human>: " + text + "\n" + "<classify> " + keys + "</classify>" + "\n<bot>: "

#   huggingface standard generation script
inputs = tokenizer(prompt, return_tensors="pt")
start_of_output = len(inputs.input_ids[0])

outputs = model.generate(inputs.input_ids.to('cpu'), eos_token_id=tokenizer.eos_token_id,
                         pad_token_id=tokenizer.eos_token_id, do_sample=True, temperature=0.3, max_new_tokens=100)

output_only = tokenizer.decode(outputs[0][start_of_output:], skip_special_tokens=True)

print("input text sample - ", text)
print("llm_response - ", output_only)

#   where it gets interesting
try:
    # convert llm response output from string to json
    output_only = ast.literal_eval(output_only)
    print("converted to json automatically")

    #   look for the key passed in the prompt as a dictionary entry
    if keys in output_only:
        if "negative" in output_only[keys]:
            print("sentiment appears negative - need to handle ...")
    else:
        print("response does not appear to include the designated key - will need to try again.")

except:
    print("could not convert to json automatically - ", output_only)

Using as Function Call in LLMWare

We envision the slim models deployed in a pipeline/workflow/templating framework that handles the prompt packaging more elegantly.

Check out llmware for one such implementation:

from llmware.models import ModelCatalog
slim_model = ModelCatalog().load_model("llmware/slim-sentiment")
response = slim_model.function_call(text,params=["sentiment"], function="classify")

print("llmware - llm_response: ", response)

Model Card Contact

Darren Oberst & llmware team