README.md

---
license: apache-2.0
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

**slim-sentiment** is part of the SLIM ("Structured Language Instruction Model") model series, providing a set of small, specialized decoder-based LLMs, fine-tuned for function-calling.  

slim-sentiment has been fine-tuned for **sentiment analysis** function calls, generating output consisting of JSON dictionary corresponding to specified keys.  

Each slim model has a corresponding 'tool' in a separate repository, e.g.,  

[**'slim-sentiment-tool'**](https://huggingface.co/llmware/slim-sentiment-tool), which a 4-bit quantized gguf version of the model that is intended to be used for inference.  

Inference speed and loading time is much faster with the 'tool' versions of the model.   

### Model Description

<!-- Provide a longer summary of what this model is. -->

- **Developed by:** llmware
- **Model type:** Small, specialized LLM
- **Language(s) (NLP):** English
- **License:** Apache 2.0
- **Finetuned from model:** Tiny Llama 1B

## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

The intended use of SLIM models is to re-imagine traditional 'hard-coded' classifiers through the use of function calls.  

Example:

    text = "The stock market declined yesterday as investors worried increasingly about the slowing economy."   

    model generation - {"sentiment": ["negative"]}

    keys = "sentiment"

All of the SLIM models use a novel prompt instruction structured as follows:

    "<human> " + text + "<classify> " + keys + "</classify>" + "/n<bot>: "


## How to Get Started with the Model

The fastest way to get started with BLING is through direct import in transformers:

    import ast
    from transformers import AutoModelForCausalLM, AutoTokenizer
    
    model = AutoModelForCausalLM.from_pretrained("llmware/slim-sentiment")
    tokenizer = AutoTokenizer.from_pretrained("llmware/slim-sentiment")

    text = "The markets declined for a second straight days on news of disappointing earnings."

    keys = "sentiment"

    prompt = "<human>: " + text + "\n" + "<classify> " + keys + "</classify>" + "\n<bot>: "

    #   huggingface standard generation script
    inputs = tokenizer(prompt, return_tensors="pt")
    start_of_output = len(inputs.input_ids[0])

    outputs = model.generate(inputs.input_ids.to('cpu'), eos_token_id=tokenizer.eos_token_id,
                             pad_token_id=tokenizer.eos_token_id, do_sample=True, temperature=0.3, max_new_tokens=100)

    output_only = tokenizer.decode(outputs[0][start_of_output:], skip_special_tokens=True)

    print("input text sample - ", text)
    print("llm_response - ", output_only)

    #   where it gets interesting
    try:
        # convert llm response output from string to json
        output_only = ast.literal_eval(output_only)
        print("converted to json automatically")

        #   look for the key passed in the prompt as a dictionary entry
        if keys in output_only:
            if "negative" in output_only[keys]:
                print("sentiment appears negative - need to handle ...")
        else:
            print("response does not appear to include the designated key - will need to try again.")

    except:
        print("could not convert to json automatically - ", output_only)

   
## Using as Function Call in LLMWare

We envision the slim models deployed in a pipeline/workflow/templating framework that handles the prompt packaging more elegantly.  

Check out llmware for one such implementation:  

    from llmware.models import ModelCatalog
    slim_model = ModelCatalog().load_model("llmware/slim-sentiment")
    response = slim_model.function_call(text,params=["sentiment"], function="classify")

    print("llmware - llm_response: ", response)

    
## Model Card Contact

Darren Oberst & llmware team