Upload README.md

README.md

@@ -1,3 +1,111 @@
 ---
 license: apache-2.0
 ---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+**slim-sentiment** is part of the SLIM ("Structured Language Instruction Model") model series, providing a set of small, specialized decoder-based LLMs, fine-tuned for function-calling.  
+
+slim-sentiment has been fine-tuned for **sentiment analysis** function calls, generating output consisting of JSON dictionary corresponding to specified keys.  
+
+Each slim model has a corresponding 'tool' in a separate repository, e.g.,  
+
+[**'slim-sentiment-tool'**](https://huggingface.co/llmware/slim-sentiment-tool), which a 4-bit quantized gguf version of the model that is intended to be used for inference.  
+
+Inference speed and loading time is much faster with the 'tool' versions of the model.   
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+- **Developed by:** llmware
+- **Model type:** Small, specialized LLM
+- **Language(s) (NLP):** English
+- **License:** Apache 2.0
+- **Finetuned from model:** Tiny Llama 1B
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+The intended use of SLIM models is to re-imagine traditional 'hard-coded' classifiers through the use of function calls.  
+
+Example:
+
+    text = "The stock market declined yesterday as investors worried increasingly about the slowing economy."   
+
+    model generation - {"sentiment": ["negative"]}
+
+    keys = "sentiment"
+
+All of the SLIM models use a novel prompt instruction structured as follows:
+
+    "<human> " + text + "<classify> " + keys + "</classify>" + "/n<bot>: "
+
+
+## How to Get Started with the Model
+
+The fastest way to get started with BLING is through direct import in transformers:
+
+    import ast
+    from transformers import AutoModelForCausalLM, AutoTokenizer
+    
+    model = AutoModelForCausalLM.from_pretrained("llmware/slim-sentiment")
+    tokenizer = AutoTokenizer.from_pretrained("llmware/slim-sentiment")
+
+    text = "The markets declined for a second straight days on news of disappointing earnings."
+
+    keys = "sentiment"
+
+    prompt = "<human>: " + text + "\n" + "<classify> " + keys + "</classify>" + "\n<bot>: "
+
+    #   huggingface standard generation script
+    inputs = tokenizer(prompt, return_tensors="pt")
+    start_of_output = len(inputs.input_ids[0])
+
+    outputs = model.generate(inputs.input_ids.to('cpu'), eos_token_id=tokenizer.eos_token_id,
+                             pad_token_id=tokenizer.eos_token_id, do_sample=True, temperature=0.3, max_new_tokens=100)
+
+    output_only = tokenizer.decode(outputs[0][start_of_output:], skip_special_tokens=True)
+
+    print("input text sample - ", text)
+    print("llm_response - ", output_only)
+
+    #   where it gets interesting
+    try:
+        # convert llm response output from string to json
+        output_only = ast.literal_eval(output_only)
+        print("converted to json automatically")
+
+        #   look for the key passed in the prompt as a dictionary entry
+        if keys in output_only:
+            if "negative" in output_only[keys]:
+                print("sentiment appears negative - need to handle ...")
+        else:
+            print("response does not appear to include the designated key - will need to try again.")
+
+    except:
+        print("could not convert to json automatically - ", output_only)
+
+   
+## Using as Function Call in LLMWare
+
+We envision the slim models deployed in a pipeline/workflow/templating framework that handles the prompt packaging more elegantly.  
+
+Check out llmware for one such implementation:  
+
+    from llmware.models import ModelCatalog
+    slim_model = ModelCatalog().load_model("llmware/slim-sentiment")
+    response = slim_model.function_call(text,params=["sentiment"], function="classify")
+
+    print("llmware - llm_response: ", response)
+
+    
+## Model Card Contact
+
+Darren Oberst & llmware team
+
+
+