vicky4s4s commited on
Commit
507fa3a
1 Parent(s): 9ca31dc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +38 -0
README.md CHANGED
@@ -36,6 +36,44 @@ tags:
36
  - Model creator: [Mistral AI_](https://huggingface.co/mistralai)
37
  - Original model: [Mistral 7B Instruct v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2)
38
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
39
  <!-- description start -->
40
  ## Description
41
 
 
36
  - Model creator: [Mistral AI_](https://huggingface.co/mistralai)
37
  - Original model: [Mistral 7B Instruct v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2)
38
 
39
+ ## Use this code
40
+
41
+ ```py
42
+ # pip install ctransformers langchain langchain-community
43
+
44
+ import os
45
+ import torch
46
+ import warnings
47
+ warnings.filterwarnings("ignore")
48
+
49
+ from langchain.llms import CTransformers
50
+ from langchain.chains import LLMChain
51
+ from langchain.prompts import PromptTemplate
52
+ from langchain.callbacks.base import BaseCallbackHandler
53
+ from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
54
+
55
+ config = {'max_new_tokens': 600,
56
+ 'repetition_penalty': 1.1,
57
+ 'context_length': 9000}
58
+
59
+ model_path = "vicky4s4s/Mistral-7B-Instruct-v0.2-GGUF" # git clone url link
60
+ llm = CTransformers(
61
+ model=model_path,
62
+ config=config,
63
+ model_type='llama',
64
+ lib='avx',
65
+ callbacks=[StreamingStdOutCallbackHandler()]
66
+ )
67
+ template = """
68
+ You are an intelligent chatbot. Help the following question with brilliant answers.
69
+ Question: {question}
70
+ Answer:"""
71
+ prompt = PromptTemplate(template=template, input_variables=["question"])
72
+ llm_chain = LLMChain(prompt=prompt, llm=llm)
73
+ response = llm_chain.run("what is python")
74
+ output = {"output_response": response}
75
+ print(output)
76
+
77
  <!-- description start -->
78
  ## Description
79