|
--- |
|
license: cc-by-nc-4.0 |
|
language: |
|
- en |
|
pipeline_tag: conversational |
|
tags: |
|
- fair-use |
|
- llama2 |
|
- ggml |
|
--- |
|
|
|
# Llama2 Movie Character Finetuned 7F Quantized |
|
|
|
Quantized by llama.cpp (https://github.com/ggerganov/llama.cpp) |
|
Either use by llama.cpp or llama cpp python wrappers |
|
|
|
with ctransformers: |
|
|
|
``` |
|
#pip install ctransformers |
|
from ctransformers import AutoModelForCausalLM |
|
|
|
llm = AutoModelForCausalLM.from_pretrained("llama2-7f-fp16-ggml-q4.bin", |
|
model_type='llama', |
|
gpu_layers=100, # you can use less (like 20) if you have less gpu ram |
|
max_new_tokens=50, |
|
stop=["###","##"], |
|
threads=4) # 4 cpu threads to limit cpu |
|
``` |
|
|
|
with llama-cpp-python: |
|
``` |
|
|
|
# !CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip install llama-cpp-python |
|
|
|
model_path="llama2-7f-fp16-ggml-q4.bin" |
|
llm = Llama(model_path=model_path,n_gpu_layers=100) |
|
output = llm("### Context: talks friendly### History: ### Morpheus: I challenge you to battle of words!### Gerald:", max_tokens=32, stop=["#","sierpeda"], echo=True) |
|
|
|
``` |
|
|
|
## Prompt format |
|
``` |
|
prompt = f"Context: {overall_context}### History: {previous_dialogue} ### {character_1}: {question}### {character_2}:" |
|
|
|
``` |
|
|
|
|
|
|
|
## Some Fun dialogues |
|
|
|
## Geralt of Witcher vs Ig-11 of Mandalorian |
|
|
|
context= "on a cruise ship going to Mars from Earth" |
|
|
|
#Gerald(Geralt Witcher, do not know why I named him incorrect) asks Ig-11 (Mandalorian) |
|
initial_text = What do you think about AI? |
|
|
|
``` |
|
Gerald : What do you think about AI? |
|
Ig-11 : I've worked with them for more than half of my operational cycle. |
|
Gerald : So, how long has the ship been here? |
|
Ig-11 : 765 years (33 days) since last port. |
|
Gerald : So this is the fourth time since I joined the ship that we've had a murderer on board? |
|
Ig-11 : My master does not believe in coincidences... He believes everything happens for a reason. The universe has a plan, and it includes me being here at the same time as you. So, what do you think about AI? |
|
Gerald : I'm not sure yet... I mean, they work on Earth, and maybe beyond. |
|
Ig-11 : You see? It is all so simple! The humans are going to Mars; we are bringing them with us. |
|
``` |
|
|
|
|
|
## Batman vs Tony Stark |
|
context= "tries to save the world" |
|
|
|
initial_text = "I am richer than you!" |
|
|
|
``` |
|
Batman : I am richer than you! |
|
Tony Stark : My car goes 100mph! |
|
Batman : My car can go at least that fast, and it's not even suped up. everybody in the car. It didn't even have a license plate! It was so loud I couldn't hear myself think. It felt like we were gonna get killed any second... |
|
Tony Stark : You need to chill out, Bruce. You can be such a bitch sometimes.~Tony Stark |
|
Batman : You don't get it do you? It's not about justice. It's about power.~Bruce Wayne |
|
Tony Stark : You know the one thing that really gets under my skin? When someone shoots himself in the foot so bad, he ends up putting on a hobble~Tony Stark |
|
Batman : "Let me tell you something. Something I've learned to my great cost..."~Batman |
|
Tony Stark : "You make it sound like a disease." |
|
``` |
|
|
|
## Darth Vader vs James T. Kirk |
|
|
|
``` |
|
Vader : I challenge you to battle of words! |
|
Kirk : What are your terms? |
|
Vader : The Emperor's terms! |
|
Kirk : I challenge you to the ultimate game, a verbal battle of wit and wile. surely you know that no mere mortal can stand against my brilliance. |
|
Vader : I accept your challenge. The terms I propose is, the winner shall have the pleasure of killing the loser! |
|
Kirk : Deal! But you will be disappointed. Though my words are powerful and sharp, they lack the strength to kill. |
|
Vader : You speak truthfully. My vocabulary lacks power. It is but a fraction of what it was 10 years ago at the height of my powers. |
|
Kirk : I've never heard you speak like this before. |
|
``` |
