how to use model with gpt4all

import pandas as pd from datasets import load_dataset import re csv_file = "/Users/admin/Downloads/GPT4_output_GPT-2_training_df.csv"

Load the dataset

dataset = load_dataset("csv", data_files=csv_file, split="train")

def stop_on_token_callback(token_id, token_string): # one sentence is enough: if '' in token_string: return False else: return True

alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

Instruction:

Find out which character said this specific quote along with their gender

Input:

{}

Response:

{}""" num_examples=10 from gpt4all import GPT4All model = GPT4All('/Users/admin/Downloads/model-unsloth.Q4_K_M.gguf') system_template = '''Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.'''

many models use triple hash '###' for keywords, Vicunas are simpler:

prompt_template = """### Instruction: Find out which character said this specific quote along with their gender

Input:

{0}

Response:

""" with model.chat_session(system_template, prompt_template): for i in range(min(num_examples, len(dataset))): row = dataset[i] #response1 = model.generate(row['formatted_input'], callback=stop_on_token_callback) print(i) response1 = model.generate(row['formatted_input']) print(response1) print() print(f"Correct output: {row['TrueSpeaker']}")

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no library tag.