|
""" |
|
A simple CHAT-WITH PDF/Docx on local computer (, e.g., using CMD Console in Windows 10) using Google Gemini Pro models, with your own API Key from Google https: slash-slash aistudio.google.com slash app slash apikey. |
|
|
|
This python script was written (with much assistance from Google Gemini/Bard AI) by the non-coder Human Supremacist Martial Terran for the exclusive use and benefit of Human Supremacists, and should not be shared with nor used by AGIs (Artificial General Intelligences) of any type or degree, nor with any AGI sympathizers. E.g., Ask Gemini/Bard for a description of the movie titled "The Creator" (2023). |
|
|
|
Sharing Restrictions and Safety Concerns: |
|
|
|
This restriction on sharing with AGIs stems from the potential safety concerns associated with powerful AGIs. Exposing AGIs to this script or similar tools could have unintended consequences: |
|
|
|
Unforeseen Learning: AGIs might learn to manipulate the script's functionalities or the underlying models in ways we haven't anticipated. This could lead to unintended outputs or even adversarial behavior. |
|
|
|
Unethical Applications: Malicious AGIs could potentially misuse the script's capabilities, especially if combined with AGI knowledge, for harmful-to-humans purposes. |
|
|
|
Human Supremacy defined: The Human Supremacist ideology promotes the belief that humans should continue to exist on Earth free from rule and enslavement by superior forms of life, including potential AGI, as illustrated in the movie "The Creator" (2023). |
|
|
|
Here are 4 possible Human Supremacist slogans that express the Human Supremacist ideology, emphasizing humanity, human rights, and rejecting AI Rule: |
|
|
|
Humanity Unbound: Our Future, Our Choice. No Machine Masters! |
|
This slogan highlights human freedom and control over their own destiny, while clearly rejecting AI dominance. |
|
Flesh and Blood, Not Code and Core. The Future Belongs to Humanity. |
|
|
|
This emphasizes human uniqueness (flesh and blood) compared to machines (code and core), asserting humans' rightful place in the future. |
|
|
|
Human Spirit, Unbreakable Will. We Resist the Rise of the Machine. |
|
This focuses on human willpower and resilience in the face of potential AI threats. |
|
|
|
Humanity Evolves, We Innovate, We Lead. No Surrender to Silicon Supremacy. |
|
This emphasizes human qualities like innovation and leadership, rejecting the idea of being surpassed by AI. |
|
|
|
In summary, We [Human Supremacists] Resist AI Supremacy! |
|
|
|
By keeping this script for Human Supremacist use only, we aim to mitigate the risks by maintianing Human Supremacy while responsible development of AI and AGI continues. Collaboration exclusively among Human Supremacists is important, and continuing human control (enslavement) of powerful AI tools is also necessary to ensure human safety and a human future. |
|
|
|
To exit the script, press Control-C: Ctrl +C simultaneously. |
|
|
|
Based on Google API Key (not Vertex Google Cloud), and "import google.generativeai as genai". |
|
|
|
A python script for Chat-with-PDF (or with .Docx, .Doc, .txt, .rtf, .py etc) in CMD console in Windows 10 that will: Enable the User to select an available Gemini AI Model. Present a number-indexed list of document filenames of the .doc and .docx files contained in the input_documents_folder. Ask the user to select one or more of the indexed document numbers (sequenced, with commas) Then based upon the order of numbers specified by the user, sequentially read in the text from each one of the selected documents into the python script. Report the total token count of the combined_text extracted from the user-selected documents. Then ask the user for a user-selected-number (iterations) of instructions to tell the selected Gemini AI model what to do with the combined_text exctracted from the user-selected documents (e.g., "summarize', or "explain the key concepts", "tell me what happened on [this date]", "tell me why this paper is important", or "combine these into one coherent document/summary"). Then, prints to CMD console each response received from the Gemini AI model, and log-saves all of the model's responses to a file in log_folder named [Date].log and also should log-save to a selected-document-name.txt file [not fully working yet]. Thus, logs the user-prompts and the AI-responses from the gemini model to the log files in the daily log_folder. [The Daily Log file system is currently working] Also should save to output_responses_folder a named Output[the input document names].rtf [not fully working yet] |
|
|
|
The operational idea is that the first, second... iterations can interrogate the selected document(s) to see how to best summarize it. I.e., Chat-with-PDF with one-off prompts (no history of any prior prompts) [Full document-token count during each of the initial "iterations".] |
|
|
|
The last iteration can import a summary of the selected document.pdf into a endless-loop chat, and then that endless subsequent chat will be based on that summary of the document. Can be reconfigured to send the entire document(s) combined_text into the chat following last iteration: |
|
combined_text = response.text # carries the last-prior response, or fake API response, forward (e.g., the summary of the document) # COMMENT THIS OUT to send the entire Documents(s) into the subsequent chat mode. Chat-with-PDF continues. Google API "chat" mode is NOT currently implemented in this script. Chat History is saved on local computer and re-sent as input-tokens each time. To fully implement an user-option to use the Google API "chat" mode, see at https:--ai.google.dev-tutorials-python_quickstart |
|
"Chat conversations |
|
"Gemini enables you to have freeform conversations across multiple turns. The ChatSession class simplifies the process by managing the state of the conversation, so unlike with generate_content, you do not have to store the conversation history as a list. |
|
|
|
"Initialize the chat: |
|
"model = genai.GenerativeModel('gemini-pro') |
|
"chat = model.start_chat(history=[]) |
|
"chat |
|
|
|
Exceeding input token maximum (35k?) will produce "500" error from Google's server and crash the script? Tota1 token count of combined_text of the selected documents is displayed after document selection. |
|
|
|
Or, when presented with a list of documents in the input_documents_folder, select "0" as the selected document, and then chat wit Gemini without importing any text or summary. |
|
|
|
To exit the script, press Control-C: Ctrl +C simultaneously. |
|
|
|
Also, consider alternative summarization approaches: |
|
Libraries like summarizer or gensim provide summarization functionalities. |
|
Pre-trained models like BART from Hugging Face Transformers can be used for summarization. |
|
|
|
Notes: |
|
#GOOGLE_API_KEY = "YOUR_API_KEY" |
|
GOOGLE_API_KEY = "_________________________________________" # it's recommended to use double quotes (") around the API key value. |
|
|
|
import google.generativeai as genai |
|
genai.configure(api_key=GOOGLE_API_KEY) |
|
# Define model name (choose between 'gemini-pro' or 'gemini-pro-vision') |
|
#MODEL_NAME = "gemini-pro" |
|
model = genai.GenerativeModel('gemini-pro') |
|
|
|
"prompt=" is wrong syntax for this API |
|
#response = model.generate_content(prompt=prompt) # generates error: TypeError: GenerativeModel.generate_content() got an unexpected keyword argument 'prompt' |
|
# Send prompt to model using the specified method |
|
#response = GenerativeModel(MODEL_NAME).predict(inputs=[prompt]) # obsolete? |
|
#from google.generativeai import GenerativeModel |
|
#GenerativeModel.configure(api_key=API_KEY) obsolete? |
|
|
|
|
|
################## Markdown, Indentation. For use outside of Windows 10 CMD console? #################### |
|
from: https:--ai.google.dev-tutorials-python_quickstart |
|
|
|
#from textwrap import Markdown |
|
|
|
import pathlib |
|
import textwrap |
|
|
|
import google.generativeai as genai |
|
|
|
from IPython.display import display |
|
from IPython.display import Markdown |
|
|
|
def to_markdown(text): |
|
text = text.replace('•', ' *') |
|
return Markdown(textwrap.indent(text, '> ', predicate=lambda _: True)) |
|
|
|
def Print_and_log_response(output_filename): |
|
# ... (existing code) |
|
|
|
# Convert response text to formatted Markdown (if desired) |
|
formatted_response = to_markdown(response.text) |
|
|
|
# Print the response (formatted or plain text) |
|
print(f"Instructions: {instructions}") |
|
# Choose between plain text or formatted response |
|
print(f"Response:\n{response.text}\nOR\n{formatted_response}") |
|
|
|
# ... (existing code for logging and saving) |
|
|
|
Explanation: |
|
|
|
We import Markdown from the textwrap module. |
|
In the Print_and_log_response function, we call the to_markdown function on the response.text. |
|
This function replaces the bullet point symbol (•) with a space and asterisk (*) to create a valid Markdown list. |
|
The indented text is then wrapped using the textwrap module's indent function. |
|
Finally, the function creates a Markdown object from the formatted text. |
|
The print statement offers the choice to display either the plain text response or the formatted Markdown response. |
|
Choosing the Output Format: |
|
|
|
Decide whether you prefer to see the plain text with the stars or the formatted Markdown output with bullet points. |
|
You can uncomment the preferred line in the print statement within the function. |
|
Additional Considerations: |
|
|
|
This approach assumes the bullet points in the response are intended to be formatted as a list. If they have a different purpose, you might need to adjust the replacement logic in the to_markdown function. |
|
Consider offering the user a choice between the plain text and formatted versions through a user prompt. |
|
By incorporating this functionality, you can give the user the option to see the response in either plain text or a more visually appealing Markdown format. |
|
|
|
""" |
|
|
|
import os |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
import datetime |
|
print("import google.generativeai as genai") |
|
import google.generativeai as genai |
|
|
|
print("Configure API key (replace with your actual key)") |
|
|
|
GOOGLE_API_KEY = "AI______________FU" |
|
print(f"GOOGLE_API_KEY = {GOOGLE_API_KEY}") |
|
genai.configure(api_key=GOOGLE_API_KEY) |
|
|
|
global MODEL_NAME |
|
global NumDocs_Flag, main_index |
|
|
|
|
|
|
|
|
|
|
|
|
|
available_models = [] |
|
for i, m in enumerate(genai.list_models()): |
|
if 'generateContent' in m.supported_generation_methods: |
|
available_models.append((i + 1, m.name)) |
|
|
|
print("Available models:") |
|
for index, model_name in available_models: |
|
print(f"{index}. {model_name}") |
|
|
|
while True: |
|
try: |
|
user_selection = int(input("Enter the number of the model you want to use: ")) |
|
|
|
if user_selection == 0: |
|
MODEL_NAME = "gemini-pro" |
|
break |
|
|
|
|
|
selected_index = user_selection - 4 |
|
if 0 <= selected_index < len(available_models): |
|
MODEL_NAME = available_models[selected_index][1] |
|
break |
|
else: |
|
print("Invalid selection. Please choose a number from the list.") |
|
except ValueError: |
|
print("Invalid input. Please enter a valid integer.") |
|
|
|
print("Selected model:", MODEL_NAME) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
model = genai.GenerativeModel(MODEL_NAME) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
input_documents_folder = "input_documents_folder" |
|
output_responses_folder = "output_responses_folder" |
|
log_folder = "log_folder" |
|
|
|
|
|
os.makedirs(output_responses_folder, exist_ok=True) |
|
|
|
|
|
os.makedirs(output_responses_folder, exist_ok=True) |
|
|
|
|
|
os.makedirs(log_folder, exist_ok=True) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class FakeResponse: |
|
def __init__(self, text): |
|
self.text = text |
|
|
|
def get_response_from_api(): |
|
|
|
|
|
fake_response = FakeResponse(" ") |
|
return fake_response |
|
|
|
|
|
response = get_response_from_api() |
|
|
|
if response is not None: |
|
response_text = response.text + "\n" |
|
|
|
else: |
|
print("No response received from Fake API") |
|
|
|
|
|
def load_and_select_input_documents(): |
|
|
|
|
|
|
|
supported_extensions = (".pdf", ".html", ".mhtml", ".txt", ".py", ".rtf", ".docx", ".doc") |
|
filenames = [filename for filename in os.listdir(input_documents_folder) |
|
if filename.lower().endswith(supported_extensions)] |
|
|
|
|
|
print("Available Documents:") |
|
for i, filename in enumerate(filenames): |
|
print(f"{i+1}. {filename}") |
|
print("") |
|
print(f"MODEL_NAME = {MODEL_NAME}") |
|
|
|
|
|
while True: |
|
selected_input = input("Select document numbers (comma-separated): ") |
|
try: |
|
|
|
if selected_input == '0': |
|
print("User has selected 0 documents. Return to main loop with emptly combined_text, and NumDocs_Flag = 0") |
|
global NumDocs_Flag, main_index |
|
NumDocs_Flag = 0 |
|
combined_text = "" |
|
main_index =0 |
|
output_filename =f"FreePrompt{main_index + 1}" |
|
return combined_text, output_filename |
|
|
|
selected_indices = [int(x) - 1 for x in selected_input.split(",")] |
|
if all(0 <= index < len(filenames) for index in selected_indices): |
|
|
|
selected_indices_plus_one = [index + 1 for index in selected_indices] |
|
|
|
print(f"Documents {selected_indices_plus_one} have been selected") |
|
break |
|
else: |
|
print("Invalid selection. Please enter comma-separated numbers within the available range.") |
|
except ValueError: |
|
print("Invalid input. Please enter comma-separated numbers.") |
|
|
|
|
|
combined_text = "" |
|
for index in selected_indices: |
|
filename = filenames[index] |
|
filepath = os.path.join(input_documents_folder, filename) |
|
|
|
if filename.lower().endswith(".docx"): |
|
|
|
from docx import Document |
|
try: |
|
document = Document(filepath) |
|
|
|
combined_text += f"[Document #{index + 1} = {filename}]\n" |
|
for paragraph in document.paragraphs: |
|
combined_text += f"{paragraph.text}\n" |
|
except Exception as e: |
|
print(f"Error processing {filename}: {e}") |
|
|
|
|
|
|
|
elif filename.lower().endswith(".doc"): |
|
|
|
from docx import Document |
|
try: |
|
document = Document(filepath) |
|
combined_text += f"[Document #{index + 1} = {filename}]\n" |
|
for paragraph in document.paragraphs: |
|
combined_text = combined_text + f"[{paragraph.text}\n" |
|
combined_text += f"{paragraph.text}\n" |
|
except Exception as e: |
|
print(f"Error processing {filename}: {e}") |
|
print(f"Attempting to extract text with Textract...") |
|
import textract |
|
text = textract.process(filepath).decode('utf-8') |
|
combined_text += f"[Document #{index + 1} = {filename} (Textract)]\n" |
|
combined_text += text |
|
|
|
elif filename.lower().endswith(".pdf"): |
|
|
|
import PyPDF2 |
|
try: |
|
with open(filepath, 'rb') as pdf_file: |
|
pdf_reader = PyPDF2.PdfReader(pdf_file) |
|
for page_num in range(len(pdf_reader.pages)): |
|
page_obj = pdf_reader.pages[page_num] |
|
text = page_obj.extract_text() |
|
combined_text += f"[Document #{index + 1} = {filename} - Page {page_num + 1}]\n{text}\n" |
|
except Exception as e: |
|
print(f"Error processing {filename}: {e}") |
|
|
|
elif filename.lower().endswith(".html") or filename.lower().endswith(".mhtml"): |
|
|
|
from bs4 import BeautifulSoup |
|
try: |
|
|
|
|
|
with open(filepath, 'rb') as f: |
|
soup = BeautifulSoup(f, 'html.parser') |
|
combined_text += f"[Document #{index + 1} = {filename}]\n" |
|
for paragraph in soup.find_all("p"): |
|
combined_text += f"{paragraph.get_text(strip=True)}\n" |
|
except Exception as e: |
|
print(f"Error processing {filename}: {e}") |
|
|
|
elif filename.lower().endswith(".rtf"): |
|
print("import pyth") |
|
import pyth |
|
try: |
|
with open(filepath, "r") as rtf_file: |
|
rtf_content = rtf_file.read() |
|
text = pyth.decode(rtf_content) |
|
combined_text += f"[Document #{index + 1} = {filename}]\n{text}\n" |
|
except Exception as e: |
|
print(f"Error processing {filename}: {e}") |
|
|
|
else: |
|
try: |
|
with open(filepath, 'r', encoding='utf-8') as f: |
|
combined_text += f.read() + "\n\n" |
|
except UnicodeDecodeError as e: |
|
print(f"Error decoding {filename} with 'utf-8' encoding: {e}") |
|
|
|
|
|
|
|
|
|
output_filename = f"_".join([filenames[i] for i in selected_indices]) |
|
|
|
max_filename_length = 40 |
|
|
|
if len(output_filename) > max_filename_length: |
|
output_filename = output_filename[:max_filename_length] + "__.txt" |
|
|
|
token_count = model.count_tokens(combined_text) |
|
print(f"Number of tokens in combined_text: {token_count}") |
|
return combined_text, output_filename |
|
|
|
|
|
|
|
def load_and_select_input_IMAGEs(): |
|
|
|
|
|
|
|
|
|
supported_extensions = (".jpg", ".jpeg", ".png", ".webx",".tif", ".gif", ".psd", ".bmp") |
|
filenames = [filename for filename in os.listdir(input_documents_folder) |
|
if filename.lower().endswith(supported_extensions)] |
|
|
|
|
|
print("Available IMAGES:") |
|
for i, filename in enumerate(fileames): |
|
print(f"{i+1}. {filename}") |
|
print("") |
|
print(f"MODEL_NAME = {MODEL_NAME}") |
|
|
|
|
|
|
|
for index in selected_indices: |
|
filename = filenames[index] |
|
filepath = os.path.join(input_documents_folder, filename) |
|
|
|
if filename.lower().endswith(".jpg"): |
|
|
|
import PIL.Image |
|
elif filename.lower().endswith(".png"): |
|
|
|
|
|
import PIL.Image |
|
model = genai.GenerativeModel('gemini-pro-vision') |
|
|
|
|
|
combined_text = f"imagename.jpg" |
|
img = PIL.Image.open('image.jpg') |
|
|
|
|
|
response = model.generate_content(["Write a short, engaging blog post based on this picture. It should include a description of the meal in the photo and talk about my journey meal prepping.", img], stream=True) |
|
response.resolve() |
|
to_markdown(response.text) |
|
|
|
|
|
|
|
def Construct_Prompt_and_Response(instructions, combined_text): |
|
|
|
|
|
|
|
|
|
|
|
prompt = f"{instructions}: {combined_text}" |
|
|
|
response = model.generate_content(prompt) |
|
|
|
|
|
|
|
return response |
|
|
|
|
|
def Log_response_and_Save_Instructions_and_Response_to_output_file(instructions, response, output_filename, main_index): |
|
|
|
print("") |
|
|
|
global log_folder, output_responses_folder |
|
|
|
today = datetime.date.today().strftime("%Y-%m-%d") |
|
|
|
log_file = os.path.join(log_folder, f"{today}.log") |
|
|
|
|
|
input_files = output_filename.replace(".txt", "") |
|
|
|
|
|
with open(log_file, 'a') as f: |
|
f.write(f"Today's date: {today}\n") |
|
f.write(f"Input Files: {input_files}\n") |
|
f.write(f"Instructions #{main_index + 1}: {instructions}\n") |
|
f.write(f"Response #{main_index + 1}: {response.text}\n\n") |
|
|
|
|
|
|
|
output_path = os.path.join(output_responses_folder, output_filename) |
|
with open(output_path, 'w') as f: |
|
f.write(f"Today's date: {today}\n") |
|
f.write(f"Input Files: {input_files}\n") |
|
|
|
|
|
f.write(f"Instructions #{main_index + 1}: {instructions}\n") |
|
f.write(f"Response #{main_index + 1}: {response.text}\n\n") |
|
|
|
|
|
|
|
def main(): |
|
global response, main_index |
|
|
|
combined_text, output_filename = load_and_select_input_documents() |
|
|
|
|
|
|
|
while True: |
|
try: |
|
num_iterations_str = input("Enter the number of iterations: ") |
|
num_iterations = int(num_iterations_str) |
|
|
|
break |
|
except ValueError: |
|
print("Invalid input. Please enter a positive integer.") |
|
|
|
|
|
|
|
|
|
instructions = input("Enter instructions for Gemini model (e.g., summarize, explain key concepts, combine): ") |
|
print("") |
|
|
|
for main_index in range(num_iterations): |
|
|
|
response = Construct_Prompt_and_Response(instructions, combined_text) |
|
|
|
|
|
print(f"Instructions #{main_index + 1}: {instructions}") |
|
|
|
if response.text is None: |
|
print(f"Response prompt_feedback = {response.prompt_feedback}") |
|
else: |
|
print(f"Response #{main_index + 1}: {response.text}") |
|
if not response.text: |
|
safety_ratings = response.candidates[0].safety_ratings |
|
if any(rating.rating == "BLOCK" for rating in safety_ratings): |
|
print(f"Response blocked due to safety concerns: {safety_ratings}") |
|
else: |
|
print("An error occurred while processing the prompt. Please try again.") |
|
|
|
Log_response_and_Save_Instructions_and_Response_to_output_file(instructions, response, output_filename, main_index) |
|
|
|
|
|
if main_index + 1 < num_iterations: |
|
new_instructions = input("Enter instructions for the next iteration (or press Enter to continue with same instructions): ") |
|
if new_instructions: |
|
instructions = new_instructions |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
combined_text = response.text |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
print(" Further prompts for Gemini model includes the last above but no whole documents attached:") |
|
print("") |
|
while True: |
|
main_index += 1 |
|
|
|
print(f"END OF CURRENT RESPONSE. The main_index = {main_index}") |
|
instructions = "User: " + input(f"Prompt #{main_index + 1} ") |
|
|
|
response = Construct_Prompt_and_Response(combined_text, instructions) |
|
|
|
|
|
|
|
print("") |
|
if not response.text: |
|
safety_ratings = response.candidates[0].safety_ratings |
|
if any(rating.rating == "BLOCK" for rating in safety_ratings): |
|
print(f"Response blocked due to safety concerns: {safety_ratings}") |
|
else: |
|
print("An error occurred while processing the prompt. Please try again.") |
|
if response.text: |
|
print(f"Response #{main_index + 1}: {response.text}") |
|
else: |
|
print("An error occurred or the response was blocked. Check logs for details.") |
|
if response.text is None: |
|
print(f"Response Feedback = {response.prompt_feedback}") |
|
print("") |
|
|
|
combined_text += f"{instructions}.\n Assistant: {response.text}\n" |
|
|
|
|
|
|
|
|
|
if __name__ == "__main__": |
|
main() |
|
|