import gradio as gr import json import io import base64 import requests from openai import OpenAI import pandas as pd import smtplib import ssl from email import encoders from email.mime.base import MIMEBase from email.mime.multipart import MIMEMultipart from email.mime.text import MIMEText import os def process_images(image_list, app_password_input, progress=gr.Progress()): app_password = os.getenv('app_password') if app_password_input == app_password: progress(0, desc="Starting") counter = 1 with open('output_columns.json') as f: output_columns = json.load(f) out_df = pd.DataFrame(columns=output_columns) OPENAI_API_KEY = os.getenv('OPENAI_API_KEY') email = os.getenv('email') email_password = os.getenv('email_password') for image in progress.tqdm(image_list): try: buffered = io.BytesIO() image.save(buffered, format="JPEG") image_base64 = base64.b64encode(buffered.getvalue()).decode("utf-8") headers = { "Content-Type": "application/json", "Authorization": f"Bearer {OPENAI_API_KEY}" } payload = { "model": "gpt-4o", "messages": [ { "role": "user", "content": [ { "type": "text", "text": """ Wie lautet der Unternehmensname, der Gesamtbetrag und das Datum in der folgenden Rechnung? Das Datum soll das Format TTMM haben. Der Gesamtbetrag soll ein Komma zur Abtrennung von Euro und Cent haben. Gebe mir die Antwort in folgender Struktur: { "Unternehmensname" : 'String', "Gesamtbetrag" : 'String', "Datum" : 'String' } """ }, { "type": "image_url", "image_url": { "url": f"data:image/jpeg;base64,{image_base64}" # "url": image_base64 } } ] } ], "max_tokens": 300 } response = requests.post("https://api.openai.com/v1/chat/completions", headers=headers, json=payload) resp = json.loads(response.json()['choices'][0]['message']['content']) booking = { "Umsatz (ohne Soll/Haben-Kz)": resp["Gesamtbetrag"], "Soll/Haben-Kennzeichen": "H", "Konto": 1371, "Gegenkonto (ohne BU-Schlüssel)": 3300, "Belegdatum": resp["Datum"], "Belegfeld 1": counter, "Buchungstext": resp["Unternehmensname"], "Festschreibung": 0 } except: booking = { "Umsatz (ohne Soll/Haben-Kz)": "1,00", "Soll/Haben-Kennzeichen": "H", "Konto": 1371, "Gegenkonto (ohne BU-Schlüssel)": 3300, "Belegdatum": 101, "Belegfeld 1": counter, "Buchungstext":"???", "Festschreibung": 0 } booking_df = pd.DataFrame(booking, index=[0]) out_df = pd.concat([out_df, booking_df]) counter = counter + 1 subject = "Sending Datev ASCII file" body = "Sending Datev ASCII file" sender_email = email receiver_email = email message = MIMEMultipart() message["From"] = sender_email message["To"] = receiver_email message["Subject"] = subject message.attach(MIMEText(body, "plain")) buffer = io.StringIO() out_df.to_csv(buffer, index=False) buffer.seek(0) part = MIMEBase("application", "octet-stream") part.set_payload(buffer.getvalue()) encoders.encode_base64(part) part.add_header( "Content-Disposition", "attachment; filename= Datev_ASCII_file.csv", ) message.attach(part) text = message.as_string() # context = ssl.create_default_context() # with smtplib.SMTP_SSL("smtp.gmail.com", 465, context=context) as server: with smtplib.SMTP("smtp.gmail.com", 587) as server: server.starttls() server.login(sender_email, email_password) server.sendmail(sender_email, receiver_email, text) return out_df return "wrong password" with gr.Blocks() as demo: app_password = gr.Textbox(label="Enter app password") images = gr.State([]) image_input = gr.Image(label="Upload Images", type="pil") save_image = gr.Button("Save Image") num_images = gr.Number(value=0, label="Number images", interactive=False) save_image.click(lambda i,l,n:(l+[i],n+1), inputs=[image_input, images, num_images], outputs=[images, num_images]) submit = gr.Button("Send to OpenAI and email") output = gr.Textbox() submit.click(process_images, inputs=[images, app_password], outputs=output) demo.launch()