Myashka/SO-Python_QA-filtered-2023-tanh_score-after_2023_02 huggingface dataset:
Title: Running alembic command cause ImportError: cannot import name '_BindParamClause' from 'sqlalchemy.sql.expression'
Tags: python,sqlalchemy,alembic

Question: This happens whenever I ran any alembic command. I am using sqlalchemy version 2.0.3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ado/anaconda3/lib/python3.8/site-packages/alembic/__init__.py", line 8, in <module>
    from . import op  # noqa
  File "/home/ado/anaconda3/lib/python3.8/site-packages/alembic/op.py", line 1, in <module>
    from .operations.base import Operations
  File "/home/ado/anaconda3/lib/python3.8/site-packages/alembic/operations/__init__.py", line 1, in <module>
    from .base import Operations, BatchOperations
  File "/home/ado/anaconda3/lib/python3.8/site-packages/alembic/operations/base.py", line 3, in <module>
    from .. import util
  File "/home/ado/anaconda3/lib/python3.8/site-packages/alembic/util/__init__.py", line 9, in <module>
    from .sqla_compat import (  # noqa
  File "/home/ado/anaconda3/lib/python3.8/site-packages/alembic/util/sqla_compat.py", line 8, in <module>
    from sqlalchemy.sql.expression import _BindParamClause
ImportError: cannot import name '_BindParamClause' from 'sqlalchemy.sql.expression' (/home/***/anaconda3/lib/python3.8/site-packages/sqlalchemy/sql/expression.py)

Answer: Solved after uninstalling alembic and reinstalling it afresh
I ran:
pip3 uninstall alembic
pip3 install alembic
Title: Single quotes around types in Python variable declarations
Tags: python

Question: Python accepts this statement:
stop:bool = False

I have been told this is better:
stop:'bool' = False

Does it matter? Is there a kind of beautifier that will make my code use a single consistent style?

Answer: It's not that the quoted style is better, but that it's necessary if you need to use a type that hasn't been defined yet.
Type hints have to be valid expressions at runtime, which means any identifier has to be defined before it can be used as a hint. If that's not possible you can supply a string that contains the name of the type.
One option for a consistent style that doesn't require everything to be quoted is to use from __future__ import annotations, which treats all type hints as strings implicitly. With that, you needn't quote anything.
Title: How do i get the name of the author who is using a modal?
Tags: python,discord,discord.py

Question: I'm making a password protected notes command for my bot where the user can enter their password and a note and it gets stored in a txt file
Im trying to get the display_name of the user who was in the modal.
Despite looking everywhere I've been unsuccessful.
Any help would be appreciated.
This is my Code
@Client.tree.command(name="add", description="Adds a new note")
async def add(interaction: discord.Interaction):
    
    file_name = f"{interaction.user.name}.txt"
    if not os.path.isfile(file_name):
        await interaction.response.send_message("You do not have a personal Notes file.")
        return
    

class Add(ui.Modal, title='Add Note'):
    Password = ui.TextInput(label='Enter password')
    Note = ui.TextInput(label='Enter your Note', style=discord.TextStyle.paragraph)
    
    #####I want the name of the user in the curly brackets in the next line#####
    with open(f'{}.txt','r') as file:
      pwd = file.readline()
    print(ui.Member.display_name)
  
    async def on_submit(self, interaction: discord.Interaction):
      if Password != pwd:
        await interaction.response.send_message(f'You entered a wrong password', ephemeral=True)
      else:
        with open(f'{file_name}','a') as file:
          file.write(Note)
        await interaction.response.send_message(f'Note Added', ephemeral=True)

I tried ui.interaction.display_name and several other methods I thought would work to no avail

Answer: You can get the name from the Interaction instance, as you've figured out in the code above. If you want to access it in the Modal, but before submission (e.g., in the __init__), you add it as an __init__ argument & pass it in. A modal is just a class, so basic OOP principles work just fine.
Title: Regex: Match decimal numerals with no digits before decimal point
Tags: python,regex,expression,leading-zero

Question: I am trying to match decimal numerals with no digits before the decimal point in a string using regex. For example, I have the strings:
The dimensions of the object-1 is: 1.4 meters wide, .5 meters long, 5.6 meters high
The dimensions of the object-2 is: .8 meters wide, .11 meters long, 0.6 meters high

I want to capture only the decimal numbers without integer digits and prefix leading zeros to them. So my final desired output will be:
The dimensions of the object-1 is: 1.4 meters wide, 0.5 meters long, 5.6 meters high
The dimensions of the object-2 is: 0.8 meters wide, 0.11 meters long, 0.6 meters high

This is what I have tried so far:
(\d+)?\.(\d+)

This expression is capturing all the decimal numbers such as: 1.4, .5, 5.6, .8, .11, 0.6.
But I need to capture only decimal numbers without integer digits: .5, .8, .11.

Answer: Use a negative lookbehind: (?<!\d)(\.\d+)
Title: opencv error conv_winograd_f63.cpp ubuntu 22 python3.10 cv2 although works on 2nd machine
Tags: python-3.x,opencv,cv2,haar-classifier

Question: I have a development machine working with ubuntu 22, python 3.10 and cv2; when I try to replicate on another machine then I get runtime error:
from section calling age prediciton from haarclassifier and age_net.caffemodel:
line 118 return age_net-forward()
cv2.error: OpenCV(4.7.0-dev)
(note this after building files from source orcv2 4.7.0 or 4.6.0 same result)
/home/art/opencv_build/opencv/modules/dnn/src/layers/cpu_kernels/conv_winograd_f63.cpp:401: error: (-215:Assertion failed) CONV_WINO_IBLOCK == 3 && CONV_WINO_ATOM_F32 == 4 in function 'winofunc_BtXB_8x8_f32'
I have tried various permutations of installing opencv-python or opencv-contrib-python or building and compiling the opencv files from source, always with the same result.  Works on the original machine but always throws this error on the second machine when running the same python code.
I have searched online generally and in stackoverflow and I don't see anyone noting this error.
Anyone know?
Tried to duplicate machine where it is working and various permuations of installing opencv either directly:
pip3 install opencv-python
or
pip3 install opencv-contrib-python
or build the opencv files from source,
which is generally to build the dependencies:
sudo apt install build-essential cmake git pkg-config libgtk-3-dev \    libavcodec-dev libavformat-dev libswscale-dev libv4l-dev \    libxvidcore-dev libx264-dev libjpeg-dev libpng-dev libtiff-dev \    gfortran openexr libatlas-base-dev python3-dev python3-numpy \    libtbb2 libtbb-dev libdc1394-dev


clone the repositories:
    $ mkdir ~/opencv_build && cd ~/opencv_build
    $ git clone https://github.com/opencv/opencv.git
    $ git clone https://github.com/opencv/opencv_contrib.git

make:
    Sudo ~/opencv_build/opencv/cmake -D CMAKE_BUILD_TYPE=RELEASE \  -D WITH_VTK=OFF -D    BUILD_opencv_viz=OFF
  -D CMAKE_INSTALL_PREFIX=/usr/local \    -D INSTALL_C_EXAMPLES=ON \    -D INSTALL_PYTHON_EXAMPLES=ON \    -D OPENCV_GENERATE_PKGCONFIG=ON \    -D OPENCV_EXTRA_MODULES_PATH=~/opencv_build/opencv_contrib/modules \    -D BUILD_EXAMPLES=ON ..

and then install
I check all the python code, file folder setup etc is the same, both running ubuntu 22, amd 64 bit.
Works in the original machine, always throws the error in the second.  The python code correctly captures an image, recognizes the face, crops the image and saves it before encountering the error, so the error is specifically to the haarclassifier and age prediction.
I can't find any documentation or comments on the subject.

Answer: yes I reverted to 4.6.0.66 by
'sudo pip install opencv-contrib-python==4.6.0.66'
to call the specific version and this version works whereas 4.7 does not work so I'm sure they will figure out whatever bug exists in their newest release.
Title: opencv error conv_winograd_f63.cpp ubuntu 22 python3.10 cv2 although works on 2nd machine
Tags: python-3.x,opencv,cv2,haar-classifier

Question: I have a development machine working with ubuntu 22, python 3.10 and cv2; when I try to replicate on another machine then I get runtime error:
from section calling age prediciton from haarclassifier and age_net.caffemodel:
line 118 return age_net-forward()
cv2.error: OpenCV(4.7.0-dev)
(note this after building files from source orcv2 4.7.0 or 4.6.0 same result)
/home/art/opencv_build/opencv/modules/dnn/src/layers/cpu_kernels/conv_winograd_f63.cpp:401: error: (-215:Assertion failed) CONV_WINO_IBLOCK == 3 && CONV_WINO_ATOM_F32 == 4 in function 'winofunc_BtXB_8x8_f32'
I have tried various permutations of installing opencv-python or opencv-contrib-python or building and compiling the opencv files from source, always with the same result.  Works on the original machine but always throws this error on the second machine when running the same python code.
I have searched online generally and in stackoverflow and I don't see anyone noting this error.
Anyone know?
Tried to duplicate machine where it is working and various permuations of installing opencv either directly:
pip3 install opencv-python
or
pip3 install opencv-contrib-python
or build the opencv files from source,
which is generally to build the dependencies:
sudo apt install build-essential cmake git pkg-config libgtk-3-dev \    libavcodec-dev libavformat-dev libswscale-dev libv4l-dev \    libxvidcore-dev libx264-dev libjpeg-dev libpng-dev libtiff-dev \    gfortran openexr libatlas-base-dev python3-dev python3-numpy \    libtbb2 libtbb-dev libdc1394-dev


clone the repositories:
    $ mkdir ~/opencv_build && cd ~/opencv_build
    $ git clone https://github.com/opencv/opencv.git
    $ git clone https://github.com/opencv/opencv_contrib.git

make:
    Sudo ~/opencv_build/opencv/cmake -D CMAKE_BUILD_TYPE=RELEASE \  -D WITH_VTK=OFF -D    BUILD_opencv_viz=OFF
  -D CMAKE_INSTALL_PREFIX=/usr/local \    -D INSTALL_C_EXAMPLES=ON \    -D INSTALL_PYTHON_EXAMPLES=ON \    -D OPENCV_GENERATE_PKGCONFIG=ON \    -D OPENCV_EXTRA_MODULES_PATH=~/opencv_build/opencv_contrib/modules \    -D BUILD_EXAMPLES=ON ..

and then install
I check all the python code, file folder setup etc is the same, both running ubuntu 22, amd 64 bit.
Works in the original machine, always throws the error in the second.  The python code correctly captures an image, recognizes the face, crops the image and saves it before encountering the error, so the error is specifically to the haarclassifier and age prediction.
I can't find any documentation or comments on the subject.

Answer: Today I tried out OpenCV 4.5.5.
Works for me now.
Title: pySerial sending data from Python to arduino
Tags: python,arduino,pyserial

Question: I have a question about sending data using the pySerial Python library.
I'm new to this library and sorry about my bad english.
How can i send the data from python to Arduino continuously? For example, i want to send a string that contain 1 digit (0 or 1), can i put it in while loop in python?
For example:
import serial
import time
import cv2
import mediapipe as mp
from cvzone.HandTrackingModule import HandDetector
ser = serial.Serial('COM5', 9600)

#initiate webcam
cap = cv2.VideoCapture(0)
cap.set(3, 1280)
cap.set(4, 720)

x3 = 0
y3 = 0
xgrab = 0
ygrab = 0
grab = 0 
# hand detector
detector = HandDetector(detectionCon=0.8, maxHands=1)

while True:
  success, image = cap.read()
  hands = detector.findHands(image, draw=False)
  if hands:
    lmList = hands[0]['lmList']
    xgrab, ygrab = lmList[4][:2]
    x3, y3 = lmList[3][:2]
    if xgrab < x3:
        grab = 1
    else:
        grab = 0
  else:
      grab = 0
  grab = str(grab)
  ser.write(grab.encode())
  cv2.imshow('Control window', cv2.flip(image,1))
  if cv2.waitKey(5) & 0xFF == 27:
      break
cap.release()
        

This code will detect your hand gesture and send 1 or 0 to Arduino. Since i put the ser.write command in while loop to send it continuously, but somehow it doesn't work as i expected. My arduino doesn't turn on/off the LED though.
Here is my Arduino code:
String Recstring;
void setup() {

Serial.begin(9600);
pinMode(LED_BUILTIN, OUTPUT);
}

void loop() {
if (Serial.available()){
  Recstring = Serial.readStringUntil('\n');
  if (Recstring == "1"){
    digitalWrite(LED_BUILTIN, HIGH);
  }
  else{
    digitalWrite(LED_BUILTIN, LOW);
  }
  }
}

I don't know if Arduino doesn't read the data fast enough to turn on/off LED. I've tried put some time.sleep to delay the data but it still like that.

Answer: So i need to add \n to grab = str(grab) + '\n' in my python code and it will work.
Thanks to Mark Setchell.
Title: Select specific values from JSON output
Tags: python,json,api,rest,python-requests

Question: I am querying a REST API and I need to select 2 fields from the adapter output below.
I basically need to make variables from OUT_Detailed Description and OUT_Vendor Ticket Number:
Code:
headers = {'content-type': 'application/json', 'Authentication-Token': authToken}
response = requests.post('http://dev.startschools.local:2031/baocdp/rest/process/:ITSM_Interface:IncidentManagement:QueryIncident/execute', headers=headers, data=json.dumps(get_query_inc_json()))
 
print(response.text)
json_format = json.loads(response)
Description = (json_format['OUT_Detailed Decription'])
Ref_Number = (json_format['OUT_Vendor Ticket Number'])

response.text printed Output:
[{"name":"OUT_HPD_CI","value":"001"},{"name":"OUT_Detailed Description","value":"Student needs a new card issued"},{"name":"OUT_Vendor Ticket Number","value":"INC0000019"}]

Error:
 in loads
    raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not Response
PS C:\Program Files\SB_Python\AD Extract> 

I tried several methods to get just the OUT_Detailed Description and OUT_Vendor Ticket Number values from the output but its all failing.

Answer: Have you tried doingjson_format = json.loads(response.content.decode('utf-8')) to translate your response into a string?
Title: PyCharm cannot see my newly compiled .pyc see on import
Tags: python,import,pycharm,pyc

Question: I'm using PyCharm and I have file foo.py. foo.py has a username and base64 representation of a password inside of it. I'm compiling foo.py so it's no longer readable by human eyes.
I'm running a command via the terminal
python -m py_compile foo.py

I can see the compiled .pyc file in the pychache folder named foo.cpython-39.pyc.
I remove foo.py.
When I add the line import foo, PyCharm indicates it cannot find the module.
If foo.py is retained, then of course the import line is fine.
If I copy and rename the .pyc file from the pycache folder to the root folder, calling it foo.pyc, PyCharm still indicates that it cannot find the module.
I have done this before so I know it's possible, but there is obviously a step I'm missing. Does anyone have any idea what that might be?

Answer: It is a cache issue or maybe a configuration issue,
You could try to clear the cach, go File then Invalidate Caches / Restart... then Invalidate and Restart
You can also check that the .pyc files are in the pycache directory
Title: unable to pull python from official docker registry
Tags: python,docker,docker-registry

Question: When I use docker pull python:3 it outputs:
3: Pulling from library/python
32fb02163b6b: Retrying in 1 second
167c7feebee8: Retrying in 1 second
d6dfff1f6f3d: Retrying in 1 second
e9cdcd4942eb: Waiting
ca3bce705f6c: Waiting
5e1c6c4f8bbf: Waiting
2da42ff3382c: Waiting
86f9457966ab: Waiting
896264e2a03c: Waiting
error pulling image configuration: download failed after attempts=6: x509: certificate signed by unknown authority

How is that possible if per default it pulls from the official docker registry?

Answer: I had company software installed that was messing with my ssl-certificates
Title: How to convert a string list to (object) list in Pandas?
Tags: python,pandas,string

Question: I have the dictionary below in which the values are string type:
data = {'object_1':"['abc']",
'object_2':"['def']",
"object_3": "['xyz']",
"object_4": "['abc']"}

I want to convert the values of the dictionary from a string type to a list type. I tried to use literal_eval and eval() but without any success to get pure python lists:
Desired output:
data = {'object_1':['abc'],
'object_2':['def'],
"object_3": ['xyz'],
"object_4": ['abc']}

Thanks for any advice.

Answer: saving the pandas file with different separator saved the day.
Title: Unable to save excel workbook on mac using xlwings
Tags: python,xlwings

Question: I'm running the same Xlwings code with jupyter notebook on both MAC and Windows to save a new Excel workbook in a folder.
import xlwings as xw
import os

wb = xw.Book()
wb.save(os.path.join(os.getcwd(),r'pro/fi/g.xlsx'))
wb.close()

It runs on Windows fine but gives the following error on MAC;
CommandError: Command failed:
                OSERROR: -50
                MESSAGE: Parameter error.
                COMMAND: app (pid=71190). workbooks [ 'Book2'].save_workbook_as (filename='Macintosh HD: Users: mohit: Desktop: pro:fi:g.xlsx', overwrite=True, file_format=k.Excel_XML_file_format, timeout=-1, password=None)

Answer: I've experienced the same issue and I noticed that was because I chose One Drive folder to save the file.
When I chose a folder outside One Drive the problem disappear.
Well, at the first instance the excel ask me to grant permission to save a file in the folder. But it saves without a problem on the subsequent safes.
Title: Is it ok to keep TCP connections alive throughout the serving time?
Tags: python,sockets,server,tcp,keep-alive

Question: I'm making a socket TCP server with python that handles multiple connections simultaneously.
so I defined a dictionary to hold the clients socket addresses, so I could access them and traffic the information between the clients easily. but I wander if there is a better way to connect them together without holding for example a hundred connection at the same time, which is really close to keep-alive in http connections that I believe we shouldn't use excessively and throughout the connection time. so do you guys have any ideas?
That's who the code looks like:
def run(self, server, port):
    while True:
        server.listen(5)
        conn, addr = server.accept()
        self.connectionMap[str(conn.getpeername())] = conn
        print('New connection:', conn.getpeername())
        Thread_(target=self.manageIndividual, args=([conn, addr, port]))

Answer: There's nothing wrong with keeping 100 connections open, especially if they are mostly idle.
In the past, having 100 threads open used to be a problem since (on many operating systems) each thread reserves 1MB of memory. So it was desirable to handle many connections per thread. But memory is plentiful now so it's still not a problem.
In the past, having 1000 connections open on one thread used to be a problem as well, since there weren't good ways to do that, but now there is epoll (Linux) and I/O Completion Ports (Windows).
But those are historical problems. Nowadays you can get away with thousands of threads, and there is also no problem handling tens of thousands of connections on one thread.
This answer is necessarily shallow since it's hard to prove a universal negative. In order to really show why it's not a bad idea I'd have to know why you think it is, and then disprove that.
Title: How can my python program do work and serve results via FastAPI at the same time?
Tags: python,fastapi,uvicorn,asgi

Question: I have a Python program that reads financial market data to do some analysis. I want to offer the results of the analysis via FastAPI. The problem is that when I start the uvicorn server for fastAPI, the rest of my Python program, especially the main loop, is nnot executed properly.
It is unclear to me how I can use FastAPI in a Python program that also has to do other work. This is what my main.py looks like (left out imports for simplicity...):
# Add fastAPI.
app = FastAPI()

# Start market scanner.
ms1 = MarketScanner(name="ms1", sample_size=10)

# Get the root.
@app.get("/")
def read_root():
    return {"Welcome": "to analyst."}

# For test purpose, get length of the current market data cache.
@app.get("/mdata/")
def read_market_data_len():
    mdata = ms1.get_market_data_len()
    #print(f"mdata: {mdata}")
    return {"Market data length": mdata}


if __name__ == "__main__":
    ms1.start()
    while True:
        ms1.get_new_market_data()
        print(f"market data length: {ms1.get_market_data_len()}")
        time.sleep(3)
   

I start the uvicorn server from a command line:
uvicorn main:app --reload

When the unicorn server is running, I start the Python program with
python main.py

I can see the market data length from the print in the while-loop increase, so my MarketScanner is fetching data from the financial data provider, but when I navigate to 127.0.0.1:8000/mdata in my browser, I always see "Market data length: 0", it just never increases.
I am a bit lost here, how can I built a python app with a fastAPI, but also a lot of other features that it handles in the background?

Answer: I gather that you are running two different processes, one serving the web app and the other running this module alone.  Those two programs are completely separate from one-another.  Each process has its own MarketScanner instance that is completely independent of the other.
One option is to move your MarketScanner into a function and run that function in a thread at module import time.
Title: Google Ads API "RefreshError: ('unauthorized_client: Unauthorized', {'error': 'unauthorized_client', 'error_description': 'Unauthorized'})"
Tags: python,api,google-ads-api

Question: I am trying to make a call to Google Ads API to access campaign data in python. I have completed the following steps:

Created a web app and enabled Google-Ads API
Saved json file that contains client id and secret information
Generated refresh token
Updated google-ads yaml file based on the client id, client secret, refresh token, developer token and customer id

However when I try this:
from google.ads.googleads.client import GoogleAdsClient
client = GoogleAdsClient.load_from_storage("google-ads.yaml")

I get the error: RefreshError: ('unauthorized_client')
I have rechecked for any typos or white-spaces. I have the "Standard Access" on my email id: on Google-Ads account under the Access and Security section. Any help would be really great.

Answer: You need to generate your refresh token again.
Is your app set to testing/public? Are you an added as an authorised user for if it's testing?
Title: Accessing model when creating new python package
Tags: python,python-packaging

Question: I'm making a python package where the main functionality is using a model that I've trained. I want it to be as easy as possible to access the model for end users so they can just use the main functions without needing to actively go find and download the model separately.
I initially thought I would just include the model file in the package but it is on the large side (~95MB) and Github warns me about it so my feeling is I should try find an alternative way.
I've tried compressing the model by zipping it but that doesn't make much of a dent in the file size. The model itself is fixed so I can't reduce the size by training an alternative version with different hyperparameters.
My solution at the moment is to not include the model and instead download it from s3 when the relevant class is used (this is an internal package, so everybody using it would hypothetically have access to the s3 bucket). However, I don't really like this solution because it requires users to have things set up to access AWS with a specific role, and even if I include examples / documentation / hints in error messaging, I can imagine the experience not being ideal. They also have the option of passing a file path if they have the model saved somewhere, but again that requires some initial setup.
I've tried researching ways to access / package models but haven't come up with much.
So my questions are:

Is there a way to include this model in the package?
Are there other ways to access the model that I haven't thought about?

Answer: Is there a way to include this model in the package?

Yes, you can include any file—no matter the size—in a Python package.

Are there other ways to access the model that I haven't thought about?

If the user is going to have to download it anyway, why not in the package?
One reason: if the model rarely changes, but the code around it does, the users will have to repeatedly download a large package.
Could you have two Python packages?  One that is just the model and the other that is the code?  That way the user will only need to download the model again if a new version is available.
Title: Django form field uploading error bad filename
Tags: python,django

Question: In that code I try to get the first page to make preview image. I use PyMuPDF library
def form_valid(self, form):
    text_book = form.save(commit=False)
    text_book.owner_id = self.request.user.pk
    # i'm trying to get the first page to make preview image
    my_file = self.request.FILES['book_file'].file
    pdf_file = fitz.open(my_file)
    page = pdf_file.load_page(0)
    pix = page.get_pixmap()
    preview_image = Image.frombytes('RGB', [pix.width, pix.height], pix.samples)
    preview_image.thumbnail((200, 200))
    image_io = BytesIO()
    preview_image.save(image_io, format='JPEG')
    image_data = image_io.getvalue()
    content_file = ContentFile(image_data)
    text_book.preview_image.save(f'text_book_preview_image{Path(text_book.book_file.name).stem}_preview.jpg',
                                    content_file,
                                    save=False)

    text_book.save()
    return HttpResponseRedirect(self.success_url)

Everything works. But when I put that in settings:
DATA_UPLOAD_MAX_MEMORY_SIZE = 1024 * 1024 * 500
FILE_UPLOAD_MAX_MEMORY_SIZE = DATA_UPLOAD_MAX_MEMORY_SIZE

I get "bad filename".
When code works without MAX_MEMORY_SIZE settings, my_file is
<tempfile._TemporaryFileWrapper object at 0x7f6491f1ae20>
But when I put that settings my_file is
<_io.BytesIO object at 0x7f69b6945950>
What is wrong?

Answer: your code is trying to open a file using a BytesIO object instead of a file path, which is likely causing the "bad filename" error. The difference between tempfile._TemporaryFileWrapper and _io.BytesIO is that the former represents a temporary file on disk, while the latter represents a file-like object in memory.
When you set DATA_UPLOAD_MAX_MEMORY_SIZE and FILE_UPLOAD_MAX_MEMORY_SIZE, you are limiting the size of files that can be uploaded to your server in memory rather than on disk. This means that larger files will not be saved to a temporary file on disk but instead will be held in memory as a BytesIO object.
To fix the "bad filename" error, you may need to modify the code that opens the PDF file to accept a BytesIO object instead of a file path. One way to do this is to pass the my_file object directly to the fitz.open method, like this:

pdf_file = fitz.open(stream=my_file.read())

This should allow your code to work even when DATA_UPLOAD_MAX_MEMORY_SIZE and FILE_UPLOAD_MAX_MEMORY_SIZE are set. However, be aware that if you allow large files to be uploaded and held in memory, you may run into performance issues or memory errors on your server.
Title: Telegram bot that can play in emulator (BizHawk)
Tags: python,bots,telegram,emulation,pyautogui

Question: I'm trying to make a bot that can read all messages sent in a group, and play the game. Something close to Twitch Plays Pokemon but for every game.
Right now I have this code, the bot works fine in Telegram but once it get the window focused and should move it doesn't...
import logging
import pyautogui  # Importa la biblioteca pyautogui
import pygetwindow as gw
from telegram import Update
from telegram.ext import Updater, CommandHandler, MessageHandler, Filters, CallbackContext

# Configurar el registro
logging.basicConfig(level=logging.INFO)

TOKEN = "TOKEN"

MOVEMENTS = {
    "up": "w",
    "Up": "W",
    "down": "s",
    "left": "a",
    "right": "d"
}

def focus_pokemon_window(window_title):
    try:
        window = gw.getWindowsWithTitle(window_title)[0]
        window.restore()
        window.activate()
    except IndexError:
        print(f"No se encontró ninguna ventana con el título '{window_title}'.")

def move_character(direction):
    focus_pokemon_window("Pokemon - FireRed Version (USA) [Gameboy Advance] - BizHawk")
    pyautogui.hotkey(MOVEMENTS[direction]) 


def start(update: Update, context: CallbackContext):
    pass

def handle_text(update: Update, context: CallbackContext):
    bot_username = context.bot.username
    text = update.message.text.lower()

    if f"@{bot_username.lower()}" not in text:
        return

    # Eliminar la mención del bot del texto
    text = text.replace(f"@{bot_username.lower()}", "").strip()

    if text in MOVEMENTS:
        move_character(text)
    else:
        update.message.reply_text("Comando desconocido. Por favor, envía una dirección válida (up, down, left, right).")

def main():
    updater = Updater(TOKEN)
    dispatcher = updater.dispatcher

    # Agregar manejadores de comandos y mensajes
    dispatcher.add_handler(CommandHandler("start", start))
    dispatcher.add_handler(MessageHandler(Filters.text & ~Filters.command, handle_text))

    # Iniciar el bot
    updater.start_polling()
    updater.idle()

if __name__ == "__main__":
    main()

the game will be streamed by OBS privately.
I tried all kind of solutions but seems that when windows pop-ups it doesn't make any movement.

Answer: In my experience, anything that involves pulling up a specific window by name using something like pygetwindow or pywin32 doesn't really work consistently, sometimes works and sometimes doesn't. I blame Microsoft for changing the Windows API constantly without much documentation. You could try using Pyautogui to get the window. To click an image in Pyautogui, just use pyautogui.click(x = image_name). Make sure that your screencap is accurate, if you're on Windows hit the Windows button, then type "snip". Drag over the part you want to click then click save. If you can't crop the image small enough, download GIMP and crop it with that.
Title: Capture strings between groups with groups aren't fixed (RegEx)
Tags: python,regex

Question: I'm trying to capture a group of strings between two groups of strings. The RegEx is mostly working, but it doesn't capture all when there is a change in pattern.
The string is:
2023-03-20 / 10:56:58 4737 Security-Enabled Global Group Modified 73 high SRVDC2 john.smi.admin 10.7.3.252 1

After the date and time is a four-digit number, which sometimes is not present, so the log show N/A instead. That's when I'm having trouble. The RegEx must be able to capture both the four-digit number and the N/A message.
Here's what I've tried:
import re
string = '2023-03-20 / 10:56:58 4737 Security-Enabled Global Group Modified 73 high SRVDC2 john.smi.admin 10.7.3.252 1'
pattern = '(?<=\d{4} )(.*?)(?=\s\d{2}\s)'
res = re.findall(pattern,string,re.MULTILINE)
print(res)

Answer: Use '(?<=\d{4} | N/A )(.*?)(?=\s\d{2}\s)' instead.
It looks behind to 4 digits and space OR N/A between spaces. This is needed to keep lookbehind group fixed length.
Title: How to create PJSUA2 Python 3 package?
Tags: python,windows,pjsip,pjsua2

Question: I have a very hard time creating a PJSUA2 package for Python 3.x on Windows 10. I downloaded the source code from the pjsip site and I'm able to compile the C++ code without problems, but I cannot build the PJSUA2 module for Python. The docs mention using SWIG, but I had no luck so far.
When I run make from ./pjsip-apps/src/swig I get the following:
Makefile:1: ../../../build.mak: No such file or directory
make: *** No rule to make target '../../../build.mak'.  Stop.

When I run make from ./pjsip-apps/src/swig/python I get:
sed: -e expression #1, char 8: unterminated `s' command
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='cp1250'>
OSError: [Errno 22] Invalid argument
python3 setup.py build --compiler=mingw32
helper.mak:2: /build/common.mak: No such file or directory
make[1]: *** No rule to make target '/build/common.mak'.  Stop.
helper.mak:2: /build/common.mak: No such file or directory
make[1]: *** No rule to make target '/build/common.mak'.  Stop.
helper.mak:2: /build/common.mak: No such file or directory
make[1]: *** No rule to make target '/build/common.mak'.  Stop.
helper.mak:2: /build/common.mak: No such file or directory
make[1]: *** No rule to make target '/build/common.mak'.  Stop.
running build
running build_py
running build_ext
building '_pjsua2' extension
x86_64-w64-mingw32-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -O2 -pipe -fno-ident -I/c/mingw-builds/ucrt64-seh-posix/x86_64-1220-posix-seh-ucrt-rt_v10-rev2/mingw64/opt/include -I/c/mingw-builds/ucrt64-seh-posix/prerequisites/x86_64-zlib-static/include -I/c/mingw-builds/ucrt64-seh-posix/prerequisites/x86_64-w64-mingw32-static/include -D__USE_MINGW_ANSI_STDIO=1 -IC:/ProgramData/chocolatey/lib/mingw/tools/install/mingw64/opt/include/python3.9 -c pjsua2_wrap.cpp -o build/temp.mingw_x86_64_ucrt-3.9/pjsua2_wrap.o
pjsua2_wrap.cpp:3841:10: fatal error: pjsua2.hpp: No such file or directory
 3841 | #include "pjsua2.hpp"
      |          ^~~~~~~~~~~~
compilation terminated.
error: command 'C:\\ProgramData\\chocolatey\\bin/x86_64-w64-mingw32-gcc.exe' failed with exit code 1
make: *** [Makefile:37: _pjsua2.so] Error 1

I don't understand what could be wrong, as I literally only downloaded the source code and did not touch anything.

Answer: When I was trying to install the PJSIP/PJSUA2 package firstly on windows it ran into constant build errors which would never go away, instead try to switch to a Linux based operating system. When that's not possible try to use the Windows Linux subsystem\WSL2 which gave me great success.
(Note that the use of audio is not possible inside WSL2, kind of defeating the purpose depending on what your project is about)
I'm hoping to have helped you out!
Title: Value interpolation issue with Python SQLite3 library
Tags: python,flask,sqlite

Question: I have an odd problem with an SQL query and I'm finding it difficult to debug unfortunately.
I have a query that inserts a user into a db, if I run this debug statement I get the values returned from the form.
app.logger.debug(f'''{request.form.get("firstName").capitalize()}, {request.form.get("lastName").capitalize()}, {request.form.get("userName").lower()}, {request.form.get("inputEmail").lower()}, {generate_password_hash(request.form.get("inputPassword"), method="sha256")}''')

which are as follows
DEBUG: David, Taylor, taylord, dave@imagetaylor.com, sha256$xXfFJg6EKm9V7O5D$f8eadba22c3ad99c4133ac9ae4d2d25dfe9665b70df04d51417d1caec708ab4c

if I then plug those into the SQL query everythings works
dbexe("INSERT INTO users(firstname,lastname,username,email,hash) VALUES('David','Taylor','taylord','dave@imagetaylor.com','sha256$04meQSacFSYL12aq$b17d53e8a3de54e0dcf66c11cf36858f551b9b72a771b18c9b8f3bf24599f8b2')")

However when I use safe interpolation through the sqlite3 library it fauls and I'm having trouble finding a way to see the issue.
dbexe("INSERT INTO users(firstname,lastname,username,email,hash) VALUES('?','?','?','?','?')", request.form.get("firstName").capitalize(), request.form.get("lastName").capitalize(), request.form.get("userName").lower(), request.form.get("inputEmail").lower(), generate_password_hash(request.form.get("inputPassword"), method="sha256"))

The dbexec function is as follows.
def dbexe(query):
    try:
        retval = []
        db = sqlite3.connect("main.db")
        db.row_factory = dict_factory
        for row in db.execute(query):
            retval.append(row)
        db.commit()
        db.close()
    except:
        return "Failed"
    return retval

Answer: Use VALUES(?,?,?,?,?) instead of VALUES('?','?','?','?','?').
There's no safe interpolation. The code is using a query with parameters, not string interpolation of any kind. ? is a parameter, not a format placeholder and shouldn't be quoted. The parameter values are included in the query itself. They're sent as separate, strongly-typed values to the database alongside the query. The database then compiles the query into a parameterized execution plan which gets executed using those values.
This way you can send numbers, dates, or strings containing anything to the server without worrying about formats or injection risks.
Title: Time Complexity of Binary Search Tree Inorder Traversal
Tags: python,python-3.x,time-complexity,binary-search-tree,inorder

Question: I found the following python code for inorder traversal of a binary search tree, with an explanation that its time complexity is O(n), where n is the number of nodes in the tree. I understand that the code visits every node in the tree, so its time complexity is linear in the number of nodes, but at each level of recursion, isn't it also performing an addition operation on lists that should take O(n) time? Shouldn't the time complexity be O(n^2)?
def inorder(r):
   return inorder(r.left) + [r.val] + inorder(r.right) if r else []

Answer: Yes. List concatenation in Python seems to be an O(m + n) operation, as values are copied each time. Therefore, the work per node is not constant but depends on the length of its children. So you are correct in your thinking and this code is not O(n). However, it could easily be implemented in such a way that it would be O(n).
The worst-case complexity of this code would indeed be O(n^2), as you claim. It is easiest to think of the extreme case if the tree is so unbalanced that it is just a 1D linked list. Then, you would obtain a complexity of O(1 + 2 + ... + n) = O(n^2).
Title: Statsmodels Clustered Logit Model With Robust Standard Errors
Tags: python,logistic-regression,statsmodels,stderr

Question: I have the following dataframe:
df.head()

        id    case   volfluid   map   rr    o2    fluid
1044    36    3      3.0        1.0   3.0   2.0   0.0
1045    37    3      2.0        3.0   1.0   2.0   1.0
1046    38    3      3.0        2.0   2.0   1.0   0.0
1047    36    4      2.0        3.0   1.0   3.0   1.0
1048    37    4      1.0        1.0   3.0   3.0   1.0
.
.
.

I want to run a logistic regression model clustered on id and with robust standard errors. Here is what I have for the equation
smf.logit('''fluid ~ C(volfluid) + C(map, Treatment(reference = 3.0)) + 
             C(o2, Treatment(reference = 3.0)) + C(rr) + 
             C(case, Treatment(reference = 4))''',
             data = df).fit(cov_type='cluster', cov_kwds={'groups': df['id']})

I'm not sure if this accomplishes both the clustering, and the robust std. errors. I understand that setting cov_type = 'hc0' provides robust std. errors, but if I do that can I still cluster on id? And do I need to do that, or are clustered standard errors inherently robust?
Thank you!

Answer: Cluster robust standard errors are also heteroscedasticity robust (HC). A HC cov_types do not take any correlation into account.
Related aside: Using GEE with independence correlation has the same underlying model as Logit but has the option of bias-reduced cluster robust standard errors (similar so CR3, the HC3 analogue for cluster correlations)
Title: Get the name of columns from rows distinct from zero python
Tags: python,pandas,dataframe

Question: I have this dataframe:
df0 = pd.DataFrame({'points': [0, 0, -3, 16, 0, 5, -3, 14],
                    'assists': [0, 0, 2, 0, 1, -7, 0, 6],
                    'numbers': [0, 0, 1, 6, 10, 5, 8, 7]})

and my desired dataset looks like this:
points assists numbers colX
0      0       0        0
0      0       0        0
-3     2       1       'points-assists-numbers'
16     0       6       'points-numbers'
0      1       10      'assists-numbers'
5      7       5       'points-assists-numbers'
-3     0       8       'points-numbers'
14     8       7       'points-assists-numbers'

A function that create a string from columns names that have values distinct from zero.
Any help?

Answer: This kind of operation is well suited to a lambda expression.
Something like this should work:
df0['colX'] = df0.apply(lambda x: '-'.join(c for c in df0.columns if x[c] != 0), axis=1).replace('', 0)

first it gets a list of the columns that are not 0
joins the names of those columns with a "-"
after that, fills blank names with a 0
Title: python keras.preprocessing.sequence has no attribute pad_sequences
Tags: python

Question: import:from keras.preprocessing import sequence
but:
AttributeError: module 'keras.preprocessing.sequence' has no attribute 'pad_sequences'
Why?
How can I edit this?

Answer: Seems like your keras version is greater than 2.8 that's why getting error as
from keras.preprocessing import sequence
works for earlier version. Instead, replace with the below given code:
from keras.utils.data_utils import pad_sequences
You can also use:
from tensorflow.keras.preprocessing.sequence import pad_sequences
They both worked for me.
Title: Python render svg image with the python only modules
Tags: python,svg,rendering

Question: The question is simple, but I have googled a lot of methods, and there no such solution as:
import svg-render-library
figure = svg-render-library.open('test.svg')
figure.render()

Is there any simple methods to display an SVG image using only python libraries?
I am asking about rendering the SVG image without any conversion to another formats and to render using pure python, without any 3rd party software. As I have tried, this seems impossible for now.
As built-in python I mean - only python packages available through pip, so it is not necessary to install/compile anything else. And to render I mean to show inside window which part of the python, not the browser or any external software.

Answer: Currently, there is no method to render natively cross-platform with just the standard library (ie. some python distributions for OSX do not include tkinter by default). Ergo, there is no good way to do this.
AFAIK, there are no other ways to do this maintaining your described API without writing your own code or reaching out to non-standard library modules.
If you still are 100% set on doing it with pure python and the standard library, you have tkinter, and don't care about writing your own implementation, then proceed.
If you are talking about rendering in the context of displaying an SVG in a window, then your best bet would be to utilize the tkinter and xml modules.
SVG is just an XML file, so xml.minidom should be able to parse an svg file. You can then extract the height and width from the dom, draw each element onto a tkinter.Canvas widget, and then have tkinter render the window. You will need to either pre-calculate transparencies or handle that while managing the layering.
Another option is to use the turtle package which wraps around tkinter. If the SVG is just a path, then this should be able to draw that path fairly straight forward.
If you are willing to reach out beyond the standard library, then cairosvg or svglib both can easily handle this task. cairosvg is a bit of a bear if you aren't used to installing system libraries, but svglib is a pure python implementation.
Title: Does DataFrame.index.empty imply DataFrame.empty?
Tags: python,pandas,dataframe

Question: If I have a DataFrame, df, for which df.index.empy is True, will this ALWAYS imply that df.empty is also True?
My intend is to test only df.index.empy when I need to test both conditions (lazy programming style).

Answer: yes, if index of DataFrame is empty it will always satisfies condition of DataFrame.empty
For e.g

DataFrame.empty = True

works in both condition for index as well as for columns.
Hence if you want to check any of these are empty then you can go with

DataFrame.empty

else need to be specific

DataFrame.index.empty
Title: Select row in dataframe containing a specific UUID
Tags: python,pandas,uuid

Question: I have a dataframe, one column of which holds a UUID:
import numpy as np
import pandas as pd
import uuid

df = pd.DataFrame(
    data=[[1, 2, 3], [4, 5, 6]],
    columns=['a', 'b', 'c']
)

df['d'] = np.NaN
df['d'] = df['d'].apply(
    lambda x: uuid.uuid4()
)

Preview:
df

-------     a   b   c   d
------- 0   1   2   3   31abc2af-117d-4fe8-b43f-e68fa429187f
------- 1   4   5   6   f63b36c8-bb4e-4148-ace9-a89fa117e15c

I now want to select rows based on a UUID. But the following returns an empty set of rows:
df.loc[
    df['d'] == '31abc2af-117d-4fe8-b43f-e68fa429187f'
]

How do I select rows using UUID as the match criteria?

Answer: I see you found a solution to you problem. Another solution would be to convert the column values to strings:
df.loc[df['d'].astype(str) == '31abc2af-117d-4fe8-b43f-e68fa429187f']
Title: Sparse matrix multiplication in pytorch
Tags: python,deep-learning,pytorch,linear-algebra

Question: I want to implement the following formula in pytorch in a batch manner:
x^T A x

where x has shape: [BATCH, DIM1]
and A has shape: [BATCH, DIM1, DIM1]
I managed to implement it for the dense matrix A as follows:
torch.bmm(torch.bmm(x.unsqueeze(1), A), x.unsqueeze(2)).squeeze().
However, now I need to implement it for a SPARSE matrix A and I am failing to implement it.
The error that I am getting is {RuntimeError}bmm_sparse: Tensor 'mat2' must be dense, which comes from the torch.bmm(x.unsqueeze(1), A) part of the code.
In order to reproduce my work you could run this:
import torch
sparse = True # switch to dense to see the working version

batch_size = 10
dim1 = 5
x = torch.rand(batch_size, dim1)
A = torch.rand(batch_size, dim1, dim1)

if sparse:
   A = A.to_sparse_coo()
xTAx = torch.bmm(torch.bmm(x.unsqueeze(1), A), x.unsqueeze(2)).squeeze()

My pytorch version is 1.12.1+cu116

Answer: The solution is as simple as changing the order of multiplications from
(xT A) x to xT (Ax).