Myashka/SO-Python_QA-filtered-2023-tanh_score-after_2023_02 huggingface dataset:
Title: When installing an old version of a package, can I install only compatible versions of dependent packages?
Tags: python,python-3.x,sqlalchemy,teradata

Question: I'm using Python 3.7.4 in a venv environment.
I ran pip install teradataml==17.0.0.3 which installs a bunch of dependent packages, including sqlalchemy.
At the time, it installed SQLAlchemy==2.0.2.
I ran the below code, and received this error:
ArgumentError: Additional keyword arguments are not accepted by this function/method.  The presence of **kw is for pep-484 typing purposes
from teradataml import create_context 

class ConnectToTeradata:
    def __init__(self):
        
        host = 'AWESOME_HOST'
        username = 'johnnyMnemonic'
        password = 'keanu4life'

        self.connection = create_context(host = host, user = username, password = password)

    def __del__(self):
        print("Closing connection")
        self.connection.dispose()

ConnectToTeradata()

If I install SQLAlchemy==1.4.26 before teradataml, I no longer get the error and successfuly connect.
This suggests SQLAlchemy==2.0.2 is not compatible with teradataml==17.0.0.3.
I expected installing an older version of teradataml would also install older, compatible versions of dependent packages.
When I install teradataml==17.0.0.3, can I force only install compatible versions of dependent packages?

Answer: We are aware of the compatibility issues that were introduced in SQLAlchemy package 2.0.x versions. The new 2.0.x package directly affects the Teradata SQL dialect in the teradatasqlalchemy package. As a temporary measure, please downgrade SQLAlchemy to 1.4.46.
Teradata Engineering is working on making the teradatasqlalchemy package compatible with the newer versions and a new package is slated to be released in March 2023.
Title: 'ReadOnlyWorksheet' object has no attribute 'defined_names'
Tags: python,exception

Question: Whenever I try to read Excel using
part=pd.read_excel(path,sheet_name = mto_sheet)

I get this exception:

<class 'Exception'> 'ReadOnlyWorksheet' object has no attribute 'defined_names'

This is if I use Visual Studio Code and Python 3.11. However, I don't have this problem when using Anaconda. Any reason for that?

Answer: Possible workaround: create new excel file, with default worksheet name ("Sheet1" etc.) and copy and paste data here.
(tested on Python 3.10.9 + openpyxl==3.1.1)
Title: pandas json dictionary to dataframe, reducing columns by creating new columns
Tags: python,pandas,dataframe

Question: Following JSON File (raw data how I am getting it back from an API call):
{
    "code": "200000",
    "data": {
        "A": "0.43221600",
        "B": "0.02311155",
        "C": "0.55057515",
        "D": "2.15957924",
        "E": "0.03818908",
        "F": "0.26853420",
        "G": "0.15007500",
        "H": "0.00685843",
        "I": "0.08500848"
    }
}

Will crate this output in Pandas by using this code (one column per data set in "data"). The result is a dataframe with many columns:
import pandas as pd
import json 
f = open('file.json', 'r')
j1 = json.load(f)
pd.json_normalize(j1)

    code    data.A  data.B  data.C  data.D  data.E  data.F  data.G  data.H  data.I
0   200000  0.43221600  0.02311155  0.55057515  2.15957924  0.03818908  0.26853420  0.15007500  0.00685843  0.08500848


I guess that Pandas should provide a built in function of the data set in the attribute "data" could be split in two new columns with names "name" and value, including a new index. But I cannot figure out how that works.
I would need this output:
    name    value
0   A       0.43221600
1   B       0.02311155
2   C       0.55057515
3   D       2.15957924
4   E       0.03818908
5   F       0.26853420
6   G       0.15007500
7   H       0.00685843
8   I       0.08500848

Answer: pd.DataFrame.from_dict(j1)
should give you the result you need
Title: Pulsar producer send_async() with callback function acknowledging the sent message
Tags: apache-pulsar,pulsar,python-pulsar

Question: I have a use case where messages from an input_topic gets consumed and sent to a list of topics. I'm using producers[i].send_async(msg, callback=callback)  where callback = lambda res, msg: consumer.acknowledge(msg). In this case, consumer is subscribed to the input_topic. I checked the backlog of input_topic and it has not decreased at all.  Would appreciate if you could point out how to deal with this? What would be the best alternative?
Thanks in advance!

Answer: Have you checked the consumer.acknowledge(msg) has actually been called? One possibility is the producer cannot write messages to the topic, and if the producer with infinite send timeout, you will never get the callback.
Title: How to reverse the shape of a numpy array
Tags: python,python-3.x,numpy,numpy-ndarray

Question: I have a numpy array with a shape of (3, 4096). However, I need it's shape to be (4096, 3). How do I accomplish this?

Answer: Use:
arr=arr.T
(or)
arr=np.transpose(arr)
(or)
arr= arr.reshape(4096, 3)
where arr is your array with shape (3,4096)
Title: Python execute code in parent shell upon exit
Tags: python,posix

Question: I have a search program that helps users find files on their system.  I would like to have it perform tasks, such as opening the file within editor or changing the parent shell directory to the parent folder of the file exiting my python program.
Right now I achieve this by running a bash wrapper that executes the commands the python program writes to the stdout.  I was wondering if there was a way to do this without the wrapper.
Note:
subprocess and os commands create a subshell and do not alter the parent shell.  This is an acceptable answer for opening a file in the editor, but not for moving the current working directory of the parent shell to the desired location on exit.
An acceptable alternative might be to open a subshell in a desired directory
example
#this opens a bash shell, but I can't send it to the right directory
subprocess.run("bash")

Answer: This, if doable, will require quite a hack. Because the PWD is passed from the shell into the subprocess - in this case, the Python process, as a subprocess owned variable, and changing it won't modify what is in the super program.
On Unix, maybe it is achievable by opening a detachable sub-process that will pipe keyboard strokes into the TTY after the main program exits - I find this the most likely to succeed than any other thing.
Title: How to convert python file to exe? The 'pathlib' package is an obsolete backport of a standard library package and is incompatible with PyInstaller
Tags: python,python-3.x,anaconda,conda,exe

Question: I am receiving following error while converting python file to .exe
I have tried to uninstall and intsall pyinstaller but it didnt helped out. i upgraded conda but still facing same error. Please support to resolve this issue
Command
(base) G:>pyinstaller --onefile grp.py
Error
The 'pathlib' package is an obsolete backport of a standard library package and is incompatible with PyInstaller. Please remove this package (located in C:\Users\alpha\anaconda3\lib\site-packages) using conda remove then try again.
Python Version
(base) G:>python --version
Python 3.9.16

Answer: The error message you received suggests that the 'pathlib' package installed in your Anaconda environment is causing compatibility issues with PyInstaller. As a result, PyInstaller is unable to create a standalone executable from your Python script.
Title: How to convert python file to exe? The 'pathlib' package is an obsolete backport of a standard library package and is incompatible with PyInstaller
Tags: python,python-3.x,anaconda,conda,exe

Question: I am receiving following error while converting python file to .exe
I have tried to uninstall and intsall pyinstaller but it didnt helped out. i upgraded conda but still facing same error. Please support to resolve this issue
Command
(base) G:>pyinstaller --onefile grp.py
Error
The 'pathlib' package is an obsolete backport of a standard library package and is incompatible with PyInstaller. Please remove this package (located in C:\Users\alpha\anaconda3\lib\site-packages) using conda remove then try again.
Python Version
(base) G:>python --version
Python 3.9.16

Answer: I face with the same problem, and I input the 'conda remove pathlib', it didn't work. The result is Not found the packages, so I found the lir 'lib', there was a folder named 'path-list-....', finally I delete it, and it began working!
Title: How to convert python file to exe? The 'pathlib' package is an obsolete backport of a standard library package and is incompatible with PyInstaller
Tags: python,python-3.x,anaconda,conda,exe

Question: I am receiving following error while converting python file to .exe
I have tried to uninstall and intsall pyinstaller but it didnt helped out. i upgraded conda but still facing same error. Please support to resolve this issue
Command
(base) G:>pyinstaller --onefile grp.py
Error
The 'pathlib' package is an obsolete backport of a standard library package and is incompatible with PyInstaller. Please remove this package (located in C:\Users\alpha\anaconda3\lib\site-packages) using conda remove then try again.
Python Version
(base) G:>python --version
Python 3.9.16

Answer: I've experienced the same problem. I managed to solve it by downgrading pyInstaller to 5.1 (from 5.8) without touching pathlib. An additional possibility to consider.
Title: I am trying to test a program that prints a shipping rate based on yes or no answers
Tags: python

Question: The problem with this program is that the if/else statements are not working properly. When the answer is "yes", the problem also prints the question for when the answer is "no". Another problem is that it's not printing the rate1 when it's supposed to.
# This program calculates the shipping cost as shown in the slide
international = input("Are you shipping internationally (yes or no)? ")
rate1 = 5
rate2 = 10

if international.upper() == "yes":
    shippingRate = rate2
else:
    continental = input("Are you shipping continental (yes or no)? ")
    if continental.upper() == "yes":
        shippingRate = rate1
    else:
        shippingRate = rate2
        
print("The shipping rate is " + ("%.2f" % shippingRate))

Answer: I notice you're using a .upper() that would not ever equal "yes"
Cause upper() won't ever return lowercase letters.
But this code might work with == "YES".
Title: Solving nonlinear differential equations in python
Tags: python,scipy,differential-equations,odeint

Question: I am trying to solve the differential equation 4(y')^3-y'=1/x^2 in python. I am familiar with the use of odeint to solve coupled ODEs and linear ODEs, but can't find much guidance on nonlinear ODEs such as the one I'm grappling with.
Attempted to use odeint and scipy but can't seem to implement properly
Any thoughts are much appreciated
NB: y is a function of x

Answer: The problem is that you get 3 valid solutions for the direction at each point of the phase space (including double roots). But each selection criterion breaks down at double roots.
One way is to use a DAE solver (which does not exist in scipy) on the system y'=v, 4v^3-v=x^-2
The second way is to take the derivative of the equation to get an explicit second-order ODE y''=-2/x^3/(12*y'^2-1).
Both methods require the selection of the initial direction from the 3 roots of the cubic at the initial point.
Title: Is there a way to find (potentially) multiple results with re.search?
Tags: python,regex

Question: While parsing file names of TV shows, I would like to extract information about them to use for renaming. I have a working model, but it currently uses 28 if/elif statements for every iteration of filename I've seen over the last few years. I'd love to be able to condense this to something that I'm not ashamed of, so any help would be appreciated.
Phase one of this code repentance is to hopefully grab multiple episode numbers.  I've gotten as far as the code below, but in the first entry it only displays the first episode number and not all three.
import re

def main():
    pattern = '(.*)\.S(\d+)[E(\d+)]+'
    strings = ['blah.s01e01e02e03', 'foo.s09e09', 'bar.s05e05']

    #print(strings)
    for string in strings:
        print(string)
        result = re.search("(.*)\.S(\d+)[E(\d+)]+", string, re.IGNORECASE)
        print(result.group(2))

if __name__== "__main__":
    main()

This outputs:
blah.s01e01e02e03
01
foo.s09e09
09
bar.s05e05
05

It's probably trivial, but regular expressions might as well be Cuneiform most days. Thanks in advance!

Answer: re.findall instead of re.search will return a list of all matches
Title: Receiving Error not all arguments converted during string formatting
Tags: python,sqlalchemy

Question: I am new to working on Python. I m not able to understand how can I send the correct input t0 the query.
      list_of_names = []

    for country in country_name_list.keys():
        list_of_names.append(getValueMethod(country))

    sql_query = f"""SELECT *  FROM table1
                            where name in (%s);"""
                           

    db_results = engine.execute(sql_query, list_of_names).fetchone()


Give the error " not all arguments converted during string formatting"

Answer: If I know right, there are a simpler solution. If you write curly bracets {}, not bracets (), and you place inside the bracets a variable, which contains the %s value, should work. I don't know, how sql works, but you should use one " each side, not three.
Sorry, I'm not english. From this, maybe I wasn't help with the question, because I don't understand correctly.
Title: TypeError: Failed to execute 'evaluate' on 'Document': The result is not a node set, and therefore cannot be converted to the desired type
Tags: python,selenium-webdriver,xpath,selenium-chromedriver

Question: I need to find elements on a page by looking for text(), so I use xlsx as a database with all the texts that will be searched.
It turns out that it is showing the error reported in the title of the publication, this is my code:
        search_num = str("'//a[contains(text()," + '"' + row[1] + '")' + "]'")
        print(search_num)
        xPathnum = self.chrome.find_element(By.XPATH, search_num)
        print(xPathnum.get_attribute("id"))

print(search_num) returns = '//a[contains(text(),"0027341-66.2323.0124")]'
Does anyone know where I'm going wrong, despite having similar posts on the forum, none of them solved my problem. Grateful for the attention

Answer: Looks like you have extra quotes here
str("'//a[contains(text()," + '"' + row[1] + '")' + "]'")
Try changing to f"//a[contains(text(),'{row[1]}')]"
Title: Would df.sort_values('A', kind = 'mergesort').sort_index(kind = 'mergesort') be a stable and valid way to sort by index and column?
Tags: python,pandas,sorting

Question: I have a Pandas dataframe equivalent to:
              'A' 'B'
'i1' 'i2' 'i3'
 1    2   4    3   0
 1    1   2    3   3
 1    1   2    1   0
 1    2   4    0   9
 1    1   2    2   6
 2    1   1    1   8

where ix are index columns and 'A', and 'B' are normal columns. I want to make sure that the indexes are strictly ordered and, when indexes are duplicated, then it is ordered by column 'A'
              'A' 'B'
'i1' 'i2' 'i3'
 1    1   2    1   0
 1    1   2    2   6
 1    1   2    3   3
 1    2   4    0   9
 1    2   4    3   0
 2    1   1    1   8
 

Would df.sort_values('A', kind = 'mergesort').sort_index(kind = 'mergesort') do it? And if so, would do it in a stable way? or could the .sort_index() operation disrupt the previous .sort_values() operation in such a way that, for the duplicated indexes, the values of 'A' are no longer ordered?

Answer: When you sort by multiple keys, only the last one is guaranteed to be sorted. The others will be sorted within the previous groups. Finally, the non-key columns will remain sorted in the original order in case of a stable sort such as the mergesort.
To answer your question, yes, your method will maintain the original order in case of duplicated keys.
Title: Sending a word document without saving it on the flask server
Tags: python,flask

Question: Good day. Today I'm trying to send a document generated on the server to the user on the click of a button using Flask.
My task is this:
Create a document (without saving it on the server). And send it to the user.
However, using a java script, I track the button click on the form and use fetch to make a request to the server. The server retrieves the necessary data and creates a Word document based on it. How can I form a response to a request so that the file starts downloading?
Code since the creation of the document. (The text of the Word document has been replaced)
python Falsk:
document = Document()
document.add_heading("Some head-title")
document.add_paragraph('Some text')
f = BytesIO()
document.save(f)
f.seek(0)
return send_file(f, as_attachment=True, download_name='some.docx')

However, the file does not start downloading.
How can I send a file from the server to the user?
Edits
This is my js request.
fetch('/getData', {
    method : 'POST',
    headers: {
        'Accept': 'application/json',
        'Content-Type': 'application/json'
    },
    body: JSON.stringify({
        someData: someData,
    })
})
.then(response => 
    response.text()
)
.then(response =>{
    console.log(response);
});

This is my html
<form action="" name="getData" method="post" enctype="multipart/form-data">
<button type = "submit" name = "Download">Download</button>
</form>

Answer: You need to specify the mimetype, It tries to detect the mimetype from the filename but since we are not saving it we need to specify the mimetype.
return send_file(f, mimetype='application/msword', as_attachment=True, download_name='output.doc')
Title: DLT Stream Error - Queries with streaming sources must be executed with writeStream.start();
Tags: python,streaming,databricks,delta-live-tables

Question: I'm trying to parse incoming variable length stream records in databricks using Delta Live Tables. I'm getting the error:
Queries with streaming sources must be executed with writeStream.start();
Notebook code
@dlt.table (
    comment="xAudit Parsed"
)
def b_table_parsed():
    df = dlt.readStream("dlt_table_raw_view")
          
    for i in range(df.select(F.max(F.size('split_col'))).collect()[0][0]):
        df = df.withColumn("col"+str(i),df["split_col"][i])
    
    df = (df
          .drop("value","split_col")
         )
      
    return df

This all works fine against the actual source text files or a delta table using the interactive cluster but when I put it in DLT and and the source is streaming files from autoloader, it doesn't like it. I assume it's stream related.
I saw a different post about using .foreach maybe but that was using writeStream and not sure if I can or how to use it to return in a DLT table, or if there is another solution.
I'm very new to python, streaming and DLT so would appreciate if anyone can walk me through a detailed solution.
Trying to parse out variable length rows in a streaming source using a delta live table notebook in databricks.  Works on the interactive cluster but not streaming in DLT

Answer: The problem is in this piece of code: df.select(F.max(F.size('split_col'))).collect()[0][0] - you're trying to find a max + collect it from the stream that is by definition doesn't have start & end.  Your code most probably works with batch DF or inside a function called from .foreachBatch that isn't supported by DLT.
Title: Getting "ValueError: A PythonObject is not attached to a node" even when wrapped in try/except block but this works fine if run in Nuke Script editor
Tags: python,nuke

Question: My question is Foundary Nuke specific.
I have a tab added to Project Settings, that contains some data I can later access via the root node. Now since I have callback invoked by a checkbox knob I added to enable disable a custom knob I added to that tab I added to Project Settings Panel. It works fine. The problem is when I close nuke I get error:
Traceback (most  recent call last):
File "/system/runtime/plugins/nuke/callbacks.py", line 127, in knobChanged
    _doCallbacks(knobChangeds)
File "/system/runtime/plugins/nuke/callbacks.py", line 44, in _doCallbacks
  for f in list:
ValueError: A PythonObject is not attached to a node

Now this error happens if I have a callback function added to the checkbox knob like this:
my_callbacks.py
import nuke

def on_checkbox_clicked():
    try:
        root_node = nuke.root()
        if not root_node:
            return
    except ValueError as er:
        print(er)

nuke.addKnobChanged(on_checkbox_clicked, nodeClass='Root', node=nuke.root())
nuke.addonScriptClose(lambda: nuke.removeKnobChanged(on_checkbox_clicked, nodeClass-'Root', node=nuke.root())

but if I create a grade node named Grade1 and run the below code in script editor it works fine.
try:
    node = nuke.toNode('Grade1') 
    nuke.delete(node)
    node.fullname() # <-- should throw error
except ValueError:
    print(error caught.)

Answer: Certainly seems like an internal nuke issue. Which nuke are you running? I know 11 and 12 will almost always spit out some kind of python error on close - either threading or something like this.
If your my_callbacks.py is being loaded by init/menu, try just adding the callback to the root node itself (rather than the global knob change process) with node.knob('knob_changed').setValue(YOUR CODE in string format)
In this case of course, the knob changed code will only fire on the Root node, and you'll have to run that setValue code in each script you want. You might be able to use init/menu and another callback (onScriptLoad) to accomplish that.
Title: Getting "ValueError: A PythonObject is not attached to a node" even when wrapped in try/except block but this works fine if run in Nuke Script editor
Tags: python,nuke

Question: My question is Foundary Nuke specific.
I have a tab added to Project Settings, that contains some data I can later access via the root node. Now since I have callback invoked by a checkbox knob I added to enable disable a custom knob I added to that tab I added to Project Settings Panel. It works fine. The problem is when I close nuke I get error:
Traceback (most  recent call last):
File "/system/runtime/plugins/nuke/callbacks.py", line 127, in knobChanged
    _doCallbacks(knobChangeds)
File "/system/runtime/plugins/nuke/callbacks.py", line 44, in _doCallbacks
  for f in list:
ValueError: A PythonObject is not attached to a node

Now this error happens if I have a callback function added to the checkbox knob like this:
my_callbacks.py
import nuke

def on_checkbox_clicked():
    try:
        root_node = nuke.root()
        if not root_node:
            return
    except ValueError as er:
        print(er)

nuke.addKnobChanged(on_checkbox_clicked, nodeClass='Root', node=nuke.root())
nuke.addonScriptClose(lambda: nuke.removeKnobChanged(on_checkbox_clicked, nodeClass-'Root', node=nuke.root())

but if I create a grade node named Grade1 and run the below code in script editor it works fine.
try:
    node = nuke.toNode('Grade1') 
    nuke.delete(node)
    node.fullname() # <-- should throw error
except ValueError:
    print(error caught.)

Answer: Have you tried using nuke.thisNode() in your callback?
and reduce to nuke.addKnobChanged(on_checkbox_clicked, nodeClass='Root')
Like you I'm confused by this error, sometimes it appears but it shouldn't and when it should appear it doesn't...
Title: Is there a way to specifically web scrape and get the data of heights that is not listed in text?
Tags: python,html,web-scraping,hidden

Question: I'm web scraping a bunch of heights for listed athletes. I have written the code to get the heights but after inspecting element, I noticed that under text the height is written in feet, but in "data-sort" that height is listed in inches. Both of these are in the td tag in class "heights". However when I use "get_text()" or .text to remove the html elements it only prints out the height in feet and removes the hidden height in inches. Is there a way I can get the height listed in inches because that will make it easier to the do math.
Here is an example of what I'm web scraping, I want remove everything and only get the height in inches which will be [79,85,74... in this case.
<td class="height" data-sort="79">6-7</td>
<td class="height" data-sort="85">7-1</td>
<td class="height" data-sort="74">6-2</td>

#This is my code

from bs4 import BeautifulSoup
import requests 

urls=['https://goduke.com/sports/mens-basketball/roster']

ListData=[]
for x in range(len(urls)):
    page=requests.get(urls[x]).text
    pagesoup=BeautifulSoup(page,'html.parser')
    h=pagesoup.find_all('td', class_="height")
    ListData.append(h)
NewList=[]
for b in range(len(ListData)):
    new=[]
    for x in ListData[b]:
        print(x.text)

Answer: If you use css selector you can simply pass the first class name.
from scrapy.selector import Selector
Title: Tkinter filename float not incrementing correctly
Tags: python,tkinter

Question: I am trying to create a program that loads the next file numerically upon the press of a button. In order to check what the next file is I simply run a while loop checking for a file name in .1 increments. I.E, I have a file labeled 1.1, 1.3, and 1.4, I want the button to load them in that order, skipping 1.2 because it doesn't exist.
The problem arising is that for some reason instead of just incrementing the value that gets checked by .1, it is incrementing it by .10000000000(some random number here), making it so I can't check the files properly
import tkinter as tk
from tkinter import *

mainscreen = tk.Tk()

cuename = tk.Entry(mainscreen)


def loadnext_cue():
    cued = float(cuename.get())
    next_cue = cued + .1
    while os.path.exists("%s .txt" % str(next_cue)) == False:
        print(next_cue)
        next_cue += 0.1
        if next_cue >= 4:
            error = Label(mainscreen, text="No Higher Cue")
            error.pack(side=TOP)
            time.sleep(2)
            error.destroy()
            break
    if os.path.exists("%s .txt" % str(next_cue)):
        callup = next_cue
    
        with open("%s.txt" % str(callup), "r") as loadentry:
            quote = loadentry.read()
        T.delete("1.0", END)
        T.insert(END, quote)


Because of what I'm using the program for I know for now at least the file names will only go into the 10's place decimal (I.E. 1.2)
So, starting with file number 1.0, looking for the next file up, the output I'm getting looks something like this:
1.1
1.2000000000000002
1.3000000000000003
1.4000000000000004
1.5000000000000004
1.6000000000000005
1.7000000000000006
1.8000000000000007
1.9000000000000008
2.000000000000001
2.100000000000001
2.200000000000001
2.300000000000001
2.4000000000000012
2.5000000000000013
2.6000000000000014
2.7000000000000015
2.8000000000000016
2.9000000000000017
3.0000000000000018
3.100000000000002
3.200000000000002
3.300000000000002
3.400000000000002
3.500000000000002
3.6000000000000023
3.7000000000000024
3.8000000000000025
3.9000000000000026
4.000000000000003

It seems to go up by exactly .1 on the first iteration but then does whatever it wants after that. Any and all help would be appreciated

Answer: Your filenames are of the form x.y where x and y are both integers, but you are using a Python float to store the filename. Floats in Python represent decimal numbers with finite precision, and when you perform arithmetic you can experience a loss of precision. That is why when you perform 1.1 + 0.1 you get a very small error in the result -- the 'random number'.
I suggest to modify your code to store the parts x and y in separate variables as integers. You can increment the parts separately according to your logic, and then compose them into the string filename when needed.
Title: What algorithm would be most efficient when trying to find the nearest city given a set of coordinates?
Tags: python,algorithm,data-structures,nearest-neighbor

Question: I have a dataset which contains the longitude and latitude of the 1000 largest US cities. I'm designing an API which returns the user's nearest city, given an input of the user's longitude/latitude.
What is the most efficient algorithm I can use to calculate the nearest city? I know that I can use the haversine formula to calculate the distance between the user's coordinate and each cities, but it seems inefficient to have to do this for all 1000 cities. I've previously used a k-d tree to solve nearest neighbour problems on a plane - is there a similar solution that can be used in the context of a globe?
Edit: keeping this simple - distance I'm looking for is as the crow flies. Not taking roads or routes into account at this stage.

Answer: This answer is very similar to that of ckc.
First, spilt the 1000 cities in 2 groups : a big one located located between Canada and Mexico and the few others cities located outside this rectangle (i.e Alaska, Hawai, ...).
When processing coordinates, check if they belong to the small group : in this case, no optimisation needed.
To optimize the other case, you may divide the map in rectangles (example 5°lat x 7° lon) and associate to each rectangle the list of cities belonging to each rectangle.
To find the nearest city, consider the rectangle R containing the point.
Compute the distance to the cities of the rectangle.
Process the 8 rectangles adjacent to R by computing the distance of the point to each rectangle : you may then eliminate the adjacent rectangles whose distance is greater than the best distance already found.
Iterate the process to a next level, i.e. the next crown (rectangles located on the outside of the area composed of 5x5 rectangles whose center is R).
Title: What algorithm would be most efficient when trying to find the nearest city given a set of coordinates?
Tags: python,algorithm,data-structures,nearest-neighbor

Question: I have a dataset which contains the longitude and latitude of the 1000 largest US cities. I'm designing an API which returns the user's nearest city, given an input of the user's longitude/latitude.
What is the most efficient algorithm I can use to calculate the nearest city? I know that I can use the haversine formula to calculate the distance between the user's coordinate and each cities, but it seems inefficient to have to do this for all 1000 cities. I've previously used a k-d tree to solve nearest neighbour problems on a plane - is there a similar solution that can be used in the context of a globe?
Edit: keeping this simple - distance I'm looking for is as the crow flies. Not taking roads or routes into account at this stage.

Answer: You can split the map into squares that do not overlap and they cover the whole US map (i.e., you will have a grid). You will number the squares using the coordinates of their upper left corner (i.e., each one will have a unique ID) and you will do a preprocessing where each city will be assigned with the ID of the square where it belongs. You will find the square where the user lies into and then you will check only the cities that lie into this square and the ones that are one step from this (total: 9 squares). If these are empty of cities, you will check the ones that are two steps of it etc. In this way, on average you will check much less cities to find the closest
Title: Python compare type of (str, enum) classes
Tags: python,python-3.x,enums

Question: I have multiple enums defined with
from enum import Enum

class EnumA(str, Enum):
    RED = "red"

class EnumB(str, Enum):
    BLUE = "blue"

How do I compare the type of these classes/enums with say x=EnumA.RED? The following doesn't work.
type(x) is enum
type(x) is EnumType
type(x) is Enum

I don't want to compare the classes directly, since I have a lot of enums.

Answer: To know the type of the variable you should compare it with EnumA or EnumB, example :
type(x) is EnumA
Title: python vending machine program-
Tags: python,printing

Question: The program allows the user to enter money and select an item that outputs the price.
Towards the end, the if statements, if purchase_choice == 1: print("************ The item costs $1.00 ************") and the following statements after that one, is not printing in the output. Can someone help me?
Here's the code.
print("*********************************")
print("Welcome to Vending Machine Bravo!")
print("*********************************")
print("If you would like to make a selection, please insert the appropriate currency into the machine.")


Currency deposit 
num_5Dollars = 5.00
num_Dollars = 1.00
num_Quarters = .25
num_Dimes = .10
num_Nickels = .05
num_Pennies = .01 

print()
   print("Please enter:")
if num_5Dollars == 5.00:
   print("5.00 for $5 bills")
if num_Dollars == 1.00:
   print("1.00 for $1 bills")
if num_Quarters == .25:
   print(".25 for Quarters")
if num_Dimes == .10:
   print(".10 for dimes")
if num_Nickels == .05:
   print(".05 for nickels")
if num_Pennies == .01:
   print(".01 for pennies")

user_val = float(input())


if int(user_val) == 0:
   print("0 to cancel")

print("At any point if you wish to cancel operation or go back to last menu, please enter  0. Thank you!:")
print()
print(int(user_val))
print()
print("************ Total money in machine is: ", user_val , "************")

purchase item selection  
Skittles = {'type' '1''Price': 1.00}
Reeses = {'type' '2' 'Price': 1.19}
M_and_M = {'type' '3' 'Price': 1.50}
Chex_Mix = {'type' '4' 'Price': 0.99}
Honey_Bun = {'type' '5' 'Price': 1.99}
types = [Skittles, Reeses, M_and_M, Chex_Mix, Honey_Bun]

global type
type = [Skittles, Reeses, M_and_M, Chex_Mix, Honey_Bun]

print()
print("At any point if you wish to cancel operation or go back to last menu, please enter 0. Thank you!:")
print()

print("If you would like to purchase:")
print()
print("Skittles - type '1', (Price = $1.00)")
print("Reeses - type '2', (Price = 1.19)")
print("M_and_M - type '3', (Price = $1.50)")
print("Chex_Mix - type '4', (Price = $0.99)")
print("Honey_Bun - type '5', (Price = $1.99)")

print()
purchase_choice = input()
print("Your enter is:", purchase_choice)

if user_val == 0:
    print('Item selection stopped')

else:
    if purchase_choice == 1:
        print("************ The item costs $1.00 ************")
    if purchase_choice == 2:
        print("************ The item costs $1.19 ************")
    if purchase_choice == 3:
       print("************ The item costs $1.50 ************") [tag:tag-name]
    if purchase_choice == 4:
       print("************ The item costs $0.99 ************")
    if purchase_choice == 5:
       print("************ The item costs $1.99 ************")

Answer: You need to convert the user input string to an integer.
purchase_choice = int(input())
Title: attrs - how to validate an instance of a Literal or None
Tags: python,python-attrs

Question: This is what I have. I believe there are two problems here - the Literal and the None.
from attrs import frozen, field
from attrs.validators import instance_of

OK_ARGS = ['a', 'b']

@field
class MyClass:
    my_field: Literal[OK_ARGS] | None = field(validator=instance_of((Literal[OK_ARGS], None)))

Error:
TypeError: Subscripted generics cannot be used with class and instance checks

Edit: I've made a workaround with a custom validator. Not that pretty however:
def _validator_literal_or_none(literal_type):
    def inner(instance, attribute, value):
        if (isinstance(value, str) and (value in literal_type)) or (value is None):
            pass
        else:
            raise ValueError(f'You need to provide a None, or a string in this list: {literal_type}')
    return inner

Answer: You can’t do isinstance() checks on Literals/Nones and that’s what the is_instance Validator is using internally (it predates those typing features by far).
While we’ve resisted adding a complete implementation of the typing language due to its complexity, having one dedicated to such cases mind be worth exploring if you’d like to open an issue.
Title: Selenium Python script works only command line php, but not in browsers
Tags: python,php,nginx,selenium-webdriver

Question: I have nginx, php server, and python selenium installed, my python selenium script works perfectly in command line php, but not via nginx server browser. Tried some python codes, all work  in  browsers except selenium code.
No error in browsers, curl.
exec.php
$selenium = ('python3 /var/www/html/selenium/test.py');
echo shell_exec($selenium);


test.py
#!/usr/bin/env python3
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('--headless')
driver = webdriver.Chrome(options=options)
driver.get('http://localhost/info.php')
print(driver.title)
driver.quit()


Tried in nginx  root directory, with php  unix and tcp socket

Answer: Downgraded from selenium 4 to selenium 3 and chrome, chromderiver 74. now it works in browser too.
Title: print the pixel coordinates of a specific point in the raster
Tags: python,rasterio

Question: To print the pixel coordinates of a specific point in the raster, i used the index() method assuming that The index() method takes the x and y coordinates of the point in geographic coordinates and returns the corresponding row and column indices of the point in the raster.
I want to double-check that.
Is this the best way to handle it?
I'm a beginner, and I'm still unsure about when and how to use the affine transformation.Is it necessary to perform the affine transformation before printing the pixel coordinate?
import rasterio

with rasterio.open("LC08_L2SP_190037_20190619_20200827_02_T1_ST_B10.TIF") as data:
    print(data.crs)
    longitude, latitude = 13.3886, 52.5174
    row, col = data.index(longitude, latitude)
    print("Pixel coordinates of point ({}, {}): ({}, {})".format(longitude, latitude, col, row))

Answer: This code uses the rasterio package to read in a GeoTIFF file and extract information about its coordinate reference system (CRS) and the pixel coordinates of a specified latitude and longitude point.
The code first opens the GeoTIFF file using the rasterio.open() function and stores the resulting DatasetReader object in the variable data. It then prints out the CRS of the raster using the crs attribute of the DatasetReader object.
Next, the code specifies a longitude and latitude point (longitude, latitude = 13.3886, 52.5174) and uses the index() method of the DatasetReader object to convert the point's coordinates to the corresponding row and column indices in the raster. The resulting pixel coordinates are then printed out using string formatting.
Note that the index() method assumes that the input coordinates are in the same CRS as the raster. If the input coordinates are in a different CRS, you may need to transform them using the rasterio.transform module's transform() or reproject() functions before calling index().
Title: Spoof ongoing datetime in Python
Tags: python,datetime

Question: I am looking for the simplest solution to spoof live datetime, specifically, I would like it to start at a specific time, say 2023-01-03 15:29, and make it go on, so that the clock is ticking, so to speak.
There are plenty of ways to spoof current datetime, but I haven't find a way that would do so continuously, so the fake time keeps moving.

Answer: My first thought is, instead of trying to 'spoof' datetime, just perform a "translation".
Basically, you just need to calculate the datetime you need to subtract from the current datatime.now() to reach your desired date.
desired_time = datatime.now - translation ==>
translation = datetime.now - desired_time
After that you can simply call datetime.now() - translation which will effectively progress the clock of your desired date.
Hope that makes sense and can work for you!
Title: Is there any python equivalent of google guava loading cache?
Tags: python-3.x,caching

Question: I am looking for an in-memory loading cache which is compatible with python and provides specially these functionalities:

Time of reloading
Method of reloading
Thread safe

I found only in-built python libraries.

Answer: You included the guava tag on your question, but Guava is a Java library, so it's almost certainly not relevant to what you need. I would suggest removing that tag.
Title: Can't add ManyToMany relation in admin
Tags: python,django,django-models,django-views,django-templates

Question: I have following model created in Django:
class Content(models.Model):
    publish_date = models.DateTimeField(auto_now_add=True)
    name = models.CharField(max_length = 100, blank = False, default='name')
    summary = models.CharField(max_length=400, blank=True, default='summary')
    description = models.TextField(blank = True, max_length=5000, default = LOREM_IPSUM)
    author = models.ForeignKey(User, default=None, on_delete=models.CASCADE)
    rules = models.TextField(blank = True, default='rule set')
    parent = models.ManyToManyField(to = 'self', related_name="child", symmetrical = False, blank=True)

I added four Content objects through Django admin:
Project1
Task1
Task2
Task3

And set parent to Project1 for all TaskX. When I want to display all the content with detailed parent atrribute it turns out to be None.
views.py
def display_ideas(request):
    ideas = Content.objects.filter(name="Task3")
    return render(request, 'display_ideas.html', context = { 'ideas' : ideas })

** display_ideas.html **
<div class="container bg-success">
{% for idea in ideas %}
    <div class="container bg-danger">
      <h2>Name:</h2>
        {{ idea.name }} has parent: {{ idea.parent.name }}
    </div>
{% endfor %}
</div>

The output is:
Name:
Task3 has parent: None
What am I doing wrong? All migrations are done and the site is up and running.

Answer: Since its a ManyToMany field content object may have more than 1 parent, thats why {{ idea.parent.name }} Does not work. You would have to iterate through each parent as you have done here for ideas {% for idea in ideas %}, to show the parent attributes.
Title: Replacing/substituting part of a string using Regex in Python
Tags: python,regex

Question: I'm tidying a string in Python and need to substitute some of the text (following a certain rule) using Regex. In the string (copied below), a place is usually mentioned followed by a comma and then the city's associated mortality rate. The next place is separated with a semi-colon. However there are some examples where the semi-colon is missing and I need to use Regex to add that semi-colon back in (e.g. 'Plymouth, 19 Portsmouth, 15' should be 'Plymouth, 19; Portsmouth, 15').
The text is as follows:

Birkenhead, 16; Birmingham, 15; Blackburn, 16; Bolton, 18 ; Bradford, 16 ; Brighton, 14 Bristol, 20; Cardiff, 25 ; Derby, 12 ; Halifax, 20; Biddersfield, 21 ; Hull, 19 ; Leeds, 22 ; Leicester, 18 ; London, 17; Manchester,15 ; Norwich, 24; Nottingham, 21; Oldham, 18 ; Plymouth, 19 Portsmouth, 15 ; Preston, 23 ; Salford, 14 ; Sheffield, 16 ; Sunderland, 18; Wolverhampton. 30. The rate in Edinburgh was 14 ;in Glasgow, 23 ; and in Dublin. 22.

I've tried using re.sub() for this using the following formula and using non-capture sets but am doing something very horribly wrong!
mystring = [the string here]
re.sub("(?:[0-9])?\s(?:[A-Z0-9]?)", ";", mystring)

Is anyone able to help me fix this?
Thank you!

Answer: Thanks all! In the end, with some experimenting, I have found a different solution:
re.sub("(?<=\d)[ ;]+", "; ", mystring)
Effectively, we just look for all cases of one or more spaces and/or semi-colons which are preceded by a number (using a look-back) and then replace the match with ; .
Title: choose checkbox in QListWidgetItem
Tags: python,pyqt5

Question: I have this code with pyqt5:
elif (content1 == "next"):
        todos = ["one" , "two" , "three" , "four", "five"]
        self.todo_listWidget.show()
        for todo in todos:
            item = QListWidgetItem(todo)
            item.setFlags(item.flags() | QtCore.Qt.ItemIsUserCheckable)
            item.setCheckState(QtCore.Qt.Unchecked) 
 
            self.todo_listWidget.addItem(item)

And I want if i chose one item make something
like if item (x) is checked then do something
how can i do this?
thanks

Answer: Connect the itemChanged signal of self.todo_listWidget with a function that takes the item as its argument.

This signal is emitted whenever the data of item has changed.

In the function, you can check the resp. item's checkState and proceed accordingly.
An alternative is itemActivated. Which one works better for you depends on if you want to react to all (also programmatical) changes of the check state or only to user interaction.

This signal is emitted when the item is activated. The item is activated when the user clicks or double clicks on it, depending on the system configuration. It is also activated when the user presses the activation key (on Windows and X11 this is the Return key, on Mac OS X it is Command+O).
Title: Time Complexity of this program solving the coin change problem
Tags: python,loops,recursion,iteration,time-complexity

Question: I have created a program shown below, and I am confused as to how to figure out its time complexity. Is it O(ntarget/min(coins)) because a for loop is created each time the function is called, and the function is called target/min(coins) times?
The program solves the coin change problem (although not in a very efficient way!):
You are given an array of coins with varying denominations and an integer sum representing the total amount of money; you must return the fewest coins required to make up that sum.
For this problem
Code:
def count(coins: list[int], target):
    def helper(coins, target, vals = [], answers = set()):
        if target<0:
            vals.pop()
        elif target==0:
            vals.sort()
            answers.add(tuple(vals))
            vals.clear()
        else:
            for coin in coins:
                helper(coins, target-coin, vals+[coin])
            return len(answers)
        
    coins.sort()
    if (answer:=helper(coins, target)): 
        return answer
    else:
        return -1

print(count([2, 5, 3, 6], 10)) # 5
print(count([1, 3, 5, 7], 8)) # 6


Attempt at code explanation (explained for print statement 2):
It starts with finding the most amount of coins that is needed to reach the target or go higher( (1, 1, 1, 1, 1, 1, 1, 1) in this case).
From there it iterates over all the possible values for the last row ((1, 1, 1, 1, 1, 1, 1, 3), (1, 1, 1, 1, 1, 1, 1, 5), (1, 1, 1, 1, 1, 1, 1, 7)). If the sum of the numbers is negative, it removes that value and tries again with a different value.
It adds any solutions (where sum(vals)==target)to the set answers (after sorting it) to prevent duplicates from being counted.
Then it moves to the higher row ((1, 1, 1, 1, 1, 1, 1), (1, 1, 1, 1, 1, 1, 3), (1, 1, 1, 1, 1, 1, 5), (1, 1, 1, 1, 1, 1, 7)). And repeats.
Can anyone explain what is the time complexity of the program along with why? Thanks!

Answer: I have realized that my initial thought was correct: the time complexity is indeed O(nceil(target/min(coins))).
If you take the smallest value in coins, then it will have to call itself ceil(target/min(coins)) times, and with each call start a new for loop, which is an O(n) operation. Therefore, it becomes O(nceil(target/min(coins))) time complexity
Title: Get all  contents in Python Selenium
Tags: python,html,selenium-webdriver

Question: Say that I have a piece of HTML code that looks like this:
<html>
    <body>
        <thspan class="sentence">He</thspan>
        <thspan class="sentence">llo</thspan>
    </body>
</html>

And I wanted to get the content of both  and connect them into a string in Python Selenium.
My current code looks like this:
from selenium import webdriver
from selenium.webdriver.common.by import By

browser = webdriver.Chrome()

thspans = browser.find_elements(By.CLASS_NAME, "sentence")
context = ""
for thspan in thspans:
    context.join(thspan.text)

The code can run without any problem, but the context variable doesn't contain anything. How can I get the content of both  and connect them into a string in Python Selenium?

Answer: context += thspan.text instead of using context.join(thspan.text) just like @Rajagopalan said