Myashka/SO-Python_QA-filtered-2023-tanh_score-after_2023_02 huggingface dataset:
Title: PySide6: How to clear the values of all QPlainTextEdit and QComboBox elements after inserting data into a database?
Tags: python,sql,pandas

Question: Hy, i want to clear all qtextedit,qcombobox etc.., when a product is inserted in database,but i dont know, how to do it.
Right now this is my function
 def commit(self):
   
    self.inserebd()
    tree_data=self.dadostree()
    self.contarserie()
 

    sql = """INSERT INTO "Teste"("DataEntrada","Código de Barras","Numero de    serie","Categoria","Marcasss","Modelo","Fornecedores","Estado","Projeto","Valor/Uni","Quantidade")       VALUES  (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)"""`
    
    for paren_data, children_data in tree_data:   
        
        codigo_barras = paren_data
        if not children_data:
            
            dados=(self.data,codigo_barras,'No',self.idC,self.id,self.modelo,self.idf,self.estado,self.projeto,self.valor,self.quantidade)
            cursor.execute(sql,dados)
            con.commit()
            QtWidgets.QMessageBox.information(self,"Sucesso",f" {self.total_serie} Produtos Inseridos")```
                
   

Tree_data is this:
[('b', ['b']), ('c', ['c'])]
the problem is i dont know were to put this part when i delete all things:
 self.ui.treeWidget.clear()
    self.ui.textmodelo.clear()
    self.ui.textEdit.clear()
    self.ui.textEdit_Data.clear()
    self.ui.textEdit_Preco.clear()
    self.dodo=[] 
    self.ui.comboBox.setCurrentIndex(self.default_item_index)
  self.ui.comboBox_Marca.setCurrentIndex(self.default_item_index)

Because if i put inside the FOR its gonna cause problems when i insert more than one product, but if i put outside the for its gonna cause problems to, because if the insert fails, its gonna delete all data to.
And i just want to delete the textedit, comboBox etc.. when the data is already inserted.
I know its a bit confuse, and its my first time posting a question, but if you have any question ask, im gona try clarify as much as I can.
thank you for the help.

Answer: I finaly got it, to resolve my problem i created an numofrows = cursor.rowcount, the rowcount count how many rows were inserted, for those o have the same problem try use it
Title: How to set `filterset_fields` in Django-Filter and Django REST Framework?
Tags: python,django,django-rest-framework,django-filter

Question: When I set the fiterset_fields like below,
class SubCategoryViewSet(viewsets.ReadOnlyModelViewSet):
    filter_backends = [DjangoFilterBackend]
    filterset_fields = ["category_id"] # single underscore

I get this response when a category with the specified ID doesn't exist
{
    "category_id": [
        "Select a valid choice. That choice is not one of the available choices."
    ]
}

But when I set it like
class SubCategoryViewSet(viewsets.ReadOnlyModelViewSet):
    filter_backends = [DjangoFilterBackend]
    filterset_fields = ["category__id"] # notice the double underscore

I get a [] when a category with the specified ID doesn't exist.
Why does this happen?
What is the right way to do this?

Answer: Lets first see the difference between them.
Case 1
You are defining a category_id field for filtering, and django is assuming that category_id is a field of Model. The viewset here expects the category_id field to be present in the request and to contain a valid choice from the available choices. Otherwise it will generate the error.
Case 2
You are defining a category__id field for filtering, and django is assuming that Category Model has a field called id that will be used for filtering purposes. It filters results based on id attribute in Category model and if the result set is empty, it returns empty list as []. Note: this approach is recommended due to the flexibility it provides.
Title: How to set `filterset_fields` in Django-Filter and Django REST Framework?
Tags: python,django,django-rest-framework,django-filter

Question: When I set the fiterset_fields like below,
class SubCategoryViewSet(viewsets.ReadOnlyModelViewSet):
    filter_backends = [DjangoFilterBackend]
    filterset_fields = ["category_id"] # single underscore

I get this response when a category with the specified ID doesn't exist
{
    "category_id": [
        "Select a valid choice. That choice is not one of the available choices."
    ]
}

But when I set it like
class SubCategoryViewSet(viewsets.ReadOnlyModelViewSet):
    filter_backends = [DjangoFilterBackend]
    filterset_fields = ["category__id"] # notice the double underscore

I get a [] when a category with the specified ID doesn't exist.
Why does this happen?
What is the right way to do this?

Answer: Why does this happen? What is the right way to do this?

This is partly one of the limitations of Django. If you use a fieldname, like category_id, is one, it will first retrieve a collection if distinct values. If the entered value is not in it, then it can raise the exception.
If you work with JOINs, however, as is the case with category__id, where you join on an extra model, that trick apparently is not implemented. It might be possible to do this, although it can be tricky for multiple JOINs, especially if there is some other filtering active that filters on another JOIN that partly overlaps. So since Django in that case can not derive a unique set of values, it fallsback on returning an empty list.
I would advise to use the first one, so category_id. Indeed, if someone enters a wrong category_id, it is more informative to say that the category_id does not exist, and not return an empty list. In the latter case it gives the impression that the category exists, but that there is simply no subcategory with that category. Usually providing more information is better, of course if it is not "sensitive".
Title: How to import Python methods of existing classes selectively?
Tags: python,pandas

Question: Consider the following example that imports additional Pandas methods from the pyjanitor package.
import pandas as pd
import janitor

df = pd.DataFrame({
    'Column One': [1, 2, 3],
    'Column-Two': [4, 5, 6],
    'Column@Three': [7, 8, 9],
})

df_cleaned = df.clean_names()
print(df_cleaned)
#>    column_one  column_two  column@three
#> 0           1           4             7
#> 1           2           5             8
#> 2           3           6             9

Is there a way to import methods from Python packages, modules, or other sources more transparently/explicitly (to know which method came from which package) and/or selectively (to import only the methods of interest, e.g., only .clean_names() but not the remaining ones)?
Note: I do not want to import a method as a function and use it as a function. I would like to control what methods are imported.

Answer: pyjanitor is extremely unusual. Adding methods to a class defined somewhere else is technically legal in Python, but almost no one does it. Most of the time, you can safely assume that any methods of a class come from the module that defines the class, or are inherited from parent classes.
It is not possible to selectively import only some of an object's methods. An object has the same methods no matter what code is looking at it. (It might suddenly grow more methods if a module like pyjanitor adds more, but if so, those new methods will be available to any code that accesses the object after that point.)
Title: Simple discord.py mp3 bot not playing music
Tags: python,ffmpeg,discord,discord.py,pynacl

Question: Discord bot dosent play anykind of music, neither mine nor other bots codes work he just dosent play music. idk maybe i am just stupid BUT IT DOSENT WORK WHY
import discord
import time
import os
from discord.ext import commands

BOT_TOKEN = "" # put token here

intentss = discord.Intents.default()
intentss.message_content = True
intentss.voice_states = True

bot = commands.Bot(command_prefix=";;", intents = intentss)

OPUS_LIBS = ['libopus-0.x86.dll', 'libopus-0.x64.dll', 'libopus-0.dll', 'libopus.so.0', 'libopus.0.dylib']

@bot.command()
async def join(ctx):
    channelVC = ctx.author.voice.channel
    await channelVC.connect()

@bot.command()
async def leave(ctx):
    await ctx.voice_client.disconnect()

@bot.command()
async def play(ctx):
    voice = ctx.guild.voice_client
    mloc = 'C:/Users/Lukas/Desktop/Bot Bethoven/Youtube/test.mp3'
    voice.play(discord.FFmpegPCMAudio(executable = "C:/ffmpeg/bin/ffmpeg.exe", source = mloc))


bot.run(BOT_TOKEN)

it just give error codes:
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: ClientException: Not connected to voice.
INFO     discord.player ffmpeg process 22896 has not terminated. Waiting to terminate...
INFO     discord.player ffmpeg process 22896 should have terminated with a return code of 1.

intresting thing when i put join and play in the same command it dosent show not connected to voice error, actually it dosent show anything and neither plays mp3 file
i tryed reinstalling: PyNaCl, FFmpeg, Visual Code, updating python.
nothing helps. this problem started after i did 1 month break from coding bot, before that it worked fine.
i'm thinking problem is something with my pc, maybe path or something dosent work(because it worked month ago and i didnt do anything to code), i tryed checking but everything seems normal,

Answer: For me it was something with diffrent versions of python's i used, they did something to eatch other soo it just broke
fix for me was to DELETE everything asociated with Python (that includes modules) and reinstall it.
Title: What is the advantage of using a hash table for storing strings in spaCy's vocabulary?
Tags: python,dictionary,nlp,hashtable,spacy

Question: Why not instead just have a dictionary of keys as the strings and their values as their properties/attributes? Is it a time-complexity advantage, or is there also a memory-saving advantage?
I have a basic/general understanding of hash-tables and dictionaries.
I am currently going through chapter 1 of the spaCy course course.spacy.io where it says, "Pipeline packages include a strings.json that stores the entries in the pipeline’s vocabulary and the mapping to hashes. This allows spaCy to only communicate in hashes and look up the corresponding string if needed."
I basically want some more clarification on what this looks like.

Answer: It's basically a time-complexity advantage，meanwhile reduce memory usage.
When using key-value pairs, a linear search of the entire collection is often required to find a specific value, resulting in a time complexity that increases with the size of the data set.
I think lots of NLP models use hash to increase efficiency.
Title: Code producing incorrect numbers in Heat Exchange Simulation in Python
Tags: python,numpy,calculation

Question: I am coding a heat exchange simulation. The values that should be producced are T_hot_out = 65, T_cold_out = 35, and A = 22.26.
The following is the code I've been using. It prints T_hot_out = 36.9 and T_cold_out = 64.32. No error is raised, so I'm confused on where the computational error is coming from and how to fix it.
#problem 4
import numpy as np

# from text (original design assumptions)
T_cold_out0 =35
T_hot_out0 = 65

# from Table 1
T_hot_in = 160.0
T_cold_in = 20.0
Cp_hot = 5.2
Cp_cold = 2.6
w_hot = 1.0
k = 0.05
l = 0.002

# from problem 2
w_cold = 5.555555
A_design = 22.26

def integrate(f, x0, x1, N):
  h = (x1 - x0) / (N-1)
  xvalues = np.linspace(x0,x1,N)
  yvalues = f(xvalues)
  F = np.zeros(N)
  for i in range (1,N):
      F[i] = h/2 * (yvalues[i-1]+yvalues[i])+F[i-1]
  return xvalues,F
  

def bisection(f, y, x0, x1, eps, max_iter=1000):
  x1_n = x1
  x0_n = x0
  T = (x1+x0) / 2.0
  f_T = f(T)
  for i in range(max_iter):
    T_n = (x0_n+x1_n)/2
    f_T_n = f(T_n)
    error = f_T_n - y
    if abs(error) <= eps:
      break
    elif  error > 0:
      x0_n = T_n
      x1_n = x1_n
    elif error < 0:
      x0_n = x0_n
      x1_n = T_n
  return T_n, error, (abs(error)<=eps)

def h_hot(T_n):
    return 0.9*(1+0.015*(T_n-70))
    
def h_cold(T_n):
    return 3.0*(1+0.02*(T_n-40))
    
def h_overall(T_cold, T_hot):
    return 1/((h_hot(T_hot))**-1 + (h_cold(T_cold))**-1 +(k/l)**-1)
    # equation (2) using h_hot() and h_cold()
    

def T_hot(T_cold, T_hot_out):
    return  T_hot_out + ((Cp_cold*w_cold)/(Cp_hot*w_hot))*(T_cold - T_cold_in) # from problem 3
    
def T_cold_out(T_hot_out):
    return  T_cold_in + (Cp_hot*w_hot)/(Cp_cold*w_cold)*(T_hot_in-T_hot_out) # similar to T_hot()
  
def A(T_hot_out):
    def f2(T_cold):
      return ((Cp_cold*w_cold)/h_overall(T_cold, T_hot(T_cold, T_hot_out)))*(1/(T_hot(T_cold, T_hot_out)-T_cold))  # from equation (9)
    T_cold, area = integrate(f2,T_cold_in,T_cold_out(T_hot_out), 1000)
    return area[-1]
  
T_hot_out, err, success = bisection(A, A_design, T_cold_in, T_hot_in, 0.001, max_iter=1000)
if not success:
    print("Bisection has failed!")
    
print("T_hot_out = {:.2f} (instead of {:.2f})".format(T_hot_out, T_hot_out0))
print("T_cold_out = {:.2f} (instead of {:.2f})".format(T_cold_out(T_hot_out), T_cold_out0))
print("A = {:.2f} (error = {:.4e})".format(A(T_hot_out), A(T_hot_out) - A_design))

Answer: It seems like your T_hot_out and T_cold_out values are being switched at some point, and it also seems like you have a rounding error somewhere.
Title: Invalid Element State Exception - import a file
Tags: python,user-interface,selenium-webdriver,xpath,automated-tests

Question: I'm writing an automated test that is supposed to import a series of files through the UI of a web application.  You click the "Import" button and it opens Windows Explorer.  I use the following XPATH in my test:
filename_field = //div[contains(@class, 'DocumentGrid')]//input[contains(@type, 'file')]
I'm getting an InvalidElementStateException.
Message: invalid element state: Element is not currently interactable and may not be manipulated
Here is a sample of my test code:
element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, Tags.filename_field))

element.send_keys(Keys.importFilePath)

I'm not sure why I'm getting this error.  This code worked with a different web application, but this one I'm working on now was built using the React framework and the other was not. Not sure if that makes a difference.

Answer: I faced some similar issues before.
For me it was another element blocking the element i wanted to click. In one case the cookiebar added a layer above the button so i could only interact with my desired element after accepting the cookies.
Since i dont know the structure of the page i can only suggest how to find such 'blocking' elements.
First i would recommend the browser dev tools and inspect every possible layout the page can have. Of course only the part you want to interact with. Check for Popup´s, cookiebars, userDialog´s so on.
After finding the issue the workaround will probably be self explainatory.
If that is not the solution it is also possible that the element did not finish loading entirely and needs some more time to complete, but since you are already waiting for 10 seconds i doubt that.
Title: How to set __len__ method for an itertools.chain object?
Tags: python,attributeerror

Question: Let's say I'm building an itertools.chain instance as follows:
from itertools import chain

list_1 = list(range(5, 15))
list_2 = list(range(20, 30))
chained = chain(list_1, list_2)

Now, since I already know the length of the lists contained in chained I can easily get the length of chained. How can I add the __len__ to chained?
I tried this:
full_len = len(list_1) + len(list_2)
setattr(chained, '__len__', lambda: full_len)

but it fails with the error
AttributeError: 'itertools.chain' object has no attribute '__len__'

Edit:
I need this to be able to display the progress of a long process with tqdm, which relays in the __len__ method to be able to show the progress bar

Answer: Create a new class, define the function for the new class and use it instead of the original.
Title: hangman game list help (python)
Tags: python,list

Question: I just would need a way to be able to end my hangman man game through check if every letter the has entered is in the list.
What I was thinking was whenever I append the letter they type and if it is in the actual letter list I would remove it from the actual letter list until there are no more elements in the list and compare it in an if statement to when it zero, and the user would win the game.
Would this be the most effective way?
import random 

Words = ["set", "scar", "cat"] 

Letters_Guessed= ""

Guess = random.choice(Words) 

ActualLetter = list(Guess) #use this to divide up the random word
print(ActualLetter)

Numb_guess = 0
max = 6 

HANGER = [
'''
  +---+
  |   |
      |
      |
      |
      |
=========
''',
'''
  +---+
  |   |
  O   |
      |
      |
      |
=========
''',
'''
  +---+
  |   |
  O   |
  |   |
      |
      |
=========
''',
'''
  +---+
  |   |
  O   |
 /|   |
      |
      |
=========
''',
'''
  +---+
  |   |
  O   |
 /|\  |
      |
      |
=========
''',
'''
  +---+
  |   |
  O   |
 /|\  |
   \  |
      |
=========
''',
'''
  +---+
  |   |
  O   |
 /|\  |
 / \  |
      |
=========
''']













letterAmount = ""
for letter in Guess:
        letterAmount += "_"
        print(letterAmount)


game = True
def guess(UserguessL): 
    global Numb_guess, game, letterAmount, ActualLetter, Letters_Guessed, max
    while game == True: 
        print(letterAmount)
        if UserguessL in ActualLetter: 
            UserguessL = input("Good job! What another word? ")
            Letters_Guessed += UserguessL
            letterAmount = ""
            print("Letters you have guessed: ", Letters_Guessed,)
            

        elif UserguessL not in ActualLetter:
            UserguessL = input("try again! ")
            Letters_Guessed += UserguessL
            Numb_guess += 1
            print("Letters you have guessed: ", Letters_Guessed, )
        print(HANGER[Numb_guess])
        print(letterAmount)
    if Numb_guess == max: 
            print("the word was", Guess)
            game = False
        


UserguessL= input("Welcome to Hangman, player! In this game you must guess a word with letters, correctly within a limited amount of guesses. \n What is the first letter you like to guess? ")
guess(UserguessL)

Answer: This is a method that probably takes more steps, but saves your original string as you may need it for something else:
Have two strings, "answer" and "guess". If a letter guessed by the user is in the "answer" string, add it to the "guess" string. Then, continue until the user guesses another letter. Do as last time, and repeat this process until all the letters are guessed. The string will only add letters if they are correct, so no reason to place it in the same location. When the two string lengths match, the player wins.
Hope this helps!
Title: How do i get my code to only select a .csv attachment and not the image from within the email txt?
Tags: python,email,outlook,email-attachments,office-automation

Question: I currently have the below code that works perfectly if the email is blank but if it comes as part of a chain and it has images (ie company logos in peoples signature etc) then it will only find those. How do i filter it to only select the .csv?
    import win32com.client
    import os
    import time
    outlook = win32com.client.Dispatch("Outlook.Application").GetNamespace("MAPI")
    folder = outlook.Folders.Item("myemail@gmail.com")
    inbox = folder.Folders.Item("Inbox")
    msg = inbox.Items
    msgs = msg.GetLast()
    print (msgs)
    print (msgs.Subject)

    Filter = "[SenderEmailAddress] = 'senderemail@gmail.com'"
    Filter = ("@SQL=" + chr(34) + "urn:schemas:httpmail:subject" +
                        chr(34) + " Like 'subjectofemail' AND " +
                        chr(34) + "urn:schemas:httpmail:hasattachment" +
                        chr(34) + "=1")

    Items = inbox.Items.Restrict(Filter)
    for Item in Items:
        for attachment in Item.Attachments:
            print(attachment.FileName)
            attachment.SaveAsFile("U:\Downloads\Foldername\Otherfolder\\" + attachment.FileName)
      
            timestr = time.strftime("%Y%m%d")

            old_name = r"U:\Downloads\Foldername\Otherfolder\\attachmentfilename.csv"
            new_name = r"U:\Downloads\Foldername\Otherfolder\\attachmentfilename " + timestr + ".csv"

            os.rename(old_name, new_name)


I have tried a few things ive seen on here but i am very new and have basically created this code from snippets of code i found and got it working with trial and error so im not sure what im doing really. Im trying to learn as i go but assume i need an ELI5 if possible. Thanks

Answer: There are two kind of attachments that should be detected:

Attachments with a specific extension - .csv. You can filter them by checking the Attachment.FileName property like you have already figured it out.

Embedded images that are used for the message body markup. You can check the message body content for the PR_ATTACH_CONTENT_ID MAPI property value (the DASL name is  "http://schemas.microsoft.com/mapi/proptag/0x3712001F"). Use the Attachment.PropertyAccessor.GetProperty method for retrieving the value and then check the attachment in the message body using the src attribute that matches the value of PR_ATTACH_CONTENT_ID set on the attachment. PR_ATTACH_CONTENT_ID corresponds to the Content-ID MIME header when the message is sent.
Title: Set values in pandas series to a random value based on condition
Tags: python,pandas

Question: I have a pandas series, and I'd like to set all the values that are 0 to a random number between 100 and 1000. Is there a direct way of doing that? I've only seen a roundabout way like so:
nums = np.random.choice(range(100, 1100), size = 924)

df.loc[df['Target']==0] = nums

The error I get back here is "Must have equal len keys and value when setting with an iterable".
Size is 924 because I know that's how many values I want to change.

Answer: You just have to specify the column as well. Switch the last line to
df.loc[df['Target']==0, 'Target'] = nums
Title: how to find exact modules of python which are there on my windows machine
Tags: python,installation,module

Question: I work on python. My laptop is going to change to a new laptop.
Is there a command which i can run on my old laptop to see which python modules are installed,
and by which commands have i installed them?
so that I can ran that file on my new laptop to install all of them.

Answer: Open a Powershell terminal and use pip like this pip freeze > requirements.txt this will list all your modules inside the file requirements.txt, then copy this file to your new computer and then on it run this to install all the previous modules pip install -r requirements.txt
Title: "GET / HTTP/1.1" 405 Method Not Allowed
Tags: python,python-3.x,rest,fastapi

Question: I had an issue with that I cann't use my Post request in FastAPI , I used the Post request but the terminal displayed the Get request and I don't know why.
And when I used the Body(...) to declare the dict type but it looked didn't work
from fastapi.params import Body
from pydantic import BaseModel

app = FastAPI()
# payload : input parameter of the function
# dict: type of data
#  được truyền vào bằng cách sử dụng FastAPI's Body dependency. ... được sử dụng để chỉ định rằng tham số 
# này là bắt buộc và không thể bỏ qua.
@app.post("/create")
def create_posts(payload: dict = Body(...)):
    # It will extract all the field from the body to the dict
    print(payload)
    return {"message": f"successfully created posts: {payload['title']}"}

@app.get("/")
def get_data():
    return {"message": "Hello World"} 

Here is my error
INFO:     Will watch for changes in these directories: ['T:\\FastAPI']
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [14464] using WatchFiles
INFO:     Started server process [3468]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     127.0.0.1:60812 - "GET /create HTTP/1.1" 405 Method Not Allowed
INFO:     127.0.0.1:60812 - "GET /cart.json HTTP/1.1" 404 Not Found
INFO:     127.0.0.1:60812 - "GET /create HTTP/1.1" 405 Method Not Allowed```

Answer: Your code works just fine. Most probably the only issue is that you in fact don't actually send a POST request.
Open http://localhost:8000/docs in your browser and try the request yourself.
Title: Export Jupyter Notebook to HTML with VS Code & Plotly figures
Tags: python,html,jupyter,vscode-extensions,plotly-python

Question: I'm struggling with the export of my ipynb-file to HTML via VS Code due to following requirements which I'm not able to combine:

The file contains plotly-figures which need to be rendered

The code should not be shown in the export file


My two approaches:

To render properly in HTML, I set
import plotly.io as pio
pio.renderers.default='notebook'
exporting via VS Code "Jupyter:Export to HTML" results in showing the figures properly, but also the input.

To hide the code-cells in the export file I can execute
jupyter nbconvert --to html --no-input file.ipynb

in the CLI.
This hides the input as required, but does not render the plotly figures, since the correct renderer-option is missing (I guess).
Adding
--WebPDFExporter.embed_images=True
did not help.


Each approach works as expected, but how to combine both?
First option would be to add an argument to VS Code Export function not to show input.
Second option would be to add an argument to cmd line that sets the correct renderer.
Any help appreciated

Answer: Silly me.
When using CLI-approach, renderer-variable from nb is taken into account (no matter if set gobally via plotly.io or within fig.show().
But notebooks needs to be saved (of course) when exported "externally" by CLI (which is not necessary when exporting directly via VS Code).
Since I'm quite new to VS Code, my assumption was that the console uses the active Kernel of the notebook which it does not.
Title: pip doesn't respond in the cmd or from venv, it just returns to the directory without error message
Tags: python,pip,python-venv

Question: i installed python3 along with its pip. but pip doesn't respond when called in CMD or from a venv. it just returns back to the directory without giving error message.
i tried to uninstall python and reinstall other versions but the problem persists.

Answer: There's a couple things that we can try, plus some of these commands can help us pinpoint the issue:

This one seems really obvious, but it's happened to me before.  Have you tried closing your command prompt and starting a fresh one?  Sometimes pip doesn't work until I start a new one command prompt session.  Some other times it got to the point where i had to reboot the whole computer first.

does anything happen when you type pip --version?  If this doesn't display then pip might not be installed correctly.

you mentioned that you added python to PATH but did you add pip?
3.1 Find the dir containing the pip executable, usually the Scripts directory
3.2 add the path to the Scripts directory to the PATH environment variable.  open up System Properties (type sysdm.cpl in the search or in run, Advanced tab) > Environment Variables > and add the path to the PATH variable
3.3 again, don't forget to close and reopen your command prompt and try pip --version again.


If none of this works try reinstalling python again, making sure to select the option to add Python to the PATH during install.
Title: What does socket.setdefaulttimeout have to do with Windows services written in Python?
Tags: python,windows,sockets,service

Question: I'm familiarizing myself with creating Windows services in Python, and all of the examples I've found here or elsewhere call socket.setdefaulttimeout(60.0) (or something similar). But none of these examples explain why this is being done...and I can't find any info regarding whether or not this has to do with Windows services or with the way they're implemented in Python.
I'm just trying to get a better handle of what's going on in my code. If anyone can explain why this is done / if it's needed, I'd appreciate the help.
For reference, here's a (partial) boilerplate implementation of a service class in Python
import socket
import win32event
from win32serviceutil import ServiceFramework
# some of the usual imports have been left out for the sake of brevity


class ExampleService(ServiceFramework):
    _svc_name_ = 'example_service'
    _svc_display_name_ = 'Example Service'
    _svc_description_ = 'Lorem ipsum dolor sit amet'

    def __init__(self, *args):
        super().__init__(*args)
        socket.setdefaulttimeout(60.0)  # <- this is the line in question
        self.svc_stop_event = win32event.CreateEvent(None, 0, 0, None)

    def SvcStop(self):
        ...

    def SvcDoRun(self):
        ...


Update:
I've removed the call to socket.setdefaulttimeout() from my code, and unsurprisingly nothing was broken. Good times!

Answer: Sockets have nothing to do with writing services.  But, sockets can be used inside of services. Obviously, you would have to study how the services are using the sockets in order to understand why a default socket timeout is being set.  Setting the timeout won't affect the service itself.
Title: Python: Importing pandas into main and modules?
Tags: python,python-3.x,pandas

Question: I have a python script with a structure that looks like the following:
/
  __init__.py
  main.py
  /modules
    fbm.py


I am attempting to split some functions out from main.py into fbm.py and then import fbm.py as a module using.
sys.path.extend([f'{item[0]}' for item in os.walk("./app/modules/") if os.path.isdir(item[0])])
import fbm

This is working.
What I am having trouble with is where I need to call pandas (or another import) from a module.
I'm currently importing pandas in main.py with import pandas as pd.
When I call the function from fbm an error is thrown as soon as it hits a reference to pd stating that NameError: name 'pd' is not defined. pd is defined in main.py, but not fbm.py. It thus works in main, but not in fbm.
So, my question is: Is it good and appropriate to import pandas as pd in both the main.py and each module which requires pandas?

Will this have an impact on memory usage, ie loading multiple copies of pandas
is there a better way to handle this?

Answer: It is appropriate to import pandas (or any other library) in both the main script and each module that requires it. This is a common practice and ensures that each module can function independently, which makes the code more modular and easier to maintain.
Regarding your concerns about memory usage: Python is smart enough to handle imports efficiently. When a module is imported for the first time, it's loaded and cached in the sys.modules dictionary. So, if the same module is imported again (even with a different alias), Python will use the cached module instead of loading it again and again. So, there won't be any significant impact on memory usage.
Title: Compiling Mitsuba 2 on Visual Studio 2022 (SCons error)
Tags: python,visual-studio-2022,scons

Question: I try to build the mitsuba-msvc2010.sln using Visual Studio 2022. I fail to successfully build this solution; most probably due to some changes in SCons. This is the error I obtain

AttributeError: 'SConsEnvironment' object has no attribute 'has_key':
1>  File "SConstruct", line 15: 1>    env =
SConscript('build/SConscript.configure') 1>  File "C:\Program Files
(x86)\Microsoft Visual
Studio\Shared\Python39_64\lib\site-packages\SCons\Script\SConscript.py",
line 662: 1>    return method(*args, **kw) 1>  File "C:\Program Files
(x86)\Microsoft Visual
Studio\Shared\Python39_64\lib\site-packages\SCons\Script\SConscript.py",
line 598: 1>    return _SConscript(self.fs, *files, **subst_kw) 1>
File "C:\Program Files (x86)\Microsoft Visual
Studio\Shared\Python39_64\lib\site-packages\SCons\Script\SConscript.py",
line 285: 1>    exec(compile(scriptdata, scriptname, 'exec'),
call_stack[-1].globals) 1>  File "build\SConscript.configure", line
111: 1>    if env.has_key('BOOSTINCLUDE'):

Was anybody able to build this and/or knows how to fix this problem?

Answer: Python 3 dictionary objects no longer supports the has_key method, so that SConscript is out of date.  The membership test "in" can be used instead, as in if 'BOOSTINCLUDE' in env:.  There may be other things that need to be "ported" to use modern Python and SCons, usually not too complicated.
Title: Model dump for XGBoost with categorical variable
Tags: python,machine-learning,xgboost,categorical-data

Question: I am trying to run XGBoost on a mix of categorical and numeric data.  While I am able to train the model and predict, I am unable to dump the model output to a df or json. Instead I get the error:
Check failed: is_categorical: A in feature map is numerical but tree node is categorical.

I have no issues if only using numeric data and same problem occurs if using numpy and setting feature_type directly.
The toy example below shows the error I am receiving.  Any suggestions on what I am doing wrong would be appreciated!
import numpy as np
import pandas as pd
import xgboost as xgb

X1 = [0, 2, 3, 1, 4, 5]
X2 = np.random.rand(6)
X3 = np.random.rand(6)*2
X = pd.DataFrame(data=np.column_stack((X1, X2, X3)), columns=['A', 'B', 'C'])
X['A'] = X['A'].astype('category')
X[['B', 'C']] = X[['A', 'B']].astype('float')
y = pd.DataFrame(data=np.random.rand(6)).astype('float')

XGBParams = {'booster': 'gbtree'}
d = xgb.DMatrix(X, label=y, missing=np.NaN, enable_categorical=True)
model = xgb.train(XGBParams, d,  num_boost_round=20, verbose_eval=True)
print(model.trees_to_dataframe())

Answer: Answering my own question:

Updated to latest XGBoost (1.7.3) -- not sure if this was critical but good practice.
Had to explicitly use hist for tree method -->  XGBParams = {'booster': 'gbtree', 'tree_method': 'hist'}

With these two changes, code above runs fine as does my actual code, with categoricals used in trees.
Title: Unable to uninstall setuptools to reinstall it
Tags: python-3.x,raspberry-pi4,adafruit-circuitpython

Question: I’m trying to set up my Raspberry Pi, and blinka is not installing, and I have been backtracking to reinstall my setuptools since I have the error where the board is not importing. And I am receiving an error:

13 permission denied ‘/usr/local/lib/python3.9/dist-packages/_distutils_hack/‘

Please I am losing my mind.
I have tried deleting the files to see if I can get a fresh restart and it then denies permission from that same file.

Answer: I ended up factory restarting the raspberry pi cause I was worried I had corrupted something at some point, and it is working fine now.
Title: Convert PySpark Dataframe to Pandas Dataframe fails on timestamp column
Tags: python,pandas,dataframe,apache-spark,pyspark

Question: I create my pyspark dataframe:
from pyspark.sql import SparkSession
from pyspark.sql.types import StructType, StructField, BinaryType, ArrayType, StringType, TimestampType
    input_schema = StructType([
            StructField("key", StringType()),
            StructField("headers", ArrayType(
                StructType([
                    StructField("key", StringType()),
                    StructField("value", StringType())
                ])
            )),
            StructField("timestamp", TimestampType())
        ])
    
        input_data = [
            ("key1", [{"key": "header1", "value": "value1"}], datetime(2023, 1, 1, 0, 0, 0)),
            ("key2", [{"key": "header2", "value": "value2"}], datetime(2023, 1, 1, 0, 0, 0)),
            ("key3", [{"key": "header3", "value": "value3"}], datetime(2023, 1, 1, 0, 0, 0))
        ]
    
        df = spark.createDataFrame(input_data, input_schema)

I want to use Pandas' assert_frame_equal(), so I want to convert my dataframe to a Pandas dataframe.
df.toPandas() will throw TypeError: Casting to unit-less dtype 'datetime64' is not supported. Pass e.g. 'datetime64[ns]' instead.
How can I successfully convert the "timestamp" column in order to not lose detail of the datetime value? I need them to remain to 2023-01-01 00:00:00 and not 2023-01-01.

Answer: I had the same problem, solved it by downgrading pandas to 1.5.3. It seems like the changes in pandas 2 break compatibility with pyspark (tested with pyspark 3.3).
If you're using pip you should be able to fix it with pip install pandas==1.*
Title: python jsonschema - get the reason of validation failure
Tags: python,jsonschema

Question: I have a json file which I need to compare with official F5 schema. Everything works but when I do any mistake it does not show me a path or an error, it just shows me that its valid under any of the given schemas.
My json file:
{
   "schema": "https://raw.githubusercontent.com/F5Networks/f5-appsvcs-extension/master/schema/latest/as3-schema.json",
   "class": "AS3",
   "action": "deploy",
   "persist": true,
   "declaration": {
       "class": "ADC",
       "schemaVersion": "3.40.0",
       "target": {
           "hostname": "test.test.com"
       },
       "id": "version_21.0",
       "label": "211008",
       "remark": "CO_version_21.0"
   }
}

Here is the Python script:
import json
import requests
from jsonschema import Draft7Validator, ValidationError

def load_json_from_url(url):
   response = requests.get(url)
   return response.json()


def main():
   schema_url = 'https://raw.githubusercontent.com/F5Networks/f5-appsvcs-extension/master/schema/latest/as3-schema.json'
   schema = load_json_from_url(schema_url)
  
   schema['$id'] = schema['$id'].replace('urn:uuid:', '')

   with open('gtm1.json', 'r') as f:
       validJson = json.load(f)

   validator = Draft7Validator(schema)

   try:
       validator.validate(validJson)
       print('The JSON file is valid according to the schema.')
   except ValidationError as e:
       message = e.schema["error_msg"] if "error_msg" in e.schema else e.message
       print(f'Invalid data: {message}')

if __name__ == '__main__':
   main()

If I were to change the json example from
{
    ...
    "declaration": {
        "class": "ADC",
        ...
    }
}

To:
{
    ...
    "declaration": {
        "class": "ADC1",
        ...
    }
}

Inside the script I have tried
message = e.schema["error_msg"] if "error_msg" in e.schema else e.message

e.path or e.json_path but nothing works
Here is the output:
Invalid data: {'schema': 'https://raw.githubusercontent.com/F5Networks/f5-appsvcs-extension/master/schema/latest/as3-schema.json', 'class': 'AS3', 'action': 'deploy', 'persist': True, 'declaration': {'class': 'ADC1', 'schemaVersion': '3.40.0', 'target': {'hostname': 'test.test.com'}, 'id': 'version_21.0', 'label': '211008', 'remark': 'CO_version_21.0'}} is not valid under any of the given schemas
How can I get the path to an item that caused validation to fail? I'd like to obtain something like .declaration.class, because this item is the invalid one (this specific output format is not important, e.g. list of path components would be also fine).

Answer: problem solved
from jsonschema import Draft7Validator  does not work
I used from jsonschema import validate
Title: Is there a SQLAlchemy session.add_all() equivalent on GORM?
Tags: python,postgresql,go,go-gorm

Question: I am new to Golang and I am trying to bulk update models for a table.
I have a "owners" table where I am trying to update multiple rows.
my go model (not complete):
type Owners struct {
    OwnerID      int64   `gorm:"primaryKey;not null"`
    OwnerName    string
    Eligibility  bool
}

I have a logic that updated the Eligibility to true or false for multiple rows
Using Python SQLAlchemy, I can create a session and use add_all() method to bulk update the rows.
Is there something similar on GORM.
I tried the following code
const batchSize = 100

func bulkUpdate(owners *[]Owners) {
    for i := 0; i < len(*owners); i += batchSize {
        end := i + batchSize
        if end > len(*owners) {
            end = len(*owners)
        }
        batch := (*owners)[i:end]
        log.Println("batch=", len(batch))
        //result := config.DB.Table("owners").Updates(batch)
        result := config.DB.Model(owners).Updates(batch)
        log.Println("result=", result)
        if result.Error != nil {
            log.Println(result.Error)
        }
    }
    config.DB.Commit()
}


But this gives me error "unsupported data"
I can do the following, which works fine:
for _, ownership := range owners {
    config.DB.Save(&ownership)
}

but since I have 4000+ rows, it takes lot of time.

Answer: I'm still pretty new to Go and Gorm but I do not believe there is a similar function to the add.all() function in SQLAlchemy.
For GORM I believe you have to use the Create() method with a slice of structs as input which will insert multiple records in a single transaction.  Though that may not be exactly what you're looking for :).
Title: AttributeError: module 'virtualenv.create.via_global_ref.builtin.cpython.mac_os' has no attribute 'CPython2macOsArmFramework'
Tags: python,macos,pip,modulenotfounderror

Question: I was trying to configure pre-commit hooks and while running pre-commit run --all-files I got this error:
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('/Users/dark_matter88/opt/anaconda3/bin/python3.8', '-mvirtualenv', '/Users/dark_matter88/.cache/pre-commit/repojfgkwtv7/py_env-python3.8')
return code: 1
stdout:
    AttributeError: module 'virtualenv.create.via_global_ref.builtin.cpython.mac_os' has no attribute 'CPython2macOsArmFramework'

I've tried to upgrade pip to resolve this issue pip install --upgrade pip and I received another error:
Traceback (most recent call last):
  File "/Users/admin/opt/anaconda3/bin/pip", line 5, in <module>
    from pip._internal.cli.main import main
  File "/Users/admin/.local/lib/python3.8/site-packages/pip/_internal/cli/main.py", line 9, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/Users/admin/.local/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py", line 10, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/Users/admin/.local/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py", line 9, in <module>
    from pip._internal.build_env import get_runnable_pip
  File "/Users/admin/.local/lib/python3.8/site-packages/pip/_internal/build_env.py", line 15, in <module>
    from pip._vendor.packaging.requirements import Requirement
ModuleNotFoundError: No module named 'pip._vendor.packaging'

I tried to check versions of both pip and pip3, and now I'm also getting the same No module named 'pip._vendor.packaging' error. I've tried all the solutions I could find, but nothing's helped.
I'm thinking if it's related to having several versions of python installed.

Answer: The same issue recently appeared on a BitBucket pipeline (while having no issue locally on a WSL pre-commit run).
The problem was solved by downgrading virtualenv from 20.23.0 to 20.19.0
Title: Imported python class does not have access to loaded package, Jupyter Notebook
Tags: python,jupyter-notebook

Question: I have written several functions which uses tensorflow and stored them in a separate file called modelDefinition.py. Currently I'm working on a notebook under the same directory as modelDefinition.py. I imported tensorflow as tf in my notebook and tried to call functions defined in modelDefinition.py. I got the following error:
NameError: name 'tf' is not defined

Here's what's inside modelDefinition.py
def modelDef(ckpt_dir,l2,dr,img_size=80,n_bands=3,nf=32):
    regularizer = tf.keras.regularizers.l2(l2)
    initializer = tf.keras.initializers.glorot_normal()
    # model definitions
    ...

    return model

Here's what I have in my notebook
import numpy as np
import tensorflow as tf

from modelDefinition import *

# Create model and load pre-trained weights 
model = modelDef(ckpt_dir='checkpoints',l2=0.01,dr=0.1)


Here's the full Error message:
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[8], line 2
      1 # Create model and load pre-trained weights 
----> 2 model = modelDef(ckpt_dir='checkpoints',l2=0.01,dr=0.1)


File D:\Data\Yale University\2023SP\Deep Learning Theory and Applications\Final_proj\modelDef.py:103, in modelDef(ckpt_dir, l2, dr, img_size, n_bands, nf)
    102 def modelDef(ckpt_dir,l2,dr,img_size=80,n_bands=3,nf=32):
--> 103     regularizer = tf.keras.regularizers.l2(l2)
    104     initializer = tf.keras.initializers.glorot_normal()
    105     common_args = {"kernel_initializer": initializer}

NameError: name 'tf' is not defined


Any help is appreciated!

Answer: I think you must import tensorflow module in modelDefinition.py file, i am not sure if you did that.
Title: Getting a website CSS url from the HTML url
Tags: python,web-scraping,colors,css-parsing

Question: I am building a Python code for scraping colors from a website and linking them to the elements that have that color (for example, I would like to associate all the <p> with their color).
The way I am doing it is that I am trying to access the website CSS and from there scrape all the hex colors present, and eventually linking it to its selector.
The problem is, there seems to be a problem with getting the CSS url. I am using Beautiful Soup to parse the html, but when trying to get the CSS I previously got the error:
MissingSchema: Invalid URL '/_next/static/css/19cb64e37006115a.css': No scheme supplied. Perhaps you meant http:///_next/static/css/19cb64e37006115a.css?
So I added a snippet that allows to add the protocol http when not present.
Anyway, I get the new error: InvalidURL: Invalid URL 'http:///_next/static/css/19cb64e37006115a.css': No host supplied.

There still is some problem with getting the right CSS url.
Here is the full code:
import re
import cssutils
import requests
from bs4 import BeautifulSoup
from urllib.parse import urlsplit, urlunsplit


url = 'https://www.endy.com/'

# Retrieve the HTML code of the website
response = requests.get(url)
html = response.text

# Use BeautifulSoup to find the CSS file(s) and extract the URLs
soup = BeautifulSoup(html, 'html.parser')
css_urls = [link['href'] for link in soup.find_all('link', rel='stylesheet')]

# Create a dictionary to store the color and corresponding selectors
color_dict = {}

# Retrieve the CSS file(s) and parse the CSS rules
for css_url in css_urls:
    # Add default scheme if none is provided
    if not urlsplit(css_url).scheme:
        css_url = urlunsplit(('http',) + urlsplit(css_url)[1:])

    css_response = requests.get(css_url)
    css_text = css_response.text
    sheet = cssutils.parseString(css_text)

    # Extract CSS rules containing hexadecimal color codes
    for rule in sheet:
        if rule.type == rule.STYLE_RULE:
            css_text = rule.selectorText + ' {' + rule.style.cssText + '}'
            hex_colors = re.findall(r'#(?:[0-9a-fA-F]{3}){1,2}\b', css_text)
            if hex_colors:
                for color in hex_colors:
                    if color not in color_dict:
                        color_dict[color] = []
                    color_dict[color].append(rule.selectorText)

# Print the color and corresponding selectors
for color, selectors in color_dict.items():
    print(f"Color: {color}")
    print(f"Selectors: {', '.join(selectors)}")
    print("------------------------------")

Any help will be greatly appreciated. Thank you!

Answer: Anyway, I get the new error: InvalidURL: Invalid URL 'http:///_next/static/css/19cb64e37006115a.css': No host supplied.

You need to set the host, like if the website is http://localhost:4200 you need to add that before ‘/_next/static/css/19cb64e37006115a.css‘
Title: How to combine dictionaries using one's values as the other's keys?
Tags: python,dictionary

Question: I have two dictionaries
data_map = {"field_1": "name_in_data"}
data_set = {"name_in_data": "data_value"}

I need to combine these two dictionaries into:
{"field_1" : "data_value"}

I wrote a simple loop to do this:
for key in data_map:
   data_map[key] = data_set[data_map[key]]

Is there a more pythonic way to accomplish the same thing?

Answer: A dictionary comprehension:
new_dict = {k1:v2 for v1, v2 in data_set.items() for k1, v1 in data_map.items()}