Myashka/SO-Python_QA-filtered-2023-tanh_score-after_2023_02 huggingface dataset:
Title: How to connect to the active session of mainframe from Python
Tags: python,win32com,mainframe

Question: How to connect to the active session in mainframe from Python -
import win32com.client

passport = win32com.client.Dispatch("PASSPORT.Session")

session = passport.ActiveSession
if session.connected():
    print("Session active")
else:
    print("Session not active")

It is giving a DDL file missing error when activating the .zws file, and when the session is not active it's giving the message, "Session not active".
Can someone please help?
I am using Python3.8 version

Answer: Discovered a possible solution -
we can create active session in mainframe using VBA code.
And connect the macro through python code.
Title: How can I write a python selenium webscraping script using chromedriver on github codespaces?
Tags: python,selenium-webdriver,codespaces

Question: I have got github codespaces environment and I have installed both selenium and the necessary chromedriver-binary using pip
pip install selenium chromedriver-binary
Here's an example of the Python web scraper I'm writing
import json
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service

class PriceScraper:

    def scrape(self):

        input_url = "https://www.google.com"
        chrome_options = Options()
        chrome_options.add_argument("--headless")
        chrome_options.add_argument("--disable-dev-shm-usage")
        chrome_options.add_argument("--no-sandbox")
        service = Service('/usr/bin/chromedriver')
        driver = webdriver.Chrome(service=service, options=chrome_options)
        driver.get(input_url)

if __name__ == '__main__':
    scraper = PriceScraper()
    scraper.scrape()


I have installed all the necessary pip packages and I have confirmed the installation of the chromium and the chromedriver by running:
(venv) $   sudo apt-get install -y chromium-browser chromium-chromedriver python3-selenium
Reading package lists... Done
Building dependency tree       
Reading state information... Done
python3-selenium is already the newest version (4.0.0~a1+dfsg1-1.1).
chromium-browser is already the newest version (1:85.0.4183.83-0ubuntu0.20.04.3).
chromium-chromedriver is already the newest version (1:85.0.4183.83-0ubuntu0.20.04.3).

And checking by running
ls -l /usr/bin/chromedriver
But when I try to execute the python from my vscode codespaces terminal as follows:
python3 scrape_prices.py
It returns the following error:
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally.
  (unknown error: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)

Interestingly when I try to run chromedriver from the command line it says:
Command '/usr/bin/chromedriver' requires the chromium snap to be installed.
Please install it with:

snap install chromium

And when I attempt to install snap with snap install chromium
I get the following error
error: cannot communicate with the server: Post http://localhost/v2/snaps/chromium: dial unix /run/snapd.socket: connect: no such file or directory

I'm unsure how to get this working

Answer: The issue was that I hadn't installed chromium locally, and once that was done properly from within my .venv virtual environment, everything worked properly.
Title: Set AppDynamics integration with Python
Tags: python,dockerfile,appdynamics

Question: I'm trying to set up my Python application to send data to AppDynamics. I have the AppDynamics controller up and running and on my local my application works, but no data is found on AppDynamics
I've been told to use this repo as a template (and I can confirm it works, sending data to the AppDynamics instance I'm working on) https://github.com/jaymku/py-k8s-init-scar/blob/master/kube/web-api.yaml
I have some doubts though, and they might be the cause of the issues that I'm having.
I had in my Dockerfile a CMD at the end like first.sh && python3 second and I've changed it to be ENTRYPOINT "first.sh && python3 second". Note no [] format here and also that there are two concatenated commands.
On the value of the APP_ENTRY_POINT variable I'm trying just the same.
There are no errors when I run this, my application works correctly, except the data is not sent to AppDynamics. Nothing seems to fail, I can't find any error messages. Any ideas what I'm missing?
Also, where can I find out, within AppDynamics, the value that we need to set for the APPDYNAMICS_CONTROLLER_PORT variable? I'm pretty sure it will be 443 in our case, since we seem to be using that in other proyects in AppDynamics that are working, but checking it would be a good idea. It might also be related with this issue, I don't know.

Answer: I managed to have this working by using CMD instead of ENTRYPOINT, using the command that is found inside the suggested entrypoint. So, I did the same that was supposed to done the entrypoint but inserting the command myself
Title: How To Prevent Stimuli Overlap Prevention?
Tags: python,python-3.x,psychopy

Question: new to the scene.
I've created a code for a Visual Search task with a randomized number of Distractors (1/5/9) and a Target that may or may not appear (50% chance).
The coordinates for the placements of both Distractors and Target are randomized, but naturally that on its own doesn't prevent overlap, and both Distractors and Target keep clashing with each other.
Is there any way to prevent it?
I've considered trying to add each new random set of positions to a list and somehow make sure the list doesn't allow duplicates/close numbers (if duplicate/close then randomize again), but no idea how to implement.
Code provided: (D = Distractor, T = Target):
for i in range(20):

    ClosedD_num = random.choice([1, 5, 9])
    ClosedD_list = []

    for i in range(ClosedD_num):
        x_pos = random.uniform(-0.5, 0.5)
        y_pos = random.uniform(-0.5, 0.5)

        ClosedD = visual.ImageStim(
            win=window,
            name='ClosedD',
            image='images/closed_d.png', mask=None, anchor='center',
            ori=randint(1, 360), pos=(x_pos, y_pos), size=(0.1, 0.1),
            color=[1, 1, 1], colorSpace='rgb', opacity=None,
            flipHoriz=False, flipVert=False,
            texRes=128.0, interpolate=True, depth=-3.0)
        ClosedD_list.append(ClosedD)

        for ClosedD in ClosedD_list:
            ClosedD.setAutoDraw(True)

    r_pos = random.uniform(-0.5, 0.5)
    s_pos = random.uniform(-0.5, 0.5)

    ClosedT = visual.ImageStim(
        win=window,
        name='ClosedT',
        image='images/closed_t.png', mask=None, anchor='center',
        ori=randint(1, 360), pos=(r_pos, s_pos), size=(0.1, 0.1),
        color=[1, 1, 1], colorSpace='rgb', opacity=None,
        flipHoriz=False, flipVert=False,
        texRes=128.0, interpolate=True, depth=-3.0)

    ClosedT_present = random.choice([0, 1])
    if ClosedT_present == 0:
        ClosedT.setAutoDraw(False)
    else:
        ClosedT.setAutoDraw(True)
    window.flip()

P.S can provide further information if needed

Answer: To avoid overlap in many stimuli you basically need to constrain the randomness. I'd recommend you take a different approach of creating a grid of coordinates, with 1 element per grid position, and then applying a random jitter to the positions of the elements that is smaller than the size of your imaginary boxes. This will still give an appearance of random positions but means a) no overlaps and b) a more even distribution with no clumps of elements
Title: Incompatible types in assignment (expression has type "List[str]", variable has type "List[Union[str, int]]")
Tags: python,mypy,python-typing

Question: If I run this code through mypy:
if __name__ == '__main__':
    list1: list[str] = []
    list2: list[str | int] = []
    list2 = list1

It gives me following error:
error: Incompatible types in assignment (expression has type "List[str]", variable has type "List[Union[str, int]]")

Why? Isn't a list[str] a subset of list[str | int]? If yes, then why can't I assign list1 which doesn't have wider range of possible types to list2?

Answer: This is the covariant vs contravariant problem.
Let X be a subtype of Y be a subtype of Z.
If I have a method that expects list[Y], and only ever reads that list, then it is perfectly safe to pass list[X]. The method expects to read an X, and it gets a Y instead, which is a subtype of X. Everything is okay. This is called a covariant argument.
If I have a method that expects list[Y], and only ever writes to that list, then it is perfectly safe to pass list[Z].  Values of type Y can fit well into list[Z].  This is called a contravariant argument.
If the method expects to both read and write to the list, then I have to pass an argument of precisely type list[Y].
Title: Invalid Token when using Fernet library in python
Tags: python,fernet

Question: I was trying to use a library I found in python called fernet and make a simple program encrypting and decrypting in python and it worked but when I tried to make it GUI it kept spiting out the error "invalid token". I don't know what I did wrong but it seems that it is only throwing a error in the GUI, Can someone help?
(Just so you know I am not the smartest in python so can you explain in simple terms)
Here is the full error:
line 121, in _get_unverified_token_data
raise InvalidToken
cryptography.fernet.InvalidToken
from cryptography.fernet import Fernet
from tkinter import *
import tkinter as tk


def Encrypt():
    get=tk.StringVar()
    message=get.get()



    byte_message = str.encode(message)

    f = Fernet(b'U_v31LOlh9SOe3eN-18CNUropIalPD02eqxgTxUnuAI=')
    encrypted_message = f.encrypt(byte_message)
    enmess.config(text = encrypted_message.decode(), font=('Poppins',5))

def Decrypt():
    get=tk.StringVar()
    message=get.get()
    f = Fernet(b'U_v31LOlh9SOe3eN-18CNUropIalPD02eqxgTxUnuAI=')
    message = str.encode(message)
    encrypted_message = f.decrypt(message)
    print(encrypted_message.decode())


#decrypt_message = f.decrypt(b'gAAAAABkTNLJxD-1doUad45Srrb0vku4QdC0sQ-KZLr4-Z5OdgJ5R118ApojM-3HQTc3_YcYLJ6FYNQgYylc1V9QE2G_lTt1_A==')
#print(decrypt_message)
root=tk.Tk()
root.geometry("600x400")
message=tk.StringVar()


    
name_label = tk.Label(root, text = "Enter Message", font=('Poppins',10, 'bold'))
name_entry = tk.Entry(root,textvariable = message, font=('Poppins',10,'normal'))
enmess = tk.Label(root, text = "will be updated soon", font=('Poppins',10))
encrypt_btn=tk.Button(root,text = 'Encrypt', command = Encrypt)
decrypt_btn=tk.Button(root,text = 'Decrypt', command = Decrypt)



name_label.grid(row=0,column=1)
name_entry.grid(row=0,column=2)
encrypt_btn.grid(row=2,column=1)
decrypt_btn.grid(row=2,column=2)
enmess.grid(row=3, column=0)
root.mainloop()




I fiddled around a lot and so the code may look a little weird but thank you for your help (:

Answer: The error message you're seeing is related to the cryptography library.
InvalidToken exception is raised when the token passed to Fernet.decrypt() method is either invalid or the message was tampered with.
Here are a few things you can check:
Make sure you are entering the correct token to decrypt. It should be the exact same token that was generated during encryption.
Ensure that the token is being passed correctly to the Fernet.decrypt() method.
Check if there is any change in the message or token during transfer or storage.
Title: Secure Password Storage in Flask-based RESTful API using Python
Tags: python,security,flask,restful-authentication

Question: I am building a Flask-based RESTful API in Python, and I need to securely store and encrypt user passwords for authentication. Currently, I am using a simple hashing algorithm to hash passwords with a salt, but I'm not sure if this is secure enough.
Here is an example of my current implementation for password hashing:
import hashlib

password = 'password123'
salt = 'somesalt'
hashed_password = hashlib.sha256((password + salt).encode('utf-8')).hexdigest()
print(hashed_password)

Can anyone suggest a more secure way to store and encrypt passwords for user authentication in a Flask-based API? Specifically, I would like to know which password hashing algorithm to use and how to use it in a Flask application. Any advice or suggestions would be greatly appreciated.

Answer: sha256 is good enough, but you're using the "salt" as a pepper
the difference is, you'll be using this salt for every password - the value will be 'somesalt' every time. This is actually called a pepper.
If you want to use a salt correctly, you have to randomize it for every password, and then save it into your database along with the password. (then retrieve it, and re-add it when checking the password's hash)
Ideally, you should be using both salt and pepper. The pepper, which is never saved into the database and cannot be figured out if someone just has your database values, and the salt, which makes sure every password's hash is different from every other password's hash, even if the passwords are the same. You should also not put the salt as clear text into your codebase either.
I would say that
cleartext = very bad
hashed = not good enough
hashed + peppered or salted = mediocre / ok
hashed + peppered + salted = standard
Title: Unexpected uint64 behaviour 0xFFFF'FFFF'FFFF'FFFF - 1 = 0?
Tags: python,numpy,uint64

Question: Consider the following brief numpy session showcasing uint64 data type
import numpy as np
 
a = np.zeros(1,np.uint64)
 
a
# array([0], dtype=uint64)
 
a[0] -= 1
a
# array([18446744073709551615], dtype=uint64)
# this is 0xffff ffff ffff ffff, as expected

a[0] -= 1
a
# array([0], dtype=uint64)
# what the heck?

I'm utterly confused by this last output.
I would expect 0xFFFF'FFFF'FFFF'FFFE.
What exactly is going on here?
My setup:
>>> sys.platform
'linux'
>>> sys.version
'3.10.5 (main, Jul 20 2022, 08:58:47) [GCC 7.5.0]'
>>> np.version.version
'1.23.1'

Answer: The behavior you are seeing is due to how unsigned integer arithmetic works in numpy. When an unsigned integer is decremented, if the result is negative, it "wraps around" to the maximum value of the data type.
In your example, a[0] starts at the value 0xFFFFFFFFFFFFFFFF, which is the maximum value for a 64-bit unsigned integer. When you subtract 1 from it, the result is 0xFFFFFFFFFFFFFFFE, as you expected. However, when you subtract 1 from it again, the result is -1 (which is represented as 0xFFFFFFFFFFFFFFFF in binary). Since this value is negative, it wraps around to the maximum value of the data type, which is 0.
So, the behavior you are seeing is expected due to the properties of unsigned integer arithmetic. If you want to avoid this behavior, you can use a signed integer data type instead.
Title: Unexpected uint64 behaviour 0xFFFF'FFFF'FFFF'FFFF - 1 = 0?
Tags: python,numpy,uint64

Question: Consider the following brief numpy session showcasing uint64 data type
import numpy as np
 
a = np.zeros(1,np.uint64)
 
a
# array([0], dtype=uint64)
 
a[0] -= 1
a
# array([18446744073709551615], dtype=uint64)
# this is 0xffff ffff ffff ffff, as expected

a[0] -= 1
a
# array([0], dtype=uint64)
# what the heck?

I'm utterly confused by this last output.
I would expect 0xFFFF'FFFF'FFFF'FFFE.
What exactly is going on here?
My setup:
>>> sys.platform
'linux'
>>> sys.version
'3.10.5 (main, Jul 20 2022, 08:58:47) [GCC 7.5.0]'
>>> np.version.version
'1.23.1'

Answer: a[0] - 1 is 1.8446744073709552e+19, a numpy.float64. That can't retain all the precision, so its value is 18446744073709551616=264. Which, when written back into a with dtype np.uint64, becomes 0.
Title: Input python using sys
Tags: python

Question: Is it possible to read the character of an input without consuming it?
Example:
while state < 100:
    if read: c = sys.stdin.read(1)
    else: read = True
    if c=='/':
        next_c=sys.stdin.peek(1)
        if next_c=='/':
            state = MT[state][filter('//')]
        else:
            state = MT[state][filter(c)]
    else:
        state = MT[state][filter(c)]
    if state < 100 and state !=0: lexeme += c

I want that next_c doesn't consume the input character so that in the rest of the runs it is complete.
For example, if I give a/b it just prints a/ because the b was read.

Answer: In Python, you can use the io.BufferedReader class to wrap the standard input and provide the peek() functionality.
Title: TypeError on Generator object with Linux Mint. Code works fine in Win11
Tags: python,generator

Question: I have code for running a google search from terminal.
The weird thing is it works just fine in my win11, but when I launch it in Linux Mint I get a TypeError.
The code creates an error in line 13
Exception has occurred: TypeError
'generator' object is not subscriptable
    for i in result[:num_results]:
TypeError: 'generator' object is not subscriptable

Any ideas why this is occuring and how to fix it?
Code below:
from googlesearch import search
import webbrowser, sys
import time

searching_for = input(("Input search words: "))

num_results = int(input("How many results : ") or "3")

result = search(searching_for)

for i in result[:num_results]:
    webbrowser.open(i)
    time.sleep(1)

Answer: search returns a generator.
Use:
for i in list(result)[:num_results]:
Title: Customtkinter: Change text color of switch
Tags: python,tkinter,customtkinter,tkinter-text

Question: I want to change the text color of the switch widget text in customtkinter
I tried to use configure in with text_color but it showed me that there's no attribute of a switch called text_color...
Btw.. when creating the switch text_color works
minimal reproducible example:
import customtkinter as ctk  
root = ctk.CTk()
switch = ctk.CTkSwitch(master=root, text='This is a switch', text_color='yellow')
switch.pack()
switch.configure(text_color='red')
root.mainloop()

Answer: Configuring CTkSwitch.configure(text_color=...) works in CustomTkinter 5.1.3.
Title: While loop stops for a reason I did not anticipate in python
Tags: python,while-loop

Question: Am very new to python, I'm just using it to solve some of Euler's problems. I'm stuck on problem 4 as my code keeps stopping for no apparent reason. Its attempting to find the largest palindrome product of two three digit numbers.
x = 100
y = 100

while x != 1000:
    test = x * y
    if str(test)[::-1] == str(test):
        ans = test
    elif y<1000:
        y += 1
    else:
        x += 1
        y = 100
        print(x)
print(ans)

If I run it all I see is that 1 is added to x only one time, though it should happen 899 times. I know the rest of the code is working as it finds 10201 which is 101 * 101 but nothing after that.
I've tried changing the loop to while a variable stop is false and setting it true when x == 999 but it still stops after x == 101. Why does my loop stop?

Answer: this line elif y<1000: should be an if not an elif: you rporgram is stuck in an infite loop not increasing x neither y once it finds the first palindrome.
Your while loop is not "stopping": you'd see your program finish, and get back to console, or whatever: as it is, it will keep running forever. Even for getting this one correct result, you likely had a different program than this: this won't print even the answer for "101".
After you fix that, move your print to inside the while loop, unless you want to see just the last result (although, IIRC the largest number is usually what Project Euler problems require - it would be ok then)
Title: strptime unconverted data remains
Tags: python,pandas,datetime,strptime

Question: I have a string I want to convert to a datetime in the following format:
'31-12-2022:24'

The last two digits being the hour. I tried the following to convert it:
dt = datetime.strptime(z[1], '%d-%m-%Y:%H').date()

But get the following error:
ValueError: unconverted data remains: 4

Why is it only selecting the first digit? I have a workaround, splitting the string and converting them separately before rejoining them but I'd like to just do it in one line.
What I would ideally like is a datetime object that allows me to do some maths on the difference between two dates (edit* and times), which I assume is easiest in datetime format?

Answer: the problem is with your date value, as there is no such hour as 24.
If your string will be correct, as e.g.:
'31-12-2022:12' it should be ok.
Title: Want to run a Python "print()" script from Java, but getting no output
Tags: python,java

Question: I have a simple Python script:
def main():
    print("Hello world")

if __name__ == "__main__":
    main()

And I want to get the response in Java. The endpoint is:
@RestController
@CrossOrigin(origins = "http://localhost:4200")
public class TestPython {

    @PostMapping("/endpoint")
    public ResponseEntity<String> endpoint() throws IOException {
        ProcessBuilder pb = new ProcessBuilder("path-to-venv", "path-to-python-script");
        pb.redirectErrorStream(true);
        Process p = pb.start();

        BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()));
        StringBuilder sb = new StringBuilder();
        String line;
        while ((line = reader.readLine()) != null) {
            sb.append(line);
        }
        String jsonResponse = sb.toString();

        BufferedReader errorReader = new BufferedReader(new InputStreamReader(p.getErrorStream()));
        StringBuilder errorSb = new StringBuilder();
        String errorLine;
        while ((errorLine = errorReader.readLine()) != null) {
            errorSb.append(errorLine);
        }
        String errorOutput = errorSb.toString();

        if (!errorOutput.isEmpty()) {
            System.out.println("Error output: " + errorOutput);
        }

        System.out.println(jsonResponse);
        System.out.println(errorOutput);
        return ResponseEntity.ok(jsonResponse);
    }
}


jsonResponse and errorOutput are both empty. My virtual environment is active and I am using it. When I execute the Python script via terminal, a cmd pops up and prints Hello World, but using Java is empty. I also tried to explicitly set the charset in java, but I get the same problem. Also I get a lot of these \u0000 characters when executing reader.cb in debug mode.
Any help is welcome.

Answer: Not 100%, but you can try to flush the output stream of your Python script. It's possible that the output is not being sent until the script completes, so flushing the stream may help. You can try adding sys.stdout.flush() at the end of your Python script to force the output to be flushed.
edit (I few other things to try):

Make sure that the Python script has execute permissions. If the Python script is not marked as executable, you may need to set the execute permission using the chmod command in your terminal.

Triple check the path's you're using in the ProcessBuilder, that object also has a few other constructors you could try that may work better.

You can also try using a different method for executing the Python script, such as Runtime.getRuntime().exec()
Title: Scrapy xpath error: 'Selector' object has no attribute '_default_type'
Tags: python,xpath,scrapy

Question: I'm using scrapy shell to run some sample test
after I enter 'response.xpath('//h1/text()').get()'
and it gives me AttributeError: 'Selector' object has no attribute '_default_type'
Can anyone help with this?
Thank you very much
below are the things that I entered in the terminal
scrapy shell
fetch("https://www.worldometers.info/world-population/population-by-country/")
r = scrapy.Request(url="https://www.worldometers.info/world-population/population-by-country/")
fetch(r)
response.xpath('//h1/text()').get()
Everything went good before the last line.
After I entered 'response.xpath('//h1/text()').get()'
it throw an error:
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-5-7a322073163e> in <module>
----> 1 response.xpath('//h1/text()').get()

~/anaconda3/envs/virtual_work/lib/python3.7/site-packages/scrapy/http/response/text.py in xpath(self, query, **kwargs)
    117 
    118     def xpath(self, query, **kwargs):
--> 119         return self.selector.xpath(query, **kwargs)
    120 
    121     def css(self, query):

~/anaconda3/envs/virtual_work/lib/python3.7/site-packages/scrapy/http/response/text.py in selector(self)
    113         from scrapy.selector import Selector
    114         if self._cached_selector is None:
--> 115             self._cached_selector = Selector(self)
    116         return self._cached_selector
    117 

~/anaconda3/envs/virtual_work/lib/python3.7/site-packages/scrapy/selector/unified.py in __init__(self, response, text, type, root, _root, **kwargs)
     84                             % self.__class__.__name__)
     85 
---> 86         st = _st(response, type or self._default_type)
     87 
     88         if _root is not None:
AttributeError: 'Selector' object has no attribute '_default_type'

Answer: What version of Scrapy are you on? Upgrading to the latest solved it for me.
Title: Pycharm/Python OpenCV and CV2 importing error
Tags: python,opencv

Question: Even after installing opencv using:
pip install opencv-python

I'm getting this error while importing it in pycharm
ERROR: Could not find a version that satisfies the requirement cv2 (from versions: none)
ERROR: No matching distribution found for cv2

I tried downloading it in differnt ways but it dosen't import

Answer: I had the same problem once when I used VS code. I tried to run the same code in PyCharm (with included opencv-python library) and it worked with no problems!
Title: Jupyterlite: accessing functions and libraries in a separate file?
Tags: jupyter,python-module,pyodide,jupyterlite

Question: I have a JupyterLite/pyodide session running a nodebook called "Workshop1.ipynb`. In the same (root) directory as that notebook, I have a file containing useful functions which also does some imports, e.g.
# functions.py
import numpy as np

def a_useful_function(val):
    return np.array(val) * 2

I want to call import functions as the first line of my notebook, to use the functions defined in the notebook, but I then get
ModuleNotFoundError: The module 'numpy' is included in the Pyodide distribution, but it is not installed.
You can install it by calling:
  await micropip.install("numpy") in Python, or
  await pyodide.loadPackage("numpy") in JavaScript

Is it possible to load modules inside a file which is then imported into the notebook, without calling e.g. import numpy and/or await micropip.install("numpy") in the notebook itself?

Answer: Is it possible to load modules inside a file which is then imported into the notebook, without calling e.g. import numpy and/or await micropip.install("numpy") in the notebook itself?

The short answer is no. Similar to regular Python you need to indicate which packages should be installed before running your code.
From a technical perspective currently, in Pyodide we can't install a package (which is an async operation ) during the import (which is sync). Unless we recursively parse imports which has a performance overhead.
Beyond that, loading packages that are imported in a pyodide.runPythonAsync code snippet is a convenient functionality (which JupyterLite is using). However, I don't believe it would be a good idea to extend it to second-level import: i.e. follow your import functions and see what imports it has. For the same reasons, Python doesn't pip install packages on import in general. Both from the security and reliability perspective it's not great.
In practice, the workaround proposed by @TachyonicBytes if you don't want to specify requirements in a notebook would work.
Title: Using a function in a loop in Python
Tags: python,mysql,loops

Question: My aim is to get the last price from a histoy table (History_price) and inject it in a parameter table (param_forex).
I have defined the following function:
    mycursor.execute(f"UPDATE `param_forex` SET `Rate` = %s, `Update_Time` = CURRENT_TIME() WHERE `param_forex`.`Ticker` LIKE %s", 
                     (Rate, Ticker))
    mydb.commit()
    mycursor.close

The function works on its own. It simply fills a table with new prices on some particular lines.
Now I try to incorporate it in my loop. It's actually a double loop because of how mycursor works in mysql.connector
def fillpricetable_assetid() :
        for x in  ['EURUSD','EURJPY','EURGBP']:
            mycursor.execute(f"SELECT `Price` FROM `History_price` WHERE `Ticker` LIKE '{x}' ORDER BY `History_price`.`Time` DESC LIMIT 1")
            for y in mycursor:
                updateForexdb(x.lower(),y[0])

And it does not work anymore...
I get
  File "c:\...\library_import.py", line 116, in <module>
    fillpricetable_assetid()
  File "c:\...\library_import.py", line 89, in fillpricetable_assetid
    updateForexdb(x.lower(),y[0])
NameError: name 'updateForexdb' is not defined

How is that possible, when I have just defined it above??

Answer: I have interverted the order and now it works fine.
Title: Email address on Django API returns 404
Tags: python,django,api,backend

Question: I have an endpoint created like this:
class OTPSViewSet(viewsets.ViewSet, mixins.RetrieveModelMixin, mixins.CreateModelMixin):
    """
    A simple ViewSet for listing or retrieving users.
    """
    throttle_classes = [OncePerDayUserThrottle]
    authentication_classes = []
    permission_classes = []

    def retrieve(self, request, pk=None):
        response = self.get_otp(pk)
        return response

    def create(self, request):
        serializer = OtpsForms(data=request.data)
        if serializer.is_valid():
            response = self.verify_otp(request)
            return response
        else:
            return response_error(serializer.errors)

    def get_otp(self, username):
        """Send OTP by email

        Args:
            username (string): username
        """
        username = username.strip().lower()
        user = _get_all_by_username(username)
        ...


What happens is that when I make a call to the endpoint with a "username" such as "javi" or "javi@gmail", the call "GET /api/otps/javi/ HTTP/1.1" 200" returns a 200, and when the username is an email like "javi@gmail.com" the call is "GET /api/otps/javi@gmail.com/ HTTP/1.1" 404" returning a 404 not found status.
I think it has to do with having dot (.) inside the url but I don't know how to fix it.
Thanks in advance!

Answer: I fixed it adding lookup_value_regex = '[^/]+' at the beginning of the viewset and now it is working
Title: Time series cross-validation removing new categories from test
Tags: python,pandas,scikit-learn,cross-validation

Question: I want to use time series cross validation to evaluate my model. I'm trying to leverage TimeSeriesSplit and cross_validate from sklearn.
Let's say my model has features A, B, and others. In practice I want my model only to give predictions for categories A and B seen during training, for all other categories it will raise an error.
How I can cross validate my model enforcing this behaviour? Could I still use the sklearn implementations, adapt them with minor changes or would I have to build my cross validation from scracth?

Answer: Import TimeSeriesSplit.
Then create an instance of TimeSeriesSplit (set the test size parameter to 1)
Then define a method, which filters out all the unwanted categories from the train and test dataset and outputs your filtered data.
Import cross_validate from scikit-learn, then make sure to call your custom function for every iteration of your cross_validate.

This way you can implement cross validation with minor changes rather than implementing it all from scartch.
Title: Want to Use UpdateMany instead of UpdateOne for syncing MongoDB Collection
Tags: python,mongodb,pymongo,asyncmongo

Question: I am a new user of MongoDB. I am working with a large amount of transaction data which is updated periodically after each day or week. I created my code for this use where I am using pymongo.UpdateOne but as I have a large amount of data, I want to use UpdateMany. Here’s a snippet of my code:
client = MongoClient()
database = client['Eg']
collection = database['eg']
start = datetime.now()

df = pd.read_csv("eg.csv")
df['_id'] = df["Factory_Id"] + df["Order ID"]
data = df.to_dict(orient="records")

requests = []
for doc in data:
    filter = {'_id': doc['_id']}
    update = {'$set': doc}
    request = pymongo.UpdateOne(filter, update, upsert=True)
    requests.append(request)

if requests:
    result = collection.bulk_write(requests)

Here I am using UpdateOne for syncing but I want it to sync once and not use a for-loop with the help of UpdateMany.

Answer: You can't use UpdateMany in your example, as each filter is different for each update; UpdateMany only works where you are applying the same filter for all the updates.
As you are already using bulk_write, there isn't a performance benefit to be gained anyway.
Title: python:requests.post(url , headers=headers, data=json.dumps(order)) is work,when use C# return error
Tags: python,c#,json,post

Question: In python, it is working:
requests.post(url , headers=headers, data=json.dumps(order))

When I use C#, then the API return msg:Invalid HTTP Request Input
string serviceUrl = string.Format("{0}{1}", this._baseUri, uri);
//System.Net.ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
//serviceUrl = "http://mj.qzdyyy.com/ChsMobilePay/ChsMobilePayApi/v1/GetPayAuthNo";
HttpWebRequest myRequest = (HttpWebRequest)WebRequest.Create(serviceUrl);
byte[] buf = Encoding.GetEncoding("UTF-8").GetBytes(data);

myRequest.Method = "POST";
myRequest.ContentLength = buf.Length;
myRequest.KeepAlive = true;
myRequest.Accept = "*/*";  
if (headerDic != null && headerDic.Count != 0)
{
    foreach (var item in headerDic)
    {
        if (item.Key == "ContentType")
        {
            myRequest.ContentType = item.Value;
        }
        else
        {
            myRequest.Headers.Add(item.Key, item.Value );
        }
    }
}

//myRequest.Headers.Add("Content-Type", "application/json");
myRequest.ContentType = "application/json";
myRequest.AllowAutoRedirect = true; 
using (var stream = myRequest.GetRequestStream())
{
    stream.Write(buf, 0, buf.Length);
}

myResponse = (HttpWebResponse)myRequest.GetResponse();
using (myResponse = (HttpWebResponse)myRequest.GetResponse())
{
    StreamReader reader = new StreamReader(myResponse.GetResponseStream(), Encoding.UTF8);
    //string returnXml = HttpUtility.UrlDecode(reader.ReadToEnd());
    returnXml = reader.ReadToEnd();
    reader.Close();
}

I tried to use fillder monitor, but when I use C#, header also have "Content-Type: application/json".

Answer: I solved it. In the api,some number should be toString.
Title: Python nested dictionary on __setitem__ only. __getitem__ raises KeyError if item is not found
Tags: python,dictionary

Question: I am trying to create a flexible nested dictionary that will raise a key error if a user tries to access a key that doesn't exist.
For a fully nested dict I've used this in the past:
nested_dict = lambda: defaultdict(nested_dict)
But I don't like that when I try to access a key that doesn't exist nested_dict returns a defaultdict. I would like to be able to add key value pairs even if the key doesn't exist, but I would like it to raise a KeyError if I try to directly access a key that doesn't exist.
I've tried things like this:
class NestedDict(dict):

    def __getitem__(self, item):
        if type(self[item]) == NestedDict:
            raise KeyError
        return self[item]

But can't figure out how to break the recursion. Any ideas how to do this?

Answer: What you're looking for isn't possible. The first part of executing d[a][b] = c is reading d[a]. If you want to forbid reading d[a] when that key isn't present, then you can't write to d[a][b] when d[a] isn't present.
Title: A memory leak in a simple Python C-extension
Tags: python,c,python-c-api

Question: I have some code similar to the one below. That code leaks, and I don't know why. The  thing that leaks is a simple creation of a Python class' instance inside a C code. The function I use to check the leak is create_n_times that's defined below and just creates new Python instances and derefrences them in a loop.
This is not an MWE per-se, but part of an example. To make it easier to understand, what the code does is:

The Python code defines the dataclass and registers it into the C-extension using set_ip_settings_type.
Then, a C-extension function create_n_times is called and that function creates and destroys n instances of the Python dataclass.

Can anyone help?
In Python:
import c_api

@dataclass
class IpSettings:
    ip: str
    port: int
    dhcp: bool

c_api.set_ip_settings_type(IpSettings)
c_api.generate_n_times(100000)

In C++ I have the following code that's compiled into a Python extension called c_api (it's a part of that library's definition):
#include <Python.h>

// ... Other functions including a "PyInit" function

extern "C" {
    
PyObject* ip_settings_type = NULL;
PyObject* set_ip_settings_type(PyObject* tp)
{
    Py_XDECREF(ip_settings_type);
    Py_INCREF(tp);
    ip_settings_type = tp;
    return Py_None;
}

PyObject* create_n_times(PyObject* n)
{
   long n_ = PyLong_AsLong(n);
   for (int i = 0; i < n_ ++i)
   {
       PyObject* factory_object = ip_settings_type;

       PyObject* args = PyTuple_New(3);
       PyTuple_SetItem(args, 0, PyUnicode_FromString("123.123.123.123"));
       PyTuple_SetItem(args, 1, PyLong_FromUnsignedLong(1231));
       PyTuple_SetItem(args, 2, Py_False);

       PyObject* obj = PyObject_CallObject(factory_object, args);
       Py_DECREF(obj);
   }

    return Py_None;
}

}

Answer: PyTuple_SetItem steals the reference to the supplied object, but Py_False is a single object. When the args tuple is destroyed, the reference count for Py_False isgetting mangled.
Use PyBool_FromLong(0) to create a new reference to Py_False, like the other two calls to PyTuple_SetItem. (see docs.python.org/3/c-api/bool.html)
Title: How to make a discord bot button that can download the file to the client's local with discord.py
Tags: python,discord,discord.py,discord-buttons

Question: I have made a button in the view class like this:
# Create Download Button
downlaodButton = Button(style=discord.ButtonStyle.grey, label='Download', row=1)
downlaodButton.callback = self.download
self.add_item(downlaodButton)

async def download(self, interaction: discord.Interaction):
 # Change Button Color And Accessbility After Clicking it
 self.children[-2].style = discord.ButtonStyle.blurple
 await interaction.response.edit_message(view=self)

 response = requests.get(self.image_url)

 with open('my_photo.png','wb') as f:
   f.write(response.content)

It does download the photos to my project directory but what I really want is when a user click that button it will start downloading the photo to their computer. Is it possible to implement with discord.py?

Answer: It's not possible with the discord.py api to make a download on a client pc. you can only upload the image in discord or give the user an url where he can download the image. If you use an url you can make it download directly depending on the browser. But the Discord Client won't download anything by clicking on a button.
Title: What is the best way to handle waiting for elements that are not always present with Selenium?
Tags: python,selenium-webdriver,webdriverwait

Question: For example if I have a code like this:
<div id='parent'>
  <div class='profile-pic'>...</div>
  <p class='email'>I'm not always present</p>
  <span class='name'>John Smith</span>
</div>

Now this is a random example. But should help to visualize what I mean. I want to grab profile picture, email and the name. None of them are necessarily always present on the page.
I could set a wait for each element separately like this:
try:
  wait = WebDriverWait(driver, 10)
  wait.until(EC.visibility_of_element_located((By.CLASS_NAME, 'email')))
except:
  pass

But that would mean every time one or more elements are not present on the page I would wait 10 seconds for each of them. Now let's say I have 20 elements like this that can be present but are not always and I have 1000 pages to go through. This approach would take forever.
Now that's where my question comes in. What is the best solution to handle cases like this? Or is this something that should not be done with Selenium at all?
What I have been doing so far is selecting the parent element and waiting for it to load but as I have learned that does not guarantee that all the children are loaded.

Answer: What you can do here is to iterate over parent elements.
For each parent element wait for it's presence.
I can't know if parent element here is visible or not.
In case parent element is visible - wait for it visibility.
Now, wait for desired child element visibility.
Here you can use a short timeout since this timeout is for use in case the parent element is already present (but maybe still not fully loaded) while child element is still need to be loaded.
2-3 seconds should be more that enough here until you have very bad internet connectivity / loading is done extremmely low.
Title: can not import CursorResult from sqlalchemy
Tags: python,sqlalchemy,langchain

Question: I am using langchain tool with streamlit and after running py file getting error that cannot import CursorResult from sqlalchemy

Answer: Try this:
pip install -U sqlalchemy
Title: can not import CursorResult from sqlalchemy
Tags: python,sqlalchemy,langchain

Question: I am using langchain tool with streamlit and after running py file getting error that cannot import CursorResult from sqlalchemy

Answer: Do pip install langchain==0.0.157
It will solve the issue.
Title: Dbt: not found .dbt folder on default default location
Tags: python,database,installation,snowflake-cloud-data-platform,dbt

Question: I was creating my dbt project, but when running the command dbt init <name_project> it returned the error that it can't find profiles.yml in the .dbt folder. So I try to access the folder on default location and it's not there. Is there a problem install dbt-snowflake inside a python virtual env that changes the installation default behavior?

Created virtuaenv:
python3 -m venv venv
Actvated venv:
source venv/bin/activate
Updated pip:
pip install --upgrade pip wheel setuptools
Install dbt-snowflake:
pip install dbt-snowflake
Trying create a project:
bt init <name_project>
But I receive this error message:

Usage: dbt init [OPTIONS] [PROJECT_NAME]
Try 'dbt init -h' for help.
Error: Invalid value for '--profiles-dir': Path '/Users/brunofonseca/.dbt' does not exist.

I saw the default location is ~/.dbt, so I tried to access the folder but it is not exist there. I also tried run the command dbt debug --config-dir to find where is the folder, but returns:
04:40:15  Running with dbt=1.5.0
04:40:15  [OpenCommand]: Unable to parse dict {'open_cmd': 'open', 'profiles_dir': PosixPath('/Users/brunofonseca/.dbt')}
04:40:15  To view your profiles.yml file, run:

Finally. I tried to find the profiles.yml inside the venv folder, but it was not found.
OBS: the operation system is macos.

Answer: Access the folder through Terminal where you want to init your dbt project, then type:

dbt init <name_project> --profiles-dir .
Title: Dbt: not found .dbt folder on default default location
Tags: python,database,installation,snowflake-cloud-data-platform,dbt

Question: I was creating my dbt project, but when running the command dbt init <name_project> it returned the error that it can't find profiles.yml in the .dbt folder. So I try to access the folder on default location and it's not there. Is there a problem install dbt-snowflake inside a python virtual env that changes the installation default behavior?

Created virtuaenv:
python3 -m venv venv
Actvated venv:
source venv/bin/activate
Updated pip:
pip install --upgrade pip wheel setuptools
Install dbt-snowflake:
pip install dbt-snowflake
Trying create a project:
bt init <name_project>
But I receive this error message:

Usage: dbt init [OPTIONS] [PROJECT_NAME]
Try 'dbt init -h' for help.
Error: Invalid value for '--profiles-dir': Path '/Users/brunofonseca/.dbt' does not exist.

I saw the default location is ~/.dbt, so I tried to access the folder but it is not exist there. I also tried run the command dbt debug --config-dir to find where is the folder, but returns:
04:40:15  Running with dbt=1.5.0
04:40:15  [OpenCommand]: Unable to parse dict {'open_cmd': 'open', 'profiles_dir': PosixPath('/Users/brunofonseca/.dbt')}
04:40:15  To view your profiles.yml file, run:

Finally. I tried to find the profiles.yml inside the venv folder, but it was not found.
OBS: the operation system is macos.

Answer: you just need to create the .dbt folder in your home directory beforehand, then run the dbt init <name_project> that will automatically create a profiles.yml in your .dbt file.
Title: Having this error : Message: unknown error: Runtime.callFunctionOn threw exception: Error: LavaMoat - property "JSON"
Tags: python,selenium-webdriver,selenium-chromedriver

Question: After updating chrome to 114.0.5735.16, there is error when i try to call elements through xpath, id..
driver get function and refresh seems to work fine
is there something i need to change?
example of codes are these
ex)
element = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "password")))

or
driver.find_element(By.XPATH,"/html/body/div[1]/div/div[3]/div/div/button")

all the errors are same as below
Message: unknown error: Runtime.callFunctionOn threw exception: Error: LavaMoat - property "JSON" of globalThis is inaccessible under scuttling mode. To learn more visit https://github.com/LavaMoat/LavaMoat/pull/360.
  at get (chrome-extension://nkbihfbeogaeaoehlefnkodbefgpgknn/runtime-lavamoat.js:11200:17)
  at buildError (<anonymous>:323:13)
  (Session info: chrome=114.0.5735.16)
Stacktrace:
Backtrace:
    GetHandleVerifier [0x00496E73+48323]
    (No symbol) [0x00429661]
    (No symbol) [0x00335308]
    (No symbol) [0x003384D6]
    (No symbol) [0x00339991]
    (No symbol) [0x00339A30]
    (No symbol) [0x003607BC]
    (No symbol) [0x00360CDB]
    (No symbol) [0x0038E3D2]
    (No symbol) [0x0037A924]
    (No symbol) [0x0038CAC2]
    (No symbol) [0x0037A6D6]
    (No symbol) [0x0035847C]
    (No symbol) [0x0035957D]
    GetHandleVerifier [0x006FFD5D+2575277]
    GetHandleVerifier [0x0073F86E+2836158]
    GetHandleVerifier [0x007396DC+2811180]
    GetHandleVerifier [0x005241B0+626688]
    (No symbol) [0x0043314C]
    (No symbol) [0x0042F4B8]
    (No symbol) [0x0042F59B]
    (No symbol) [0x004221B7]
    BaseThreadInitThunk [0x77000099+25]
    RtlGetAppContainerNamedObjectPath [0x77207B6E+286]
    RtlGetAppContainerNamedObjectPath [0x77207B3E+238]

all the find element functions, showing the same error

Answer: I have encountered the same problem, and my approach is to lower the Chrome version