eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: When hashing the same email address in two DataFrames, I am returned different hashes. These two dataframes, df1 and df2, each contain a column of email addresses which need to be hashed, so the hashes can be compared when they are inner joined, like this: import pandas as pd ### Boring part to import the data ### # define table 1 as df1 df1 = pd.DataFrame([[2, 'random.email@fjjd.com'], [6, 'different.email@u8888ueend.dskjs'], [7, 'random.email@dsju.c'], [8, 'same.email@dkicj.c'], [200, 'different.email@cjhs.oo'], [18, 'random.email@siidjd.dd'], [19, 'random.email@dsjds.j']]) df1 = df1.set_axis(['ID1', 'email 1'], axis=1) # define table 2 as df2 df2 = pd.DataFrame([[100, 'new.email@jsscd.d'], [6, 'different.email@uueend.dskjs'], [99, 'new.email@djjsd.conm'], [10, 'new.email@jhhs.co'], [115, 'new.email@dsjjdsds.cod'], [116, 'new.email@dsjkjds.ckk'], [8, 'same.email@dkicj.c'], [200, 'different.email@jdsjd.co']]) df2 = df2.set_axis(['ID2', 'email 2'], axis=1) ### End part to import the data ### ### Fun part now... ### # hash the emails in each row of df1? df1['hash 1'] = pd.util.hash_pandas_object(df1['email 1'].astype(str)) # hash the emails in each row of df2? df2['hash 2'] = pd.util.hash_pandas_object(df2['email 2'].astype(str)) # perform an inner join of df1 and df2 about their IDs, ID1 and ID2 respectively df3 = pd.merge(df1, df2, how='inner', left_on='ID1', right_on='ID2') # add an email comparison column df3['same email'] = df3['email 1'] == df3['email 2'] # add a hash comparison column df3['same hash'] = df3['hash 1'] == df3['hash 2'] # print the table... print(df3)  The result shows that while the email addresses in row 1 are identical (as far as I can tell) the hashes are not the same:  ID1 email 1 hash 1 ID2 email 2 hash 2 same email same hash 0 6 different.email@u8888ueend.dskjs 18381560226251184406 6 different.email@uueend.dskjs 16113553761483526335 False False 1 8 same.email@dkicj.c 5780217243550696535 8 same.email@dkicj.c 6939369575697951555 True False 2 200 different.email@cjhs.oo 13252009090739560311 200 different.email@jdsjd.co 1942861278265138167 False False  Why are these hashes of the same email address from different DataFrames different to one another?

Answer: According to the documentation, the default mode of operation is including index in hash computation. So when two same emails have different indices, the hash is different. You can try: df1["hash 1"] = pd.util.hash_pandas_object(df1["email 1"].astype(str), index=False) df2["hash 2"] = pd.util.hash_pandas_object(df2["email 2"].astype(str), index=False)  Then the result will be:  ID1 email 1 hash 1 ID2 email 2 hash 2 0 6 different.email@u8888ueend.dskjs 5185970979410096600 6 different.email@uueend.dskjs 18338061231746973003 1 8 same.email@dkicj.c 9881121729072933860 8 same.email@dkicj.c 9881121729072933860 2 200 different.email@cjhs.oo 742268446511091656 200 different.email@jdsjd.co 775994242592712264   Other method of hash computation is using the built-in hash function: df1["hash 1"] = df1["email 1"].apply(hash) df2["hash 2"] = df2["email 2"].apply(hash)
Question: using LangChain and OpenAI, how can I have the model return a specific default response? for instance, let's say I have these statement/responses Statement: Hi, I need to update my email address. Answer: Thank you for updating us. Please text it here. Statement: Hi, I have a few questions regarding my case. Can you call me back? Answer: Hi. Yes, one of our case managers will give you a call shortly.  if the input is similar to one of the above statements, I would like to have OpenAI respond with the specific answer.

Answer: You can handle this with precise set of examples and role of AI assistant. I am setting verbose = 1 in LLMChain so that you can see the observation/execution..  from langchain.prompts import PromptTemplate from langchain.prompts import FewShotPromptTemplate from langchain.chat_models import ChatOpenAI from langchain.chains import LLMChain examples = [ { "query": "What's the weather like?", "answer": "It's raining cats and dogs, better bring an umbrella!" }, { "query": "How old are you?", "answer": "Age is just a number, but I'm timeless." }, { "query":"Could you update my email address", "answer":"Thank you for updating us. Please text it here" }, { "query":"I have a few questions regarding my case. Can you call me back?", "answer":"Yes, one of our case managers will give you a call shortly" } ] example_template = """ User:{query}, AI:{answer} """ example_prompt = PromptTemplate( input_variables=["query", "answer"], template=example_template ) # prefix= """ The following are excerpts from conversations with an AI # assistant. The assistant is known for its humor and wit, providing # entertaining and amusing responses to users' questions. Here are some # examples:""" prefix= """ The following are excerpts from conversations with an AI assistant. The assistant is known for its accurate responses to users' questions. Here are some examples:""" suffix=""" User:{query}, AI: """ few_shot_template = FewShotPromptTemplate( examples=examples, example_prompt=example_prompt, prefix=prefix, suffix=suffix, input_variables=["query"], example_separator="\n\n" ) chat = ChatOpenAI(model_name="gpt-3.5-turbo-0301", temperature=0.0) chain = LLMChain(llm=chat, prompt=few_shot_template, verbose=1) print(chain.run("what's meaning of life ?")) print(chain.run("Could you update my email address ?")) print(chain.run("I have a few questions regarding my case. Can you call me back?"))
Question: I have a an expression, i.e. 6*x**3 + 2*x**(1/2) and want to select the item with the lowest exponent, in this case 2*x**(1/2). Is there a simple way to do that? Working with args gives an unsorted list of items and working with Poly is not possible here.

Answer: Try using as_ordered_terms and as_coeff_exponent with min: >>> from sympy import symbols >>> x = symbols('x') >>> expr = 6*x**3 + 2*x**(1/2) >>> terms = expr.as_ordered_terms() >>> lowest_term = min(terms, key=lambda term: term.as_coeff_exponent(x)[1]) >>> lowest_term 2*x**0.5
Question: I want to copy a file generated by a docker container, and store inside it to my local host. What is a good way of doing that? The docker container is ephemeral (i.e. it runs for a very short time and then stops.) I am working with the below mentioned scripts: Python (script.py) which generates and saves a file titled read.txt. with open('read.txt', 'w') as file: lst = ['Some\n', 'random\n', 'sentence\n'] file.writelines("% s\n" % words for words in lst)  I use the below Dockerfile: FROM python:3.9 WORKDIR /app RUN pip install --upgrade pip COPY . /app/ RUN pip install --requirement /app/requirements.txt CMD ["python", "/app/script.py.py"]  Below is my folder structure: - local - folder1 - script.py - requirements.txt - Dockerfile - folder2  Till now, I have managed to successfully build a docker container using: docker build --no-cache -t test:v1 .  When I run this docker container inside /local/folder1/ using the below command, I get the desired file, i.e. read.txt inside /local/folder1/ docker run -v /local/folder1/:/app/ test:v1  But, when I run docker run -v /local/folder2/:/app/ test:v1 inside /local/folder2/, I do not see read.txt inside /local/folder2/ and I get the below message. python: can't open file '/app/script.py': [Errno 2] No such file or directory  I want to be able to get read.txt inside /local/folder2/ when I run the docker container test:v1 inside /local/folder2/. How can I do it? I want to be able to do it without copying the contents of /local/folder1/ inside /local/folder2/. The docker container is ephemeral (i.e. it runs for a very short time and then stops.) Hence answers given in this and this Stackoverflow posts, which focus on docker cp have not worked for me. Time of running of the abovementioned container is not very essential. Even if a workable solution increases the time of running of a container, that is okay.

Answer: Since the container is ephemeral, you may want to run the container and keep it running. And then copy the file. Try the below steps: 1. Keep the container running and run the python command. docker run -it --name temp test:v1 bash -c 'python script.py'  Here temp is the name of a temporary container. 2. Copy file from temp container to local host. docker cp temp:/app/read.txt /local/folder2/.  3. Remove the temporary container. docker rm temp
Question: I'm executing a python file with powershell and I would like to get the log in a file and on console also, so tried to use the tee command. cd folder .\program.py | tee log.txt  But error was thrown back Cannot run a document in the middle of a pipeline  Python file just contains 1 line of code print "test"  How can I use this command at the right way?

Answer: You have to call python to execute the script. Try executing in the folder where program.py and log.txt is: python program.py | tee log.txt  The explanation is that ./program.py without python command tries to run or open the file, but you are not executing it. Then, you try to to pass the ouput to log.txt with the pipe operator and tee. So it is the expected behaviour. Aditionally, you are using python 2 syntax which is deprecated since ages. Try using print("test") inside the script.
Question: I have a dataset that looks like (with more columns and rows):  id type value 0 104 0 7999 1 105 1 196193579 2 108 0 245744 3 NaN 1 NaN  Some rows have NaN values, and I already have the indexes for these rows. Now I would like to concat these rows with their previous row, except for NaN values. If I say indexes=[3], then the new dataframe should be:  id type value 0 104 0 7999 1 105 1 196193579 2 108 01 245744  How can I do this? NOTE: First row never will be in the list of indexes I have. The solution must be given my list of indexes. I also know the names of the columns where NaN values are, if necessary.

Answer: If you have an external list of indices to merge with the rows above you can use: indexes=[3] out = (df .astype({'type': str}) .groupby((~df.index.to_series().isin(indexes)).cumsum()) .agg({'id': 'first', 'type': ''.join, 'value': 'first'}) )  If you have many columns build the aggregation dictionary programmatically: indexes=[3] d = {k: ''.join if t == object else 'first' for k, t in df.dtypes.items()} out = (df .astype({'type': str}) .groupby((~df.index.to_series().isin(indexes)).cumsum()) .agg(d) )  Output:  id type value 1 104.0 0 7999.0 2 105.0 1 196193579.0 3 108.0 01 245744.0
Question: I am using the toml package to read my pyproject.toml file. I want to add custom data which in this case I will read in my docs/conf.py file. When I try and add a custom section I get errors and warnings from Even Better TOML extension in Vs Code stating that my custom data is no allowed. Example TOML section in pyproject.toml [custom.metadata] docs_name = "GUI Automation for windows" docs_author = "myname" docs_copyright = "2023" docs_url= "https://someurl.readthedocs.io/en/latest/"  So, my question is: Is there a valid way of adding custom data to a pyproject.toml file?

Answer: Use a [tool.*] table. Quoting PEP 518:  The [tool] table is where any tool related to your Python project, not just build tools, can have users specify configuration data as long as they use a sub-table within [tool], e.g. the flit tool would store its configuration in [tool.flit].  Any tools that interpret pyproject.toml files will expect the [tool] sub-tables to contain arbitrary, tool-specific metadata. You should not have any issues populating a custom subtable with whatever project metadata you need. In your case, something like this should work warning-free: [tool.my_distribution_name] docs_name = "GUI Automation for windows" docs_author = "myname" docs_copyright = "2023" docs_url= "https://someurl.readthedocs.io/en/latest/"  Creating your own custom table is not permitted, since PEP 518 reserves all top-level tables in pyproject.toml:  Tables not specified in this PEP are reserved for future use by other PEPs.
Question: I have this clock app written in python. It runs successfully on Windows. Here is the source code. # Source: https://www.geeksforgeeks.org/python-create-a-digital-clock-using-tkinter/ # importing whole module from tkinter import * from tkinter.ttk import * # importing strftime function to # retrieve system's time from time import strftime # creating tkinter window root = Tk() root.title('Clock') # This function is used to # display time on the label def time(): string = strftime('%H:%M:%S %p') lbl.config(text=string) lbl.after(1000, time) # Styling the label widget so that clock # will look more attractive lbl = Label(root, font=('calibri', 40, 'bold'), background='purple', foreground='white') # Placing clock at the centre # of the tkinter window lbl.pack(anchor='center') time() mainloop()  This is how the clock app looks.  It works fine. However, I want to make the app always appear on top in Windows. How can I modify the code to make the app always appear on top? I am using Windows 11.

Answer: By adding the line root.attributes('-topmost', True), you set the window to be always on top of other windows. Update your code as follow: # importing whole module from tkinter import * from tkinter.ttk import * # importing strftime function to # retrieve system's time from time import strftime # creating tkinter window root = Tk() root.title('Clock') root.attributes('-topmost', True) # Set the window to always appear on top # This function is used to # display time on the label def time(): string = strftime('%H:%M:%S %p') lbl.config(text=string) lbl.after(1000, time) # Styling the label widget so that clock # will look more attractive lbl = Label(root, font=('calibri', 40, 'bold'), background='purple', foreground='white') # Placing clock at the centre # of the tkinter window lbl.pack(anchor='center') time() mainloop()
Question: I'm encountering different overflow behavior on Mac vs Linux with the same version of numpy. MWE: import numpy as np arr = np.arange(0, 2 * 4e9, 1e9, dtype=float) print(arr.astype(np.uint32)) print(np.__version__)  Mac (Python 3.9.13): array([ 0, 1000000000, 2000000000, 3000000000, 4000000000, 705032704, 1705032704, 2705032704], dtype=uint32) '1.22.4'  Linux (Python 3.9.7): array([ 0, 1000000000, 2000000000, 3000000000, 4000000000, 0, 0, 0], dtype=uint32) '1.22.4'  I would prefer the "Mac" behavior of the expected rollover (rather than forcing overflowed values to 0), so I would like to know how to fix this for the Linux version.

Answer: I believe that this is due to the underlying C implementation of numpy, which probably triggers undefined behaviour, which is handled differently by the compiler used for the linux and mac distributions of numpy. Looking at cast float to unsigned int in C with gcc which deals with a similar topic, we can also see link to the C standard which states on page 51  6.3.1.4 Real floating and integer 1 When a finite value of real floating type is converted to an integer type other than _Bool, the fractional part is discarded (i.e., the value is truncated toward zero). If the value of the integral part cannot be represented by the integer type, the behavior is undefined.61)  One possibility would be I guess that you try out different compilers in compiling your own numpy version and check the behaviour. Alternatively, since you want the roll-over behaviour, you could try to convert first to an unsigned integer type of larger width and then to the smaller width, since unsigned integer conversions always do a roll-over  6.3.1.3 Signed and unsigned integers 1 When a value with integer type is converted to another integer type other than _Bool, if the value can be represented by the new type, it is unchanged. 2 Otherwise, if the new type is unsigned, the value is converted by repeatedly adding or subtracting one more than the maximum value that can be represented in the new type until the value is in the range of the new type.60)  import numpy as np arr = np.arange(0, 2 * 4e9, 1e9, dtype=float) print(arr.astype(np.uint64).astype(np.uint32)) print(np.__version__)
Question: I am trying to read a flat file, assign column names to the dataframe and cast few columns as per my requirements. However while casting a column to Decimal gives me error in Polars. I implemented the same successfully in Spark, but need help if anyone can guide me how to do the same in Polars. sample data data = b""" D|120277|Patricia|167.2|26/12/1982 D|757773|Charles|167.66|08/04/2019 D|248498|Katrina|168.68|20/11/2016 D|325561|Christina|170.86|05/10/1998 D|697464|Joshua|171.41|07/09/1970 D|244750|Zachary|169.43|08/12/2014 """.strip()  Polars Script import polars as pl cols_dict = {'column_1': 'rtype', 'column_2': 'EMP_ID', 'column_3': 'EMP_NAME', 'column_4': 'SALARY', 'column_5': 'DOB'} df = pl.read_csv(data, separator='|', has_header=False) df = df.select(pl.all().name.map(lambda col_name: cols_dict.get(col_name))) df = df.with_columns( pl.col('EMP_ID').cast(pl.Decimal(scale=6, precision=0)))  It throws me exception:  InvalidOperationError: conversion from i64 to decimal[1,6] failed in column 'EMP_ID' for 5 out of 5 values: [120277, 757773, … 697464]  Spark Script: rdd = sparkContext.textFile(core_data_file).filter(lambda x: x\[0\] == "D").map( lambda x: x.split('|')) sparkDf= rdd.toDF(schema=\["rtype"\] + list(cols_dict .values())) sparkDf= sparkDf.withColumn(col_name, coalesce( col(col_name).cast(DecimalType(data_length, data_scale)), lit(0))) sarkDf.show()  +-----+------+---------+------+-------------------+ |rtype|EMP_ID|EMP_NAME |SALARY|DOB | +-----+------+---------+------+-------------------+ |D |120277|Patricia |167.20|1982-12-26 00:00:00| |D |757773|Charles |167.66|2019-04-08 00:00:00| |D |248498|Katrina |168.68|2016-11-20 00:00:00| |D |325561|Christina|170.86|1998-10-05 00:00:00| |D |697464|Joshua |171.41|1970-09-07 00:00:00| |D |244750|Zachary |169.43|2014-12-08 00:00:00| +-----+------+---------+------+-------------------+ 

Answer: I believe the scale and precision parameters are invalid. scale specifies the number of digits to the right of the decimal point, while precision specifies the total number of digits in the number. Setting precision = 0 is invalid. If you want 6 decimals, then rewrite precision as: pl.col('EMP_ID').cast(pl.Decimal(scale=6, precision=None)))  I noticed that pl.Decimal is an "experimental work-in-progress feature and may not work as expected" Polars docs. An alternative is to use pl.Float64 instead: df = df.with_columns(pl.col('EMP_ID').cast(pl.Float64))
Question: Let's say that a pytest session fixture returns a dictionary and this dictionary is then manipulated within a test. Would this affect subsequent tests which use the same fixture?

Answer: Yes. And that is trivial to verify. Exactly one of these tests will fail: import pytest @pytest.fixture(scope="session") def d(): return {} def test_one(d): assert not d d["k"] = "v" def test_two(d): assert not d d["k"] = "v"  If the fixture scope is changed to "function" (the default scope) then both tests will pass, since the dict will be created anew each time.
Question: I am getting the following error:  selenium.common.exceptions.SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 114 Current browser version is 116.0.5845.97 with binary path C:\Program Files\Google\Chrome\Application\chrome.exe  Can anyone help me? I've seen some suggestions in other posts but none worked here. I am aware of the Selenium v4.6 update My code: from selenium import webdriver import time from selenium.webdriver.chrome.service import Service from selenium.webdriver.common.by import By class ChromeAuto: def __init__(self): self.options = webdriver.ChromeOptions() self.options.add_experimental_option('excludeSwitches', ['enable-logging']) self.service = Service() self.chrome = webdriver.Chrome(service=self.service, options=self.options) self.chrome.implicitly_wait(20) 

Answer: For Chrome 116+ you'll need selenium 4.11.2 at a minimum for the Selenium Manager to download chromedriver 116+ from https://googlechromelabs.github.io/chrome-for-testing/ Then you'll be able to run basic Selenium scripts like this: from selenium import webdriver from selenium.webdriver.chrome.service import Service service = Service() options = webdriver.ChromeOptions() options.add_experimental_option('excludeSwitches', ['enable-logging']) driver = webdriver.Chrome(service=service, options=options) # Add your code here driver.quit()
Question: I'm finding that inheriting from a base class can change the derived class signature according to inspect.signature, and I would like to understand how that happens. Specifically, the base class in question is tensorflow.keras.layers.Layer: import sys import inspect import tensorflow as tf class Class1(tf.keras.layers.Layer): def __init__(self, my_arg: int): pass class Class2: def __init__(self, my_arg: int): pass print("Python version: ", sys.version) print("Tensorflow version: ", tf.__version__) print("Class1 signature: ", inspect.signature(Class1)) print("Class2 signature: ", inspect.signature(Class2))  Outputs Python version: 3.8.10 (default, Mar 23 2023, 13:10:07) [GCC 9.3.0] Tensorflow version: 2.12.0 Class1 signature: (*args, **kwargs) Class2 signature: (my_arg: int)  I tried running the code above and I expected it to print the same signature for both classes.

Answer: This is a bug behavior that is unique to earlier versions of Python, including 3.8 and early patch releases of Python 3.9/3.10 In 3.8 and 3.9.4: Class1 signature: (*args, **kwargs) Class2 signature: (my_arg: int)  In newer versions of Python, like 3.9.17 and latest versions of 3.10 and 3.11, the result is how you would expect: Class1 signature: (my_arg: int) Class2 signature: (my_arg: int)  I'm not 100% sure what exactly changed the behavior between these python versions, but my best guess is that this is related to either this issue or this issue, which were fixed in Python3.11 and backported to some earlier Python versions. However, Python3.8 did not receive any of these backport fixes.
Question: The install is successful. I receive this error when trying to import. TypeError: 'numpy._DTypeMeta' object is not subscriptable   I have tried higher and lower versions of numpy (1.22.0,1.23.0,1.24.0,1.25.0,1.25.2). I installed mediapipe via pypi as well as a downloaded whl ( mediapipe-0.10.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl) from pypi Versions numpy 1.21.5 mediapipe 0.10.3 Python 3.10.6  These questions are similar but not the answer. Importing xarray raises not subscriptable issue Cannot import mediapipe in Jupyter notebook How to prevent error message when importing import cv2?

Answer: Updating numpy to 1.23 works. I was not restarting the kernel after installing versions of numpy. With notebook-scoped libraries, use dbutils.library.restartPython(). If you are using cluster-scoped libraries, then be sure to restart your cluster. There's an issue about this that's documented on the OpenCV repo here: https://github.com/opencv/opencv/issues/23822 In short, it imported successfully with most numpy>=1.22.0 versions. Just be careful, if you use some of the most recent library versions, like numpy>=1.25.0, you start running into other compatibility issues.