eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have a dataset looks like this:     Car Make Model Engine     Toyota Rav 4 8cyl6L Toyota  8cyl6L   Mitsubishi Eclipse 2.1T Mitsubishi  2.1T   Monster Gravedigger 25Lsc Monster  25Lsc     The data was clearly concatenated from Make + Model + Engine at some point but the car Model was not provided to me. I've been trying to use Pandas to say that if we take Car, replace instances of Make with a nothing, replace instances of Engine with nothing, then trim the spaces around the result, we will have Model.     Car Make Model Engine     Toyota Rav 4 8cyl6L Toyota Rav 4 8cyl6L   Mitsubishi Eclipse 2.1T Mitsubishi Eclipse 2.1T   Monster Gravedigger 25Lsc Monster Gravedigger 25Lsc     There's something I'm doing wrong when I'm trying to reference another column in this manner. df['Model'] = df['Car'].str.replace(df['Make'],'')  gives me an error of "unhashable type: 'Series'". I'm guessing I'm accidentally inputting the entire 'Make' column. At every row I want to make a different substitution using data from other columns in that row. How would I accomplish this?

Answer: you can use: df['Model']=df.apply(lambda x: x['Car'].replace(x['Make'],"").replace(x['Engine'],""),axis=1) print(df) ''' Car Make Model Engine 0 Toyota Rav 4 8cyl6L Toyota Rav 4 8cyl6L 1 Mitsubishi Eclipse 2.1T Mitsubishi Eclipse 2.1T 2 Monster Gravedigger 25Lsc Monster Gravedigger 25Lsc '''
Question: When using SQLAlchemy (version 1.4.44) to create, drop or otherwise modify tables, the updates don't appear to be committing. Attempting to solve this, I'm following the docs and using the commit() function. Here's a simple example from sqlalchemy import create_engine, text engine = create_engine("postgresql://user:password@connection_string:5432/database_name") with engine.connect() as connection: sql = "create table test as (select count(1) as result from userquery);" result = connection.execute(text(sql)) connection.commit()  This produces the error: AttributeError: 'Connection' object has no attribute 'commit'  What am I missing?

Answer: The comment on the question is correct you are looking at the 2.0 docs but all you need to do is set future=True when calling create_engine() to use the "commit as you go" functionality provided in 2.0. SEE migration-core-connection-transaction  When using 2.0 style with the create_engine.future flag, “commit as you go” style may also be used, as the Connection features autobegin behavior, which takes place when a statement is first invoked in the absence of an explicit call to Connection.begin():
Question: I want to compare execution time of two snippets and see which one is faster. So, I want an accurate method to measure execution time of my python snippets. I already tried using time.time(), time.process_time(), time.perf_counter_ns() as well as timeit.timeit(), but I am facing the same issues with all of the them. That is: when I use any of the above methods to measure execution time of THE SAME snippet, it returns a different value each time I run it. And this variation is somewhat significant, to the extent that I cannot reliably use them to compare difference in execution time of two snippets. As an example, I am running following code in my google colab: import time t1 = time.perf_counter() sample_list = [] for i in range(1000000): sample_list.append(i) t2 = time.perf_counter() print(t2 - t1)  I ran above code 10 times and the variation in my results is about 50% (min value = 0.14, max value = 0.28). Any alternatives?

Answer: The execution time of a given code snippet will almost always be different every time you run it. Most tools that are available for profiling a single function/snippet of code take this into account, and run the code multiple times to be able to provide an average execution time. The reason for this is that there are other processes running on your computer, and resources are not always allocated the same way, so it is impossible to control every variable so that you get the same execution time for every run. One of the easiest ways to profile a given function or short snippet of code is using the %timeit "magic" command in ipython. Example: >>> %timeit 1 + 1 8.41 ns ± 0.0181 ns per loop (mean ± std. dev. of 7 runs, 100,000,000 loops each)  It also allows you to enter a multi-line block of code to time if you use %%timeit instead of %timeit. The timeit library can be used independently, but it is often easier to use in an interactive ipython session. Additional resources:  How to profile my code? What is %timeit in Python? How can I time a code segment for testing performance with Pythons timeit?
Question: Say I instantiated a random generator with import numpy as np rng = np.random.default_rng(seed=42)  and I want to change its seed. Is it possible to update the seed of the generator instead of instantiating a new generator with the new seed? I managed to find that you can see the state of the generator with rng.__getstate__(), for example in this case it is {'bit_generator': 'PCG64', 'state': {'state': 274674114334540486603088602300644985544, 'inc': 332724090758049132448979897138935081983}, 'has_uint32': 0, 'uinteger': 0}  and you can change it with rng.__setstate__ with arguments as printed above, but it is not clear to me how to set those arguments so that to have the initial state of the rng given a different seed. Is it possible to do so or instantiating a new generator is the only way?

Answer: N.B. The other answer (https://stackoverflow.com/a/74474377/2954547) is better. Use that one, not this one.  This is maybe a silly hack, but one solution is to create a new RNG instance using the desired new seed, then replace the state of the existing RNG instance with the state of the new instance: import numpy as np seed = 12345 rng = np.random.default_rng(seed) x1 = rng.normal(size=10) rng.__setstate__(np.random.default_rng(seed).__getstate__()) x2 = rng.normal(size=10) np.testing.assert_array_equal(x1, x2)  However this isn't much different from just replacing the RNG instance. Edit: To answer the question more directly, I don't think it's possible to replace the seed without constructing a new Generator or BitGenerator, unless you know how to correctly construct the state data for the particular BitGenerator inside your Generator. Creating a new RNG is fairly cheap, and while I understand the conceptual appeal of not instantiating a new one, the only alternative here is to post a feature request on the Numpy issue tracker or mailing list.
Question: I'm working on a FastAPI application, and I want to create multi-part paths. What I mean by this is I know how to create a path like this for all the REST methods: /api/people/{person_id}  but what's a good way to create this: /api/people/{person_id}/accounts/{account_id}  I could just keep adding routes in the "people" routes module to create the additional accounts paths, but I feel like there should be a separate "accounts" routes module that could be included in the "people" routes module, and I'm just missing something. Am I over-thinking this?

Answer: In addition to what I have mentioned in the comments, would something like this be of use? from fastapi import FastAPI, APIRouter app = FastAPI() people_router = APIRouter(prefix='/people') account_router = APIRouter(prefix='/{person_id}/accounts') @people_router.get('/{person_id}') def get_person_id(person_id: int) -> dict[str, int]: return {'person_id': person_id} @account_router.get('/{account_id}') def get_account_id(person_id: int, account_id: int) -> dict[str, int]: return {'person_id': person_id, 'account_id': account_id} people_router.include_router(account_router) app.include_router(people_router, prefix='/api')
Question: TorchVision Detection models have a weights and a weights_backbone parameter. Does using pretrained weights imply that the model uses pretrained weights_backbone under the hood? I am training a RetinaNet model and I'm not sure which of the two options I should use and what the differences are.

Answer: The difference is pretty simple: you can either choose to do transfer learning on the backbone only or on the whole network. RetinaNet from Torchvision has a Resnet50 backbone. You should be able to do both of:  retinanet_resnet50_fpn(weights=RetinaNet_ResNet50_FPN_Weights.COCO_V1) retinanet_resnet50_fpn(backbone_weights=ResNet50_Weights.IMAGENET1K_V1)  As implied by their names, the backbone weights are different. The former were trained on COCO (object detection) while the later were trained on ImageNet (classification). To answer your question, pretrained weights implies that the whole network, including backbone weights, are initialized. However, I don't think that it calls backbone_weights under the hood.
Question: I may have messed up some environmental path variables. I was tinkering around VS Code while learning about Django and virtual environments, and changing the directory path of my Python install. While figuring out how to point VS Code's default Python path, I deleted some User path variables. Then, isort began to refuse to run. I've tried uninstalling the extension(s), deleting the ms-python.'s, and uninstalling VS Code itself, clearing the Python Workspace Interpreter Settings, and restarting my computer. Even if it's not my path variables, anyone know the defaults that should be in the "user" paths variables?

Answer: I ended up refreshing my Windows install. Was for the best because I'm repurposing an older machine anyway.
Question: def bot_start(): application = ApplicationBuilder().token("api_key").build() async def stop(update, context): await context.bot.send_message(chat_id=update.message.chat_id, text='Terminating Bot...') await application.stop() await Updater.shutdown(application.bot) await application.shutdown() async def error(update, context): err = f"Update: {update}\nError: {context.error}" logging.error(err, exc_info=context.error) await context.bot.send_message(chat_id=user_id, text=err) application.add_handler(CommandHandler('stop', stop)) application.add_error_handler(error) application.run_polling()  I tried everything I could to stop it and I couldnt as it's not letting other lines of code that comes after calling bot_start() run. It basically never reaches them.

Answer: Application.run_polling is a convenience methods that starts everything and keeps the bot running until you signal the process to shut down. It's mainly intended to be used if the Application is the only long-running thing in your python process. If you want to run other things alongside your bot, you can instead manually call the methods listed in the docs of run_polling. You may also want to have a look at this example, where this is showcased for a setup for a custom webhook server is used instead of PTBs built-in one.  Disclaimer: I'm currently the maintainer of python-telegram-bot.
Question: I'm trying to draw a circle in a picture using open CV with Python. Here is the picture I wish I can make :  Here is the code I write : import cv2 import numpy as np import imutils text1 = "10x" text2 = "20gr" # Load image in OpenCV image = cv2.imread('Sasa.jfif') resized = imutils.resize(image, width=500) cv2.circle(resized,(350,150),65,(102,51,17),thickness=-1) # Convert the image to RGB (OpenCV uses BGR) cv2_im_rgb = cv2.cvtColor(resized,cv2.COLOR_BGR2RGB) # Pass the image to PIL pil_im = Image.fromarray(cv2_im_rgb) draw = ImageDraw.Draw(pil_im) # use a truetype font font1 = ImageFont.truetype("arial.ttf", 50) font2 = ImageFont.truetype("arial.ttf", 25) # Draw the text draw.text((310,110), text1, font=font1) draw.text((325,170), text2, font=font2) # Get back the image to OpenCV cv2_im_processed = cv2.cvtColor(np.array(pil_im), cv2.COLOR_RGB2BGR) cv2.imshow('Fonts', cv2_im_processed) cv2.waitKey(1)  But this is what my code generate :  The circle line is not precise. Is there anything I can do to make the line preciser or is there any other library that generates circle with precise line ? Any suggestion will be very appreciated !

Answer: You can use anti aliasing to make the circle look better as described here: cv2.circle(resized,(350,150),65,(102,51,17),thickness=-1,lineType=cv2.LINE_AA)
Question: Say I have 2 list columns like below: group1 = [['John', 'Mark'], ['Ben', 'Johnny'], ['Sarah', 'Daniel']] group2 = [['Aya', 'Boa'], ['Mab', 'Johnny'], ['Sarah', 'Peter']] df = pd.DataFrame({'group1':group1, 'group2':group2})  I want to compare the two list columns and remove the list elements from group1 if they are present in group2. So expected results for above:  group1 group2 ['John', 'Mark'] ['Aya', 'Boa'] ['Ben'] ['Mab', 'Johnny'] ['Daniel'] ['Sarah', 'Peter']  How can I do this? I have tried this: df['group1'] = [[name for name in df['group1'] if name not in df['group2']]]  But got errror: TypeError: unhashable type: 'list'  Please help.

Answer: You need to zip the two Series. I'm using a set here for efficiency (this is not critical if you have only a few items per list): df['group1'] = [[x for x in a if x not in S] for a, S in zip(df['group1'], df['group2'].apply(set))]  Output:  group1 group2 0 [John, Mark] [Aya, Boa] 1 [Ben] [Mab, Johnny] 2 [Daniel] [Sarah, Peter]
Question: I have a trained scikit-learn model pipeline (including a SimpleImputer) that I'm trying to put into production. However, I get the following error when running it in the production environment. SimpleImputer object has no attribute _fit_dtype  How do I solve this?

Answer: This is a result of using different versions of scikit-learn in the development and production environments. The model has been trained using one version and then it's used with a different version. This can be solved by storing the current library versions in the development environment in a requirements.txt file using: pip list --format=freeze > requirements.txt  In the production environment, install the same library versions with: pip install -r requirements.txt
Question: Sentry can track performance for celery tasks and API endpoints https://docs.sentry.io/product/performance/ I have custom script that are lunching by crone and do set of similar tasks I want to incorporated sentry_sdk into my script to get performance tracing of my tasks Any advise how to do it with https://getsentry.github.io/sentry-python/api.html#sentry_sdk.capture_event

Answer: You don't need use capture_event I would suggest to use sentry_sdk.start_transaction instead. It also allows track your function performance. Look at my example from time import sleep from sentry_sdk import Hub, init, start_transaction init( dsn="dsn", traces_sample_rate=1.0, ) def sentry_trace(func): def wrapper(*args, **kwargs): transaction = Hub.current.scope.transaction if transaction: with transaction.start_child(op=func.__name__): return func(*args, **kwargs) else: with start_transaction(op=func.__name__, name=func.__name__): return func(*args, **kwargs) return wrapper @sentry_trace def b(): for i in range(1000): print(i) @sentry_trace def c(): sleep(2) print(1) @sentry_trace def a(): sleep(1) b() c() if __name__ == '__main__': a()  After starting this code you can see basic info of transaction a with childs b and c
Question: Every time I run any pipenv command I'm getting this:  C:\Users\user_name\AppData\Local\Programs\Python\Python311\Lib\site-packages\pipenv\vendor\attr_make.py:876: RuntimeWarning: Running interpreter doesn't sufficiently support code object introspection. Some features like bare super() or accessing class will not work with slotted classes. set_closure_cell(cell, cls)  The command runs after it, but I would like to disable this message. I'm using Windows 10 19044.2194 and pipenv 2022.10.25.

Answer: I was fighting the same issue on MacOS. The problem seems to be when pipenv is installed with brew. I fixed it by uninstalling the brew version of pipenv, then installing pipenv using pip. Here are the commands: brew uninstall pipenv pip install pipenv  Worked like a charm for me. Hope it helps you.
Question: I have a dictionary that for examples sake, looks like {'a': 1, 'b': 4, 'c': 7}  I have a dataframe that has the same index values as the keys in this dict. I want to add each value from the dict to the dataframe. I feel like doing a check for every row of the DF, checking the index value, matching it to the one in the dict, then trying to add it is going to be a very slow way right?

Answer: You can use map and assign back to a new column: d = {'a': 1, 'b': 4, 'c': 7} df = pd.DataFrame({'c':[1,2,3]},index=['a','b','c']) df['new_col'] = df.index.map(d)  prints:  c new_col a 1 1 b 2 4 c 3 7
Question: Hi I am trying to plot Plotly 3D surface plot, but unfortunately it doesnt appear. When I try with Scatter3D it works though not with Surface3D. Any ideas why? # Scatter 3D p = go.Figure() p.add_trace(go.Scatter3d( x = df.X1, y = df.X2, z = df.Y3, mode = "markers", marker = dict(size = 3), name = "actual" ))   from plotly.offline import plot trace = go.Surface(x = df.X1,y = df.X2,z = df.Y3) data = [trace] p2 = dict(data = data) plot(p2)  

Answer: A surface is not just a bunch of points. To draw a surface, Plotly needs to know how to split it in elementary triangles. Sure, you may think that, seeing your scatter plot, it seems obvious how to do so. But, well, it would be way less obvious if your points were not that planar. Plus, even in obvious cases, that would imply doing things like Delaunay's triangulation of all your (x,y) points. That would be costly. So, well, in plotly at least, a surface is not just a bunch of points. It is a matrix of z values, matching a regular, implicit mesh of (x,y) values. Just see how you would draw a 3×3 plane surface with both methods (scatter and surface). import plotly.graph_objects as go p = go.Figure() p.add_trace(go.Scatter3d( x = [0,1,2,0,1,2,0,1,2], y = [0,0,0,1,1,1,2,2,2], z = [1,1,1,1,1,1,1,1,1], mode = "markers", marker = dict(size = 3), name = "actual" )) p.show()  import plotly.graph_objects as go p = go.Figure() p.add_trace(go.Surface( x = [0,1,2], y = [0,1,2], z = [[1,1,1],[1,1,1],[1,1,1]] )) p.show()  In the second case, the surface is described as a 2D-array of z-values (x and y values just set the scale. You can omit them, unless you need irregular spacing) In the first case, x,y,z together describe a bunch of points (that, in this example, happen to be regularly spaced, but plotly can't guess that, since nothing prevent me to give different values for x and y)
Question: Description of the problem: The problem I'm stuck on is when I run a code, it first opens the chrome browser and opens the google.com website and then it closes it for no reason. This is my code: from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.chrome.service import Service opts = Options() opts.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36") PATH = "C:\Program Files (x86)\chromedriver.exe" driver_service = Service(executable_path=PATH) driver = webdriver.Chrome(service=driver_service,options=opts) driver.get("https://www.google.com/") print("connected to: " + driver.title)  What I expect is: that my bot will stay on google.com and not close it! For example: according to a video https://www.youtube.com/watch?v=Xjv1sY630Uc at minute 9:37 he runs his bot, a bot that does not close the window and compared to what my bot does to me it does close the window (I don't understand why it opens and immediately closes the window with my bot) Here we see that my program finished without errors but also see that it stopped working for no reason

Answer: Add this code and try: options = Options() options.add_experimental_option("detach", True) driver = webdriver.Chrome(service=driver_service,options=options)  Don't forget to add double slash '\\' in the chromedriver.exe path.
Question: I'm using VS-Code version 1.73.1, with MS Python extension v2022.18.2, on Windows 10 Pro, Build 10.0.19045. After installing the October 2022 update of VS Code, when writing Python code I noticed nagging error diagnostics being issued by the isort extension about the import order of modules. Previously, I had never encountered such diagnostics. I traced this behaviour back to the VS Code release notes for the Update October 2022. These announce the migration of VS Code to a new stand-alone isort extension, instead of the isort support built into the Python extension, by automatically installing it alongside the Python extension. When opening a file in which the imports do not follow isort standards, the extension is intended to issue an error diagnostic and display a Code Action to fix the import order. Whilst the extension seems to work as intended, I found the issues described below: 1. Even after having executed the Code Action to fix the import order, a 'light-bulb' with the same error diagnostic and Code Action again pops up on moving the cursor to a new line of code. 2. The error diagnostic and Code Action 'light-bulb' are also displayed when moving the cursor to any new line of code, even when all lines of code in the file have been commented out; that is, effectively, there are no longer any import statements in the code, and therefore also nothing to be sorted. I'd appreciate comments on whether this is a recognised issue in VS Code, and if so, whether any workarounds are available. It defeats the purpose of having an 'error lightbulb' pop up on every line of code, just to find a code action recommending to fix the import order, even when this requires no fixing. I have opened this question on this forum as recommended on the GitHub 'Contributing to VS Code' page.

Answer: Upgrade the isort extension version to latest(v2022.8.0).
Question: A bit of context I am working with a package that allows you to calculate several things about planets (such as their speed, or position), using information stored in files. The package includes methods to load, and unload files, so its basic usage would look like this: load(["File_1", "File_2"]) try: function() finally: unload(["File_1", "File_2"])  As this is a textbook example of the utility of a context manager, and the package lacks one, I am writing my own. class file_manager: def __init__(self, file_list) -> None: self.file_list = file_list load(self.file_list) return None def __enter__(self) -> None: return None def __exit__(self, exc_type, exc_value, traceback) -> None: unload(self.file_list) return None  With the new context manager, the previous example can be rewritten as follows: with file_manager(["File_1", "File_2"]): function()  and the __exit__ method guarantees that files will still be unloaded if function raises an error. My problem The load function loads files one by one, without first checking if all of them are available. As a result, if File_1 exists, but File_2 doesn't, File_1 will be loaded, and an exception will be raised while loading File_2. According to python documentation:  The with statement guarantees that if the __enter__() method returns without an error, then __exit__() will always be called.  Therefore, in the previous case, the execution of the program will end without File_2 being unloaded. What am I looking for I can obviously fix this by using a try...except clause inside the __init__() method: def __init__(self, file_list) -> None: self.file_list = file_list try: load(self.file_list) except FileDoesNotExistError: self.__exit__(FileDoesNotExistError, False, None)  but I want to know if this is the proper way to solve this problem. For example, in Cython, classes have a __dealloc__() method, which is guaranteed to run, no matter what type of exception occurs.

Answer: You can wrap your original code using contextlib.contextmanager. from contextlib import contextmanager @contextmanager def file_manager(file_list): try: load(file_list) yield None # after this the code inside the with block is executed finally: # this is called when the with block has finished # or when load raises an exception unload(file_list)  and use it like with file_manager(["File_1", "File_2"]): function()