eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: When an infrastructure incident happens, the application will start to generate thousands of occurrences of the same error. Is it possible to configure some kind of rate limiting or anything like that on the sentry client (or server) to avoid depleting the error quota? I'm using Python, Django and Celery mostly.

Answer: Yes, there are multiple solutions to this common problem. Sentry is an amazing tool, but when there's that annoying bug it can cause quite the issue. Sentry has great documentation that covers this problem. There are two solutions I see as a good use case for you:  Rate limiting - This enables you to control the number of incidents Sentry receives per project over a given period of time, thus some redundant events won't be measured and counted against quota. Spike protection - Maybe the best way to protect your quota is by enabling the spike protection. Sentry measures your average usage and if there's a abnormal number of events coming in a short period of time they will get ignored.  Rate limiting is a good protection system, but kinda hardcoded. Spike protection is dynamic and adjusts more to your personal needs. Combination of both could provide a great way to preserve your quota until the end of the month.
Question: Whenever I try to read Excel using part=pd.read_excel(path,sheet_name = mto_sheet)  I get this exception:  <class 'Exception'> 'ReadOnlyWorksheet' object has no attribute 'defined_names'  This is if I use Visual Studio Code and Python 3.11. However, I don't have this problem when using Anaconda. Any reason for that?

Answer: The error seems to be caused by the latest version of openpyxl. You can fix it by downgrading to a lower version pip install --force-reinstall -v "openpyxl==3.1.0"
Question: Say I have a string, s1="Hey Siri open call up duty"  and another string s2="call up duty". Now I know that "call up duty" should be replaced by "call of duty". Say s3="call of duty". So what I want to do is that from s1 delete s2 and place s3 in its location. I am not sure how this can be done. Can anyone please guide me as I am new to python. The answer should be "Hey siri open call of duty"  Note--> s2 can be anywhere within the string s1 and need not be at the last everytime

Answer: In python, Strings have a replace() method which you can easily use to replace the sub-string s2 with s3. s1 = "Hey Siri open call up duty" s2 = "call up duty" s3 = "call of duty" s1 = s1.replace(s2, s3) print(s1)  This should do it for you. For more complex substitutions the re module can be of help.
Question: I can't find the correct syntax for assigning a value to a variable using its BrowseName. I am testing with the 'flag1' boolean variable because it is easier to debug. But my goal is be able to write in all variables, including the arrays. If I try to use the index number it works fine. import pyOPCClient as opc client = opc.opcConnect('192.168.5.10') opc.write_value_bool(client, 'ns=4;s="opcData"."flag1"', True) client.disconnect()  Here is my function to write boolean ##### Function to WRITE a Boolean into Bool Object Variable - Requires Object Name ##### def write_value_bool(client, node_id, value): client_node = client.get_node(node_id) # get node client_node_value = value client_node_dv = ua.DataValue(ua.Variant(client_node_value, ua.VariantType.Boolean)) client_node.set_value(client_node_dv) print("Value of : " + str(client_node) + ' : ' + str(client_node_value))  I am getting this error: PS C:\Users\ALEMAC\Documents\Python Scripts> & C:/ProgramData/Anaconda3/python.exe "c:/Users/ALEMAC/Documents/Python Scripts/opctest.py" Requested session timeout to be 3600000ms, got 30000ms instead Traceback (most recent call last): File "c:\Users\ALEMAC\Documents\Python Scripts\opctest.py", line 5, in <module> opc.write_value_bool(client, 'ns=4;s="opcData"."flag1"', True) File "c:\Users\ALEMAC\Documents\Python Scripts\pyOPCClient.py", line 49, in write_value_bool client_node.set_value(client_node_dv) File "c:\Users\ALEMAC\Documents\Python Scripts\opcua\common\node.py", line 217, in set_value self.set_attribute(ua.AttributeIds.Value, datavalue) File "c:\Users\ALEMAC\Documents\Python Scripts\opcua\common\node.py", line 263, in set_attribute result[0].check() File "c:\Users\ALEMAC\Documents\Python Scripts\opcua\ua\uatypes.py", line 218, in check raise UaStatusCodeError(self.value) opcua.ua.uaerrors._auto.BadNodeIdUnknown: "The node id refers to a node that does not exist in the server address space."(BadNodeIdUnknown)   

Answer: I see you use the pyOPCClient package. I´m not sure if this is maintaned anymore (Last Update: 2014-01-09 see here). You can switch to opcua-asyncio which can address nodes with the browse services like this: myvar = await client.nodes.root.get_child(["0:Objects",..., "4:flag1"])  And here is the complete example
Question: I am building a simple script which polls some data and then updates a spreadsheet that I am giving to my client. (It is a small project and I don't need anything fancy.) So I created a Google Cloud project, enabled the Sheets API, and got a credential for a Desktop app. When I try to run the quickstart sample, I get an error: Access blocked: <my project name> has not completed the Google verification process I have tried googling and all the solutions seem to be oriented toward what a user should do if they see this, but I am the developer. I only need to grant my own self access to this spreadsheet, since my script is the only thing that will be changing it (I will also share it with the client). What do I do?

Answer: You need to add the account as a test user under the OAuth consent screen: 1.) From the dashboard go to APIs & Services and click OAuth concent screen  2.) Under the Test users, click +Add Users. A menu will prompt on the right panel.  3.) Input the users email  4.) Reload the URL provided. Reference: https://www.youtube.com/watch?v=bkZns_VOB6I Note: I am not affiliated with the video nor the owner of the youtube channel
Question: I am installed pillow,following the documentation here, python3 -m pip install --upgrade pip python3 -m pip install --upgrade Pillow  and import Image like this: from PIL import Image  Even though I upgraded Pillow to 9.4.0, I am getting the following error in vscode  No module named 'PIL'  I am using Python 3.9.7. I am not sure what I am doing wrong here, is it my python version or is it the vscode. Can someone please enlighten me about this issue. I can see them installed in my venv folder, but cannot access it in the file I am working on (which is highlighted by yellow) 

Answer: Add this code to your script. import sys print(sys.path)  Ensure that your sys.path contains the path "$PROJECT/venv/lib/python3.9/site-packages" If it doesn't, your virtual environment is broken. Try this instead:  Use this command to remove the current environment. rm -rf venv Create it again. python -m venv venv Install all your dependencies and run pip install --no-cache-dir Pillow  Make sure your environment is functioning properly right now.
Question: I am working on my final year project, so I working on a website where a user can come and read PDF. I am adding some features such as converting currency to their country currency. I am using flask and pymuPDF for my project and I don't know how I can modify the text at a pdf anyone can help me with this problem? I heard here that using pymuPDF or pypdf can work, but I didn't find any solution for replacing text.

Answer: Using the redaction facility of PyMuPDF is probably the adequate thing to do. The approach:  Identify the location of the text to replace Erase the text and replace it using redactions  Care must be taken to get hold of the original font, and whether or not the new text is longer / short than the original. import fitz # import PyMuPDF doc = fitz.open("myfile.pdf") page = doc[number] # page number 0-based # suppose you want to replace all occurrences of some text disliked = "delete this" better = "better text" hits = page.search_for("delete this") # list of rectangles where to replace for rect in hit: page.add_redact_annot(rect, better, fontname="helv", fontsize=11, align=fitz.TEXT_ALIGN_CENTER, ...) # more parameters page.apply_redactions(images=fitz.PDF_REDACT_IMAGE_NONE) # don't touch images doc.save("replaced.pdf", garbage=3, deflate=True)  This works well with short text and medium quality expectations. With some more effort, the original font properties, color, font size, etc. can be identified to produce a close-to-perfect result. This code works well with PyMuPDF==1.24.12 (Python 3.12.5) when lastly tested..
Question: Is there any built-in function in polars or a better way to convert time durations to numeric by defining the time resolution (e.g.: days, hours, minutes)? import polars as pl df = pl.DataFrame({ "from": ["2023-01-01", "2023-01-02", "2023-01-03"], "to": ["2023-01-04", "2023-01-05", "2023-01-06"], })  My current approach: # Convert to date and calculate the time difference df = ( df.with_columns( pl.col("to", "from").str.to_date().name.suffix("_date") ) .with_columns((pl.col("to_date") - pl.col("from_date")).alias("time_diff")) ) # Convert the time difference to int (in days) df = df.with_columns( ((pl.col("time_diff") / (24 * 60 * 60 * 1000)).cast(pl.Int8)).alias("time_diff_int") )  Output: shape: (3, 6) ┌────────────┬────────────┬────────────┬────────────┬──────────────┬───────────────┐ │ from ┆ to ┆ to_date ┆ from_date ┆ time_diff ┆ time_diff_int │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ str ┆ date ┆ date ┆ duration[ms] ┆ i8 │ ╞════════════╪════════════╪════════════╪════════════╪══════════════╪═══════════════╡ │ 2023-01-01 ┆ 2023-01-04 ┆ 2023-01-04 ┆ 2023-01-01 ┆ 3d ┆ 3 │ │ 2023-01-02 ┆ 2023-01-05 ┆ 2023-01-05 ┆ 2023-01-02 ┆ 3d ┆ 3 │ │ 2023-01-03 ┆ 2023-01-06 ┆ 2023-01-06 ┆ 2023-01-03 ┆ 3d ┆ 3 │ └────────────┴────────────┴────────────┴────────────┴──────────────┴───────────────┘ 

Answer: The dt accessor lets you obtain individual components, is that what you're looking for? df.select( total_days = pl.col.time_diff.dt.total_days(), total_hours = pl.col.time_diff.dt.total_hours(), total_minutes = pl.col.time_diff.dt.total_minutes() )  shape: (3, 3) ┌────────────┬─────────────┬───────────────┐ │ total_days ┆ total_hours ┆ total_minutes │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 │ ╞════════════╪═════════════╪═══════════════╡ │ 3 ┆ 72 ┆ 4320 │ │ 3 ┆ 72 ┆ 4320 │ │ 3 ┆ 72 ┆ 4320 │ └────────────┴─────────────┴───────────────┘  docs: Temporal API reference
Question: I am trying to count the number of letters in a string in Polars. I could probably just use an apply method and get the len(Name). However, I was wondering if there is a polars specific method? import polars as pl df = pl.DataFrame({ "start_date": ["2020-01-02", "2020-01-03", "2020-01-04", "2020-01-05"], "Name": ["John", "Joe", "James", "Jörg"] })  In Pandas I can use .str.len() >>> df.to_pandas()["Name"].str.len() 0 4 1 3 2 5 3 4 Name: Name, dtype: int64  But that does not exist in Polars: df.with_columns(pl.col("Name").str.len()) # AttributeError: 'ExprStringNameSpace' object has no attribute 'len' 

Answer: You can use  .str.len_bytes() that counts number of bytes in the UTF8 string .str.len_chars() that counts number of characters  df.with_columns( pl.col("Name").str.len_bytes().alias("bytes"), pl.col("Name").str.len_chars().alias("chars") )  shape: (4, 4) ┌────────────┬───────┬───────┬───────┐ │ start_date ┆ Name ┆ bytes ┆ chars │ │ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ str ┆ u32 ┆ u32 │ ╞════════════╪═══════╪═══════╪═══════╡ │ 2020-01-02 ┆ John ┆ 4 ┆ 4 │ │ 2020-01-03 ┆ Joe ┆ 3 ┆ 3 │ │ 2020-01-04 ┆ James ┆ 5 ┆ 5 │ │ 2020-01-05 ┆ Jörg ┆ 5 ┆ 4 │ └────────────┴───────┴───────┴───────┘
Question: I have a simple project layout myproject on  main [$!?] is 📦 v1.0.0 via  v18.14.0 via 🐍 v3.10.9 ❯ tree -L 1 . ├── build ├── deploy ├── Dockerfile ├── poetry.lock ├── pyproject.toml ├── README.md ├── scripts.py └── src  The pyproject.toml is: [tool.poetry] name = "myproject" version = "0.1.0" description = "" authors = [""] [tool.poetry.scripts] test = "scripts:test"  The scripts.py is: import subprocess def test(): """ Run all unittests. """ subprocess.run( ['python', '-u', '-m', 'pytest'] ) if __name__ == '__main__': test()  When I run poetry run test: myproject on main [$!?] is 📦 v1.0.0 via  v18.14.0 via 🐍 v3.10.9   No file/folder found for package myproject 

Answer: Short answer: The directory where your pyproject.toml file sits needs to be share the same name, e.g., if the name config in pyproject.toml is name = "myproject", the directory needs to be also named myproject. Therefore, you must either:  Rename the directory to match your name configuration in the pyproject.toml or Move the pyproject.toml file to the correct directory or If you don't care about making your project packageable by Poetry and only use poetry as your package manager for this project, add package-mode = false to your pyproject.toml.  Explanation: There's a thread on the official GitHub for Poetry that discusses this issue. The gist of the issue is that whatever you choose for your project name, i.e., name = "myproject", needs to be the direct parent directory of where your pyproject.toml file lives. When there is a mismatch, you get the error you are getting. Sources:  Breakdown of the cause of the issue on thread on GitHub A Poetry contributor's response on thread on GitHub
Question: How to limit number of rows in pandas dataframe in python code. I needed last 1000 rows the rest need to delete. For example 1000 rows, in pandas dataframe -> 1000 rows in csv. I tried df.iloc[:1000] I needed autoclean pandas dataframe and saving last 1000 rows.

Answer: With df.iloc[:1000] you get the first 1000 rows. Since you want to get the last 1000 rows, you have to change this line a bit to df_last_1000 = df.iloc[-1000:] To safe it as a csv file you can use pandas' to_csv() method: df_last_1000.to_csv("last_1000.csv")  Update - Speed Comparison: Both .tail(1000) and .iloc[-1000:, :] return the last 1000 rows, so let's compare their performance: import pandas as pd import numpy as np import timeit df = pd.DataFrame(np.random.rand(1000000, 5), columns=['A', 'B', 'C', 'D', 'E']) def tail_operation(): _ = df.tail(1000) def iloc_operation(): _ = df.iloc[-1000:, :] tail_time = timeit.timeit(tail_operation, number=1000) iloc_time = timeit.timeit(iloc_operation, number=1000) print(f"Execution time for tail operation: {tail_time} seconds") print(f"Execution time for iloc operation: {iloc_time} seconds")  Execution time for tail operation: 0.0280200999986846 seconds Execution time for iloc operation: 0.07651790000090841 seconds
Question: I am working on a small flask API using flask-admin and flask-sqlalchemy. The api runs well but whenever I install a new package I am faced with an error. Error: While importing 'app', an ImportError was raised: Traceback (most recent call last): File "/Users/brandoncreed/.local/share/virtualenvs/management-System-Wyc_ba1N/lib/python3.8/site-packages/flask/cli.py", line 218, in locate_app __import__(module_name) File "/Users/brandoncreed/Desktop/Brandon Coding Projects/management-System/src/app.py", line 12, in <module> from api.admin import setup_admin File "/Users/brandoncreed/Desktop/Brandon Coding Projects/management-System/src/api/admin.py", line 5, in <module> from flask_admin.contrib.sqla import ModelView File "/Users/brandoncreed/.local/share/virtualenvs/management-System-Wyc_ba1N/lib/python3.8/site-packages/flask_admin/contrib/sqla/__init__.py", line 2, in <module> from .view import ModelView File "/Users/brandoncreed/.local/share/virtualenvs/management-System-Wyc_ba1N/lib/python3.8/site-packages/flask_admin/contrib/sqla/view.py", line 18, in <module> from flask_admin.contrib.sqla.tools import is_relationship File "/Users/brandoncreed/.local/share/virtualenvs/management-System-Wyc_ba1N/lib/python3.8/site-packages/flask_admin/contrib/sqla/tools.py", line 11, in <module> from sqlalchemy.ext.associationproxy import ASSOCIATION_PROXY ImportError: cannot import name 'ASSOCIATION_PROXY' from 'sqlalchemy.ext.associationproxy' (/Users/brandoncreed/.local/share/virtualenvs/management-System-Wyc_ba1N/lib/python3.8/site-packages/sqlalchemy/ext/associationproxy.py)  I am not sure what is causing this but it seems there is a conflict between flask_admin and sqlalchemy after the new package is installed. I deleted the pipfile.lock in my virtual environment and ran pipenv install to see if the new file would work but it did not. I also tried uninstalling the new package to see if it would resolve the issue but the same error still persists. I am wondering if it was to do with updating either flask-admin or sqlalchemy.

Answer: This issue has been fixed in Flask-Admin v1.6.1, which can be installed like this: python3 -m pip install --upgrade flask-admin  Flask-Admin versions earler than v1.6.1 are not compatible with SQLAlchemy 2.0. The was an issue for this specific problem. The recommended workaround was to install an earlier version of SQLALchemy, for example: python3 -m pip install --upgrade 'sqlalchemy<2.0'
Question: I want to use openai.embeddings_utils import get_embeddings So already install openai Name: openai Version: 0.26.5 Summary: Python client library for the OpenAI API Home-page: https://github.com/openai/openai-python Author: OpenAI Author-email: support@openai.com License: Location: /Users/lima/Desktop/Paprika/Openai/.venv/lib/python3.9/site-packages Requires: aiohttp, requests, tqdm Required-by:  This is my openai But why not use openai.embeddings_utils??

Answer: For my case, check the version of openai. openai.embeddings_utils does not exist in latest openai 1.2.0, but exists in 0.27.7
Question: This bug suddenly came up literally today after read_excel previously was working fine. Fails no matter which version of python3 I use - either 10 or 11. Do folks know the fix?  File "/Users/aizenman/My Drive/code/daily_new_clients/code/run_daily_housekeeping.py", line 38, in <module> main() File "/Users/aizenman/My Drive/code/daily_new_clients/code/run_daily_housekeeping.py", line 25, in main sb = diana.superbills.load_superbills_births(args.site, ath) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/aizenman/My Drive/code/daily_new_clients/code/diana/superbills.py", line 148, in load_superbills_births sb = pd.read_excel(SUPERBILLS_EXCEL, sheet_name="Births", parse_dates=["DOS", "DOB"]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pandas/util/_decorators.py", line 211, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pandas/util/_decorators.py", line 331, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pandas/io/excel/_base.py", line 482, in read_excel io = ExcelFile(io, storage_options=storage_options, engine=engine) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pandas/io/excel/_base.py", line 1695, in __init__ self._reader = self._engines[engine](self._io, storage_options=storage_options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pandas/io/excel/_openpyxl.py", line 557, in __init__ super().__init__(filepath_or_buffer, storage_options=storage_options) File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pandas/io/excel/_base.py", line 545, in __init__ self.book = self.load_workbook(self.handles.handle) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pandas/io/excel/_openpyxl.py", line 568, in load_workbook return load_workbook( ^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/openpyxl/reader/excel.py", line 346, in load_workbook reader.read() File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/openpyxl/reader/excel.py", line 303, in read self.parser.assign_names() File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/openpyxl/reader/workbook.py", line 109, in assign_names sheet.defined_names[name] = defn ^^^^^^^^^^^^^^^^^^^ AttributeError: 'ReadOnlyWorksheet' object has no attribute 'defined_names' 

Answer: Currently, we face the issue on python3.8 environment (Rocky 8.8 OS). We downgrade the openpyxl module to 3.0.9 to fix this issue.  sudo pip3 install openpyxl==3.0.9  Enjoy it!
Question: Is there an efficient way to get the month end date on a date column. Like if date =‘2023-02-13” to return “2023-02-28”, also beginning of the month would be great as well. Thanks! df = pl.DataFrame({'DateColumn': ['2022-02-13']}) test_df = df.with_columns([ pl.col('DateColumn').str.strptime(pl.Date).cast(pl.Date) ] ) ┌────────────┐ │ DateColumn │ │ --- │ │ date │ ╞════════════╡ │ 2022-02-13 │ └────────────┘  Two new columns would be perfect.

Answer: [Update]: Polars has since added .month_start() and .month_end() methods. See the answer from @n-maks  You could use .truncate and .offset_by test_df.with_columns( MonthStart = pl.col("DateColumn").dt.truncate("1mo"), MonthEnd = pl.col("DateColumn").dt.offset_by("1mo").dt.truncate("1mo").dt.offset_by("-1d") )  shape: (1, 3) ┌────────────┬────────────┬────────────┐ │ DateColumn | MonthStart | MonthEnd │ │ --- | --- | --- │ │ date | date | date │ ╞════════════╪════════════╪════════════╡ │ 2022-02-13 | 2022-02-01 | 2022-02-28 │ └────────────┴────────────┴────────────┘
Question: I dont know why this error occurs. pd.read_excel('data/A.xlsx', usecols=["B", "C"])  Then I get this error: "Value must be either numerical or a string containing a wild card"  So i change my code use nrows all data pd.read_excel('data/A.xlsx', usecols=["B","C"], nrows=172033)  Then there is no error and a dataframe is created. my excel file has 172034 rows, 1st is column name.

Answer: This problem is unique to the latest version of the Openpyxl library, v3.1.2. Downgrading to v.3.0.10 will fix this issue.
Question: For example, if I define this function: def open_pkl(src: str) -> ?: with open('serialized.pkl', 'rb') as f: data = pickle.load(f) return data  what type hint should I write for the return value? Now, I write the function as: def open_pkl(src: str): with open('serialized.pkl', 'rb') as f: data = pickle.load(f) return data  Is there type a hint for an uncertain return type?

Answer: There are two options: object and typing.Any. Returning an object signals to the caller of the function that nothing can be assumed about the returned object (since everything is an object, saying that something is an object gives no information). So, if a user were to do def open_pkl(src: str) -> object: ... something = open_pkl('some/file') print(len(something))  that would be a type violation, even if the object were a list, because objects per se don't have a __len__ method. typing.Any, on the other hand, is like a wild card which could hypothetically be anything. So, if you reworked the above example to have a typing.Any return type, there would be no type violation. Does a typing.Any have a __len__ method? Maybe. Who says it couldn't? To summarize, you should use object if you want to "force" (because type hints are just suggestions) your users to verify the type of any object returned by this function. Use typing.Any to be more lax.