eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am migrating to Python 3.12, and finally have to remove the last distutils dependency. I am using from distutils.util import strtobool to enforce that command-line arguments via argparse are in fact bool, properly taking care of NaN vs. False vs. True, like so: arg_parser = argparse.ArgumentParser() arg_parser.add_argument("-r", "--rebuild_all", type=lambda x: bool(strtobool(x)), default=True)  So this question is actually twofold:  What would be an alternative to the deprecated strtobool? Alternatively: What would be an even better solution to enforce 'any string' to be interpreted as bool, in a safe way (e.g., to parse args)? 

Answer: What about using argparse's BooleanOptionalAction? This is documented in the argparse docs, but not in a way that provides me with a useful link. That would look like: arg_parser = argparse.ArgumentParser() arg_parser.add_argument("-r", "--rebuild-all", action=argparse.BooleanOptionalAction, default=True)  And it would allow you to pass either --rebuild-all or --no-rebuild-all. This gets you an option that can be either true or false, but without the hassle of having to parse a string into a boolean value.
Question: I have a type that has subclasses. E.g., Event, with classes Event1(Event), Event2(Event), etc. I want to specify a type annotation for the argument to a function that accepts any union of subclasses of Event. Concretely, I want all of the following to type check: listen_for_events(Event1) listen_for_events(Event1 | Event3) listen_for_events(Event1 | Event2 | Event3)  while the following should not type check: listen_for_events(str | int)  How do I specify the type on the argument to listen_for_events? I can do this with tuple (tuple[type[Event], ...]), but I don't appear to be able to do something similar with Union. Is there a way to correctly type this?

Answer: So to answer this question, firstly let's look at how union types work. The syntax you're describing generates a types.UnionType, and you can't specify any more detail about this type. e.g. Something like the below passes for all of your examples, including the one you didn't want: def listen_for_events(event: type[Event] | types.UnionType): ... listen_for_events(Event1) listen_for_events(Event1 | Event2 | Event3) listen_for_events(str | int)  Why does python not offer a more descriptive syntax for this? Consider the features of the collection of events. What the union syntax is really expressing here is we want to be able to supply:  an unknown number of events in no particular order with no repeats  ...or in other words, a set: def listen_for_events(event: set[type[Event]]): ... listen_for_events({Event1}) listen_for_events({Event1, Event2, Event3}) listen_for_events({str, int}) # raises error  Note: for this to work you have to make event hashable, or use frozenset This is actually more than a coincidence - in some sense a union of objects is just a way of expressing the specific members in a particular set. A | B is the set that contains only A and B, it's just that normally in python we ascribe this set to a type and say: "x must be an instance of the class that features in the set A | B" Is there a better syntax? If we relax the unordered and non-repeating condition, which maybe is not so critical to the problem, there become other options available (including the tuple mentioned in the comments). Personally I think the best syntax for the problem you're trying to solve would be the below: def listen_for_events(*events: type[Event]): ... listen_for_events(Event1) listen_for_events(Event1, Event2, Event3) listen_for_events(str, int) # raises error  An advantage of this syntax is it even flags the exact member that differs from the type if you entered something like: listen_for_events(str, Event1) In summary, python may not offer an exact way to type the syntax you want, but it does off other structures that express exactly the same concept. Hope this is useful!
Question: I get following warning on RedHat 8.8 for H5PY Package: Is there any way to fix this? I dont want to use warnings.filterwarnings("ignore") bash-4.4$ python3 Python 3.11.6 (main, Oct 16 2023, 03:41:34) [GCC 8.5.0 20210514 (Red Hat 8.5.0-18)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> import h5py /scratch/Python/lib/python3.11/site-packages/numpy/core/getlimits.py:511: UserWarning: Signature b'\x00\xd0\xcc\xcc\xcc\xcc\xcc\xcc\xfb\xbfOE\xfc\x7f\x00\x00' for <class 'numpy.float128'> does not match any known type: falling back to type probe function. This warnings indicates broken support for the dtype! machar = _get_machar(dtype) >>> numpy.__version__ '1.24.3' >>> h5py.__version__ '3.10.0' >>> exit() bash-4.4$ cat /etc/os-release NAME="Red Hat Enterprise Linux" VERSION="8.8 (Ootpa)" ID="rhel" ID_LIKE="fedora" VERSION_ID="8.8" PLATFORM_ID="platform:el8" PRETTY_NAME="Red Hat Enterprise Linux 8.8 (Ootpa)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" HOME_URL="https://www.redhat.com/" DOCUMENTATION_URL="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8" BUG_REPORT_URL="https://bugzilla.redhat.com/" REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" REDHAT_BUGZILLA_PRODUCT_VERSION=8.8 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="8.8" bash-4.4$  This warning is seen only on Linux, Windows is clean.

Answer: A similar issue posted on git : https://github.com/h5py/h5py/issues/2357 moving to numpy version older than 1.24 may fix the warning. Looks like a broken support for long double/numpy.float128 data type from the OS. You can try other methods to ignore warnings as well.
Question: Here's my python code: from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.common.by import By from flask import Flask, render_template from webdriver_manager.chrome import ChromeDriverManager from selenium.webdriver.chrome.service import Service APP = Flask(__name__) @APP.route('/ohc_search_link') def ohc_search_link(): ohc_search() return render_template("finished.html") def ohc_search(): chrome_options = Options() chrome_options.add_argument("--headless") chrome_options.add_argument('--disable-dev-shm-usage') chrome_options.add_argument("--no-sandbox") driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()),options=chrome_options) driver.get('https://docs.oracle.com/') # More scraping code here ... driver.close() if __name__ == '__main__': #start OHC search APP.run()  When I run this, I see this warning on the console: [0203/171925.768:INFO:CONSOLE(0)] "Third-party cookie will be blocked. Learn more in the Issues tab.", source: https://docs.oracle.com/search/?q=recruiting&pg=1&size=10&showfirstpage=true&lang=en My script runs fine though and I'm seeing the results I expect. But can I ignore this warning? If yes, is there a way to suppress it? I wasn't seeing any warnings a few days earlier. I remember that my chrome browser updated around that time. I'm seeing this warning after that update. Do you think this is related to this news about restricting third party cookies? But strangely I don't see the Eye icon on the website I'm scraping.

Answer: I just came across this error too. Try adding the following: chrome_options.add_argument('log-level=3')
Question: I have a following dataframe called ol_axle:     Single Tridem Tandem Tridem     700 1500 1200 2700     The code to create the dataframe is as follows: weight=[700,1500,1200,2700] name=['Single','Tridem','Tandem','Tridem'] ol_axle=pd.DataFrame([weight],columns=[name])  My apologies as the query doesn't appear to be that difficult, but what I want is that if there is 'Tridem' in column name, the column repeats two more times and the value is divided by 3 and if it there is 'Tandem' in col name, it repeats once more and the value is divided by 2 such that I get the resulting dataframe:     Single Tridem Tridem Tridem Tandem Tandem Tridem Tridem Tridem     700 500 500 500 600 600 900 900 900     The code that I developed to get the required result is as follows: # Initialize a flag to track whether division has been performed tridem_divided = False tandem_divided = False # Iterate over the columns and update values based on conditions for column in ol_axle.columns: if 'Tridem' in column and not tridem_divided: ol_axle[column] = ol_axle[column] / 3 tridem_divided = True elif 'Tandem' in column and not tandem_divided: ol_axle[column] = ol_axle[column] / 2 tandem_divided = True else: ol_axle[column] = ol_axle[column] ol_axle  The issue is, that my code was working perfectly well and and I used it for a lot of data with various different combinations of these axle groups, but all of a sudden and for some reason I still do not know I started receiving the following error;  PerformanceWarning: indexing past lexsort depth may impact performance. ol_axle[column] = ol_axle[column] / 3  I just installed a Matplotlib and OSMNX library 2 days earlier and which I am sure has got nothing to do with this error popping up now. I have tried other snippets but I am still getting this error. I am not that experienced in Python as yet, so it would be great if anyone can help me get the results that I want for my dataframe.

Answer: Using numpy.repeat with map: n = {'Single': 1, 'Tandem': 2, 'Tridem': 3} rep = ol_axle.columns.map(n) out = (pd.DataFrame(np.repeat(ol_axle.div(rep, axis=1), rep, axis=1), columns=np.repeat(ol_axle.columns, rep), index=ol_axle.index) ) print(out)  Variant of @Timeless' approach using iloc: n = {'Single': 1, 'Tandem': 2, 'Tridem': 3} rep = ol_axle.columns.map(n) out = (ol_axle.div(rep) .iloc[:, np.repeat(np.arange(ol_axle.shape[1]), rep)] )  Output:  Single Tridem Tridem Tridem Tandem Tandem Tridem Tridem Tridem 0 700.0 500.0 500.0 500.0 600.0 600.0 900.0 900.0 900.0
Question: For example, I want to install fish shell using conda. But the server has no internet connection. On https://anaconda.org/conda-forge/fish/files , many versions are provided. But almost the latest few versions are always in .conda format. I downloaded linux-64/fish-3.7.0-hdab1d28_0.conda and install it using conda install linux-64/fish-3.7.0-hdab1d28_0.conda  But this does not work. It shows a long error report like ... FileNotFoundError: [Errno 2] No such file or directory: '...../miniconda3/pkgs/linux-64_fzf-0.46.1-ha8f183a_0/info/repodata_record.json' FileNotFoundError: [Errno 2] No such file or directory: '...../miniconda3/pkgs/linux-64_fzf-0.46.1-ha8f183a_0/info/index.json' .... etc  But old version in tar.gz format like linux-64/fish-3.4.1-h682823d_0.tar.bz2 installed just fine. So how to correctly conda install .conda file?

Answer: Bug with Anaconda Cloud downloads There is an outstanding bug with the Anaconda Cloud website that prepends the subdirectory (e.g., a linux-64_ prefix) to file names when downloading from a browser (i.e., clicking file links). This is fine for .tar.gz files, but for .conda the file name is coupled(!) to being able to correctly unpack the archive. Error in OP shows a reference to linux-64_fzf-0.46.1-ha8f183a_0, which implies a linux-64_fzf-0.46.1-ha8f183a_0.conda file is being unpacked. However, the file name should likely not have the linux-64_ prefix. Try renaming the file to fzf-0.46.1-ha8f183a_0.conda. Note that programmatically downloading the .conda files (e.g., with curl, wget) will not have this problem, so one may also be interested in downloading archives en masse, like what is provided for in this answer.
Question: Let's say I have a dataframe: df = pd.DataFrame({ 'ID': [1, 2, 3, 1, 2, 3], 'Value': ['A', 'B', 'A', 'B', 'C', 'A'] })  If I wanted to only remove duplicates on ID when ID is a specified value (let's say 1), how would I do that? In other words, the resulting dataframe would look like: |ID|Value| |--|-----| |1 |A | |2 |B | |3 |A | |2 |C | |3 |A |  AI assistants are having a surprisingly difficult time with this one.

Answer: Create a boolean mask with DataFrame.duplicated: mask = df.ID.eq(1) & df.duplicated(subset=["ID"]) print(df[~mask])  Prints:  ID Value 0 1 A 1 2 B 2 3 A 4 2 C 5 3 A   Steps: print(df.ID.eq(1)) 0 True 1 False 2 False 3 True 4 False 5 False Name: ID, dtype: bool print(df.duplicated(subset=["ID"])) 0 False 1 False 2 False 3 True 4 True 5 True dtype: bool mask = df.ID.eq(1) & df.duplicated(subset=["ID"]) print(mask) 0 False 1 False 2 False 3 True 4 False 5 False dtype: bool
Question: How can I get the spreadsheet values with Python's gspread? Suppose there is a cell that looks like 1/1 because m/d is specified as the cell display format, but actually contains 2024/1/1. Retrieving this cell using get_all_values() returns "1/1". I want the actual value "2024/1/1", not the value displayed on the sheet. What should I do?  I will omit the sheet acquisition part. values = workbook.worksheet(sheet_name).get_all_values() value = values[1][0] # 1/1 will be obtained

Answer: Issue and workaround: From your showing image, script, and current value, I understood that you put a value of 2024/01/01 into a cell "A2" as a date object. And, the cell value is shown as 1/1 with the number format. In the current stage, when this cell value is retrieved by Sheets API, 1/1 is retrieved as the default request of valueRenderOption: FORMATTED_VALUE. When valueRenderOption is changed to UNFORMATTED_VALUE, the serial number of 45292 is retrieved. Unfortunately, in the current stage, the inputted value of 2024/01/01 cannot be directly retrieved. So, it is required for the following flow.  Retrieve the serial number from a cell with valueRenderOption: UNFORMATTED_VALUE. Convert the serial number to the date object. Convert date format.  When this flow is reflected in your showing script, how about the following modification? Modified script: values = workbook.worksheet(sheet_name).get_all_values(value_render_option="UNFORMATTED_VALUE") value = values[1][0] date = datetime.fromtimestamp((int(value) - 25569) * 86400) # Ref: https://stackoverflow.com/a/6154953 formatted = date.strftime('%Y/%m/%d') # or date.strftime('%Y/%-m/%-d') # or date.strftime('%Y/%#m/%#d') print(formatted)   In the case that the cell "A2" has 2024/01/01 of the date object and 1/1 is shown by the number format, when this script is run, formatted is 2024/01/01.  from datetime import datetime is used.   References:  get_all_values Method: spreadsheets.values.get
Question: sympy's solve seems to generally work for inequalities. For example: from sympy import symbols, solve; x = symbols('x'); print(solve('x^2 > 4', x)); # ((-oo < x) & (x < -2)) | ((2 < x) & (x < oo))  In some cases where all values of x or no values of x are valid solutions, solve() returns True or False to indicate that. print(solve('x^2 > 4 + x^2', x)); # False print(solve('x^2 >= x^2', x)); # True  However, there are other cases where I would expect a result of True or False and instead I get []: print(solve('x - x > 0', x)); # [] print(solve('x - x >= 0', x)); # []  Why does this happen/what am I doing wrong?

Answer: Remember that expressions are not WYSIWYG: your expressions did not have x in them: print(solve('x - x > 0', x)); # [] <-- solve(0>0, x) => solve(False, x) print(solve('x - x >= 0', x)); # [] <-- solve(0>=0, x) => solve(True, x)  Since x does not appear in either expression, there are no values of x to report in the first case that can make the False become True; in the second case there is no way to represent the Universal set (since any value of x cannot change the value of True to anything else). So in both cases there is no solution to report (though maybe an error or at least x<=oo could be reported for the second case) but in general: if the requested symbol does not appear in the equation (at the outset or after simplification) then [] is returned (and x does not appear in True).
Question: I have just upgraded to PyCharm 2023.3.3 community edition and am experiencing a weird issue that hasn't occurred before. System is Windows 10 64 bit, and this has worked fine on this machine before. When I debug a script and hit a breakpoint and the program is suspended and the debugger window is opened, autocomplete doesn't seem to work on new variables i create in the debugging console, however it works on variables created in the script before the breakpoint was hit. e.g., Script  Debugging console showing autocomplete working for variable testing but not test  Any ideas why the debugging console is not autocompleting test? and how I could fix this?

Answer: Open Settings | Build, Execution, Deployment | Console and switch Code Completion to Runtime. Relevant ticket in PyCharm's issue tracker - PY-64055.
Question: I'm writing an Ansible module using Paramiko's SFTPClient class. I have the following code snippets import paramiko from ansible.module_utils.basic import * def get_sftp_client(host, port, user, password): # Open a transport transport = paramiko.Transport((host,port)) # Auth transport.connect(None,user,password) # Get a client object sftp = paramiko.SFTPClient.from_transport(transport) return (transport, sftp) def transfer_file(params): has_changed = False meta = {"upload": "not yet implemented"} user = params['user'] password = params['password'] host = params['host'] port = params['port'] direction = params['direction'] src = params['src'] dest = params['dest'] transport, sftp = get_sftp_client(host, port, user, password) stdout = None stderr = None if direction == 'download': sftp.get(src, dest) else: sftp.put(src, dest) meta = { "source": src, "destination": dest, "direction": direction, "stdout": stdout, "stderr": stderr } # Close if sftp: sftp.close() if transport: transport.close() return (has_changed, meta) def main(): fields = { "user": {"required": True, "type": "str"}, "password": {"required": True, "type": "str", "no_log": True}, "host": {"required": True, "type": "str"}, "port": {"type": "int", "default": 22}, "direction": { "type": "str", "choices": ["upload", "download"], "default": "upload", }, "src": {"type": "path", "required": True}, "dest": {"type": "path", "required": True}, } module = AnsibleModule(argument_spec=fields) # Do the work has_changed, result = transfer_file(module.params) module.exit_json(changed=has_changed, meta=result) if __name__ == '__main__': main()  How can I capture the pass/fail result of the get() and put() functions? The docs don't mention the return value for get(), and the return value for put() is an SFTPAttribute class, which I don't need. I just want to know if the transaction passed or failed, and if failed, get the error. Thanks.

Answer: The Paramiko SFTPClient.put and .get throw an exception on error. If they do not throw, then the transfer succeeded.
Question: I have a list of dictionary like below :  sample = [{'By': {'email': 'xyz@stackoverflow.com', 'name': 'xyzzy', 'id': '5859'}},{'By': {'email': 'abc@stackoverflow.com', 'name': 'abccb', 'id': '9843'}}, {'By': {'email': 'xyz@stackoverflow.com', 'name': 'xyzzy', 'id': '5859'}}]  From this, I am trying to access keys 'name' and 'id' and write distinct values into a dictionary. Below is returning id's alone : print(set(map(lambda x: x['By']['id'], sample)))  output: {'9843', '5859'} Required output : {"9843":"abccb", "5859":"xyzzy"} I tried with f-string to concatenate the 'id' and 'name' values with a colon (:) in between to give in the format "id:name" for each dictionary item in the list. set(map(lambda x: f"{x['By']['id']}:{x['By']['name']}", sample))  output : {'9843:abccb', '5859:xyzzy'} Is it possible to access values of two keys in the map lambda function ? Thank you.

Answer: You want a dict, not a set. You can get all the values in the lambda and pack them in a tuple print(dict(map(lambda x: (x['By']['id'], x['By']['name']), sample)))  You should also consider a simpler solution, you don't really need map and lambda here, dict-comprehensions will do just fine print({x['By']['id']: x['By']['name'] for x in sample})
Question: I have a list of users, that I would like to type correctly, but I'm not able to find an answer explaining how to do this. The list looks like this: listofusers = [[167, 'john', 'John Fresno', [[538, 'Privileged'], [529, 'User']]]]  I tried doing like this: def test(listofusers: list[list[str,list[list[str]]]]) -> None: ...  But VS Code only shows typing for: list[list[str]]:  I also tried using Union, like this: listofusers: list[list[Union[int,str,list[list[str]]]]]  But again, this doesn't seems to work. Can someone shed a light over this?

Answer: As already mentioned in comments and in this answer, having such complicated structures look like code smell. You can think of re-organising your code a bit and do some more explicit semantics: import dataclasses @dataclasses.dataclass class Data: data_id: int attribute: str @dataclasses.dataclass class User: user_id: int name: str fullname: str data: list[Data] user_1 = User( user_id=167, name="john", fullname="John Fresno", data=[ Data(data_id=538, attribute='Privileged'), Data(data_id=529, attribute='User') ] )  The code above is just an example (maybe I missed somewhere a detail, but I hope you get my point). The details are fully up to you. The advantage is that your listofusers becomes simply: users: list[User] = [user_1] with better readability, no? You can also use Pydantic, of course. It is very similar to dataclasses (using it behind the scenes AFAIK) and has some convenient simplifications regarding data verification.
Question: I have a polars dataframe like this one: shape: (10, 2) ┌────────┬───────┐ │ foo ┆ bar │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞════════╪═══════╡ │ 86 ┆ 11592 │ │ 109 ┆ 2765 │ │ 109 ┆ 4228 │ │ 153 ┆ 4214 │ │ 153 ┆ 7217 │ │ 153 ┆ 11095 │ │ 160 ┆ 1134 │ │ 222 ┆ 5509 │ │ 225 ┆ 10150 │ │ 239 ┆ 4151 │ └────────┴───────┘  And a sorted list of integers points: points = [0, 1500, 3000, 4500, 6000, 7500, 9000, 10500, 12000]  I want to create a new column baz, such that for each element y of bar, I find the largest x in points such that x =< y. Then the element of baz is y - x. How can I do that?

Answer: You can have a solution which fully relies on polars expressions, by using join_asof. First, create DataFrame out of the list and then join (I'm assuming that points list is pre-sorted) Here, we leave strategy argument of the join empty, so default backward strategy is going to be used.  A backward search selects the last row in the right DataFrame (in our case - list of points) whose on key is less than or equal to the left’s key:  df_points = pl.DataFrame({'point': points}) ( df.sort('bar') .join_asof(df_points.set_sorted('point'), right_on='point', left_on='bar') .with_columns(baz=pl.col('bar') - pl.col('point')) .drop('point') .sort('foo') ) ┌─────┬───────┬──────┐ │ foo ┆ bar ┆ baz │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 │ ╞═════╪═══════╪══════╡ │ 86 ┆ 11592 ┆ 1092 │ │ 109 ┆ 2765 ┆ 1265 │ │ 109 ┆ 4228 ┆ 1228 │ │ 153 ┆ 4214 ┆ 1214 │ │ 153 ┆ 7217 ┆ 1217 │ │ 153 ┆ 11095 ┆ 595 │ │ 160 ┆ 1134 ┆ 1134 │ │ 222 ┆ 5509 ┆ 1009 │ │ 225 ┆ 10150 ┆ 1150 │ │ 239 ┆ 4151 ┆ 1151 │ └─────┴───────┴──────┘