eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have a list that I'm trying to add to a dataframe. It looks something like this: list_one = ['apple','banana','cherry',' ', 'grape', 'orange', 'pineapple','']  If I add the list to a dataframe, using df = pd.DataFrame({'list_one':list_one}) it'll look like this:  list_one ------------- 0 apple 1 banana 2 cherry 3 4 grape 5 orange 6 pineapple 7  I want the combine some of the rows into one row, so that the dataframe looks something like this:  list_one ----------------------------- 0 apple, banana, cherry 1 grape, orange, pineapple  Is there a simple way to do this? Thank you for taking the time to read my question and help in any way you can.

Answer: Create mask for match words by Series.str.contains, invert by ~ and crate groups by Series.cumsum, filter only matched rows and pass to GroupBy.agg with join function: m = df['list_one'].str.contains('\w+') df = df[m].groupby((~m).cumsum(), as_index=False).agg(', '.join) print (df) list_one 0 apple, banana, cherry 1 grape, orange, pineapple
Question: How can I get the last character of this string? seed_name = "Cocoa" 

Answer: As shown in the official Python tutorial,  >>> word = 'Python'  [...] Indices may also be negative numbers, to start counting from the right: >>> word[-1] # last character 'n'
Question: Is there a way to turn off the top marginal plot in a Seaborn Jointplot? tips = sns.load_dataset('tips') g = sns.jointplot( data=tips, x="total_bill", y="tip", hue="smoker", )  

Answer: Perhaps just call remove on .ax_marg_x. g.ax_marg_x.remove()  Output:  (Note that ax_marg_y and ax_joint are the remaining plots, as detailed in the JointGrid docs).
Question: I need to read the file line by line with readline() and cannot easily change that. Roughly it is: with open(file_name, 'r') as i_file: while True: line = i_file.readline() # I need to check that EOF has not been reached, so that readline() really returned something  The real logic is more involved, so I can't read the file at once with readlines() or write something like for line in i_file:. Is there a way to check readline() for EOF? Does it throw an exception maybe? It was very hard to find the answer on the internet because the documentation search redirects to something non-relevant (a tutorial rather than the reference, or GNU readline), and the noise on the internet is mostly about readlines() function. The solution should work in Python 3.6+.

Answer: From the documentation:  f.readline() reads a single line from the file; a newline character (\n) is left at the end of the string, and is only omitted on the last line of the file if the file doesn’t end in a newline. This makes the return value unambiguous; if f.readline() returns an empty string, the end of the file has been reached, while a blank line is represented by '\n', a string containing only a single newline.  with open(file_name, 'r') as i_file: while True: line = i_file.readline() if not line: break # do something with line
Question: I have a python dictionary: d = { "config": { "application": { "payment": { "dev": { "modes": {"credit,debit,emi": {}}, "company": { "address": { "city": {"London": {}}, "pincode": {"LD568162": {}}, }, "country": {"United Kingdom": {}}, "phone": {"7865432765": {}}, }, "levels": {"0,1,2": {}}, }, "prod": {"modes": {"credit,debit": {}}, "levels": {"0,1": {}}}, } } } }  I want to change it to something like this(if the value is empty{} then make the key as value for its parent) d = { "config": { "application": { "payment": { "dev": { "modes": "credit,debit,emi", "company": { "address": { "city": "London", "pincode": "LD568162" }, "country": "United Kingdom", "phone": "7865432765" }, "levels": "0,1,2" }, "prod": { "modes": "credit,debit", "levels": "0,1" } } } } }  i tried to write the code to traverse this deep dictionary, but couldn't modify it to get the above output. Please help. def recur(json_object): for x in list(json_object.items()): print(x) recur(json_object[x]) d={'config': {'application': {'payment': {'dev': {'modes': {'credit,debit,emi': {}}, 'company': {'address': {'city': {'London': {}}, 'pincode': {'LD568162': {}}}, 'country': {'United Kingdom': {}}, 'phone': {'7865432765': {}}}, 'levels': {'0,1,2': {}}}, 'prod': {'modes': {'credit,debit': {}}, 'levels': {'0,1': {}}}}}}} 

Answer: Solution 1 We can use a non-recursive approach with queues to enqueue each inner/nested element of the document and put as value if the nested value is just {}: # d = ... queue = [d] while queue: data = queue.pop() for key, value in data.items(): if isinstance(value, dict) and list(value.values()) == [{}]: data[key] = list(value.keys())[0] elif isinstance(value, dict): queue.append(value) print(d)  Output { "config": { "application": { "payment": { "dev": { "modes": "credit,debit,emi", "company": { "address": { "city": "London", "pincode": "LD568162" }, "country": "United Kingdom", "phone": "7865432765" }, "levels": "0,1,2" }, "prod": { "modes": "credit,debit", "levels": "0,1" } } } } }  Solution 2 Here's a recursive approach # d = ... def recur(data): for key, value in data.items(): if isinstance(value, dict) and list(value.values()) == [{}]: data[key] = list(value.keys())[0] elif isinstance(value, dict): recur(value) recur(d) print(d)  Output  Same as Solution 1
Question: I know that the asterisk is used to unpack values like system args or when you unpack lists into variables. But I have not seen this syntax here before in this example of asyncio. I was reading this article here, https://realpython.com/async-io-python/#the-10000-foot-view-of-async-io , but I don't understand what the asterisk operator is doing in this context. #!/usr/bin/env python3 # rand.py import asyncio import random # ANSI colors c = ( "\033[0m", # End of color "\033[36m", # Cyan "\033[91m", # Red "\033[35m", # Magenta ) async def makerandom(idx: int, threshold: int = 6) -> int: print(c[idx + 1] + f"Initiated makerandom({idx}).") i = random.randint(0, 10) while i <= threshold: print(c[idx + 1] + f"makerandom({idx}) == {i} too low; retrying.") await asyncio.sleep(idx + 1) i = random.randint(0, 10) print(c[idx + 1] + f"---> Finished: makerandom({idx}) == {i}" + c[0]) return i async def main(): res = await asyncio.gather(*(makerandom(i, 10 - i - 1) for i in range(3))) return res if __name__ == "__main__": random.seed(444) r1, r2, r3 = asyncio.run(main()) print() print(f"r1: {r1}, r2: {r2}, r3: {r3}")  Under the async def main function right before the makerandom there is an asterisk. Could someone explain what it does in this context ? I am trying to understand how async / await works. I looked at this answer, Python asterisk before function but it doesn't really explain it.

Answer: The asterisk isn't before makerandom, it's before the generator expression (makerandom(i, 10 - i - 1) for i in range(3))  asyncio.gather doesn't take an iterable as its first argument; it accepts a variable number of awaitables as positional arguments. In order to get from a generator expression to that, you need to unpack the generator. In this particular instance, the asterisk unpacks asyncio.gather(*(makerandom(i, 10 - i - 1) for i in range(3)))  into asyncio.gather(makerandom(0, 9), makerandom(1, 8), makerandom(2, 7))
Question: I am doing some validation of the input data of one program I have created. I am doing this with assert. If assertion arises I want to know in which part of the data occurs so I want to get the value that arises the assertion. assert all(isinstance(e, int) for l1 in sequence.values() for l2 in l1 for e in l2),"Values of the dictionnary aren't lists of integers. Assert found in '{l2}'" # This code doesn't work 

Answer: Not when using all as it does not expose the "iteration variable". You will need an explicit, nested loop. You also forgot the f'' prefix to denote an f-string: for l1 in [['a']]: for l2 in l1: for e in l2: assert isinstance(e, int), f"Values of the dictionnary aren't lists of integers. Assert found in '{l2}'"  AssertionError: Values of the dictionnary aren't lists of integers. Assert found in 'a'
Question: I have a list of masks and I want to obtain the resulting mask by multiplying all of them. My Donkey Kong approach is the following: a = [[1, 1], [1, 0], [1, 0]] b = a[0] for i in range(1, len(a)): b = b * np.array(a[i])  which I think it works as returns [1,0] as value of b. Is there a nicer way of doing this?  EDIT: I am looking for common ranges in the mask. To find all the non zero ranges I do the following: label = 0 for i in range(1, len(labels)): label = label + np.array(labels[i]) label = [1 if x > 0 else 0 for x in label] 

Answer: Take a look at np.prod, which returns the product of array elements over a given axis: import numpy as np a = [[1, 1], [1, 0], [1, 0]] np.prod(a, axis=0)
Question: I am trying to install pyOpenSSl and it shows the following error  Requirement already satisfied: six>=1.5.2 in /home/tony/hx-preinstaller-venv/lib/python3.6/site-packages (from pyOpenSSL) Collecting cryptography>=3.3 (from pyOpenSSL) Using cached https://files.pythonhosted.org/packages/cc/98/8a258ab4787e6f835d350639792527d2eb7946ff9fc0caca9c3f4cf5dcfe/cryptography-3.4.8.tar.gz Complete output from command python setup.py egg_info:   =============================DEBUG ASSISTANCE========================== If you are seeing an error here please try the following to successfully install cryptography: Upgrade to the latest pip and try again. This will fix errors for most users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip =============================DEBUG ASSISTANCE========================== Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/pip-build-8toyhikv/cryptography/setup.py", line 14, in <module> from setuptools_rust import RustExtension ModuleNotFoundError: No module named 'setuptools_rust'  The command I ran: pip install pyOpenSSL 

Answer: Try upgrade pip and install setuptools-rust: pip install --upgrade pip pip install setuptools-rust
Question: The Goal I would like to get the ranges where values are not None in a list, so for example: test1 = [None, 0, None] test2 = [2,1,None] test3 = [None,None,3] test4 = [1,0,None,0,0,None,None,1,None,0] res1 = [[1,1]] res2 = [[0,1]] res3 = [[2,2]] res4 = [[0,1],[3,4],[7,7],[9,9]]  What I have tried This is my super lengthy implementation, which does not perfectly work... def get_not_None_ranges(list_): # Example [0, 2, None, 1, 4] -> [[0, 1], [3, 4]] r = [] end_i = len(list_)-1 if list_[0] == None: s = None else: s = 0 for i, elem in enumerate(list_): if s != None: if elem == None and end_i != i: r.append([s,i-1]) s = i+1 if end_i == i: if s > i: r=r elif s==i and elem == None: r=r else: r.append([s,i]) else: if elem != None: s = i if end_i == i: if s > i: r=r else: r.append([s,i]) return r  As you can see the results are sometimes wrong: print(get_not_None_ranges(test1)) print(get_not_None_ranges(test2)) print(get_not_None_ranges(test3)) print(get_not_None_ranges(test4)) [[1, 2]] [[0, 2]] [[2, 2]] [[0, 1], [3, 4], [6, 5], [7, 7], [9, 9]]  So, I was wondering if you guys know a much better way to achieve this?

Answer: Use itertools.groupby: from itertools import groupby test1 = [None, 0, None] test2 = [2, 1, None] test3 = [None, None, 3] test4 = [1, 0, None, 0, 0, None, None, 1, None, 0] def get_not_None_ranges(lst): result = [] for key, group in groupby(enumerate(lst), key=lambda x: x[1] is not None): if key: index, _ = next(group) result.append([index, index + sum(1 for _ in group)]) return result print(get_not_None_ranges(test1)) print(get_not_None_ranges(test2)) print(get_not_None_ranges(test3)) print(get_not_None_ranges(test4))  Output [[1, 1]] [[0, 1]] [[2, 2]] [[0, 1], [3, 4], [7, 7], [9, 9]]
Question: I have a DataFrame read from Excel with one of the columns of type DateTime. sales_data=pandas.read_excel(r'Sample Sales Data.xlsx')  I was able to extract substrings from other columns using str.extract/lambda functions. But I was unable to process the column "Order Date" The command sales_data['Order Date'] gives the below output  As recommended in other StackOverflow questions, I tried with sales_data['Order Date'].apply(lambda x:x.str.slice())  I got an error that : AttributeError: 'datetime.datetime' object has no attribute 'str' To check the type of the Order date column, I tried sales_data['Order Date'].apply(lambda x:type(x))  I got the type datetime.datetime  But when I tried the datetime operation sales_data['Order Date'].apply(lambda x:x.strftime("m"))  I got the error: AttributeError: 'int' object has no attribute 'strftime' I got a similar error for the command sales_data['Order Date'].apply(lambda x:x.dt.month)  Please suggest a method to extract month from the datetime object into another column without iterating through the DataFrame. I am not able to use datetime or int functions with this column since it is behaving as both a datetime and int column.

Answer: I found the issue. The sales_data['Order Date'] column had a mix of both date and int values due to some input data inaccuracy. I found this since sales_data['DateType']=sales_data['Order Date'].apply(lambda x:type(x)) sales_data['DateType'].unique()  returned array([<class 'datetime.datetime'>, <class 'int'>], dtype=object) I cleaned this DataFrame by filtering out the values without the datetype as datetime. type1=type(sales_data['DateType'][0]) new_df=sales_data[sales_data['DataType']==type1]  Now the new dataframe supports the date and string operations. pd.to_datetime(new_df['Order Date']).dt.month  This can be assigned to other columns.
Question: This seems relatively straight forward, but I have yet to find a duplicate that answers my question, or a method with the needed functionality. I have an Excel spreadsheet where each column that contains data has a unique header. I would like to use pandas to get the letter key of the column by passing this header string into a function. For example, if I pass "Output Parameters" to the function, I would like to return "M":  The closest thing that I have found is the xlsxwriter.utility.xl_col_to_name(index) method outlined in the second answer in Convert spreadsheet number to column letter This is very similar to what I am trying to do, however the column numbers within my sheets will not remain constant (unlike the headers, which will). This being said, a method that can return the column number based on the header would also work, as I would then be able to apply the above. Does pandas or xlsxwriter have a method that can handle the above case?

Answer: You can try: import xlsxwriter col_no = df.columns.get_loc("col_name") print(xlsxwriter.utility.xl_col_to_name(col_no))
Question: When I run dev_appserver.py on google-cloud-sdk, I get ImportError: No module named py27_urlquote. Traceback (most recent call last): File "/Users/user/Downloads/google-cloud-sdk/platform/google_appengine/dev_appserver.py", line 109, in <module> _run_file(__file__, globals()) File "/Users/user/Downloads/google-cloud-sdk/platform/google_appengine/dev_appserver.py", line 103, in _run_file _execfile(_PATHS.script_file(script_name), globals_) File "/Users/user/Downloads/google-cloud-sdk/platform/google_appengine/dev_appserver.py", line 83, in _execfile execfile(fn, scope) File "/Users/user/Downloads/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/devappserver2.py", line 44, in <module> from google.appengine.tools.devappserver2 import dispatcher File "/Users/user/Downloads/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/dispatcher.py", line 43, in <module> from google.appengine.tools.devappserver2 import module File "/Users/user/Downloads/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/module.py", line 39, in <module> import py27_urlquote ImportError: No module named py27_urlquote  I have tried the following:  Reinstall Cloud SDK Find out about the py27_urlquote module (I couldn't find any such information ...) Change the version of CLOUDSDK_PYTHON to 2.7 or 3.8 and execute 

Answer: Right now this is a public issue and is currently being addressed by our Google Engineering Team. A workaround was provided for you to run your local development server:  Install pip for Python 2  sudo apt update sudo apt install python-pip   Install urlquote instead of py27_urlquote  pip install urlquote   Modify module.py located on your local directory from the error message  /Users/user/Downloads/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/module.py   Replace py27_urlquote to urlquote. There are 3 lines of code that uses py27_urlquote. Specifically lines 39, 833, and 836.  You can check this public tracker similar to your issue for more information and updates.
Question: I want to move a file form one directory to another in linux with python. I wish to achieve a behavior similar to bash mv command. What is the difference in practice between the two commands os.replace() os.rename()  Is it simply that os.rename() will raise an error if file exists in destination while os.replace() will overwrite it? Also if another secondary difference that I see it that the os.replace() needs a file as a destination not just the directory. I can find a direct answer anywhere.

Answer: On POSIX systems, the rename system call will silently replace the destination file if the user has sufficient permissions. The same is not true on Windows: a FileExistsError is always raised. os.replace and os.rename are the same function on POSIX systems, but on Windows os.replace will call MoveFileExW with the MOVEFILE_REPLACE_EXISTING flag set to give the same effect as on POSIX systems. If you want consistent cross-platform behaviour you should consider using os.replace throughout.
Question: I find this answer, and I want to use pyvips to resize images. In the mentioned answer and the official documentation image resized by scale. However, I want to resize the image to a specific height and width. Is there any way to achieve this with pyvips?

Answer: The thumbnail operation in pyvips will load an image to fit a box, for example: thumb = pyvips.Image.thumbnail("some-file.jpg", 128)  Will load some-file.jpg and make an image that fits within 128x128 pixels. Variations on thumbnail can load from strings, buffers or pipes, load to fit other boxes, check the chapter on vipsthumbnail for details. If you already have a loaded image, thumbnail_image will resize to pixel dimensions in the same way, but can be quite a bit slower.
Question: Setup: # Pydantic Models class TMDB_Category(BaseModel): name: str = Field(alias="strCategory") description: str = Field(alias="strCategoryDescription") class TMDB_GetCategoriesResponse(BaseModel): categories: list[TMDB_Category] @router.get(path="category", response_model=TMDB_GetCategoriesResponse) async def get_all_categories(): async with httpx.AsyncClient() as client: response = await client.get(Endpoint.GET_CATEGORIES) return TMDB_GetCategoriesResponse.parse_obj(response.json())  Problem: Alias is being used when creating a response, and I want to avoid it. I only need this alias to correctly map the incoming data but when returning a response, I want to use actual field names. Actual response: { "categories": [ { "strCategory": "Beef", "strCategoryDescription": "Beef is ..." }, { "strCategory": "Chicken", "strCategoryDescription": "Chicken is ..." } }  Expected response: { "categories": [ { "name": "Beef", "description": "Beef is ..." }, { "name": "Chicken", "description": "Chicken is ..." } } 

Answer: Use the Config option by_alias.  from fastapi import FastAPI, Path, Query from pydantic import BaseModel, Field app = FastAPI() class Item(BaseModel): name: str = Field(..., alias="keck") @app.post("/item") async def read_items( item: Item, ): return item.dict(by_alias=False)  Given the request: { "keck": "string" }  this will return { "name": "string" }
Question: I installed git-filter-repo via scoop, tried multiple git filter-repo commands e.g. git filter-repo -h, they all logged nothing, no warning or error, just nothing. Tried rebooting, reinstalling, and installing it on another Windows 10 computer, all reproduced it. git-filter-repo: v2.33.0 git: v2.33.0.windows.2 python: v3.9.7 scoop: Current Scoop version: 09200504 (HEAD -> master, origin/master, origin/HEAD) reset: skip when app instance is running (#4359) 'main' bucket: b71f4a842 (HEAD -> master, origin/master, origin/HEAD) nunit-extension-vs-project-loader: Update to version 3.9.0  How to solve this issue?

Answer: (Now updated for newer Python installers.) When I installed git-filter-repo on Windows earlier this year, the following steps worked for me:  Download and install Python for Windows. In newer installers you need to go into the Advanced Options to make sure Python is added to your Path:   Confirm python was added to your path and that you can run either the command python --version or python3 --version from your Git command line. (I recommend Git Bash.) In my case, my executable name is python and if yours is too, you will need this in step #7 below.  Clone git-filter-repo from GitHub. git clone https://github.com/newren/git-filter-repo.git   Run the command git --exec-path to see your Git exe directory.  From the git-filter-repo repo's root directory, copy the file git-filter-repo (about 160KB) into your Git exe directory.  In your command line where you use Git, type the command git filter-repo. If it works, you should get the message "No arguments specified." and you can skip step #7. If it doesn't work, it's likely that your python exe is python instead of python3 as determined in step #2. Go to the next step.  If you get no message or an error message similar to "/usr/bin/env: ‘python3’: No such file or directory", then edit the file git-filter-repo that you copied into your Git exe directory in step #5, and change the first line from "python3" to "python".   Now be amazed at how fast and awesome git-filter-repo is. Still having problems? If you didn't add the environment variable in step #1, some people have had luck in step #7 by changing their python command to just "py". This is the python launcher which can auto-detect the highest version installed on your machine. More info here. I should point out that this did not work for me with python 3.10.7. I actually tried this first but ended up re-installing and enabling the option to "Add python to environment variables" as described above in step #1.
Question: glob.glob() does not use regex. it uses Unix path expansion rules. How can I emulate this regex in glob: ".*.jpg|.*.png" 

Answer: @U12-Forward is correct that there isn't an exact solution but depending on your use case you might be able to solve it with the [...] wildcard. For your example with .png or .jpg you could use this: .*.[jp]*  which will match any extension that starts with a j or p If you have other extensions that start with j or p you can be more specific: .*.[jp][pn]g