eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I create a dataframe with distinct column names and use rename to create columns with same name. import pandas as pd df = pd.DataFrame({ "a_1": ["x", "x"], "a_2": ["", ""], "b_1": ["", ""], "a_3": ["", "y"], "c_1": ["z", "z"], }) names = { "a_1": "a", "a_2": "a", "a_3": "a", "b_1": "b", "c_1": "c", } df2 = df.rename(columns=names)  This produces  a a b a c 0 x z 1 x y z  How to join values in the columns with same name into a list? out = pd.DataFrame({ "a": [["x"], ["x", "y"]], "b": [[""], [""]], "c": [["z"], ["z"]], }) a b c 0 [x] [] [z] 1 [x, y] [] [z]  Bad attempt I suspect this can be resolved with lambda for c in ['a', 'b', 'c']: df2[c] = df2[c].apply(lambda x: x.tolist() if x.any() else [], axis=1)  This however produces TypeError: <lambda>() got an unexpected keyword argument 'axis'  Any idea?

Answer: You could transpose and groupby.agg with a custom lambda: df2.T.groupby(level=0).agg(lambda x: x[x!=''].tolist()).T  Output:  a b c 0 [x] [] [z] 1 [x, y] [] [z]
Question: All of my Selenium scripts are raising errors after Chrome updated to version 127 because I always have to select a default search engine when the browser is being launched. I use ChromeDriver 127.0.6533.72. How to fix it?

Answer: You need to add this Chrome Option to disable the 'choose your search engine' screen: options.addArguments("--disable-search-engine-choice-screen");  If you are using selenium with Python, you'll have to use: options.add_argument("--disable-search-engine-choice-screen")
Question: I have a dataframe column containing text, and I'd like to make a new column which contains the sentences with names, but no other sentences. Hoping for an end result that looks like this:  I am able to identify cells containing names from the list of names, but I'm stumbling on the part that extracts the sentence containing the name. import re import pandas as pd import numpy as np df = pd.DataFrame({ 'ColumnA': ['Lorum ipsum. This is approved. Lorum Ipsum.', 'Lorum Ipsum. Send the contract to May. Lorum Ipsum.', 'Junk Mail from Brian.'] }) last_names_list = ['May','Brian'] df['last_names'] = '' for x in last_names_list: df['last_names'] = np.where(df['ColumnA'].str.contains(x),x,df['last_names']) def f(x,y): return re.findall(fr'[^.]{x}[^.]',y) df['col_3'] = df.apply(lambda x: f(x['last_names'],x['ColumnA']), axis=1) print(df)  When I print the dataframe, every row with a name in df[col_3'] produces an empty list. Any help appreciated.

Answer: Code pat = '|'.join(last_names_list) df['col_3'] = df['ColumnA'].str.extract(rf'([^.]*?\b(?:{pat})\b.*?\.)')  df:
Question: In Python, we can use subprocess to run bat (cmd) scripts natively, like import subprocess as sp sp.run(["D:/Temp/hello.bat"])  works fine. However, it cannot run ps1 scripts natively, codes like import subprocess as sp sp.run(["D:/Temp/hello.ps1"])  will cause `WinError 193: %1 is not a valid Win32 application" to be raised. I thought it was because the .PS1 had not been showing up in PATHEXT and added it there, but it failed again. I also tried the way provided in this answer which seems to be setting the file execution policy and it still wouldn't work. I know it would work when I add the executor in the call like sp.run(["pwsh", "D:/Temp/hello.ps1"]) or use a bat script as intermediate, but that's not desired. I am using a program which is badly ported from *nix to Windows and it just call whatever I provided as a single executable (on *nix we can use shebang, but not on windows).

Answer:  Batch files are special in that they are the only type of interpreter-based script files directly recognized as executables by the system.  For any other script files, including PowerShell scripts (*.ps1), the only way you can achieve their execution when directly invoked is:  Redefine the command that is used by the Windows shell to open *.ps1 files (i.e. the command associated with the Open shell verb) so as to pass the file path at hand to the PowerShell CLI (powershell.exe for Windows PowerShell, pwsh for PowerShell (Core) 7) for execution.  Note: The default behavior when opening *.ps1 files from outside PowerShell (which includes invoking them from cmd.exe and double-clicking in File Explorer) is to open them for editing. Changing the behavior to executing PowerShell scripts may be unexpected by other users, so it's best to limit this change to your own user account..  The SuperUser answer you link to shows how do to that, via the registry; note that the linked code requires elevation, i.e. running with administrative privileges.  However, given the above, you may prefer to define the command for the current user only, in which case elevation isn't required - see the second-to-last section of this answer.     In order for subprocess.run() to use the redefined command, shell=True must be passed (which calls via cmd.exe, which automatically invokes the default Windows shell operation on non-executable files): import subprocess as sp # Performs the "Open" Window shell operation on the *.ps1 file. sp.run(["D:/Temp/hello.ps1"], shell=True)     If you don't want to redefine the command associated with the Open shell verb, your only option is indeed to call *.ps1 files via the PowerShell CLI; e.g.: import subprocess as sp sp.run(["powershell.exe", "-NoProfile", "-File", "D:/Temp/hello.ps1"])
Question: I’m trying to create a GET route with Litestar that utilizes a Pydantic model as a query parameter. However, the serialization does not work as expected. Here is a minimal example that reproduces my issue: from pydantic import BaseModel from litestar import Litestar, get, Controller class Input(BaseModel): foo: str bar: str class RootController(Controller): path = "/" @get() def input(self, input: Input) -> str: return input.foo + input.bar app = Litestar(route_handlers=[RootController])  And the following GET request: import httpx import json params = { "input": { "foo": "test", "bar": "this" } } def prepare_encode(params: dict) -> dict: for key, value in params.items(): if isinstance(value, dict): params[key] = json.dumps(value, indent=None) return params params = prepare_encode(params) response = httpx.get("http://localhost:8000/", params=params) response.json()  The GET request results in the following error: { "status_code": 400, "detail": "Validation failed for GET /?input=%7B%22foo%22%3A%20%22test%22%2C%20%22bar%22%3A%20%22this%22%7D", "extra": [ { "message": "Input should be a valid dictionary or instance of Input" } ] }  It seems that the query parameter is not being properly serialized into the Input Pydantic model. What I've Tried:  Using json.dumps to encode the dictionary before sending it as a parameter. Debugging the Litestar model implementation where the query parameter is provided as string into the msgspec conversion. This does obviously not fit to the required type.  Expected Behavior: I expect the input query parameter to be correctly parsed and serialized into the Input model, allowing the GET request to succeed without validation errors. Question: How can I correctly pass a Pydantic model as a query parameter in a Litestar GET route? What am I missing in the serialization process? Is it possible at all? Additional Context:  Litestar version: 2.10.0 Pydantic version: 2.8.2 httpx version: 0.27.0  Any help or guidance would be greatly appreciated.

Answer: There are actually problems with both how you make and how you process your request. First, I wasn't be able to find in the docs a possibility to use Pydantic model for query params as FastAPI has. However you can implement similar logic yourself via DI mechanism: def build_input(foo: str, bar: str) -> Input: """Prepare input model from query params.""""" return Input(foo=foo, bar=bar) class RootController(Controller): path = "/" @get(dependencies={"input": Provide(build_input)}) def input(self, input: Input) -> str: return input.foo + input.bar  Secondly, if you'd like to use nested structure as an input you should better use POST method instead. { "input": { "foo": "test", "bar": "this" } }  Otherwise, the above dictionary will be converted to the following string when used as a query: %7B%22foo%22%3A%20%22test%22%2C%20%22bar%22%3A%20%22this%22%7D I assume what you wanted to do was the following: response = httpx.get( "http://localhost:8000/", params={ "foo": "test", "bar": "this" } )  With these changes everything seems working!
Question: I'm trying to get dollar, prices for my school project. So I decide to use web scraping for this but I have a problem about it. When I try to use my code on server it gives me NoneType ERROR. It works on google colab but I can't use on my pc or server. How can I solve this guys? Web Scrape code ; def dolar(): headers = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15' url = f'https://finance.yahoo.com/quote/TRY=X/' r = requests.get(url) soup = bs(r.text, 'html.parser') dolar = soup.find("div", {"class": "container yf-mgkamr"}).find_all("span")[0].text return dolar  EROOR ; Traceback (most recent call last): File "/Users/user/Desktop/API/main.py", line 38, in <module> dolar() File "/Users/user/Desktop/API/main.py", line 35, in dolar dolar = soup.find("div", {"class": "container yf-mgkamr"}).find_all("span")[0].text ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'find_all' (.venv) user@192 API %  I tried to change my main website, tried to use without ".find_all" method. It doesn't change anything.

Answer: You should probably use this import requests import time def dolar(): now = time.time() - 10 headers = { "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15" } url = f"https://query1.finance.yahoo.com/v8/finance/chart/TRY=X?period1={int(now)}&period2={int(now)}" response = requests.get(url, headers=headers) return response.json()['chart']['result'][0]['meta']['regularMarketPrice'] print(dolar())
Question: I'm trying to filter out rows of a dataframe where a column of strings under the name 'question' contains all the substrings in a given list. That is, if the given list of substrings is ['King', 'England'], then I need to retain all the rows in the dataframe where the string in the df.question contains both King and England. This code executes without any problems and prints out a boolean value: print(all([word in df.question[0] for word in ['King', 'England']]))  But this code results in the following error: print(df[all([word in df.question for word in ['King', 'England']])])  --------------------------------------------------------------------------- KeyError Traceback (most recent call last) File ~\anaconda3\Lib\site-packages\pandas\core\indexes\base.py:3805, in Index.get_loc(self, key) 3804 try: -> 3805 return self._engine.get_loc(casted_key) 3806 except KeyError as err: File index.pyx:167, in pandas._libs.index.IndexEngine.get_loc() File index.pyx:196, in pandas._libs.index.IndexEngine.get_loc() File pandas\\_libs\\hashtable_class_helper.pxi:7081, in pandas._libs.hashtable.PyObjectHashTable.get_item() File pandas\\_libs\\hashtable_class_helper.pxi:7089, in pandas._libs.hashtable.PyObjectHashTable.get_item() KeyError: False The above exception was the direct cause of the following exception: KeyError Traceback (most recent call last) Cell In[9], line 2 1 print(all([word in df.question[0] for word in ['King', 'England']])) ----> 2 print(df[all([word in df.question for word in ['King', 'England']])]) File ~\anaconda3\Lib\site-packages\pandas\core\frame.py:4102, in DataFrame.__getitem__(self, key) 4100 if self.columns.nlevels > 1: 4101 return self._getitem_multilevel(key) -> 4102 indexer = self.columns.get_loc(key) 4103 if is_integer(indexer): 4104 indexer = [indexer] File ~\anaconda3\Lib\site-packages\pandas\core\indexes\base.py:3812, in Index.get_loc(self, key) 3807 if isinstance(casted_key, slice) or ( 3808 isinstance(casted_key, abc.Iterable) 3809 and any(isinstance(x, slice) for x in casted_key) 3810 ): 3811 raise InvalidIndexError(key) -> 3812 raise KeyError(key) from err 3813 except TypeError: 3814 # If we have a listlike key, _check_indexing_error will raise 3815 # InvalidIndexError. Otherwise we fall through and re-raise 3816 # the TypeError. 3817 self._check_indexing_error(key) KeyError: False  How do I filter rows in a Dataframe based on this condition?

Answer: You can use .str.contains to check each word and then np.all over axis=0 to combine all boolean values for each row: import numpy as np import pandas as pd df = pd.DataFrame( { "question": [ "Who is the King of England?", "Where is England?", "England King", "King", ] } ) df2 = df[ np.all([df['question'].str.contains(word) for word in ["King", "England"]], axis=0) ] print(df2)  Output:  question 0 Who is the King of England? 2 England King
Question: I am using VScode 1.91.1. It seems like it doesn't want to recognize Literal outside of the __init__ method. Is there a way to have the right type throughout the class? class MyDict(TypedDict): my_var: Literal["a", "b"] class MyClass: def __init__(self, my_dict: MyDict): self.my_var = my_dict["my_var"] # hovering over `my_var` here shows Literal["a", "b"] def other_method(self): self.my_var # hovering over it here shows "str" 

Answer: Another way you could try is the following: from typing import Literal, TypedDict class MyDict(TypedDict): my_var: Literal["a", "b"] class MyClass: def __init__(self, my_dict: MyDict): self._my_var: Literal["a", "b"] = my_dict["my_var"] @property def my_var(self) -> Literal["a", "b"]: return self._my_var def other_method(self): # Now, type checkers should recognize `self.my_var` as `Literal["a", "b"] print(self.my_var) # Hovering over this should correctly show `Literal["a", "b"]  where you ensure the property method with @property that helps maintain the Literal type. By assigning _my_var with the specific Literal type and then exposing it through a property method, the type information is maintained consistently throughout the class. This approach ensures that type checkers and IDEs correctly recognize self.my_var as Literal["a", "b"], keeping the type information intact and correctly propagated. I hope this is useful, if not I will try my best again to help you. Have a good coding bro :)))
Question: I am trying to learn Pandas, and am doing a simple project in PyCharm, but PyCharm is saying there is an error because I am not using specific out-of-date versions of Pandas, NumPy, and Faker. I do not know what Faker is. You can see the error message and my code below, though my code doesn't matter much to my question.  The code still runs as expected, but I want to understand if I'm setting up problems for the future. I tried googling this, but I don't know what keywords I should be searching.

Answer: From what I've encountered in the past, this usually means that there is either a requirements.txt or a pyproject.toml, or a similar file in your project directory. These files define what versions of required packages the project was built for to let the user know that if they have a version different than that, the developer cannot guarantee the code will work. At least for the commonly used styles, regardless of the style of requirements file a project has, there will be lines included in it such as what you are seeing in that warning banner from PyCharm. These will look like numpy == 1.18.4 pandas == 1.0.4 ... other package requirements ...  The reason it is showing a warning is because these requirements are listed as strong equalities, meaning that the developer is indicating that these versions of these packages are the ONLY version that they guarantee the code will work with. But this isn't the only requirement type that can be listed - there is also >= meaning it needs to be at least the listed version, <= for at most, etc. From what I've seen though, the syntax of the requirements file may vary a bit between styles though I don't know this for certain. I would recommend looking up the syntax of whatever specific requirements file you have. Are you importing a custom package? I don't have time right now to test it to make sure, but I believe it's possible that an imported package that wasn't installed through pip or an environment manager such as conda can also trigger these kinda of warnings (such as in the case of downloading source code from GitHub and including it in your project) (As a side note, it looks like there are a few different file conventions that can be used, though I don't know what files PyCharm will autodetect the less common ones. From what I was able to find in the Pycharm docs, it seems that you can point PyCharm to whatever requirements file you have if it fails to autodetect it.)
Question: I am working with Qiskit for programming quantum circuits. Everything works fine but there is one thing which I didn't find out. Here is my Python code: from qiskit import QuantumCircuit qc = QuantumCircuit(2) qc.h(0) qc.cx(0, 1) qc.draw(output = "mpl")  This is my output:  I am only interested into the graphical representation of the Hadamard gate (red one). I need it for a description. I don't need the wires and the qubits. How can I save the Hadamard gate as png/svg file using Qiskit?

Answer: This worked with me: from qiskit import QuantumCircuit from qiskit.visualization import circuit_drawer qc = QuantumCircuit(1) qc.h(0) circuit_img = circuit_drawer(qc, output='mpl', scale=2) circuit_img.savefig('hadamard_gate.png') # save figure as PNG circuit_img.savefig('hadamard_gate.svg') # save figure as SVG  cf. https://docs.quantum.ibm.com/api/qiskit/qiskit.visualization.circuit_drawer for additional info
Question: I was trying to follow an example outlined here https://stackoverflow.com/a/61973946 The code, after small adjustments looks like this: def pos_image(x, y, pays, haut): pays = countries.get(pays).alpha2.lower() fichier = "iso-flag-png" fichier += f"/{pays}.png" im = mpimg.imread(fichier) ratio = 4 / 3 w = ratio * haut print(w) ax.imshow(im, extent=(x - w, x, y, y + haut), zorder=2) plt.style.use('seaborn') fig, ax = plt.subplots() liste_pays = [('France', 10), ('USA', 9), ('Spain', 5), ('Italy', 5)] X = [p[1] for p in liste_pays] Y = [p[0] for p in liste_pays] haut = .8 r = ax.barh(y=Y, width=X, height=haut, zorder=1) y_bar = [rectangle.get_y() for rectangle in r] for pays, y in zip(liste_pays, y_bar): pos_image(pays[1], y, pays[0], haut) plt.show()  However, the resulting plot looks like this  Showing only the last value of the list. What am i doing wrong? Thank you

Answer: I can reproduce your result. I am not exactly sure what's going wrong here: Either the answer that you refer to never worked in the first place or some matplotlib internals have changed since it was posted. That said, here is the problem: Once you call pos_image(), the internal call to ax.imshow() has the effect that the x and y limits are adjusted to the region where the flag is placed. The accepted answer to the same question actually also mentions this:  Afterwards, the plt.xlim and plt.ylim need to be set explicitly (also because imshow messes with them.)  And here is a solution (or workaround): Remember the x and y limits before calling pos_image(), and adjust them again afterwards: x_lim, y_lim = ax.get_xlim(), ax.get_ylim() for pays, y in zip(liste_pays, y_bar): pos_image(pays[1], y, pays[0], haut) ax.set_xlim(x_lim) ax.set_ylim(y_lim)  Produces (I used the default style rather than 'seaborn'):
Question: I'm looking for a way to have two updatemenu buttons on a plotly figure. One changes the data and the y-axis title and one switches from linear to log scale. Broadly the code below works but I lose something depending on which method I use. If the buttons method is update then when I switch parameters it defaults back to linear scale. If I use restyle the y-axis title stops updating. Is there a method to only partially update I've missed? import plotly.graph_objects as go import pandas as pd def main(): figure = go.Figure() df = pd.DataFrame([['sample 1', 1, 5, 10], ['sample 2', 2, 20, 200], ], columns=['sample id', 'param1', 'param2', 'param3'] ) options = ['param1', 'param2', 'param3'] figure.add_trace(go.Bar(x=df['sample id'], y=df[options[0]])) figure.update_yaxes(title=options[0]) buttons = [] for option in options: buttons.append({'method': 'restyle', ### this can be update or restyle, each has different issues 'label': option, 'args': [{'y': [df[option]], 'name': [option]}, {'yaxis': {'title': option}}, [0]] }) scale_buttons = [{'method': 'relayout', 'label': 'Linear Scale', 'args': [{'yaxis': {'type': 'linear'}}] }, {'method': 'relayout', 'label': 'Log Scale', 'args': [{'yaxis': {'type': 'log'}}] } ] figure.update_layout(yaxis_title=options[0], updatemenus=[dict(buttons=buttons, direction='down', x=0, xanchor='left', y=1.2, ), dict(buttons=scale_buttons, direction='down', x=1, xanchor='right', y=1.2, )], ) figure.show() if __name__ == '__main__': main() 

Answer: You need to use the update method for the first dropdown (restyle only updates data, update updates both data and layout - nb. those methods refer to their respective Plotly.js function, see Plotly.relayout and Plotly.update). Now to fix the issue, the key is to use "attribute strings", that is, 'yaxis.title': option instead of 'yaxis': {'title': option} in the arguments, otherwise the update will reset the yaxis params other than title to their defaults (same applies for the yaxis type) :  The term attribute strings is used to mean flattened (e.g., {marker: {color: 'red'}} vs. {'marker.color': red}). When you pass an attribute string to restyle inside the update object, it’s assumed to mean update only this attribute. Therefore, if you wish to replace and entire sub-object, you may simply specify one less level of nesting.  buttons = [] for option in options: buttons.append({ 'method': 'update', 'label': option, 'args': [{ 'y': [df[option]], 'name': [option] }, { 'yaxis.title': option }] }) scale_buttons = [{ 'method': 'relayout', 'label': 'Linear Scale', 'args': [{'yaxis.type': 'linear'}], }, { 'method': 'relayout', 'label': 'Log Scale', 'args': [{'yaxis.type': 'log'}] }]
Question: I am trying to use colorbar for an output variable circle plot in Folium colormap = cm.LinearColormap(colors=['green','red'], index=[min(df['output']), max(df['output'])], vmin=min(df['output']),vmax=max(df['output']), caption='output in units') folium.Circle(location=[row['Latitude'], row['Longitude']], radius=800*row["output"], fill=True, # opacity=0.8, # fill_opacity=1, color=colormap(row["output"])).add_to(m)  When I use this code, the caption text "output in units" is appearing very small. How to increase its font size?

Answer: AFAIK, there is no direct way to do it but, you can inject CSS using the font-size property. By inspecting the html's folium Map (with CTRL+SHIFT+I) on Chrome, the caption's text seems to be at #legend > g > text :  import folium import pandas as pd import branca.colormap as cm df = pd.DataFrame({"output": range(10, 110, 10)}) agg = df["output"].agg(vmin="min", vmax="max") m = folium.Map([39.5, -98.35], zoom_start=4) colormap = cm.LinearColormap( colors=["green", "red"], index=agg.to_list(), caption="output in units", **agg.to_dict(), ) CSS = "font-size: 16px;" # add more props if needed m.get_root().header.add_child( folium.Element(f"&ltstyle>#legend > g > text.caption {{{CSS}}}</style>") ) colormap.add_to(m)