eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am trying to setup my VS Code debugger for pytest and am getting the error discovering pytests tests from the testing tab. The test I'm running works perfectly when run from the terminal but the VS debugger is not able to run it. In the testing tab, it directs me to the output where it shows a ModuleNotFoundError for logdna in the common.py file. That file has some other seemingly syntax errors but I believe VSCode is just not interpreting them correctly because the modules are in the code base. On the left you can see I ran pip3 install logdna to ensure logdna is actually installed. Note that I am running inside a Poetry environment and have a .env in the project root directory. Note also that I have reviewed and tried all answers provided here and none of the provided answers helped in my case: VSCode pytest test discovery fails 

Answer: You are using virtualenv inside terminal. Here is clearly seeing in logs that python is installed inside /Users/.../pypoetry/virtualenvs/... folder. But VSCode using default python (/usr/local/bin/python3) or at least non virtualenv version You have to choose same interpreter in VSCode as terminal's one What is virtualenv?
Question: I'm trying to return a status code without exiting the process of my script. Is there an equivalent to sys.exit() that does not stop execution ? What I'd like to happen : If no exception is raised, status code is by default 0. For minor exceptions, I want to return status code 1, but keep the process running, unless a more critical error exits the process and returns status code 2 with sys.exit(2). In my case, I'm looping through a bunch of PDF files and if one file has not been anonymised for some reason, I want to return status code 1 while looping through the rest of the files. In other words, status code should correspond to :  0 - SUCCESS : all files have been sucessfully anonymised 1 - WARNING : at least one file has not been anonymised 2 - ERROR : critical error, interrupted process  For clarity : by status code I mean, what is returned by the Bash command echo $? after the execution of the script. What I have done : I have tried using sys.exit(1) to handle minor exceptions but it automatically stops execution, as stated in the library documentation. To sum up with code snippets : For a given main.py : import sys import os def anonymiser(file) -> Boolean : # Returns True if successfully anonymises the file, False otherwise ... def process_pdf_dir(): for file in os.listdir('pdf_files/'): if file.endswith('.pdf'): if anonymiser(file) == False: ''' Here goes the code to change status code to 1 but keeps looping through the rest of the files unlike `sys.exit(1)` that would stop the execution. ''' if criticalError = True: sys.exit(2) if __name__ == '__main__': process_pdf_dir()  After calling the script on the terminal, and echoing its status like so : $ python main.py $ echo $?  If indeed, at least one file has not been anonymised :  Expected output (if no critical error) :  1   Expected output (if critical error) :  2   Current output (if no critical error) :  0   Current output (if critical error) :  2    Hope this is clear enough. Any help would be greatly appreciated. Thanks.

Answer: Wait until the end of your loop to return the 1 exit code. I'd do it like: def process_pdf_dir() -> int: ret = 0 for file in os.listdir('pdf_files/'): if file.endswith('.pdf'): if not anonymiser(file): # remember to return a warning ret = 1 if criticalError: # immediately return a failure return 2 return ret if __name__ == '__main__': sys.exit(process_pdf_dir())  If "critical error" includes uncaught exceptions, maybe put that outside of process_pdf_dir, e.g.: if __name__ == '__main__': try: sys.exit(process_pdf_dir()) except: sys.exit(2)
Question: I have the following directory structure (a friends was so kind to put it on github while he examined it) - code - elements __init__.py type_of_car.py __init__.py car.py - tests __init__.py test_car.py  These are my launch.json settings: { "version": "0.2.0", "configurations": [ { "name": "Python: Debug Tests", "type": "python", "request": "launch", "program": "${file}", "purpose": ["debug-test"], "console": "integratedTerminal", "justMyCode": false }, { "name": "Python: Module", "type": "python", "request": "launch", "module": "main", "justMyCode": false, "cwd": "${workspaceFolder}" } ] }  The VS Python Test settings are: { "python.testing.unittestArgs": [ "-v", "-s", "./tests", "-p", "test_*.py" ], "python.testing.pytestEnabled": false, "python.testing.unittestEnabled": true, "python.testing.cwd": "${workspaceFolder}" }  The test_car.py imports module - of course - code.car. But also code.car.type_of_car When I run the test from the project root, the test can be invoked and passes. py -m unittest tests.test_car.py However, I cannot run my main code by pressing F5 (see launch.json) configuration. This fails because with No module named 'code.car'; 'code' is not a package being reported. Also, I have to debug my tests as well with Visual Studio Code:  I navigate to test_engine.py and open it Switch the "RUN AND DEBUG" to Python: Debug Tests configuration Press F5 to run the debugging.  This fails because with No module named 'code.car'; 'code' is not a package being reported. How can I resolve the module-hell so that I can run the test also from VSCode/debugger? (this thing did cost me hours. Any hint is appreciated.) Does someone has insight what the VS Code launcher considers it's root when calling the module?

Answer: You can modify the car.py and test_car.py files as follows: I only pasted the modified code。 car.py： # your code from code.elements.type_of_car import TypeOfCar  # my code from elements.type_of_car import TypeOfCar  test_car.py： # your code import unittest from code.car import Car from random import randint from code.elements.type_of_car import TypeOfCar  # my code import unittest import sys sys.path.append("./code") from car import Car from random import randint from elements.type_of_car import TypeOfCar  Results of debugging Python: Debug Tests:
Question: I need to test my telegram bot. To do this I need to create client user to ask my bot. I found telethon library which can do it. First I wrote a code example to ensure that authorisation and connection works and send test message to myself (imports omitted): api_id = int(os.getenv("TELEGRAM_APP_ID")) api_hash = os.getenv("TELEGRAM_APP_HASH") session_str = os.getenv("TELETHON_SESSION") async def main(): client = TelegramClient( StringSession(session_str), api_id, api_hash, sequential_updates=True ) await client.connect() async with client.conversation("@someuser") as conv: await conv.send_message('Hey, what is your name?') if __name__ == "__main__": asyncio.run(main())  @someuser (me) successfully receives message. Okay, now I create a test with fixtures based on code above: api_id = int(os.getenv("TELEGRAM_APP_ID")) api_hash = os.getenv("TELEGRAM_APP_HASH") session_str = os.getenv("TELETHON_SESSION") @pytest.fixture(scope="session") async def client(): client = TelegramClient( StringSession(session_str), api_id, api_hash, sequential_updates=True ) await client.connect() yield client await client.disconnect() @pytest.mark.asyncio async def test_start(client: TelegramClient): async with client.conversation("@someuser") as conv: await conv.send_message("Hey, what is your name?")  After running pytest received an error: AttributeError: 'async_generator' object has no attribute 'conversation' It seems client object returned from client fixture in "wrong" condition. Here is print(dir(client)): ['__aiter__', '__anext__', '__class__', '__class_getitem__', '__del__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__name__', '__ne__', '__new__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'aclose', 'ag_await', 'ag_code', 'ag_frame', 'ag_running', 'asend', 'athrow']  Where I loose "right" client object from generator in fixture?

Answer: Use @pytest_asyncio.fixture decorator in async fixtures according to documentation https://pypi.org/project/pytest-asyncio/#async-fixtures. Like this: import pytest_asyncio @pytest_asyncio.fixture(scope="session") async def client(): ...
Question: I am trying to schedule a crontab job to run a python script and referring to python in /usr/local/bin/python3 but getting this error SyntaxError: Non-UTF-8 code starting with '\xca' in file /usr/local/bin/python3 on line 2 What does this mean and how can I solve it? I can't open the python3 file

Answer: you're calling to python twice. Decide which interpreter you want to use and run either: * * * * * /Users/name/opt/anaconda3/envs/myenv/bin/python /Users/name/Desktop/Scrape/scraper.py  or * * * * * /usr/local/bin/python3 /Users/name/Desktop/Scrape/scraper.py
Question: I created a docker image for my FastAPI application then I created a container from that image. Now When I connect to that container using docker exec -it <container-id> through my terminal, I am able to access it but the problem is that autocomplete doesn't work when I press TAB.

Answer: What I have understood from your question is when you enter into the docker environment, you are unable to autocomplete filenames and folders. Usually when you enter into the container via shell, the autocomplete not works properly. Tried to enter into the container using bash environment i.e., docker exec -it <container-id> bash. Now you can use TAB to autocomplete files and folders.
Question: I have a text file in this format (in_file.txt): banana 4500 9 banana 350 0 banana 550 8 orange 13000 6   How can I convert this into a dictionary list in Python? Code: in_filepath = 'in_file.txt' def data_dict(in_filepath): with open(in_filepath, 'r') as file: for line in file.readlines(): title, price, count = line.split() d = {} d['title'] = title d['price'] = int(price) d['count'] = int(count) return [d]  The terminal shows the following result: {'title': 'orange', 'price': 13000, 'count': 6}  Correct output: {'title': 'banana', 'price': 4500, 'count': 9}, {'title': 'banana', 'price': 350, 'count': 0} , ....  Can anyone help me with my problem? Thank you!

Answer: titles = ["title","price","count"] [dict(zip(titles, [int(word) if word.isdigit() else word for word in line.strip().split()])) for line in open("in_file.txt").readlines()]  or: titles = ["title","price","count"] [dict(zip(titles, [(data:=line.strip().split())[0], *map(int, data[1:])])) for line in open("in_file.txt").readlines()]  your approach(corrected): in_filepath = 'in_file.txt' def data_dict(in_filepath): res = [] with open(in_filepath, 'r') as file: for line in file.readlines(): title, price, count = line.split() d = {} d['title'] = title d['price'] = int(price) d['count'] = int(count) res.append(d) return res data_dict(in_filepath)  why? because  ->   d = {} d['title'] = title d['price'] = int(price) d['count'] = int(count)  is out of for loop and run only once and when ‍‍for be finished and then you have just one element  you return your last element and didn't use others and use must create a list and append every element at the last line of for loop (saving) and at last, return result  @Rockbar approach: import pandas as pd list(pd.read_csv("in_file.txt", sep=" ", header=None, names=["title","price","count"]).T.to_dict().values())
Question: Assuming I prevent Hydra from changing the current working directory, how can I still retrieve the job's output directory (i.e., the folder Hydra created for storing the results of the particular job) from main()? Ideally, I'd like a method that works regardless of whether it's a regular run or a multi-run. I know I can access the configuration through hydra.core.hydra_config.HydraConfig.get(), but it's unclear whether any of the settings there always map to the output folder. The relevant tutorial page discusses how to set the output folder but not how to retrieve it in the case we didn't change the cwd.

Answer: Found it. hydra_cfg = hydra.core.hydra_config.HydraConfig.get() hydra_cfg['runtime']['output_dir']
Question: I have a GUI with widgets that has multiple child widgets inside, as you can see in the image:  I want to communicate "Widget 2-2" with "Widget 1-1". I have different options, but I don't know which is better. 1. Propagate signal My first idea was propagates the emit to Main Windows and then it propagates the action to the target widget:  Widget 2-2 emits a signal Widget 2 catch the emit and emit anther signal Main Windows catch the emit and call a Widget 1 controller function Widget 1 controller function calls a Widget 1-1 controller function Widget 1-1 makes the action  Pros  All the widgets only talks with its parents and children  Cons  It can be a bit messy  2. Create a singleton class with all the signals Another idea is to create the signals in a singleton class. Then the Widget 2-2 can emit this signals easily and Widget 1-1 can listen this signals easily too. The relative position of the widgets in the architecture does not care, everybody can emit and connect the signals. Pros  Emit and connect this signals is super easy. Everybody can do it.  Cons  I feel that it is a super bad idea. Probably it will explode in the future, but I don't know why. I prefer to follow the best practices than solve my problem fast and easily (for the moment...) 

Answer: Singletons, or static initialisation for that matter ought to be the last resort as they can cause all sorts of headaches, especially in multi-threaded environment. Definitely do not go for that in this case. What I have personally done in my project in this case, I emitted a signal in the first widget and then forwarded that in the first common parent to the second widget so that the second widget handled it via its slot in there. This has worked fairly well for me, at least. You can forward signals just fine in Qt, i.e. connect a signal to a signal. The first common parent will encompass both, so you can use it as a proxy for managing the two, or potentially even more widgets in your architecture.
Question: When I create a virtual environment in Python on Windows cmd, in the virtual environment folder, the following files appear:  Include Lib Scripts pyvenv.cfg  What is the goal of pyvenv.cfg creation? Can I use it in any way?

Answer: pyvenv.cfg is a configuration file that stores information about the virtual environment such as  standard libraries path Python version interpreter version virtual env flags or any other venv configs  If print content of pyvenv.cfg $ cat myenv/pyvenv.cfg home = /usr/bin include-system-site-packages = false version = 3.8.5  This is a sample output of the pyvenv.cfg file. Cheers!
Question: How can I create a TypedDict class that supports keys containing hyphens or other characters that are supported in strings, such as "justify-content" in the example below. from typing import TypedDict, Literal from typing_extensions import NotRequired class Attributes(TypedDict): width: NotRequired[str] height: NotRequired[str] direction: NotRequired[Literal["row", "column"]] justify-content: NotRequired[Literal["start", "end", "center", "equally-spaced"]] 

Answer: It is possible with the functional syntax: from typing import TypedDict, Literal from typing_extensions import NotRequired Attributes = TypedDict( "Attributes", { "width": NotRequired[ str, ], "height": NotRequired[ str, ], "direction": NotRequired[ Literal["row", "column"], ], "justify-content": NotRequired[ Literal["start", "end", "center", "equally-spaced"] ], }, )  It's mentioned in the documentation  The functional syntax should also be used when any of the keys are not valid identifiers, for example because they are keywords or contain hyphens
Question: I need a function that takes a rule/condition as an input. for example given an array of integers detect all the numbers that are greater than two, and all the numbers greater than four. I know this can be achieved easily without a function, but I need this to be inside a function. The function I would like to have is like def _select(x,rule): outp = rule(x) return outp L = np.round(np.random.normal(2,4,50),decimals=2) y = _select(x=L,rule=(>2)) y1 = _select(x=L,rule=(>4))  How should I code a function like this?

Answer: Functions are first class objects meaning you can treat them as any other variable. import numpy as np def _select(x,rule): outp = rule(x) return outp def rule_2(val): return val > 2 def rule_4(val): return val > 4 L = np.round(np.random.normal(2,4,50),decimals=2) y = _select(x=L,rule=rule_2) print(y) y1 = _select(x=L,rule=rule_4) print(y1)  In your example, the condition you want to use can be expressed as a simple expression. The python lambda keyword lets you define expressions as anonymous functions in other statements and expressions. So, you could replace the explicit def of the functions import numpy as np def _select(x,rule): outp = rule(x) return outp L = np.round(np.random.normal(2,4,50),decimals=2) y = _select(x=L,rule=lambda val: val > 2) print(y) y1 = _select(x=L,rule=lambda val: val > 4) print(y1)
Question: Hi I was wondering how to asynchronously call a function within a for-loop in Python, allowing the for-loop to execute more quickly. bar() in this case is a time intensive function, which is why I want the calls to it to be nonblocking. Here is what I want to refactor: def bar(item): //manipulate item return newItem newItems = [] for item in items: newItem = foo(item) newItems.append[newItem]  Here is what I've tried: async def bar(item): //manipulate item return newItem async def foo(): newItems = [bar(item) for item in items] newItems = await asyncio.gather(*newItems) return newItems newItems = asyncio.run(foo())  This doesn't seem to work as each function call still waits for the previous one to finish before starting. I would love tips on what I might be doing wrong. Thank you so much for any and all help!

Answer: If your tasks are really async you can do it the following way: import asyncio async def bar(item: int) -> int: # manipulate item print("Started") await asyncio.sleep(5) print("Finished") return item ** 2 async def foo(): items = range(1, 10) tasks = [bar(item) for item in items] new_items = await asyncio.gather(*tasks) return new_items if __name__ == '__main__': results = asyncio.run(foo()) print(results)
Question: Let's say we have pandas dataframe pd and a dask dataframe dd. When I want to plot pandas one with matplotlib I can easily do it: fig, ax = plt.subplots() ax.bar(pd["series1"], pd["series2"]) fig.savefig(path)  However, when I am trying to do the same with dask dataframe I am getting Type Errors such as: TypeError: Cannot interpret 'string[python]' as a data type  string[python] is just an example, whatever is your dd["series1"] datatype will be inputed here. So my question is: What is the proper way to use matplotlib with dask, and is this even a good idea to combine the two libraries?

Answer: SultanOrazbayev's is still spot on, here is an answer elaborating on the datashader option (which hvplot call under the hood). Don't use Matplotlib, use hvPlot! If you wish to plot the data while it's still large, I recommend using hvPlot, as it can natively handle dask dataframes. It also automatically provides interactivity. Example import numpy as np import dask import hvplot.dask # Create Dask DataFrame with normally distributed data df = dask.datasets.timeseries() df['x'] = df['x'].map_partitions(lambda x: np.random.randn(len(x))) df['y'] = df['y'].map_partitions(lambda x: np.random.randn(len(x))) # Plot df.hvplot.scatter(x='x', y='y', rasterize=True)
Question: When using nbconvert, how can I specify the name and directory of the new file?

Answer: Use the --output flag to change the name of the converted file Use the --output-dir flag to change the directory of the converted file jupyter nbconvert <path/to/notebook.ipynb> --to <x> --output <"name" (without file extension)> --output-dir <path/to/new/file>
Question: I have a 2D numpy array (y_array) with 3 columns (and common x values as a list, x_list) and I want to create a plot with each column plotted as a line. I can do this by simply doing matplotlib.pyplot.plot(x_list, y_array) and it works just fine. However I am struggeling with the colors. I need to assign custom colors to each line. I tried by handing a list of my custom colors to the color=key argument, but apparently it does not take a List and throws a ValueError. I find this particularly odd since giving a list for labels actually does work. I also thought about creating a custom colormap from my choosen colors, but I do not know how to switch to this colormap when plotting... What can I do to specify the colors to be used when plotting the array? I would like to avoid iterating over the array columns in a for loop. Thanks in advance! Edit: minimal working example: This throws the mentioned ValueError import matplotlib.pyplot as plt import numpy as np if __name__ == '__main__': x_list = np.linspace(0, 9, 10) y1 = np.random.uniform(10, 19, 10) y2 = np.random.uniform(20, 29, 10) y3 = np.random.uniform(30, 39, 10) y_array = np.column_stack([y1, y2, y3]) labels = ['A', 'B', 'C'] # plt.plot(x_list, y_array, label=labels) # this works just fine my_colors = ['steelblue', 'seagreen', 'firebrick'] plt.plot(x_list, y_array, label=labels, color=my_colors) # throws ValueError plt.legend() plt.show() 

Answer: You can create a custom cycler and use it to define your colors. You can get more information here import matplotlib.pyplot as plt import numpy as np from cycler import cycler if __name__ == '__main__': my_colors = ['steelblue', 'seagreen', 'firebrick'] custom_cycler = cycler(color=my_colors) x_list = np.linspace(0, 9, 10) y1 = np.random.uniform(10, 19, 10) y2 = np.random.uniform(20, 29, 10) y3 = np.random.uniform(30, 39, 10) y_array = np.column_stack([y1, y2, y3]) fig, ax = plt.subplots() ax.set_prop_cycle(custom_cycler) ax.plot(x_list, y_array) plt.show()