eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have a Django app and in one of its views I use asyncio in order to make some concurent requests to an external component. Here is the idea: import asyncio async def do_request(project): result = ... return result def aggregate_results(projects: list): loop = asyncio.new_event_loop() asyncio.set_event_loop(loop) results = loop.run_until_complete( asyncio.gather(*(do_request(project) for project in projects)) ) loop.close() return zip(projects, results)  Well, when I run the tests I get DeprecationWarning: There is no current event loop at this line:  asyncio.gather(*(do_request(project) for project in projects))  How should I interpret this warning and what do I need to change to get rid of it? Thanks!

Answer: According to the documentation, this happens because there is no event loop running at the time when you call gather.  Deprecated since version 3.10: Deprecation warning is emitted if no positional arguments are provided or not all positional arguments are Future-like objects and there is no running event loop.  As you've probably noticed, your code works. It will continue to work and you can ignore the deprecation warning as long as you use 3.10. At some point in the future, though, this may change to a runtime error. If you'll bear with me a moment, the recommended way to run an event loop is with run, not loop.run_until_complete. def aggregate_results(projects: list): results = asyncio.run(asyncio.gather(*(do_request(project) for project in projects))) return zip(projects, results)  This, however, won't actually work. You'll instead get an exception ValueError: a coroutine was expected, got <_GatheringFuture pending>  The fix is to instead await gather from another coroutine. async def get_project_results(projects: list): results = await asyncio.gather(*(do_request(project) for project in projects)) return results def aggregate_results(projects: list): results = asyncio.run(get_project_results(projects)) return zip(projects, results)  (You could also use get_project_results with your version of aggregate_results.)
Question: Im using python 3.9 and coverage 6.2 I would like to have a record of my most recent coverage but Im not sure if I should upload my .coverage file. Im guessing no since it sort of has info about my directory layout. So i would like to know how I should go about that, is it even standard to upload such a thing? If not, why not? I also generated the htmlcov folder but I didnt upload it since it has a default gitignore for the entire folder.

Answer: Most people don't upload the .coverage or HTML report, because they don't need to track it over time. But there's no harm in uploading them. You mention directory layout as if it was a secret to protect, but isn't your layout already committed to GitHub? If you want to commit the HTML report, you will need to remove the .gitignore file in the directory.
Question: Recently I came across functools.cache and didn't know how it differs from functools.lru_cache. I found posts about the difference between functools.cached_property and lru_cache but nothing specifically for cache and lru_cache.

Answer: functools.cache was newly added in version 3.9. The documentation states:  Simple lightweight unbounded function cache. Sometimes called “memoize”. Returns the same as lru_cache(maxsize=None), creating a thin wrapper around a dictionary lookup for the function arguments. Because it never needs to evict old values, this is smaller and faster than lru_cache() with a size limit.  Example from the docs: @cache def factorial(n): return n * factorial(n-1) if n else 1 >>> factorial(10) # no previously cached result, makes 11 recursive calls 3628800 >>> factorial(5) # just looks up cached value result 120 >>> factorial(12) # makes two new recursive calls, the other 10 are cached 479001600  So, in short: cache and lru_cache(maxsize=None) are exactly the same (link to cpython source). But in cases where you don't want to limit the cache size, using cache may make the code clearer, since a least recently used cache without limit doesn't make much sense.
Question: I followed a tutorial in youtube on how to create a Telegram Bot and for now it can only send messages but I want to send even files like documents or audio, video, photos etc... For now im just trying to send a file but I'm pretty confused and I don't know how to do it. The source code of the bot is divided in 2 main files. One responser.py: def responses(input_text): user_message = str(input_text).lower() if user_message in ("test", "testing"): return "123 working..." return "The command doesn't exists. Type /help to see the command options."  and main.py: import constants as key from telegram.ext import * import responser as r print("Hello. Cleint has just started.") def start_command(update): update.message.reply_text("The Bot Has Started send you command sir.") def help_command(update): update.message.reply_text(""" Welcome to the Cleint Bot. For this purchase the following commands are available: send - send command is to send the log file from the other side of computer""") def send_document(update, context): doc_file = open("image1.png", "rb") chat_id = update.effctive_chat.id return context.bot.send_document(chat_id, doc_file) def handle_message(update, context): text = str(update.message.text).lower() response = r.responses(text) update.message.reply_text(response) def error(update, context): print(f"Update {update} cause error: {context.error}") def main(): updater = Updater(key.API_KEY, use_context=True) dp = updater.dispatcher dp.add_handler(CommandHandler("start", start_command)) dp.add_handler(CommandHandler("help", help_command)) dp.add_handler(MessageHandler(Filters.text, handle_message)) dp.add_error_handler(error) updater.start_polling() updater.idle() main()  could someone help me out?

Answer: Try the following as your send_document function: def send_document(update, context): chat_id = update.message.chat_id document = open('image1.png', 'rb') context.bot.send_document(chat_id, document)  And add the command 'send' to the bot in the main function like this: dp.add_handler(CommandHandler("send", send_document))  This will make it so if you type /send in Telegram, the bot will send you the document.
Question: Given a string, typically a sentence, I want to extract all substrings of lengths 3, 4, 5, 6. How can I achieve this efficiently using only Python's standard library? Here is my approach, I am looking for one which is faster. To me it seems the three outer loops are inevitable either way, but maybe there is a low-level optimized solution with itertools or so. import time def naive(test_sentence, start, end): grams = [] for word in test_sentence: for size in range(start, end): for i in range(len(word)): k = word[i:i+size] if len(k)==size: grams.append(k) return grams n = 10**6 start, end = 3, 7 test_sentence = "Hi this is a wonderful test sentence".split(" ") start_time = time.time() for _ in range(n): naive(test_sentence, start, end) end_time = time.time() print(f"{end-start} seconds for naive approach")  Output of naive(): ['thi', 'his', 'this', 'won', 'ond', 'nde', 'der', 'erf', 'rfu', 'ful', 'wond', 'onde', 'nder', 'derf', 'erfu', 'rful', 'wonde', 'onder', 'nderf', 'derfu', 'erful', 'wonder', 'onderf', 'nderfu', 'derful', 'tes', 'est', 'test', 'sen', 'ent', 'nte', 'ten', 'enc', 'nce', 'sent', 'ente', 'nten', 'tenc', 'ence', 'sente', 'enten', 'ntenc', 'tence', 'senten', 'entenc', 'ntence']   Second version: def naive2(test_sentence,start,end): grams = [] for word in test_sentence: if len(word) >= start: for size in range(start,end): for i in range(len(word)-size+1): grams.append(word[i:i+size]) return grams 

Answer: Well, I think this is not possible to improve the algorithm, but you can micro-optimize the function: def naive3(test_sentence,start,end): rng = range(start,end) return [word[i:i+size] for word in test_sentence if len(word) >= start for size in rng for i in range(len(word)+1-size)]  Python 3.8 introduces assignment Expressions that are quite useful for performance. Thus if you can use a recent version, then you can write: def naive4(test_sentence,start,end): rng = range(start,end) return [word[i:i+size] for word in test_sentence if (lenWord := len(word)+1) > start for size in rng for i in range(lenWord-size)]  Here are performance results: naive2: 8.28 µs ± 55 ns per call naive3: 7.28 µs ± 124 ns per call naive4: 6.86 µs ± 48 ns per call (20% faster than naive2)  Note that half of the time of naive4 is spent in creating the word[i:i+size] string objects and the rest is mainly spent in the CPython interpreter (mainly due to the creation/reference-counting/deletion of variable-sized integer objects).
Question: Currently, the latest Python 3 offered through winget is 3.10.150.0: Name Id Version Match Source ----------------------------------------------------------------------------------------------------------------------------------- Python 3 Python.Python.3 3.10.150.0 Command: python winget  but I'd like to install 3.9 and keep using 3.9. Is it possible to do that with winget?

Answer: winget install -e --id Python.Python -v 3.9.0
Question: I'm using Pytorch to run Transformer model. when I want to split data (tokenized data) i'm using this code: train_dataset, test_dataset = torch.utils.data.random_split( tokenized_datasets, [train_size, test_size])  torch.utils.data.random_split using shuffling method, but I don't want to shuffle. I want to split it sequentially. Any advice? thanks

Answer: The random_split method has no parameter that can help you create a non-random sequential split. The easiest way to achieve a sequential split is by directly passing the indices for the subset you want to create: # Created using indices from 0 to train_size. train_dataset = torch.utils.data.Subset(tokenized_datasets, range(train_size)) # Created using indices from train_size to train_size + test_size. test_dataset = torch.utils.data.Subset(tokenized_datasets, range(train_size, train_size + test_size))  Refer: PyTorch docs.
Question: I have a dataframe like this: df = pd.DataFrame({'A': ['1', '2', '3'], 'B': ['aa', 'b', 'c']}) A B 0 1 aa 1 2 b 2 3 c  I want to convert each row of column B to a list. For example, my desired output is something like this:  df_new A B 0 1 [aa] 1 2 [b] 2 3 [c] 

Answer: You can use split to do stuff. import pandas as pd df = pd.DataFrame({'A': ['1', '2', '3'], 'B': ['a', 'b', 'c']}) df['B'] = df['B'].apply(lambda x: x.split(',')) print(df)
Question: I installed Anaconda. Somehow, afterwards, when I press Enter in Python files, it no longer works. This error shows up:  command 'pythonIndent.newlineAndIndent' not found  How do I solve this problem?

Answer: Try checking your keybindings.json file, also disable all VS Code extensions and install them one by one. Maybe it's caused by some buggy extension.
Question: I am using pydantic for some user/password data model. When the model is printed, I want to replace the value of password with something else (*** for example) to prevent that the password is e.g. written into log-files or the console accidentally. from pydantic import BaseModel class myUserClass(BaseModel): User = 'foo' Password = 'bar' def __str__(self): return "Hidden parameters." U = myUserClass() print(U) # > Hidden parameters. # Should be User='foo', Password='***' or similar U # This will still show my password. #> myUserClass(User='foo', Password='bar')  How can I access the string that is normally printed and replace only 'bar' with '***', but keeping all parameters? How can I also do that when just calling U? This may not be equally important as for logging and console output usually print is called.

Answer: Pydantic provides SecretStr data type. Which is replaced by *** when converted to str (e.g printed) and actual value can be obtained by get_secret_value() method: class Foobar(BaseModel): password: SecretStr empty_password: SecretStr # Initialize the model. f = Foobar(password='1234', empty_password='') # Assert str and repr are correct. assert str(f.password) == '**********' assert str(f.empty_password) == '' # Assert retrieval of secret value is correct assert f.password.get_secret_value() == '1234' assert f.empty_password.get_secret_value() == ''
Question: I have a URL that can only be accessed from within our VPC. If I enter the URL via Postman, I am able to see the content (which is a Pdf) and I'm able to save the output to a file perfectly fine. However, when trying to automate this using python requests with import requests r = requests.get(url, params=params)  I get an exception ChunkedEncodingError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))  The other Stackoverflow questions haven't really helped much with this and this is consistently reproducible with requests

Answer: I think the reason for this error is server is not correctly encoding the chunk. some of the chunk sizes are not integer b'' due to which you are getting chunk encoding error. Try below code import requests from requests.exceptions import ChunkedEncodingError response = requests.get(url, params=params, stream=True) try: for data in response.iter_content(chunk_size=1024): print(data) except ChunkedEncodingError as ex: print(f"Invalid chunk encoding {str(ex)}")
Question: When I use FastAPI , how can I sepcify a base path for the web-service? To put it another way - are there arguments to the FastAPI object that can set the end-point and any others I define, to a different root path? For example , if I had the code with the spurious argument root below, it would attach my /my_path end-point to /my_server_path/my_path ? from fastapi import FastAPI, Request app = FastAPI(debug = True, root = 'my_server_path') @app.get("/my_path") def service( request : Request ): return { "message" : "my_path" } 

Answer: You can use an APIRouter and add it to the app after adding the paths: from fastapi import APIRouter, FastAPI app = FastAPI() prefix_router = APIRouter(prefix="/my_server_path") # Add the paths to the router instead @prefix_router.get("/my_path") def service( request : Request ): return { "message" : "my_path" } # Now add the router to the app app.include_router(prefix_router)  When adding the router first and then adding paths it does now work. It seems that the paths are not detected dynamically. Note: a path prefix must start with / (Thanks to Harshal Perkh for this)
Question: I have the following pyproject.toml for configuring black: [tool.black] exclude = 'foo.py'  If I run black . from the project's root folder that only contains foo.py, I get No Python files are present to be formatted. Nothing to do �😴 as expected. However, when I save foo.py from within VS Code (I have black configured as the formatter and enabled Format On Save), the file is still formatted by black. Interestingly, VS Code seems to honor other configurations, e.g. line-length. Is there a way to make VSCode honor the exclude configuration?

Answer: The --force-exclude option also excludes files if they are explicitly listed. Thus, it also works with formatOnSave in VS Code. In the example above, simply use [tool.black] force-exclude = 'foo.py'
Question: I'm using multiprocessing in a larger code base where some of the import statements have side effects. How can I run a function in a background process without having it inherit global imports? # helper.py: print('This message should only print once!')  # main.py: import multiprocessing as mp import helper # This prints the message. def worker(): pass # Unfortunately this also prints the message again. if __name__ == '__main__': mp.set_start_method('spawn') process = mp.Process(target=worker) process.start() process.join()  Background: Importing TensorFlow initializes CUDA which reserves some amount of GPU memory. As a result, spawing too many processes leads to a CUDA OOM error, even though the processes don't use TensorFlow. Similar question without an answer:  How to avoid double imports with the Python multiprocessing module? 

Answer:  Is there a resources that explains exactly what the multiprocessing module does when starting an mp.Process?  Super quick version (using the spawn context not fork) Some stuff (a pair of pipes for communication, cleanup callbacks, etc) is prepared then a new process is created with fork()exec(). On windows it's CreateProcessW(). The new python interpreter is called with a startup script spawn_main() and passed the communication pipe file descriptors via a crafted command string and the -c switch. The startup script cleans up the environment a little bit, then unpickles the Process object from its communication pipe. Finally it calls the run method of the process object. So what about importing of modules? Pickle semantics handle some of it, but __main__ and sys.modules need some tlc, which is handled here (during the "cleans up the environment" bit).
Question: I have the following Pydantic model: from pydantic import BaseModel import key class Wallet(BaseModel): private_key: str = Field(default_factory=key.generate_private_key) address: str  I want address to have a default_factory as a function that takes a private_key as input and returns an address. My intentions would be something along the lines of the following: address: str = Field(default_factory=key.generate_address(self.private_key)  How can I achieve this?

Answer: Another option is to just use @validator because in it you can access previously validated fields. From the documentation:   validators are "class methods", so the first argument value they receive is the UserModel class, not an instance of UserModel. the second argument is always the field value to validate; it can be named as you please you can also add any subset of the following arguments to the signature (the names must match):  values: a dict containing the name-to-value mapping of any previously-validated fields     Example: class Wallet(BaseModel): private_key: str = Field(default_factory=key.generate_private_key) address: str = "" # "" seems better than None to use the correct type @validator("address", always=True) def get_address(cls, address: str, values: Dict[str, Any]) -> str: if address == "" and "private_key" in values: return key.generate_address(values["private_key"]) return address  It can be argued that @validator should be preferred over @root_validator if you just want to generate a value for a single field. There are two important aspects of this approach that must be considered:  The "previously-validated fields" from the documentation means that in your case private_key must be defined before address. The values of fields defined after the field that is validated are not available to the validator.  If the field that is validated has a default value and you still want the validator to be executed in that case, you have to use always=True.
Question: I'm on a Macbook with M1 (Apple ARM architecture) and I've tried running the following Python code using the layoutparser library, which indirectly uses pycocotools: import layoutparser as lp lp.Detectron2LayoutModel()  And I've received the error: [...] ImportError: dlopen([...]/.venv/lib/python3.9/site-packages/pycocotools/_mask.cpython-39-darwin.so, 0x0002): tried: '[...]/.venv/lib/python3.9/site-packages/pycocotools/_mask.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/_mask.cpython-39-darwin.so' (no such file), '/usr/lib/_mask.cpython-39-darwin.so' (no such file)  The crucial info for me seems to be [...] is an incompatible architecture (have 'x86_64', need 'arm64e') [...]. Indeed, I am using the Apple ARM architecture, and sometimes it is not supported by some software. This is usually solved by using Rosetta, which simulates an Intel-x64 architecture. So I start a terminal with Rosetta (arch -x86_64 zsh), create a new virtual environment, make a fresh install of the dependencies, and try to run the code again ... ... and I receive the same error that I had also had without Rosetta: [...] is an incompatible architecture (have 'x86_64', need 'arm64e') [...] 🥲 I've double-checked that Rosetta is really activated: > uname -m x86_64  Rosetta seems to be working. And yet, according to the error message, it seems not to be working. Any ideas what could be the problem with Rosetta, or the library, or whatever, and how I could try fixing it?

Answer: Charles Duffy explained the problem in the comments, thank you! 😃 When I checked the platform in Python, it was indeed ARM: > python -c 'import platform; print(platform.platform())' macOS-12.0.1-arm64-i386-64bit  So I had been using a Python installation for ARM. Now I installed brew and then python3 from the Rosetta terminal and used the newly installed Python to initiate a fresh virtual environment, and this fixed it. (This article helped me a bit with it.) Update: When creating Python environments with conda, it is possible to specify whether they should use Apple ARM or Intel-x64:  CONDA_SUBDIR=osx-arm64 conda create -n my_env python makes an ARM environment CONDA_SUBDIR=osx-64 conda create -n my_env python makes an x64 environment
Question: I have wrote the following code: import matplotlib.pyplot as plt import numpy as np x = np.linspace(0, 5, 100) y = x**2 plt.plot(x, y) plt.xlabel("X-axis") plt.ylabel("Y-axis") plt.title("graph of $x^2$") plt.show()  When I run the code, it shows the following error:  I have installed NumPy, successfully, but still getting this warning. How can I get rid of this?

Answer: Try this in VSCode IDE: (1) ctrl-shift-p (2) In search box, type python and then select Python: Select Interpreter (3) select Python 3.10.4 or any one matches your verion the problem usually caused by IDE didn't pick up the right interpreter
Question: I make creating a discord asynchronous library that is fully typed. I have a method that create objects from a aiohttp get requests such as the following example: async def get_bans(self): """|coro| Fetches all the bans in the guild. """ data = await self._http.get(f"guilds/{self.id}/bans") for ban_data in data: yield Ban.from_dict(construct_client_dict(self._client, ban_data))  I was wondering about the return type of this code snippet and whether it should be a AsyncGenerator[Ban, None] or AsyncIterator[Ban, None]. To be honest i have been searching for a bit, and i could find any information that would gave me a clear idea on the subject.

Answer: Official documentation describes AsyncIterator as:  ... that provide __aiter__ and __anext__ methods ...  AsyncGenerator leads from documentation to PEP525.  ... generator is any function containing one or more yield expressions ...   The result of calling an asynchronous generator function is an asynchronous generator object ...   Asynchronous generators define both of these methods (__aiter__ and __anext__)  So I think, because of yield in your function it's better to use AsyncGenerator.