eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I know that Python maintains an internal storage of small-ish integers rather than creating them at runtime: id(5)  4304101544  When repeating this code after some time in the same kernel, the id is stable over time: id(5)  4304101544  I thought that this wouldn't work for floating point numbers because it can't possibly maintain a pre-calculated list of all floating point numbers. However this code returns the same id twice. id(4.33+1), id(5.33)  (5674699600, 5674699600)  After some time repeating the same code returns some different location in memory: id(4.33 + 1), id(5.33)  (4962564592, 4962564592)  What's going on here?

Answer: It's not just that the object is garbage collected and and the new object stored in the same location as the previous one after garbage collection. Something different is at work here. We can use the dis module to look at the bytecode generated: import dis def f(): one, two = 4.3333333, 3.3333333 + 1. a, b = id(one), id(two) return one, two, a, b dis.dis(f) one, two, a, b = f()  shows us the bytecode generated:  1 0 RESUME 0 2 2 LOAD_CONST 1 ((4.3333333, 4.3333333)) 4 UNPACK_SEQUENCE 2 8 STORE_FAST 0 (one) 10 STORE_FAST 1 (two) 3 12 LOAD_GLOBAL 1 (NULL + id) 24 LOAD_FAST 0 (one) 26 PRECALL 1 30 CALL 1 40 LOAD_GLOBAL 1 (NULL + id) 52 LOAD_FAST 1 (two) 54 PRECALL 1 58 CALL 1 68 STORE_FAST 3 (b) 70 STORE_FAST 2 (a) 4 72 LOAD_FAST 0 (one) 74 LOAD_FAST 1 (two) 76 LOAD_FAST 2 (a) 78 LOAD_FAST 3 (b) 80 BUILD_TUPLE 4 82 RETURN_VALUE (4.3333333, 4.3333333, 12424698960, 12424698960)  The id of one and two are also stable over time: >>> id(one), id(two) (12424698960, 12424698960)  They are indeed the same object, because the interpreter optimizes the addition before the bytecode is generated.
Question: I have this data [[ 2.696000e+00 0.000000e+00 0.000000e+00 0.000000e+00] [ 2.577000e+00 0.000000e+00 0.000000e+00 0.000000e+00] [ 0.000000e+00 -2.560096e+03 0.000000e+00 0.000000e+00] ...  and want to print it as such, for each row 2.696000e+00 & 0.000000e+00 & 0.000000e+00 & 0.000000e+00 // 2.577000e+00 & 0.000000e+00 & 0.000000e+00 & 0.000000e+00 // ...  I have done is for k in range(1): for i in range(len(data[k])): print(data[k][i],'&') if i == len(data[k])-1: print(data[k][i],'//')  Which outputs 2.696 & 0.0 & 0.0 & 0.0 & 0.0 //  How to get rid of the new line between each entry of the row?

Answer: You can use str.join to join the numbers by & and then print // as last string: data = [ [2.696000e00, 0.000000e00, 0.000000e00, 0.000000e00], [2.577000e00, 0.000000e00, 0.000000e00, 0.000000e00], [0.000000e00, -2.560096e03, 0.000000e00, 0.000000e00], ] for row in data: print(' & '.join(map(str, row)), '//')  Prints: 2.696 & 0.0 & 0.0 & 0.0 // 2.577 & 0.0 & 0.0 & 0.0 // 0.0 & -2560.096 & 0.0 & 0.0 //
Question: My dataframe contains a multivariate time series per user id. The first column id is the user id (there are N users), the second dt is the date (each user has T days worth of data, i.,e T rows for each user) and the other columns are metrics (basically, each column is a time series per id.) Here's a code to recreate a similar dataframe import pandas as pd from datetime import datetime import numpy as np N=5 T=100 dfs=[] datelist = pd.date_range(datetime.today(), periods=T).tolist() for id in range(N): test = pd.DataFrame(np.random.randint(0,100,size=(100, 4)), columns=list('ABCD')) test['dt'] = datelist test['id']=id dfs.append(test) dfs = pd.concat(dfs)  The output would look something like this, where 'A','B' and so on are metrics (like total purchases):      A B C D dt id     0 58 3 52 5 2023-07-05 14:34:12.852460 0   1 6 34 28 88 2023-07-06 14:34:12.852460 0   2 27 98 74 81 2023-07-07 14:34:12.852460 0   3 96 13 7 52 2023-07-08 14:34:12.852460 0   4 80 69 22 12 2023-07-09 14:34:12.852460 0     I want to transform this data into a numpy matrix X, of shape N x T x F, where N is the number of users, T is the number of days in the time series (T is constant for all ids) and F is the number of metrics (in the example above, F=4.) This means that X[0] returns a TxF array, that should look exactly like the output of dfs.query('id==0')[['A','B','C','D']].values So far, I've tried using pivot and reshape but the elements in the final matrix are not arranged as I would like. Here's what I've tried: # Pivot the dataframe df_pivot = dfs.sort_values(['id','dt']).pivot(index='id', columns='dt') # Get the values from the pivot table X = df_pivot.values.reshape(dfs['id'].nunique(), -1, len([x for x in dfs.columns if x not in ['dt','id']]))  If I do X[0], the result I get it: [58, 6, 27, 96], [80, 65, 41, 39], [30, 26, 38, 13], [50, 60, 60, 73], ...  From which you can see that the result is not what I would want. This is what I need: [58, 3, 52, 5], [ 6, 34, 28, 88], [27, 98, 74, 81], [96, 13, 7, 52], ...  Any help appreciated!

Answer: Group the dataframe by id then for each unique id export the metrics as numpy array inside a list comprehension df = df.set_index(['id', 'dt']).sort_index() np.array([g.values for _, g in df.groupby(level=0)])   array([[[58, 3, 52, 5], [ 6, 34, 28, 88], [27, 98, 74, 81], [96, 13, 7, 52], [80, 69, 22, 12]]])
Question: I have a unit test I'm running with a particular fixture. The fixture is pretty simple and looks like so: @pytest.fixture def patch_s3(): # Create empty bucket bucket = s3_resource().create_bucket(Bucket=BUCKET) bucket.objects.all().delete() yield bucket.objects.all().delete()  and the test isn't all that complicated either. Here is the signature for the test: def test_task_asg(patch_s3, tmp_path, monkeypatch)  However, when I run pytest, I get the following error: tests/test_task_asg.py:313: in <module> @pytest.fixture home/ubuntu/miniconda/envs/pipeline/lib/python3.7/site-packages/_pytest/fixtures.py:1150: in fixture fixture_function home/ubuntu/miniconda/envs/pipeline/lib/python3.7/site-packages/_pytest/fixtures.py:1011: in __call__ "fixture is being applied more than once to the same function" E ValueError: fixture is being applied more than once to the same function  This is weird to me, as it seems to imply I'm somehow including patch_s3 more than once in test_task_asg. What could be causing this?

Answer: If you scroll up from your definition of patch_s3, you'll probably find another duplicate line like @pytest.fixture, causing the issue. That line can only be used once per fixture method definition. The easiest way to reproduce this issue is with: import pytest @pytest.fixture(scope="session") @pytest.fixture def my_fixture(): yield def test_abc(my_fixture): pass  Running that with pytest yields ValueError: fixture is being applied more than once to the same function. But if you remove one of the two @pytest.fixture lines, then the issue goes away. Here are some examples where other people had the same issue and fixed it:  https://github.com/pytest-dev/pytest/issues/3518 https://github.com/simonw/datasette/commit/969771770fcf795daace72e2310804e699067cfe https://github.com/RhodiumGroup/rhg_compute_tools/pull/67
Question: I have a basic chain that classifies some text based on the Common European Framework of Reference for Languages. I'm timing the difference between normal chain.apply and chain.aapply but can't get it to work. What am I doing wrong? import os from time import time import openai from dotenv import load_dotenv, find_dotenv from langchain.chains import LLMChain from langchain.chat_models import ChatOpenAI from langchain.prompts import ChatPromptTemplate _ = load_dotenv(find_dotenv()) openai.api_key = os.getenv('OPENAI_API_KEY') llm = ChatOpenAI(temperature=0) prompt = ChatPromptTemplate.from_template( 'Classify the text based on the Common European Framework of Reference ' 'for Languages (CEFR). Give a single value: {text}', ) chain = LLMChain(llm=llm, prompt=prompt) texts = [ {'text': 'Hallo, ich bin 25 Jahre alt.'}, {'text': 'Wie geht es dir?'}, {'text': 'In meiner Freizeit, spiele ich gerne Fussball.'} ] start = time() res_a = chain.apply(texts) print(res_a) print(f"apply time taken: {time() - start:.2f} seconds") print() start = time() res_aa = chain.aapply(texts) print(res_aa) print(f"aapply time taken: {time() - start:.2f} seconds")  Output [{'text': 'Based on the given text "Hallo, ich bin 25 Jahre alt," it can be classified as CEFR level A1.'}, {'text': 'A2'}, {'text': 'A2'}] apply time taken: 2.24 seconds <coroutine object LLMChain.aapply at 0x0000025EA95BE3B0> aapply time taken: 0.00 seconds C:\Users\User\AppData\Local\Temp\ipykernel_13620\1566967258.py:34: RuntimeWarning: coroutine 'LLMChain.aapply' was never awaited res_aa = chain.aapply(texts) RuntimeWarning: Enable tracemalloc to get the object allocation traceback 

Answer: Changing res_aa = chain.aapply(texts)  to res_aa = await chain.aapply(texts)  did the job! Now it works (damn these methods are much faster than doing it sequentially) [{'text': 'Based on the given text "Hallo, ich bin 25 Jahre alt," it can be classified as CEFR level A1.'}, {'text': 'A2'}, {'text': 'A2'}] apply time taken: 2.87 seconds [{'text': 'Based on the given text "Hallo, ich bin 25 Jahre alt," it can be classified as CEFR level A1.'}, {'text': 'A2'}, {'text': 'A2'}] aapply time taken: 1.34 seconds
Question: If I have commands for special roles (e.g. /ban, /kick, /mute), every user can see them. It would be possible to display these commands only for those users who can execute them? You could check for permissions, roles, or id, but then people would still see these commands in the list

Answer: You can prevent users without the required permissions from seeing a command, by using the default_member_permissions parameter found here, when creating slash commands. @client.slash_command(name="ping", description="Ping the bot", default_member_permissions=(nextcord.Permissions(administrator=True))) async def ping(interaction: nextcord.Interaction): await interaction.send("Pong!", ephemeral=True)  You can allow and deny certain users and roles from seeing specific commands in the server settings.  To do so, go to Server Settings -> Integrations -> Bot/Application and click the command you want to configure roles and members for.
Question: I have a date type of the format 15-Jul-1996. And I am trying to define a Pydantic model to use for input data validation in a FastAPI application as such: from pydantic import BaseModel from datetime import date class Profile(BaseModel): name: str DOB: date # type of 15-Jul-1996 gender: str  Is there a way to constrain the DOB to that particular format? I can't seem to find a way to do that in the Pydantic docs. The attempt above is not quite what I want as it constrains to 05-07-2023.

Answer: Try this. You can use @validator from pydantic import BaseModel, validator from datetime import datetime class Profile(BaseModel): name: str DOB: str # change this to str gender: str @validator('DOB') def parse_dob(cls, v): return datetime.strptime(v, '%d-%b-%Y').date() # convert string to date
Question: I'm using the cachetools library and I would like to wrap the decorator form this library and add a class self argument to enable/disable the caching at the class level e.e. MyClass(enable_cache=True) An example usage would be something like: class MyClass(object): def __init__(self, enable_cache=True): self.enable_cache = enable_cache self.cache = cachetools.LRUCache(maxsize=10) @cachetools.cachedmethod(operator.attrgetter('cache')) def calc(self, n): return 1*n  I'm not sure how to keep the cache as a shared self class object and allow for the enable_cache flag within my own wrapper decorator using this library.

Answer: When you use cachetools the answer is actually quite simple - you set the cache to None. import cachetools import operator class MyClass(object): def __init__(self, enable_cache=True): self.cache = cachetools.LRUCache(maxsize=10) if enable_cache else None @cachetools.cachedmethod(operator.attrgetter('cache')) def calc(self, n): print("Calculating", n) return 1*n m1 = MyClass(True) m1.calc(2) # print m1.calc(2) # should not print m1.calc(3) # print m1.calc(3) # should not print print("now without") m2 = MyClass(False) m2.calc(2) # print m2.calc(2) # print m2.calc(3) # print m2.calc(3) # print  Output: Calculating 2 Calculating 3 now without Calculating 2 Calculating 2 Calculating 3 Calculating 3   More flexible you can do it do it by wrapping the cache or by making a whole new decorator: import cachetools import operator def flexible_cache(cache): def cache_wrapper(self): if self.enable_cache: return cache(self) return None return cache_wrapper def optional_cache(cache, *args, **kwargs): return cachetools.cachedmethod(flexible_cache(cache), *args, **kwargs) class MyClass(object): def __init__(self, enable_cache=True): self.enable_cache = enable_cache self.cache = cachetools.LRUCache(maxsize=10) # Now the None part is handled by the decorators @cachetools.cachedmethod(flexible_cache(operator.attrgetter('cache'))) def calc2(self, n): print("Calculating2", 2*n) return 2*n @optional_cache(operator.attrgetter('cache')) def calc3(self, n): print("Calculating3", 2*n) return 2*n
Question: The polars dataframe has a top_k method that can be used to select rows which contain the k largest values when sorting on a column. For example, the following code selects the two rows with the largest and second largest entry in the val column: df = pl.DataFrame({'grp':['a','a','a','b','b','b'], 'val':[1,2,3,10,20,30], 'etc':[0,1,2,3,4,5]}) grp val etc str i64 i64 "a" 1 0 "a" 2 1 "a" 3 2 "b" 10 3 "b" 20 4 "b" 30 5 df.top_k(2, by='val') grp val etc str i64 i64 "b" 30 5 "b" 20 4  My question is: how do I get the rows with top k values for each group? Specifically, I want the entire row and not just the value in the val column. I want to do something like this, but this doesn't work in polars because polars GroupBy doesn't have a top_k method: df.groupby('grp').top_k(2, by='val') # doesnt work in polars grp val etc str i64 i64 "b" 30 5 "b" 20 4 "a" 3 2 "a" 2 1  I was able to come up with two ways: one using map_groups and another using sorting. Both of these are not desirable for performance reasons. map_groups is generally not recommended because it's almost always significantly slower. The sorting option is also not desirable as getting the top k elements uses a faster algorithm than sorting (for small k and large n, it's basically O(n) vs O(n log n)). So even though the following below work, I'm looking for other approaches. Is there any way to directly use a top_k method with polars groupby? That would be my ideal solution. # works, but at expense of using map_groups method df.group_by('grp').map_groups(lambda df: df.top_k(2, by='val')) grp val etc str i64 i64 "b" 30 5 "b" 20 4 "a" 3 2 "a" 2 1  # works, but at expense of sorting entire groups df.group_by('grp').agg(pl.all().sort_by('val', descending=True).head(2)).explode('val','etc') grp val etc str i64 i64 "a" 3 2 "a" 2 1 "b" 30 5 "b" 20 4   df.group_by('grp').top_k(2, by='val'), which doesn't work in polars df.group_by('grp').map_groups(lambda df: df.top_k(2, by='val')), which works at the cost of using map_groups df.group_by('grp').agg(pl.all().sort_by('val', descending=True).head(2)).explode('val','etc'), which works at the cost of sorting 

Answer: The latest release (version 0.20.24) of polars introduced pl.Expr.top_k_by (and also pl.Expr.bottom_k_by) with optimised runtime complexity O(n + k log n - k / 2) for precisely the use-case mentioned in the question. It can be used jointly with pl.Expr.over and mapping_strategy="explode" to obtain the desired result. df.select( pl.all().top_k_by("val", k=2).over("grp", mapping_strategy="explode") )  shape: (4, 3) ┌─────┬─────┬─────┐ │ grp ┆ val ┆ etc │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╡ │ a ┆ 3 ┆ 2 │ │ a ┆ 2 ┆ 1 │ │ b ┆ 30 ┆ 5 │ │ b ┆ 20 ┆ 4 │ └─────┴─────┴─────┘  Note. The call to pl.Expr.over with mapping_strategy="explode" is equivalent to the following aggregation. df.group_by("grp").agg(pl.all().top_k_by("val", k=2)).explode(pl.exclude("grp"))
Question: I tried to run the following code to generate a word cloud. text = 'word1 word2 word2 word3 word3 word3' from wordcloud import WordCloud wordcloud = WordCloud(width=480, height=480).generate(text)  But I ran into this error: --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) <ipython-input-17-659d2fbc8555> in <module> 8 9 # Generate the word cloud ---> 10 wordcloud.generate(text) 11 12 # Display the word cloud ~\anaconda3\lib\site-packages\wordcloud\wordcloud.py in generate(self, text) 637 self 638 """ --> 639 return self.generate_from_text(text) 640 641 def _check_generated(self): ~\anaconda3\lib\site-packages\wordcloud\wordcloud.py in generate_from_text(self, text) 619 """ 620 words = self.process_text(text) --> 621 self.generate_from_frequencies(words) 622 return self 623 ~\anaconda3\lib\site-packages\wordcloud\wordcloud.py in generate_from_frequencies(self, frequencies, max_font_size) 451 font_size = self.height 452 else: --> 453 self.generate_from_frequencies(dict(frequencies[:2]), 454 max_font_size=self.height) 455 # find font sizes ~\anaconda3\lib\site-packages\wordcloud\wordcloud.py in generate_from_frequencies(self, frequencies, max_font_size) 506 font, orientation=orientation) 507 # get size of resulting text --> 508 box_size = draw.textbbox((0, 0), word, font=transposed_font, anchor="lt") 509 # find possible places using integral image: 510 result = occupancy.sample_position(box_size[3] + self.margin, ~\anaconda3\lib\site-packages\PIL\ImageDraw.py in textbbox(self, xy, text, font, anchor, spacing, align, direction, features, language, stroke_width, embedded_color) 565 font = self.getfont() 566 mode = "RGBA" if embedded_color else self.fontmode --> 567 bbox = font.getbbox( 568 text, mode, direction, features, language, stroke_width, anchor 569 ) AttributeError: 'TransposedFont' object has no attribute 'getbbox'  This is the full error trace I got when I run the code in my jupyter notebook. What is this error and how to resolve this? I can't understand what is that TransposedFont when I haven't used any argument of that name in the WordCloud function.

Answer: The solution was to upgrade pip and pillow, as based on an issue posted at the word_cloud GitHub repo. See the comments below the post above for how it solved the error featured here.
Question:  I am trying to have images in my Tkinter GUI, hence I am using PIL. Image.ANTIALAIS is not working. However, Image.BILINEAR works  Here's some sample code: import tkinter as tk from PIL import Image, ImageTk window = tk.Tk() image = Image.open(r"VC.png") image = image.resize((20, 20), Image.ANTIALIAS) tk_image = ImageTk.PhotoImage(image) image_label = tk.Label(window, image=tk_image) image_label.pack() window.mainloop()  Here's the error: Traceback (most recent call last): File "<module1>", line 19, in <module> AttributeError: module 'PIL.Image' has no attribute 'ANTIALIAS'   I tried reinstalling pip and Pillow. It didn't work. I asked ChatGPT about this, and it advised me to upgrade to Pillow's latest version. I am on the latest version (10.0.0). 

Answer: The problem is with Pillow 10.0. Trying to uninstall Pillow might give some errors. Just put this in cmd: pip install Pillow==9.5.0
Question: I want to train the YOLO v8 in transfer learning on my custom dataset. I have different classes than the base training on the COCO dataset. Yet I don't want to learn again the feature extraction. Hence I though following the Ultralytics YOLOv8 Docs - Train. Yet, When I train on my small dataset I want to freeze the backbone. How can I do that? I looked at the documentation and couldn't find how to do so.

Answer: You can do the following def freeze_layer(trainer): model = trainer.model num_freeze = 10 print(f"Freezing {num_freeze} layers") freeze = [f'model.{x}.' for x in range(num_freeze)] # layers to freeze for k, v in model.named_parameters(): v.requires_grad = True # train all layers if any(x in k for x in freeze): print(f'freezing {k}') v.requires_grad = False print(f"{num_freeze} layers are freezed.")  Then add this function as a custom callback function to the model model = YOLO("yolov8x.pt") model.add_callback("on_train_start", freeze_layer) model.train(data="./dataset.yaml")  Original answer is provided in one of the issues in ultralytics repo Freezing layers yolov8 #793
Question: I'm using reactable, Datatable or rhandsontable in R Shiny to display an interactive table and as a user input for selecting rows. Given there are a few packages for doing this in R, I thought there would be even more libraries in python for selecting rows from an interactive table - however I haven't found one yet. Please let me know if one exists.

Answer: I also love the way Datatable works for shiny R, and have been searching for an equivalent in shiny Python. The latest release of shiny Python (0.4) has a new feature "data grid / data table" that looks promising ! Here are the links to the release announcement and the API documentation: https://shiny.posit.co/blog/posts/shiny-python-0.4.0/ https://shiny.posit.co/py/api/render.DataTable.html They show an example of getting the selected rows in a callback (as you asked) Hope this helps
Question: How do I make a Categorical column which has:  elements: ['a', 'b', 'a', 'a'] categories ['a', 'b', 'c']  in polars? In pandas, I would do: In [31]: pd.Series(pd.Categorical(['a', 'b', 'a', 'a'], categories=['a', 'b', 'c'])) Out[31]: 0 a 1 b 2 a 3 a dtype: category Categories (3, object): ['a', 'b', 'c']  I have no idea how to do this in polars, the docs for Categorical look completely empty: https://pola-rs.github.io/polars/py-polars/html/reference/api/polars.Categorical.html

Answer: You can use the StringCache with pl.StringCache(): pl.Series(['a', 'b', 'c'], dtype=pl.Categorical()) s = pl.Series(['a', 'b', 'a', 'a','z'], dtype=pl.Categorical())  Everything in the StringCache context will share the same index/value mapping so the first line initialized the mapping with the categories you want. The second line is the Series you want to keep. I added an extra 'z' so that we can see: s.to_physical() shape: (5,) Series: '' [u32] [ 0 1 0 0 3 ]  Note that the s series skips 2 as it doesn't have a c value in it.