eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have a Pandas dataframe: import pandas as pd import numpy as np np.random.seed(150) df = pd.DataFrame(np.random.randint(0, 10, size=(10, 2)), columns=['A', 'B'])  I want to add a new column "C" whose values ​​are the combined-list of every three rows in column "B". So I use the following method to achieve my needs, but this method is slow when the data is large. >>> df['C'] = [df['B'].iloc[i-2:i+1].tolist() if i >= 2 else None for i in range(len(df))] >>> df A B C 0 4 9 None 1 0 2 None 2 4 5 [9, 2, 5] 3 7 9 [2, 5, 9] 4 8 3 [5, 9, 3] 5 8 1 [9, 3, 1] 6 1 4 [3, 1, 4] 7 4 1 [1, 4, 1] 8 1 9 [4, 1, 9] 9 3 7 [1, 9, 7]  When I try to use the df.apply function, I get an error message: df['C'] = df['B'].rolling(window=3).apply(lambda x: list(x), raw=False) TypeError: must be real number, not list  I remember that Pandas apply doesn't seem to return a list, so how do I do this? I searched the forum, but couldn't find a suitable topic about apply and return.

Answer: You can use numpy's sliding_window_view: from numpy.lib.stride_tricks import sliding_window_view as swv N = 3 df['C'] = pd.Series(swv(df['B'], N).tolist(), index=df.index[N-1:])  Output:  A B C 0 4 9 NaN 1 0 2 NaN 2 4 5 [9, 2, 5] 3 7 9 [2, 5, 9] 4 8 3 [5, 9, 3] 5 8 1 [9, 3, 1] 6 1 4 [3, 1, 4] 7 4 1 [1, 4, 1] 8 1 9 [4, 1, 9] 9 3 7 [1, 9, 7]
Question: Error: File "/app/.venv/lib/python3.11/site-packages/anthropic/_client.py", line 386, in __init__ super().__init__( File "/app/.venv/lib/python3.11/site-packages/anthropic/_base_client.py", line 1437, in __init__ self._client = http_client or AsyncHttpxClientWrapper(^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.11/site-packages/anthropic/_base_client.py", line 1334, in __init__ super().__init__(**kwargs) TypeError: Async`Client.__init__() got an unexpected keyword argument 'proxies'  I know that to fix this problem i need to downgrade httpx to another version, but what if im using fasthx = "2.0.1". Whats the solution for Antropic AI? My code: load_dotenv() log = logging.getLogger(__name__) anthropic_api_key = os.getenv("ANTHROPIC_API_KEY") anthropic_client = anthropic.Anthropic(api_key=anthropic_api_key) # <-- Eror happens here anthropic_client = instructor.from_anthropic(anthropic_client)  I want to fix the error TypeError: AsyncClient.init() got an unexpected keyword argument 'proxies'

Answer: Just upgrade anthropic to the latest version, the problem was fixed in version 0.45.2 and above.
Question: When the consumer (which is a very simple confluent-kafka-python consumer), we see this log message after the assignment  %6|1739802885.947|GETSUBSCRIPTIONS|<consumer id>#consumer-1| [thrd:main]: Telemetry client instance id changed from AAAAAAAAAAAAAAAAAAAAAA to <some random string>   I tried running the consumer locally (in contrast to the Kubernetes cluster) and see no such logs.  I tried googling for this log message but found no bugs or help avoiding this (though I am not the only person with such logs)  

Answer: As per my previous comment, from confluent_kafka import Consumer conf = { 'bootstrap.servers': 'your broker', 'group.id': 'your group', 'enable.metrics.push': False } consumer = Consumer(conf)
Question: trying to run a function with 'X' cores simultaneously that has multiple parameters. with PROCESS I cant set the number of cores, and with POOL I cant set multiple parameters for a function. def funct(a, b): #run .bat file with params (a,b) if __name__ == '__main__': cores = int(cores) #set with input pool = Pool(processes=cores) pool.map(funct(3, 44))  Im getting this error: TypeError: Pool.map() missing 1 required positional argument: 'iterable'  I cant use this, because I cant set Cores p = Process(target=funct, args=(1,2)) 

Answer: Note - Multiprocessing.py doesn't work in IDLE, you have to use software like intellij or something. def funct(a, b): #run .bat file with params (a,b) if __name__ == "__main__": # run as many cores as in the range processes = [] for _ in range(4): p = multiprocessing.Process(target=funct, args=[2, "hello"]) p.start() processes.append(p) for process in processes: process.join()
Question: Using music21, how can I transpose a pitch down an octave? If I use p.transpose(-12), flats get changed to sharps: import music21 p = music21.pitch.Pitch('D-4') print(p.transpose(-12))  output C#3 

Answer: Instead of -P12, use -P8, like this: import music21 p = music21.pitch.Pitch('D-4') print(p.transpose('-P8'))
Question: CPython implements arbitrary-precision integers as PyLongObject, which extends PyObject. On 64-bit platforms they take at least 28 bytes, which is quite memory intensive. Also well-known is that it keeps a cache of small integer objects from -5 to 256. I am interested in seeing how PyPy implements these, in particular what optimizations there are optimizations for limited size integer objects. It is difficult to find documentation online. The PyPy docs mention a tagged pointer optimization for "small ints" of 63 bits (signed?). The most obvious to me is treating an integer as a primitive instead of a general purpose object if possible.

Answer: The PyPy docs mention a tagged pointer optimization as something that you need to enable explicitly, and it's never enabled by default because it comes with a large performance cost. Instead, the story is:  There are two different internal representations, one for 64-bit signed integers and one for larger integers.  The common, small representation is a "PyObject"-like object, but only 2x8 bytes in total (including headers etc.). (The reason is that our production GC on 64-bit machines adds only a single 64-bit header, which packs a number to identify the RPython class and a number of GC flags; and then the W_IntObject class only adds one 64-bit field for the integer value.)  There is a separate optimization for "list of small integers", which is implemented as an array of 64-bit ints instead of an array of objects (so 8 bytes per item instead of 16-plus-8-for-the-pointer, plus increased locality).
Question: I'm trying to extract and save the 32x32 icon from a .ico file that contains multiple icons with multiple sizes and color depths using Pillow. The 32x32 icon is available in the following color depths: 32-bit, 8-bit and 4-bit. I tried opening the icon file using Image.open(), then set its size to 32x32, and then save it as a .png file. I expect to get the icon with the highest color depth possible, but I'm getting the one with the lowest color depth possible. Here's a minimal, reproducible example: from PIL import Image icon = Image.open("icon.ico") icon.size = (32, 32) icon.save("icon.png")  I'm expecting to get this:  But, I'm getting this:  You can get the .ico file here: https://www.mediafire.com/file/uls693wvjn3njqa/icon.ico/file I also tried looking for similar questions on the internet, but I didn't find anything. Is there a way I can tell Pillow to extract the 32x32 icon from a .ico file at the highest color depth possible without modifying my .ico file to exclude icons with lower color depths?

Answer: WHile an ICO image loaded in PIL provide access to the individual images through the img.ico.frame call, there is one problem: upon loading, PIL will convert all loaded individual frames which are indexed into the RGBA format. Fortunately, it will preserve a bpp attribute listing the original bitcount for each frame in a headers list stored in the img.ico.entry attribute. Which means a small function which iterates the frame headers can find the needed information: from PIL import Image def get_max_bpp(ico, size=(32,32)): frames = [(index, header, ico.ico.frame(index)) for index, header in enumerate(icon.ico.entry) if header.width==size[0] and header.height == size[1] ] frames.sort(key=lambda frame_info: frame_info[1].bpp, reverse=True) return frames[0][2] icon = Image.open("icon.ico") max_32 = get_max_bpp(icon) max_32.save("icon.png")  So, a straight look at the .mode attribute of each frame won't help - all will show up as RGBA (at least for an ICO file which contains at least one 24 or 32bit frame. I had not tested with a file with indexed-only frames).
Question: I have a class A with a property prop and a method method that uses this property: class A: @property def prop(self) -> int: return 1 def method(self) -> int: return self.prop * 2 # Uses self.prop  Then I have a wrapper class that tries to override this property like this: class B: def __init__(self, a: A): self._a = a @property def prop(self) -> int: return 2 # Override A's property def __getattr__(self, attr): return getattr(self._a, attr) # Delegate attribute access to A if attr not in B  When b.method() is called, I'd like the calculation to be done with B's prop, and get 4 as result. What is happening instead is that prop is resolving within A, and b.method() returns 2. b = B(a) b.method() # Returns 2. Wanted 4  Is there any way I can override the property prop in A, when used on A's code, when it is being wrapped by B?  I can't use inheritance for these classes I'd like to avoid replicating the method() logic in B I also tried setting __getattribute__ in B, but it seems it is not even called when resolving prop inside A  EDIT: I also want the property to be overridden for other properties too. So if I have other_prop in A as follows: class A: @property def prop(self) -> int: return 1 @property def other_prop(self) -> int: return self.prop * 4 # Uses self.prop def method(self) -> int: return self.prop * 2 # Uses self.prop  I'd like b.other_prop to return 8 Any way to achieve this?

Answer: You can test if the attribute obtained from _a is a method and re-bind the underlying function (accessible via the __func__ attribute) to the B instance so that the method can have access to B's properties: from inspect import ismethod class B: def __getattr__(self, attr): value = getattr(self._a, attr) if ismethod(value): value = value.__func__.__get__(self) return value  Demo: https://ideone.com/jLJma8 Note that it is unnecessary for __getattr__ to test if attr in self.__dict__: because __getattr__ is called only when an attribute name is not found in __dict__ in the first place. EDIT: Similarly, if you need properties of _a to be able to access the B instance, you can test if the attribute obtained from _a's type is a property and call the the property's getter function with the B instance instead. So together with the method re-binding logics above, B.__getattr__ shall become: class B: def __getattr__(self, attr): value = getattr(type(self._a), attr, None) if isinstance(value, property): return value.fget(self) value = getattr(self._a, attr) if ismethod(value): value = value.__func__.__get__(self) return value  Demo: https://ideone.com/uE3WHl
Question: I have a simple dataframe: data = [[2025, 198237, 77, 18175], [202, 292827, 77, 292827]]  I only want the 1st and 4th columns and I don't want header or index labels: df = pd.DataFrame(data).iloc[:,[0,3]] print(df.to_string(index=False, header=False))  Output is the following: 2025 18175 202 292827  How do I line up my first column in column 3 (left-justified) and line up my second column in column 10 (left-justified)? Since i'm calling the to_string method, which is converting the dataframe to a string representation, shouldn't I be able to use ljust? I'm not able to produce the desired output, which would be:  2025 18175 202 292827 

Answer: Define a formatting function with a single arg and apply to relevant columns import pandas as pd mw=7 def left_align(x): return f"{x: <{mw}}" data = [[2025, 198237, 77, 18175], [202, 292827, 77, 292827]] df = pd.DataFrame(data).iloc[:,[0,3]] # get length of max value mw = len(str(df.max(numeric_only=True).max())) #print(mw) print(df.to_string(index=False, header=False, formatters=[left_align, left_align]))  Result 2025 18175 202 292827
Question: I'm using something similar to this as input.txt header 040525 $$$$$ 9999 12345 random stuff 040525 $$$$$ 8888 12345 040525 $$$$$ 7777 12345 random stuff 040525 $$$$$ 6666 12345 footer  Due to the way this input is being pre-processed, I cannot correctly use pd.read_csv. I must first create a list from the input; Then, create a DataFrame from the list. data_list = [] with open('input.txt', 'r') as data: for line in data: data_list.append(line.strip().split()) df = pd.DataFrame(data_list)  I only want to append lines that contain '$$$' in the second column. Desired output would be:  0 1 2 3 0 40525 $$$$$ 9999 12345 1 40525 $$$$$ 8888 12345 2 40525 $$$$$ 7777 12345 3 40525 $$$$$ 6666 12345 

Answer: data_list = [] with open('input.txt', 'r') as data: for line in data: #create split_row to check split_row = line.strip().split() #check if the second substring in split_row starts with "*****" if split_row[1].startswith("*****"): data_list.append(split_row) df = pd.DataFrame(data_list)
Question: I am trying to use regular expressions to replace numeric ranges in text, such as "4-5", with the phrase "4 to 5". The text also contains dates such as "2024-12-26" that should not be replaced (should be left as is). The regular expression (\d+)(\-)(\d+) (attempt one below) is clearly wrong, because it falsely matches dates. Using a negative lookahead expression, I came up with the regex (?!\d+\-\d+\-)(\d+)(\-)(\d+) instead (attempt two below), which correctly matches "4-5" while rejecting "2024-12-26". However, attempt_two does not behave correctly in a re.subn() context, because although it rejects "2024-12-26", the search continues on to match (and replace) the substring "12-26": import re text = """ 2024-12-26 4-5 78-79 """ attempt_one = re.compile(r"(\d+)(\-)(\d+)") attempt_two = re.compile(r"(?!\d+\-\d+\-)(\d+)(\-)(\d+)") print("Attempt one:") print(re.match(attempt_one, "4-5")) # Match: OK print(re.match(attempt_one, "2024-12-26")) # Match: False positive new_text, _ = re.subn(attempt_one, r"\1 to \3", text) # Incorrect substitution print(new_text) print("Attempt two:") print(re.match(attempt_two, "4-5")) # Match: OK print(re.match(attempt_two, "2024-12-26")) # Doesn't match: OK new_text, _ = re.subn(attempt_two, r"\1 to \3", text) # Still incorrect print(new_text)  Output: Attempt one: <re.Match object; span=(0, 3), match='4-5'> <re.Match object; span=(0, 7), match='2024-12'> 2024 to 12-26 4 to 5 78 to 79 Attempt two: <re.Match object; span=(0, 3), match='4-5'> None 2024-12 to 26 4 to 5 78 to 79  What regular expression can I use so that the substitution returns the following instead? 2024-12-26 4 to 5 78 to 79  (As my goal is to learn about regular expressions, I am not interested in workarounds such as matching the whitespace or newline after "12-26".)

Answer: You need both a negative lookbehind and a negative lookahead, to prohibit an extra hyphen before or after the match. (?<![-\d])(\d+)-(\d+)(?![-\d])  The lookarounds also have to match digits, so it won't match part of the date, e.g. 024-1 from 2024-12-26.
Question: Consider the following code. I have a base and derived class, both dataclasses, and I want to call a method of the base class in the derived class via super(): import abc import dataclasses import typing SLOTS = False @dataclasses.dataclass(slots=SLOTS) class Base: @abc.abstractmethod def f(self, x: int) -> int: return x @dataclasses.dataclass(slots=SLOTS) class Derived(Base): @typing.override def f(self, x: int) -> int: return super().f(x) d = Derived() d.f(2)  When setting SLOTS = False, this runs fine. When setting SLOTS = True, I get an error: --------------------------------------------------------------------------- TypeError Traceback (most recent call last) Cell In[2], line 24 20 return super().f(x) 23 d = Derived() ---> 24 d.f(2) Cell In[2], line 20, in Derived.f(self, x) 18 @typing.override 19 def f(self, x: int) -> int: ---> 20 return super().f(x) TypeError: super(type, obj): obj (instance of Derived) is not an instance or subtype of type (Derived).  Why is that? Note that it works if I use a regular slotted class instead of a dataclass: import abc import typing class Base: __slots__ = tuple() @abc.abstractmethod def f(self, x: int) -> int: return x class Derived(Base): __slots__ = tuple() @typing.override def f(self, x: int) -> int: return super().f(x) d = Derived() d.f(2) 

Answer: WHen one uses the slots=True option on dataclasses, it will create a new class using the namespace of the decorated class - while not passing it makes it modify the class "in place". (This is needed because using slots really change how the class is built - its "layout" as it is called) The simplest thing to do is not to use __slots__ altogether - the gains slots gives one in modern Python are small since some of the optimizations that went on Python 3.11 (check it here: Are Python 3.11 objects as light as slots? ) Anyway, the workaround is quite simple - instead of the parameterless version of super(), which uses a mechanism introduced in Python 3.0, just use the explicit version of it, where you pass the class and instance as arguments. The thing is that the parameterless version uses implicitly the class body where the super() call is actually written in: at compile time, the current class is frozen into the method, in a non-local like variable named __class__. The new class created by dataclass with slots can't (or simply does not do) update this value - so parameterless super will try to call super on the original (pre dataclass decorator ) Derived class and will fail. This version of the code will work:  @dataclasses.dataclass(slots=SLOTS) class Derived(Base): @typing.override def f(self, x: int) -> int: return super(Derived, self).f(x)
Question: I'm using something similar to this as input.txt  040525 $$$$$ 9999 12345 040525 $$$$$ 8888 12345 040525 $$$$$ 7777 12345 040525 $$$$$ 6666 12345  Due to the way this input is being pre-processed, I cannot correctly use pd.read_csv. I must first create a list from the input; Then, create a DataFrame from the list. data_list = [] with open('input.txt', 'r') as data: for line in data: data_list.append(line.strip()) df = pd.DataFrame(data_list)  This results in each row being considered 1 column print(df.shape) print(df) print(df.columns.tolist()) (4, 1) 0 0 040525 $$$$$ 9999 12345 1 040525 $$$$$ 8888 12345 2 040525 $$$$$ 7777 12345 3 040525 $$$$$ 6666 12345 [0]  How can I create 4 columns in this DataFrame? Desired output would be: (4, 4) a b c d 0 40525 $$$$$ 9999 12345 1 40525 $$$$$ 8888 12345 2 40525 $$$$$ 7777 12345 3 40525 $$$$$ 6666 12345 ['a', 'b', 'c', 'd'] 

Answer: In your loop, you should split the strings into a list of substrings for the fields: for line in input_txt: data_list.append(line.strip().split())  This will give you the correct number of columns. Alternatively, keep your loop as it is, but create a Series and str.split with expand=True. This might be less efficient, but could be more robust if you don't have a consistent number of fields: data_list = [] with open('input.txt', 'r') as data: for line in data: data_list.append(line.strip()) df = pd.Series(data_list).str.split(expand=True)  Output:  0 1 2 3 0 040525 $$$$$ 9999 12345 1 040525 $$$$$ 8888 12345 2 040525 $$$$$ 7777 12345 3 040525 $$$$$ 6666 12345  For the first approach, if you want column names: df = pd.DataFrame(data_list, columns=['a', 'b', 'c', 'd'])  Output:  a b c d 0 040525 $$$$$ 9999 12345 1 040525 $$$$$ 8888 12345 2 040525 $$$$$ 7777 12345 3 040525 $$$$$ 6666 12345
Question: I'm developing an indicator in Indie. I’m trying to calculate Bollinger Bands and Keltner Channels, but I’m running into an error when adding a Series[float] to a float value. Here’s the error message:  Error: 20:20 cannot apply operator + to operands of types: <class 'indie.Series[float]'> and <class 'float'>  Here’s the relevant code snippet: sDev = StdDev.new(self.close, length) mid_line_bb = Sma.new(self.close, length) lower_band_bb = mid_line_bb + num_dev_dn * sDev[0] # <-- ERROR HERE upper_band_bb = mid_line_bb + num_dev_up * sDev[0] # <-- ERROR HERE  What I’ve Tried:  Wrapping sDev[0] with MutSeriesF.new(), but that caused another issue. Using sDev.value_or(0) (which doesn’t seem to exist in Indie v4). Looking for an explicit type conversion function in Indie’s documentation.  I assume the issue is that mid_line_bb is a Series[float], while sDev[0] is a single float. How should I properly handle this type of mismatch in Indie?

Answer: The problem is that you cannot add a Series container with a float number, you must extract a specific value from the Series and perform an arithmetic operation with it. You can extract the value from the Series using square brackets mid_line_bb[0] or using the get method mid_line_bb.get(offset=0, default=0). The source code of the BB algorithm on Indie may also help you: # indie:lang_version = 5 from indie import algorithm, SeriesF, MutSeriesF from indie.algorithms import Sma, StdDev @algorithm def Bb(self, src: SeriesF, length: int, mult: float) -> tuple[SeriesF, SeriesF, SeriesF]: '''Bollinger Bands''' middle = Sma.new(src, length)[0] dev = mult * StdDev.new(src, length)[0] lower = middle - dev upper = middle + dev return MutSeriesF.new(lower), MutSeriesF.new(middle), MutSeriesF.new(upper)  If we adapt this to your example, we will get the following indicator code: # indie:lang_version = 5 from indie import indicator, param from indie.algorithms import StdDev, Sma @indicator('Std dev', overlay_main_pane=True) @param.int('length', default=5) @param.float('num_dev_dn', default=1.0) @param.float('num_dev_up', default=1.0) def Main(self, length, num_dev_dn, num_dev_up): sDev = StdDev.new(self.close, length) mid_line_bb = Sma.new(self.close, length) lower_band_bb = mid_line_bb[0] + num_dev_dn * sDev[0] upper_band_bb = mid_line_bb[0] + num_dev_up * sDev[0] return mid_line_bb[0], lower_band_bb, upper_band_bb  This code on Indie v4 would have looked the same, switching to v5 did not affect working with Series type.
Question: I need to create a pdf with header, footer and background color. Tge following code is generating all 3, but it seems the footer is getting behind the pdf rect from fpdf import FPDF class PDF(FPDF): def header(self): self.set_font(family='Helvetica', size=8) self.cell(0, 10, 'test_header', align='L') def footer(self): self.set_y(-15) self.set_font(family='Helvetica', size=8) self.cell(0, 10, 'test_footer', align='L') pdf = PDF() pdf.add_page() pdf.set_font("Times", size=12) # BG pdf.set_fill_color(r=249, g=247, b=242) pdf.rect(h=pdf.h, w=pdf.w, x=0, y=0, style="F")  With the above only footer is visible but without it, both are visible. How can I achieve the desired outcome?

Answer: It looks like the issue is that you're setting the background colour after you draw the page. The way you're doing it paints the background colour over everything, like what would happen if you painted a room without taking the posters off the wall. From a quick google search, FPDF doesn't have a method for modifying the background colour, so it might be best to squeeze the method into a component that will be going on all pages (in this case, i'll do it with the header). from fpdf import FPDF class PDF(FPDF): def header(self): # drawing the background self.set_fill_color(249, 247, 242) self.rect(0, 0, self.w, self.h, style="F") # then drawing the header self.set_font("Helvetica", size=8) self.cell(0, 10, "test_header", align="L") def footer(self): self.set_y(-15) self.set_font("Helvetica", size=8) self.cell(0, 10, "test_footer", align="L") pdf = PDF() pdf.add_page() pdf.set_font("Times", size=12)  I know it's a rudimentary fix, but if you're using python to create a PDF then I'm assuming this isn't meant for production-level code, and this would be a strong enough bandaid fix.