eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have class Book(models.Model): title = models.CharField(max_length=32) class Table(models.Model): book = models.ForeignKey(Book, related_name='tables') class TableEntry(models.Model): table = models.ForeignKey(Table, related_name='entries') value = models.FloatField()  and want a queryset of books that has annotated for each book, the sum of all table entries of this book. I have tried Book.objects.all().annotate(sum=Sum('tables__entries'))  but this does not seem to work. [Edit: Seems to work, after all, as expected.] Note that the idea works when the sum of all entries of each table should be annotated: Table.objects.all().annotate(sum=Sum('entries')) 

Answer: You can annotate the prefetched Tables, like: from django.db.models import Prefetch, Sum Book.objects.prefetch_related( Prefetch('tables', Table.objects.annotate(sum=Sum('entries__value'))) )  If you then access a Book instance (named book for example), the book.tables.all() is a QuerySet of Tables with each an extra sum attribute that is the sum of value of the entries of the Table.
Question: I want to change the column labels of a Polars DataFrame from ['#a', '#b', '#c', '#d'] to ['a', 'b', 'c', 'd']

Answer: The following approach is most effective when renaming all columns that follow a consistent pattern, such as a common prefix or suffix. df.columns = [col.lstrip("#") for col in df.columns]  Alternatively, the approach outlined below is most suitable when renaming specific columns or when no consistent pattern is present: df = df.rename({ "#a": "a", "#b": "b", "#c": "c", "#d": "d" })  Output: ┌─────┬─────┬─────┬─────┐ │ a ┆ b ┆ c ┆ d │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╪═════╡ │ 1 ┆ 3 ┆ 5 ┆ 7 │ │ 2 ┆ 4 ┆ 6 ┆ 8 │ └─────┴─────┴─────┴─────┘
Question: I have a Polars DataFrame below: import polars as pl df = pl.DataFrame({"a":[1, 2, 3], "b":[4, 3, 2]}) >>> df a b i64 i64 1 4 2 3 3 2  I can subset based on a specific value: x = df[df["a"] == 3] >>> x a b i64 i64 3 2  But how can I subset based on a list of values? - something like this: list_of_values = [1, 3] y = df[df["a"] in list_of_values]  To get:  a b i64 i64 1 4 3 2 

Answer: y = df.filter(pl.col("a").is_in(list_of_values))  Output: ┌─────┬─────┐ │ a ┆ b │ │ --- ┆ --- │ │ i64 ┆ i64 │ ╞═════╪═════╡ │ 1 ┆ 4 │ │ 3 ┆ 2 │ └─────┴─────┘
Question: from selenium import webdriver from selenium.webdriver.chrome.service import Service from selenium.webdriver.chrome.options import Options import json import time service = Service(executable_path=f"chromedriver.exe", log_path=f"seleniumlog.txt") selenium_options = Options() selenium_options.add_argument("--disable-blink-features=AutomationControlled") selenium_options.add_argument("--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36") selenium_options.add_experimental_option("excludeSwitches", ["enable-automation"]) selenium_options.add_experimental_option('useAutomationExtension', False) chrome_driver = webdriver.Chrome(service=service, options=selenium_options) chrome_driver.get("https://www.tiktok.com/login") time.sleep(5) cookie = { 'domain': '.tiktok.com', 'expiry': 1734578412, 'httpOnly': True, 'name': 'sessionid', 'path': '/', 'secure': True, 'value': 'abc123' } # Step 2: Load cookies with open("flamencocookies.json", "r") as f: cookies = json.load(f) try: chrome_driver.add_cookie(cookie) except Exception as e: print("Could not add cookie") print(chrome_driver.get_cookies())  I'm just testing Selenium and chose TikTok.com to test it on. But in the above code, I'm unable to add the cookie to the browser session. The cookie format seems correct, and I'm not getting an exception, but it's also not added to the browser session. Does anyone know why? Could it be something to do with the switches I added to the browser? Read selenium docs, everything seems correct

Answer: Your cookie won't add as it is expired. Update the time and it adds in fine:  I used this: cookie = { 'domain': '.tiktok.com', 'expiry': 1744835794, # <-- updated here 'httpOnly': True, 'name': 'sessionid', 'path': '/', 'secure': True, 'value': 'abc123' }  Cookies work on an epoch time. Which is number seconds since 1st Jan 1970. There are lots of handy converters available online. Your expiry time was set to: Thursday, 19 December 2024 03:20:12
Question: I have the following 3d-numpy.ndarray: import numpy as np X = np.array([ [[0.0, 0.4, 0.6, 0.0, 0.0], [0.6, 0.0, 0.0, 0.0, 0.0], [0.4, 0.0, 0.0, 0.0, 0.0], [0.0, 0.6, 0.0, 1.0, 0.0], [0.0, 0.0, 0.4, 0.0, 1.0]], [[0.1, 0.5, 0.4, 0.0, 0.0], [0.6, 0.0, 0.0, 0.0, 0.0], [0.2, 0.0, 0.0, 0.0, 0.0], [0.1, 0.6, 0.0, 1.0, 0.0], [0.0, 0.0, 0.4, 0.0, 1.0]] ])  I want a new array where all the rows and columns are dropped where the diagnonal is equal to 1. idx = np.diag(X[0]) == 1 # for my implementation it is sufficient to look at X[0]  Important to note is that X.shape[1] == X.shape[2], so I try to use the mask as follows Y = X[:, ~idx, ~idx]  The above returns something different than my desired output:   [[0.0, 0.4, 0.6], [0.6, 0.0, 0.0], [0.4, 0.0, 0.0]], [[0.1, 0.5, 0.4], [0.6, 0.0, 0.0], [0.2, 0.0, 0.0]]   Please advice

Answer: Probably you can try X[:, ~idx,:][:,:, ~idx]  or np.ix_ X[:, *np.ix_(~idx, ~idx)]  which gives array([[[0. , 0.4, 0.6], [0.6, 0. , 0. ], [0.4, 0. , 0. ]], [[0.1, 0.5, 0.4], [0.6, 0. , 0. ], [0.2, 0. , 0. ]]])
Question: I'm attempting to use real constants in Python—values which cannot be modified once defined. I realize Python has no native support for constants like certain other languages (e.g., final in Java or const in C++), but I want the most Pythonic or safest manner of achieving this behavior.  class Constants: PI = 3.14159 GRAVITY = 9.8 Constants.PI = 3 # This still allows reassignment  I want something that raises an error or prevents the value from being changed. I’ve also heard of using custom classes or metaclasses, but I’m not sure what the best practice is. What’s the best way to implement unchangeable constants in Python?

Answer: You're correct—Python natively supports neither true constants like C, C++, nor Java. There is no const keyword, and even the hack of creating variables in all caps (PI = 3.14) is completely dependent on developer discipline. Although workarounds such as metaclasses or special classes can be used to mimic immutability, none are able to prevent reassignment at the module level, or prevent an astute user from simply overwriting your values. Therefore, I created something that does. Introducing setconstant: Immutability in Python Constants Python is not immutable, and that can have unintended value mutations—particularly in big codebases. So I made setconstant, a light Python package allowing you to define true constants that cannot be mutated after declaration. How It Works import setconstant setconstant.const_i("PI", 3) setconstant.const_f("GRAVITY", 9.81) setconstant.const_s("APP_NAME", "CodeHaven") print(setconstant.get_constant("PI")) # Output: 3 print(setconstant.get_constant("GRAVITY")) # Output: 9.81 print(setconstant.get_constant("APP_NAME")) # Output: CodeHaven # Try to change the value  Once declared, constants are locked. Any attempt to overwrite them throws a ValueError, protecting your code from bugs or unintended logic changes. Why Not Just Use a Class or Metaclass? class Constants: PI = 3.14 def __setattr__(self, name, value): raise AttributeError("Cannot reassign constants")  Or even using metaclasses like: class ConstantMeta(type): def __setattr__(cls, name, value): if name in cls.__dict__: raise AttributeError(f"Cannot modify '{name}'") super().__setattr__(name, value)  But these solutions have their limitations: They can be circumvented with sufficient effort. They are not intuitive to beginners. They need boilerplate or more advanced Python functionality such as metaclasses. setconstant circumvents that. It's simple, strict, and does what constants are supposed to do. Installation pip install setconstant  Feedback is always welcome! Anuraj R, Creator of setconstant
Question: Without python3.13 free-threading, cv2 importing numpy is fine. But when python3.13 free-threading is turned on, when cv2 tries to import numpy, numpy gives this error: ImportError: Error importing numpy: you should not try to import numpy from its source directory; please exit the numpy source tree, and relaunch your python interpreter from there.  Does this mean cv2 and python3.13 free-threading are not compatible? I'm relatively new to cv2, python3.13 and threading :)

Answer:  Starting with the 3.13 release, CPython has experimental support for a build of Python called free threading where the global interpreter lock (GIL) is disabled. [...] The free-threaded mode is experimental and work is ongoing to improve it: expect some bugs and a substantial single-threaded performance hit.  (From: https://docs.python.org/3/howto/free-threading-python.html. Emphasis mine.) I think that counts as "some bugs". You were right to try this. That's why they're offering this mode now, for users to test with all the things they care about. You could file issues about this with the Python, OpenCV, and Numpy projects. Then they will know to investigate and work on this. CPython (the canonical C Python implementation) tracks issues there: https://github.com/python/cpython/issues?q=is%3Aissue%20state%3Aopen%20label%3Atopic-free-threading It looks like pandas also has trouble importing numpy: https://github.com/python/cpython/issues/120653
Question: I have data that looks like: lf = pl.LazyFrame( { "points": [ [ [1.0, 2.0], ], [ [3.0, 4.0], [5.0, 6.0], ], [ [7.0, 8.0], [9.0, 10.0], [11.0, 12.0], ], ], "other": ["foo", "bar", "baz"], }, schema={ "points": pl.List(pl.Array(pl.Float32, 2)), "other": pl.String, }, )  And I want to make all lists have the same number of elements. If it currently has more than I need, it should truncate. If it has less than I need, it should repeat itself in order until it has enough. I managed to get it working, but I feel I am jumping through hoops. Is there a cleaner way of doing this? Maybe with gather? target_length = 3 result = ( lf.with_columns( needed=pl.lit(target_length).truediv(pl.col("points").list.len()).ceil() ) .with_columns( pl.col("points") .repeat_by("needed") .list.eval(pl.element().explode()) .list.head(target_length) ) .drop("needed") )  EDIT The method above works for toy examples, but when I try to use it in my real dataset, it fails with: pyo3_runtime.PanicException: Polars' maximum length reached. Consider installing 'polars-u64-idx'.  I haven't been able to make a MRE for this, but my data has 4 million rows, and the "points" list on each row has between 1 and 8000 elements (and I'm trying to pad/truncate to 800 elements). These all seem pretty small, I don't see how a maximum u32 length is reached. I appreciate any alternative approaches I can try. The closest I have (which doesn't panic) is: But this doesn't pad repeating the list in order. It just pads repeating the last element. target_length = 3 result = ( lf.with_columns( pl.col("points") .list.gather( pl.int_range(target_length), null_on_oob=True, ) .list.eval(pl.element().forward_fill()) ) .drop("needed") ) 

Answer: The repr defaults for lists are quite small, so we will increase them for the example. pl.Config(fmt_table_cell_list_len=8, fmt_str_lengths=120)  If you use pl.int_ranges() (plural) and modulo arithmetic, you can generate the indices. target_length = 5 lf.select(pl.int_ranges(target_length) % pl.col("points").list.len()).collect()  shape: (3, 1) ┌─────────────────┐ │ literal │ │ --- │ │ list[i64] │ ╞═════════════════╡ │ [0, 0, 0, 0, 0] │ │ [0, 1, 0, 1, 0] │ │ [0, 1, 2, 0, 1] │ └─────────────────┘  Which you can pass to .list.gather() lf.with_columns( pl.col("points").list.gather( pl.int_ranges(target_length) % pl.col("points").list.len() ) ).collect()  shape: (3, 2) ┌──────────────────────────────────────────────────────────────────┬───────┐ │ points ┆ other │ │ --- ┆ --- │ │ list[array[f32, 2]] ┆ str │ ╞══════════════════════════════════════════════════════════════════╪═══════╡ │ [[1.0, 2.0], [1.0, 2.0], [1.0, 2.0], [1.0, 2.0], [1.0, 2.0]] ┆ foo │ │ [[3.0, 4.0], [5.0, 6.0], [3.0, 4.0], [5.0, 6.0], [3.0, 4.0]] ┆ bar │ │ [[7.0, 8.0], [9.0, 10.0], [11.0, 12.0], [7.0, 8.0], [9.0, 10.0]] ┆ baz │ └──────────────────────────────────────────────────────────────────┴───────┘
Question: I am working with an undirected graph G where the maximum degree is bounded by a constant d. My goal is to check whether G contains a complete bipartite subgraph K_{i,j} as a subgraph, for small values of i, j (specifically, i, j < 8). I currently use the following brute-force approach to detect a K_{i,j} subgraph: nodes = [u for u in self.g if len(self.g[u]) >= j] set_combinations = combinations(nodes, i) for A in set_combinations: common_neighbors = set(self.g) for u in A: common_neighbors &= self.g[u] if len(common_neighbors) >= j: return False # K_ij found return True  This method works correctly but becomes slow as the graph size increases. The main bottleneck seems to be:  Enumerating all subsets of i nodes (O(n^i) complexity). Intersecting neighbor sets iteratively, which can be costly for large graphs.  Given that the maximum degree is bounded and i, j are small, I wonder if there are more efficient approaches.

Answer: I incorporated an idea from the comments. I’m iterating through the nodes and, for each node, only consider its neighbors to construct the subgraph K_ij. This reduces the complexity of generating all possible combinations, but it’s still exponential for graphs with a high degree. def is_k_ij_free(self, i: int, j: int) -> bool: if i < 1 or j < 1: raise ValueError("i and j must be greater than 0") if i < j: # swap i and j for better runtime i, j = j, i nodes = {u for u in self.g if len(self.g[u]) >= j} for node in nodes: neighbor_i = {u for u in self.g[node] if len(self.g[u]) >= i} for A in combinations(neighbor_i, j): common_neighbors = set(self.g) for a in A: common_neighbors &= self.g[a] if len(common_neighbors) >= i: return False return True  Feel free to further improve it, or point out any errors I have overseen.
Question: With normal classes you have **kwargs in the __init__ so non-matching keyword arguments can be ignored: class MyClass: def __init__(self, a, **kwargs): self.a=a my_class = MyClass(20, **{"kwarg1" : 1})  Is there an equivalent for @dataclass that can gracefully ignore non-matching keyword arguments without having to include an __init__? from dataclasses import dataclass @dataclass class MyClass: a: int my_class = MyClass(20, **{"kwarg1" : 1}) # TypeError: MyClass.__init__() got an unexpected keyword argument 'kwarg1' 

Answer: TL;DR No, it's impossible This scenario was even quoted in the original pep introducing dataclass. I think it's against the original assumptions around dataclass - which would be simplify semantics at the expense of making functionality less flexible. vide pep-0557  Sometimes the generated init method does not suffice. For example, suppose you wanted to have an object to store *args and **kwargs:  @dataclass(init=False) class ArgHolder: args: List[Any] kwargs: Mapping[Any, Any] def __init__(self, *args, **kwargs): self.args = args self.kwargs = kwargs a = ArgHolder(1, 2, three=3)
Question: I would like to generate a surface plot with discrete arc-shaped cells on a 2D cartesian plane. I am able to get decent results by plotting a 3D surface plot (using plot_surface) and viewing it from the top, but matplotlib can be a bit finicky with 3D, so I'd prefer to do it in 2D. I can also get similar results using pcolormesh on a polar plot, but again, I want a 2D cartesian plane. How can I do this in matplotlib? MRE: import numpy as np import matplotlib.pyplot as plt r = np.linspace(2, 5, 25) theta = np.linspace(0, np.pi, 25) R, Theta = np.meshgrid(r, theta) X = R*np.cos(Theta) Y = r*np.sin(Theta) U = R*np.cos(Theta)*np.exp(R*Theta/500) fig, ax = plt.subplots(figsize=(8,6), subplot_kw={"projection":"3d"}) surf = ax.plot_surface(X, Y, U, cmap="viridis", rstride=1, cstride=1) ax.view_init(elev=90, azim=-90) ax.set_proj_type("ortho") ax.zaxis.line.set_lw(0.) ax.set_zticks([]) ax.set_aspect("equalxy") fig.colorbar(surf, shrink=0.5, aspect=5) fig.tight_layout() fig, ax = plt.subplots(figsize=(8,6), subplot_kw={"projection":"polar"}) ax.pcolor(Theta, R, U, shading="nearest") ax.set_xlim(0, np.pi) ax.grid(False) fig.tight_layout()  3D plot version:  2D polar plot version: 

Answer: One solution could be to put together the plot from wedge patches.  import numpy as np import matplotlib.pyplot as plt from matplotlib.patches import Wedge from matplotlib.collections import PatchCollection r = np.linspace(2, 5, 25) theta = np.linspace(0, np.pi, 25) r_mid = 0.5 * (r[:-1] + r[1:]) theta_mid = 0.5 * (theta[:-1] + theta[1:]) R, Theta = np.meshgrid(r_mid, theta_mid) U = R * np.cos(Theta) * np.exp(R * Theta / 500) patches = [] color = [] for i in range(len(theta) - 1): for j in range(len(r) - 1): t0, t1 = np.degrees(theta[i]), np.degrees(theta[i+1]) #angle bounds in degrees r0, r1 = r[j], r[j+1] #radial bounds wedge = Wedge(center=(0, 0), r=r1, theta1=t0, theta2=t1, width=r1 - r0) patches.append(wedge) color.append(U[i, j]) #coloring fig, ax = plt.subplots(figsize=(8, 6)) collection = PatchCollection(patches, array=np.array(color), cmap='viridis', edgecolor=None) ax.add_collection(collection) ax.set_xlim(-5.5, 5.5) ax.set_ylim(-0.5, 5.5) ax.set_aspect('equal') ax.spines['top'].set_visible(False) ax.spines['right'].set_visible(False) ax.spines['bottom'].set_color('black') ax.spines['left'].set_color('black') fig.colorbar(collection, ax=ax, shrink=0.5) plt.tight_layout() plt.show()
Question: class L1Dist(Layer): def __init__(self, **kwargs): super().__init__() def call(self, input_embedding, validation_img): return tf.math.abs(input_embedding - validation_img)  Signature of method 'L1Dist.call()' does not match signature of the base method in class'Layer' how can i fix this problem?

Answer: I found the solution : class L1Dist(Layer): def __init__(self, **kwargs): super(L1Dist, self).__init__(**kwargs) def call(self, inputs, *args, **kwargs): input_embedding, validation_embedding = inputs return tf.math.abs(input_embedding - validation_embedding)  Instead of having input_embedding and validation_embedding as separate parameters, the method now accepts a single inputs argument, which will be a tuple or list containing both embeddings. as per a tansorflow document and it matches with the base method. found in base_layer.py line 465. sorry i'm bad at explaining things.
Question: I have a Polars dataframe corpus with one string column, and millions of rows. I also have a list of substrings substrings. I can take a substring and query in how many rows that substring appears with: corpus.select(pl.col('contents').str.contains(substrings[0]).sum()).item()  This works well for one substring, but I have 10,000 substrings to check. What is the most efficient way in Polars to check all of them? I have considered converting substrings into its own polars dataframe, and then performing an inner-join on substring presence, grouping by keyword, then counting the size of the groups. However, this seems very expensive from a RAM overhead perspective, and I am limited on RAM. Is there a better/cleaner way? Current slow approach: import polars as pl substrings = pl.DataFrame({'substring': ['a', 'b', 'c']}) corpus = pl.DataFrame({'contents': ['aBMMmcICmY', 'ORqkIJCwjV', 'JTQHufYApo', 'SNoqiJxpMY', 'SYbEsasrzt', 'XLinDPSRld', 'iInkOGqBDU', 'vBtykwGOqN', 'ZIpOdkkXBd', 'iUokuiefBS']}) def count_occurrences(substring): return corpus.select(pl.col('contents').str.contains(substring).sum()).item() substrings = substrings.with_columns(pl.col('substring').map_elements(count_occurrences).alias('frequency'))  Outputting: shape: (3, 2) ┌───────────┬───────────┐ │ substring ┆ frequency │ │ --- ┆ --- │ │ str ┆ i64 │ ╞═══════════╪═══════════╡ │ a ┆ 2 │ │ b ┆ 1 │ │ c ┆ 1 │ └───────────┴───────────┘ 

Answer: substrings = ["ab", "abc", "c"] df = pl.DataFrame({ "contents": ["abcMMmcICm", "ckIJCwjVab", "JTQHufYcpo", "SNoqabcpMY", "SYbEsasrzt"] })  .str.extract_many() has been the fastest way I've found to do this. df.with_columns( pl.col("contents").str.extract_many(substrings).alias("substring") )  shape: (5, 2) ┌────────────┬──────────────────┐ │ contents ┆ substring │ │ --- ┆ --- │ │ str ┆ list[str] │ ╞════════════╪══════════════════╡ │ abcMMmcICm ┆ ["ab", "c", "c"] │ # <- did not find 'abc' │ ckIJCwjVab ┆ ["c", "ab"] │ │ JTQHufYcpo ┆ ["c"] │ │ SNoqabcpMY ┆ ["ab", "c"] │ │ SYbEsasrzt ┆ [] │ └────────────┴──────────────────┘  You need to pass overlapping=True to get "all" the matches. shape: (5, 1) ┌─────────────────────────┐ │ substring │ │ --- │ │ list[str] │ ╞═════════════════════════╡ │ ["ab", "abc", "c", "c"] │ # <- "c" is also found twice │ ["c", "ab"] │ │ ["c"] │ │ ["ab", "abc", "c"] │ │ [] │ └─────────────────────────┘  We don't want to count the same row twice, so we add a row index, .explode() and .unique() If you .group_by() the substring, the length of the group is the count. ( df.select( pl.col("contents").str.extract_many(substrings, overlapping=True) .alias("substring") ) .with_row_index() .explode("substring") .unique() .group_by("substring") .len() .drop_nulls() # empty lists will be null )  shape: (3, 2) ┌───────────┬─────┐ │ substring ┆ len │ │ --- ┆ --- │ │ str ┆ u32 │ ╞═══════════╪═════╡ │ c ┆ 4 │ │ abc ┆ 2 │ │ ab ┆ 3 │ └───────────┴─────┘  Substrings that did not match will not be in the output, you would combine them in if required.
Question: I'm very baffled and a little concerned to discover the following behaviour where I have two tests and two fixtures. import pytest @pytest.fixture def new_object(): return list() @pytest.fixture def a_string(new_object): # Change this instance of the object new_object.append(1) return "a string" def test_1(new_object): assert len(new_object) == 0 def test_2(a_string, new_object): assert len(new_object) == 0  The first test passes but the second one fails.  def test_2(a_string, new_object): > assert len(new_object) == 0 E assert 1 == 0 E + where 1 = len([1]) tests/test_pytest_list.py:21: AssertionError ================================================ short test summary info ================================================= FAILED tests/test_pytest_list.py::test_2 - assert 1 == 0 ============================================== 1 failed, 1 passed in 0.36s ===============================================  I expected fixtures to pass new instances of an object (unless specified otherwise), not the same object that some other fixture has modified. According to the documentation about the scope of fixtures it says:  the default is to invoke once per test function  Does another fixture not qualify as a function? UPDATE Based on the comments I now understand the issue, although I still think it's a dangerous behaviour for a unit-testing tool. Here's another invalid use of fixtures which a naive person like myself might not realize is wrong: @pytest.fixture def new_object(): """I want to test instances of this class""" return list() @pytest.fixture def case_1(new_object): new_object.append(1) return new_object @pytest.fixture def case_2(new_object): new_object.append(2) return new_object def test_cases(case_1, case_2): assert sum(case_1) + sum(case_2) == 3 # fails: assert (3 + 3) == 3 

Answer: The fixture new_object is invoked for each test as you clearly stated from the documentation. The issue lies within your second fixture and the usage of the combination of both in your second test. As pytest allows you to use fixtures more than once per test without affecting each other by using cached returns. That means as the fixture a_word uses the fixture new_object the second time new_object is encountered in your second test it uses the cached return value from the a_word fixture call instead of giving you another empty list.