eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: How do I get the type tuple[A[int], A[int]] for b.b? import typing T = typing.TypeVar('T') Ts = typing.TypeVarTuple('Ts') class A(typing.Generic[T]): a: T class B(typing.Generic[*Ts]): b: tuple[*A[Ts]] b: B[int, int] = B() typing.reveal_type(b.b) # tuple[A[*tuple[int, int]]] i got # but i need a tuple[A[int], A[int]]  I tried something like tuple[*(A[T1] for T1 in Ts)], but it doesn't work too UPD: I want it to be like in c++ #include <iostream> #include <tuple> #include <typeinfo> template<typename T> struct A { T a; }; template<typename... Ts> struct B { std::tuple<A<Ts>...> b; }; int main() { B<int, int> b; std::cout << typeid(b.b).name() << std::endl; // std::__1::tuple<A<int>, A<int>> return 0; } 

Answer: Now (Python 3.12) there is no such possibility. But the issue was created on github python/typing. And I hope that they will add such an opportunity! My suggestion: import typing T = typing.TypeVar('T') Ts = typing.TypeVarTuple('Ts') class A[T]: a: T class B[*Ts]: b: tuple[*A[Ts]] b: B[int, int] = B() typing.reveal_type(b.b) # tuple[A[int], A[int]]  class SerializationWrapper[*Ts]: def __init__( self, types: *type[Ts], additional_encoding_hooks: Callable | None = None, additional_decoding_hooks: Callable | None = None, ): self._types: *type[Ts] = types # create stateful en/decoders, register these types, and raise if the types aren't understood or # if the types have ambiguous overlap from the perspective of encoding or decoding # after the registration of optional hooks def encode(obj: *type[Ts]) -> bytes: ... def decode(bytes) -> Union[*Ts]: ...
Question: I'm getting an error while using tkinter. I have installed python using pyenv. >>> import tkinter >>> tkinter._test() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/hrj/.pyenv/versions/3.8.16/lib/python3.8/tkinter/__init__.py", line 4557, in _test root = Tk() File "/Users/hrj/.pyenv/versions/3.8.16/lib/python3.8/tkinter/__init__.py", line 2272, in __init__ self._loadtk() File "/Users/hrj/.pyenv/versions/3.8.16/lib/python3.8/tkinter/__init__.py", line 2288, in _loadtk raise RuntimeError("tk.h version (%s) doesn't match libtk.a version (%s)" RuntimeError: tk.h version (8.5) doesn't match libtk.a version (8.6)  The error is similar to this question, but my tk.h version is lower. Any way to upgrade this? tkinter._test() should display a test window but I get a error. In stackoverflow and GitHub people usually have a lower version of libtk.a but mine is the reverse. So those solutions won't work.

Answer: This worked in Python versions 3.7 to 3.11.  brew update brew install tcl-tk echo 'export PATH="/usr/local/opt/tcl-tk/bin:$PATH"' >> ~/.zshrc export LDFLAGS="-L/usr/local/opt/tcl-tk/lib" export CPPFLAGS="-I/usr/local/opt/tcl-tk/include" export PKG_CONFIG_PATH="/usr/local/opt/tcl-tk/lib/pkgconfig" brew install pyenv --head python --version >> py_ver sed -i '' "s/Python //g" py_ver export PY_VER="$( cat py_ver )" env \ PATH="$(brew --prefix tcl-tk)/bin:$PATH" \ LDFLAGS="-L$(brew --prefix tcl-tk)/lib" \ CPPFLAGS="-I$(brew --prefix tcl-tk)/include" \ PKG_CONFIG_PATH="$(brew --prefix tcl-tk)/lib/pkgconfig" \ CFLAGS="-I$(brew --prefix tcl-tk)/include" \ PYTHON_CONFIGURE_OPTS="--with-tcltk-includes='-I$(brew --prefix tcl-tk)/include' --with-tcltk-libs='-L$(brew --prefix tcl-tk)/lib -ltcl8.6 -ltk8.6'" \ pyenv install $PY_VER pyenv global $PY_VER  I have tried it on Github Actions.
Question: My most recent env prints traceback like this ╭───────────────────── Traceback (most recent call last) ──────────────────────╮  which is beyond useless. I've already looked at this How to make typer traceback look normal but it doesn't help. My hunch is it may be about Huggingface but maybe something else like datasets or evaluate but I can't find anything useful so far. How to make stacktrace print everything it need to again?

Answer: It is this thing that causes the issue. So pip uninstall rich solved everything. Edit July 3rd, 2023 This seems to be the issue with accelerate as I originally anticipated. In this case it default to use rich when available. So ACCELERATE_DISABLE_RICH=1 should fix the issue in case other libs require rich for something else.
Question: In the Plotly Sankey diagram, you are able to see the 'value' of a link/node by hovering over it. I want the image to display the values without hovering though. I've looked through the documentation and see virtually no way of doing this beside replacing the labels themselves with the desired value. That is not a good option, as nothing would then by labeled. Short of making dynamic labels that include both and and value, I'm not sure how to approach this. Examples below... Sample Sankey Diagram (source): import plotly.graph_objects as go fig = go.Figure(data=[go.Sankey( node = dict( pad = 15, thickness = 20, line = dict(color = "black", width = 0.5), label = ["A1", "A2", "B1", "B2", "C1", "C2"], customdata = ["Long name A1", "Long name A2", "Long name B1", "Long name B2", "Long name C1", "Long name C2"], hovertemplate='Node %{customdata} has total value %{value}<extra></extra>', color = "blue" ), link = dict( source = [0, 1, 0, 2, 3, 3], target = [2, 3, 3, 4, 4, 5], value = [8, 4, 2, 8, 4, 2], customdata = ["q","r","s","t","u","v"], hovertemplate='Link from node %{source.customdata}<br />'+ 'to node%{target.customdata}<br />has value %{value}'+ '<br />and data %{customdata}<extra></extra>', ))]) fig.update_layout(title_text="Basic Sankey Diagram", font_size=10) fig.show()  Actual Output:  Desired Output: 

Answer: The node positions are determined by non-trivial algorithms and I am afraid that Plotly does not make the coordinates explicit, as of now, see Extract X and Y coordinates from Plotly Sankey diagram. I think you can:  Pass locations computed on your own (may be tricky!), and use the locations to anchor custom annotations Plotly: how to write a text over my Sankey diagram columns? Inject your information into displayable labels (what you described as an alternative), see Add node count to Plotly Sankey diagram
Question: I've thought that poetry add package would simply add the package to pyproject.toml but it seems it doesn't just add but also installs it in a virtual environment. But what does poetry install do? When I run it after I added the deps with add, I am getting the following message: Installing dependencies from lock file No dependencies to install or update  Note that I started a project from scratch with mkdir new_dir; cd new_dir; poetry init.

Answer: poetry add library_name installs the library and adds it to the pyproject.toml file. Note - both installs the library and adds it to the file. poetry install is used when you've directly edited the pyproject.toml file and added the dependency names manually. In that case, they aren't installed yet, so, poetry install takes care of that.
Question: What is the difference between permission_classes = [A, B] permission_classes = [A & B]  ?? I understand that the first one processes permission sequentially, and the second one does it at once. Is there a difference in the result? Also, I wonder which method is preferred and why.

Answer: In short: two or more elements in a list or tuple is advisable. Semantically the two are nearly the same. Indeed, Django checks the permissions in the .check_permissions(…) method [GitHub]:  def check_permissions(self, request): """ Check if the request should be permitted. Raises an appropriate exception if the request is not permitted. """ for permission in self.get_permissions(): if not permission.has_permission(request, self): self.permission_denied( request, message=getattr(permission, 'message', None), code=getattr(permission, 'code', None), )  It thus enumerates over the items in the list and when one of the permissions fails, it invokes the .permission_denied(…) method with the message and code of the permission that failed. If you use the & operator on the other hand, it will construct a new permission, with the .__and__(…) method [GitHub]:  class OperationHolderMixin: def __and__(self, other): return OperandHolder(AND, self, other) # … class OperandHolder(OperationHolderMixin): def __init__(self, operator_class, op1_class, op2_class): self.operator_class = operator_class self.op1_class = op1_class self.op2_class = op2_class def __call__(self, *args, **kwargs): op1 = self.op1_class(*args, **kwargs) op2 = self.op2_class(*args, **kwargs) return self.operator_class(op1, op2) # … class AND: def __init__(self, op1, op2): self.op1 = op1 self.op2 = op2 def has_permission(self, request, view): return self.op1.has_permission(request, view) and self.op2.has_permission( request, view ) def has_object_permission(self, request, view, obj): return self.op1.has_object_permission( request, view, obj ) and self.op2.has_object_permission(request, view, obj)  This is thus essentially just some meta-programming logic to first run the .has_permission(…) on the first operand and if that succeeds, on the second operand. So it seems that the two are equivalent? Well not exactly, by using an &, the .message and .code are gone, so one can not trace back which permission check exactly has been denied. While most (all) builtin permissions have no message or code, if you thus specify one yourself for a custom permission check, and that permission check is used in an operator, the result of that expression loses the code and message. So while this is likely, a small detail, there is a small advantage in using [perm1, perm2] over [perm1 & perm2]
Question: I'm encountering tweepy.errors.Forbidden: 403 Forbidden When authenticating requests to the Twitter API v2 endpoints, you must use keys and tokens from a Twitter developer App that is attached to a Project. You can create a project via the developer portal.  while trying to run the following code that fetches a user's post history using the Twitter API and Tweepy:  client = tweepy.Client(bearer_token=bearer_token) tweets = client.search_recent_tweets(query=f'from:{user_handle}')  My app does seem to be connected to a project (see img) I have come across some links that suggest it might be a Twitter issue related to API authentication. However, I would like to confirm if this is indeed the case and if there are any possible solutions or workarounds for this problem. Links indicating it might be a Twitter issue: https://github.com/twitterdev/Twitter-API-v2-sample-code/issues/58 https://twittercommunity.com/t/when-authenticating-requests-to-the-twitter-api-v2-endpoints-you-must-use-keys-and-tokens-from-a-twitter-developer-app-that-is-attached-to-a-project-you-can-create-a-project-via-the-developer-portal/189699 I would greatly appreciate any insights, explanations, or potential solutions to resolve this issue. 

Answer: Your app has the Free access tier which only allows:  Posting tweets with the Twitter API v2 Media Upload and Login With Twitter with the Twitter API v1.1  To do anything else, you need at least the Basic access tier, which costs some money. The access tiers are documented in https://developer.twitter.com/en/docs/twitter-api/getting-started/about-twitter-api And of course you should use a recent version of Tweepy that supports the V2 API.
Question: The following function is supposed to work similarly to pow(x, 1/k) but to be symmetric around the line y = 1 - x as well as not having a 0 or 1 slope at either end of [0, 1]: def sym_gamma(x, k): if k == 1.0: return x a = 1.0 / k - 1.0 b = 1.0 / a c = k + 1.0 / k - 2.0; return 1.0 / (a - c * x) - b  As can be seen, it is not defined when k = 1 so when that is the case, I simply return x. However, this special case handling is not enough since the function also behaves poorly when x is not equal to but very close to 1.0. For example sym_gamma(0.5, 1.00000001) yields 0.0 while it's supposed to return something very close to 0.5. How can achieve the same thing without the poor stability? I know that I can introduce a tolerance with respect to k equaling 1.0 but it feels like a hack and I would also want to make sure that the function is perfectly smooth with regards to k.

Answer: Simplifying your expression seems to help with the precision. Numerical errors tends to accumulate in each operation. Thus, reducing the number of operation will reduce the chance of numerical errors. We can notice that: a = (1 - k) / k b = k / (1 - k) c = (1 - k) ** 2 / k a - c * x = (1 - k) * (1 + x*k - x) / k 1.0 / (a - c * x) - b = x*k / (1 - x * (1 - k))  Then you can simply rewrite your method: def sym_gamma(x, k): return x*k / (1 - x * (1 - k))  Instead of performing several division, only one division is computed. This method returns 0.5000000025 for sym_gamma(0.5, 1.00000001).
Question: I have a complex matrix multiplication with several hundreds of thousands of rows and columns. At some point the memory usage grows to 100% and then the computer is freezed and I have to restart it manually. I have tried with Numba (writing the code inside a function with a decorator) and Dask (transforming the numpy arrays as da.from_array(var, chunk)) without success. I am not an expert in any of those. I have read plenty of similar questions to this without finding a good solution for my problem. A minimum reproducible example could be m = 100000 n = 100000 a1 = np.random.rand(m) a2 = np.random.rand(n) c = np.random.rand(m)+1j*np.random.rand(m) b = np.random.rand(n)+1j*np.random.rand(n) A = np.exp(1j*np.outer(a1,a2)) d = c*np.dot(A,b)  What would be the best option of solving it in terms of memory usage? (not neccesarily the fastest)

Answer: Main issue The main issue is that 1j*np.outer(a1,a2) takes 100_000 * 100_000 * (8 * 2) = 149 GiB. On top of that, np.exp needs to read this matrix and produce another one of the same size so you need at least ~300 GiB of RAM just for this. This is HUGE and inefficient. You should avoid creating the matrix A at any price (including similar temporary matrices).  Fast memory-efficient solution Numba can help in this case : you can compute the array d on the fly avoiding huge temporary matrices. Here is an optimized Numba code doing this: import numba as nb import numpy as np @nb.njit('(float64[::1], float64[::1], complex128[::1], complex128[::1])', parallel=True) def compute(a1, a2, b, c): m, n = a1.size, a2.size assert n == m # seems already mantatory in the initial code tmpDot = np.zeros(n, dtype=np.complex128) for i in nb.prange(n): for j in range(n): tmpDot[i] += np.exp(1j * (a2[j] * a1[i])) * b[j] return c * tmpDot m = 100000 n = 100000 a1 = np.random.rand(m) a2 = np.random.rand(n) c = np.random.rand(m)+1j*np.random.rand(m) b = np.random.rand(n)+1j*np.random.rand(n) d = compute(a1, a2, b, c)  This code only take a very small amount of memory compared to the initial one : only few MiB. So it takes about 100_000 times less memory! Besides, I also expect it to run significantly faster (because it is multi-threaded and better use the CPU caches as well as the RAM). It takes only 17.1 seconds on my machine (while I cannot even run the initial code)!
Question: I have this df:  df = pd.DataFrame( {'loss': [0.044, 0.044, 0.038, 0.037, 0.036], 'code': ["('ac',)", "('ac', 'be')", "('ab', 'ac', 'be')", "('ab', 'ac', 'be', 'fi')", "('ab', 'ac', 'be', 'de', 'fi')"]} ) df loss code 0 0.044 ('ac',) 1 0.044 ('ac', 'be') 2 0.038 ('ab', 'ac', 'be') 3 0.037 ('ab', 'ac', 'be', 'fi') 4 0.036 ('ab', 'ac', 'be', 'de', 'fi')  Now I want add a new column added-code, the new value introduce in the code column. Expected results:  loss code added-code 0 0.044 ('ac',) ac 1 0.044 ('ac', 'be') be 2 0.038 ('ab', 'ac', 'be') ab 3 0.037 ('ab', 'ac', 'be', 'fi') fi 4 0.036 ('ab', 'ac', 'be', 'de', 'fi') de 

Answer: Assuming there is one new value per row, you can convert to tuples, explode and drop_duplicates: from ast import literal_eval df['added'] = (df['code'] .apply(literal_eval) .explode() .drop_duplicates() )  Output:  loss code added 0 0.044 ('ac',) ac 1 0.044 ('ac', 'be') be 2 0.038 ('ab', 'ac', 'be') ab 3 0.037 ('ab', 'ac', 'be', 'fi') fi 4 0.036 ('ab', 'ac', 'be', 'de', 'fi') de  Alternative, as suggested in comment, to use set operations: df['added'] = (df['code'] .apply(lambda x: set(literal_eval(x))) .diff() )  Output:  loss code added 0 0.044 ('ac',) NaN 1 0.044 ('ac', 'be') {be} 2 0.038 ('ab', 'ac', 'be') {ab} 3 0.037 ('ab', 'ac', 'be', 'fi') {fi} 4 0.036 ('ab', 'ac', 'be', 'de', 'fi') {de}
Question: I want to have error bars in my bar plots when more than 3 data points are available (Condition A) but omit error bars when there are less than 3 data points for that specific condition (Condition B). I've only found options to show or hide error bars for all bars, not for specific conditions. import pandas as pd import seaborn as sns import numpy as np df = pd.DataFrame(np.random.randint(0,100,size=(15)), columns=["Value"]) df["Label"]="Condition A" df.Label[13:]="Condition B" sns.barplot(data=df, x="Label", y="Value", errorbar="sd")  Actual Outcome: Error bars on all bars:  Desired outcome: Error bars on condition A only: 

Answer: You can use a custom errorbar function in sns.barplot. It should return a [y1, y2] iterable with the position of the min/max error: # defining a custom function to only compute # the error if more than 3 values def cust_error(s): if len(s)<3: return [None, None] else: avg = s.mean() std = s.std() return [avg-std, avg+std] sns.barplot(data=df, x="Label", y="Value", errorbar=cust_error)  Another option could be to plot the error bars manually: ax = sns.barplot(data=df, x="Label", y="Value", errorbar=None) g = df.groupby('Label', sort=False) error = g['Value'].std().where(g.size()>=3) plt.errorbar(range(len(s)), g['Value'].mean(), error, ls='', color='#5F5F5F', lw=3)  Output:
Question: I recently updated my python install from 3.11.2 to 3.11.3, pandas version is 2.0.2 I am now getting this error: TypeError: Cannot cast DatetimeArray to dtype datetime64[D]  When I try to perform this: df = df[df['CancelDate'].astype('datetime64[D]') >= (datetime.now() - relativedelta(years=2))]  On this dataframe: mydataset = { 'CancelDate': ["2021-09-07", "2021-07-26", "2021-11-01","2015-06-15"] } df = pandas.DataFrame(mydataset)  Prior to the update, I was not getting the given error. Could anyone help me realize the error in my ways?

Answer: To solve this issue, you can use the pd.pd.to_datetime function to convert the 'CancelDate' column to a DatetimeArray before performing the comparison. df['CancelDate'] = pd.to_datetime(df['CancelDate']) # Convert to DatetimeArray df = df[df['CancelDate'] >= (datetime.now() - relativedelta(years=2))]
Question: I am having problems with importing the pandas package. I used the following command to import it: import pandas as pd  However, I receive the following error message: Traceback (most recent call last): Cell In[54], line 1 import pandas as pd File ~\AppData\Local\anaconda3\lib\site-packages\pandas\__init__.py:48 from pandas.core.api import ( File ~\AppData\Local\anaconda3\lib\site-packages\pandas\core\api.py:27 from pandas.core.arrays import Categorical File ~\AppData\Local\anaconda3\lib\site-packages\pandas\core\arrays\__init__.py:1 from pandas.core.arrays.arrow import ArrowExtensionArray File ~\AppData\Local\anaconda3\lib\site-packages\pandas\core\arrays\arrow\__init__.py:1 from pandas.core.arrays.arrow.array import ArrowExtensionArray File ~\AppData\Local\anaconda3\lib\site-packages\pandas\core\arrays\arrow\array.py:60 from pandas.core.arraylike import OpsMixin File ~\AppData\Local\anaconda3\lib\site-packages\pandas\core\arraylike.py:21 from pandas.core.ops.common import unpack_zerodim_and_defer File ~\AppData\Local\anaconda3\lib\site-packages\pandas\core\ops\__init__.py:38 from pandas.core.ops.array_ops import ( File ~\AppData\Local\anaconda3\lib\site-packages\pandas\core\ops\array_ops.py:57 from pandas.core.computation import expressions File ~\AppData\Local\anaconda3\lib\site-packages\pandas\core\computation\expressions.py:20 from pandas.core.computation.check import NUMEXPR_INSTALLED File ~\AppData\Local\anaconda3\lib\site-packages\pandas\core\computation\check.py:5 ne = import_optional_dependency("numexpr", errors="warn") File ~\AppData\Local\anaconda3\lib\site-packages\pandas\compat\_optional.py:157 in import_optional_dependency version = get_version(module_to_get) File ~\AppData\Local\anaconda3\lib\site-packages\pandas\compat\_optional.py:84 in get_version raise ImportError(f"Can't determine version for {module.__name__}") ImportError: Can't determine version for numexpr  I am using the following version of Python: Python 3.10.10 | packaged by Anaconda, Inc. | (main, Mar 21 2023, 18:39:17) [MSC v.1916 64 bit (AMD64)]  Is there any way to solve this problem? Some important information is maybe that this computer has remote access to the server that I'm using via VPN. So I should only have access to the program when I am logged into the VPN.

Answer: If you are on ubuntu linux, you can try sudo apt-get install python-numexpr. Refer this answer - https://askubuntu.com/questions/446644/why-do-i-get-importerror-when-trying-to-import-pandas-python-module After installing numexpr and bottleneck, you can try pip install --force-reinstall pandas or pip install --upgrade --force-reinstall pandas to ensure pandas is installed properly.
Question: Debain stable wants me to install Python modules using pipx. So I do $ pipx install auditwheel $ pipx ensurepath $ python3 -m pipx ensurepath $ python3 Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import auditwheel Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'auditwheel' >>>  What am I doing wrong?

Answer: From Python 3.11 onward, Debian encourages the users to create a separate Python virtual environment to install Python packages. Because Debian declares its Python install to be externally-managed, pip (and other installers) will refuse to install packages system-wide. Installation is only possible in virtual environments or separate Python installs. This is because Python package installers (like pip) are unaware of the constraints that APT-managed packages have on libraries and versions. See PEP-668 for a full discussion of the problems that can occur when multiple installers operate on the same Python install. Therefore, the optimal way is to create a virtual environment, say MyEnv, and install packages therein: $ mkdir -p $HOME/.venvs # create a folder for all virtual environments $ python3 -m venv $HOME/.venvs/MyEnv # create MyEnv  This will create a directory $HOME/.venvs/MyEnv with a configuration file pyvenv.cfg which includes some details for this virtual environment, such as the Python executable and Python version. Verify the version of the Python in the virtual environment: $HOME/.venvs/MyEnv/bin/python --version  The executables of the created virtual environment are found under $HOME/.venvs/MyEnv/bin. To install a package into the virtual environment, use $HOME/.venvs/MyEnv/bin/python -m pip install <some-package>  To 'activate' the virtual enviroment, i.e. adding its configuration variables into the shell environment, use source $HOME/.venvs/MyEnv/bin/activate  Consult Python's guide to virtualenv and pip at https://packaging.python.org/en/latest/guides/installing-using-pip-and-virtual-environments.
Question: I Have the following Data frame:  id flag col_1 col_2 name 0 1 1 11 13 a 1 2 0 62 14 b 2 1 0 13 15 a 3 2 1 74 16 b 4 3 1 25 17 c 5 3 0 22 18 c  I need this as the output -  id col_3 col_4 name 0 1 2 2 a 1 2 -12 -2 b 2 3 -3 1 c  I need to group by id, name and take flag[0] of col_1 - flag[1] of the col_1 which has id, name in common. Thanks in advance.

Answer: Using simple indexing with a temporary index (set_index and reset_index) tmp = df.set_index(['flag', 'id', 'name']) out = (tmp.loc[0] - tmp.loc[1]).reset_index()  Output:  id name col_1 col_2 0 1 a 2 2 1 2 b -12 -2 2 3 c -3 1  Used input: df = pd.DataFrame({'id': [1, 2, 1, 2, 3, 3], 'flag': [1, 0, 0, 1, 1, 0], 'col_1': [11, 62, 13, 74, 25, 22], 'col_2': [13, 14, 15, 16, 17, 18], 'name': ['a', 'b', 'a', 'b', 'c', 'c']})