eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Our Django project sends GeoFeatureModelSerializer responses and we want to include an additional value in this response for JS to access. We figured out how to do this in serializers.py: from rest_framework_gis import serializers as gis_serializers from rest_framework import serializers as rest_serializers from core.models import Tablename class MarkerSerializer(gis_serializers.GeoFeatureModelSerializer): new_value = rest_serializers.SerializerMethodField('get_new_value') def get_new_value(self, foo): return True class Meta: fields = ("date", "new_value") geo_field = "geom" model = Tablename  JS can get this value with geojson.features[0].properties.new_value where const geojson = await response.json(), but it's unnecessarily added with every entry. We'd like for it to be included only once so JS can access it with something like newResponse.new_value and existing functionality can continue getting the same data via newResponse.geojson or similar. How can we include a single additional value in this response? We thought maybe wrapping our serializer in another, but they seem to be asking a different thing we don't understand. Can we append this somehow? In the serializer can we do something like newResponse = {'new_value': new_value, 'geojson': geojson} somewhere? We've had a dig through the Django Rest Framework serializers docs and couldn't work it out, so perhaps we're missing something. Other SO threads seem to only ask about adding data for every entry. edit: we should have mentioned we're using viewsets.py, which looks like: class MarkerViewSet(viewsets.ReadOnlyModelViewSet): bbox_filter_field = "location" filter_backends = (filters.InBBoxFilter,) queryset = Marker.objects.all() serializer_class = MarkerSerializer 

Answer: Figured it out. This answer is if you're using views, but for viewsets see list() and include this in your viewset: def list(self, request, *args, **kwargs): queryset = self.filter_queryset(self.get_queryset()) serializer = self.get_serializer(queryset, many=True) # idk what this code does but as we're overriding probably best to keep it page = self.paginate_queryset(queryset) if page is not None: serializer = self.get_serializer(page, many=True) return self.get_paginated_response(serializer.data) # adjust things as desired here like so, defining self.new_value elsewhere returned_object = {'new_value': self.new_value, 'geojson': serializer.data} return Response(returned_object)
Question: I was updated my Pandas from I think it was 1.5.1 to 2.0.1. Any how I started getting an error on some code that works just fine before. df = df.groupby(df['date'].dt.date)['Lake', 'Canyon'].mean().reset_index()   Traceback (most recent call last): File "f:...\My_python_file.py", line 37, in df = df.groupby(df['date'].dt.date)['Lake', 'Canyon'].mean().reset_index() File "C:\Users...\Local\Programs\Python\Python310\lib\site-packages\pandas\core\groupby\generic.py", line 1767, in getitem raise ValueError( ValueError: Cannot subset columns with a tuple with more than one element. Use a list instead. 

Answer: Versions before Pandas < 2.0.0 raises a FutureWarning if you don't use double brackets to select multiple columns  FutureWarning: Indexing with multiple keys (implicitly converted to a tuple of keys) will be deprecated, use a list instead  From Pandas >= 2.0.0, it raises a ValueError:  ValueError: Cannot subset columns with a tuple with more than one element. Use a list instead.  For example: # Pandas < 2.0.0 # Missing [[ ... ]] --v --v >>> df.groupby(df['date'].dt.date)['Lake', 'Canyon'].mean().reset_index() ... FutureWarning: Indexing with multiple keys (implicitly converted to a tuple of keys) will be deprecated, use a list instead. df.groupby(df['date'].dt.date)['Lake', 'Canyon'].mean().reset_index() # Pandas >= 2.0.0 >>> df.groupby(df['date'].dt.date)['Lake', 'Canyon'].mean().reset_index() ... ValueError: Cannot subset columns with a tuple with more than one element. Use a list instead.  Fix this using [[col1, col2, ...]]: >>> df.groupby(df['date'].dt.date)[['Lake', 'Canyon']].mean().reset_index() date Lake Canyon 0 2023-05-02 1.5 3.5  Minimal Reproducible Example: import pandas as pd df = pd.DataFrame({'date': ['2023-05-02 12:34:56', '2023-05-02 12:32:12'], 'Lake': [1, 2], 'Canyon': [3, 4]}) df['date'] = pd.to_datetime(df['date']) print(df) # Output date Lake Canyon 0 2023-05-02 12:34:56 1 3 1 2023-05-02 12:32:12 2 4
Question: I have a dataframe where the first column is a list, how can I iterate through the list and add a value to the relevant pre defined column: workflow cost cam gdp ott pdl ['cam', 'gdp', 'ott'] $2,346 ['pdl', 'ott'] $1,200  should convert to: workflow cost cam gdp ott pdl ['cam', 'gdp', 'ott'] $2,346 782 782 782 ['pdl', 'ott'] $1,200 600 600  I can get the length of the list, but I can't work out how to iterate over the list in order to match it to a column heading. Basically the cost is simply split evenly between the number of processes in the list.

Answer: Another alternative: df1 = ( df.assign(cost= df["cost"].str.replace(r"\$|,", "", regex=True).astype("float") / df["workflow"].str.len() ) .explode("workflow") .pivot(columns="workflow", values="cost") ) df = pd.concat([df[["workflow", "cost"]], df1], axis=1)  Result for the sample:  workflow cost cam gdp ott pdl 0 [cam, gdp, ott] $2,346 782.0 782.0 782.0 NaN 1 [pdl, ott] $1,200 NaN NaN 600.0 600.0
Question: In the following code: import threading def infinite_loop(): while True: pass def huge_sum(): return sum(range(2**100)) thread = threading.Thread(target=huge_sum) thread.start() thread.join(1) print("Done")  I expect the script to print "Done" after one second since join() will timeout, but instead the script hangs. If you replace the call to huge_sum with infinite_loop, it works fine. The problem seems to be with the built-in sum() function. Is there a way I can reliably get something like join()'s timeout behavior no matter what the thread is doing? I don't mind wacky metaprogramming solutions, this is for a very niche application. However, for the most part I cannot modify the code being executed inside the thread (e.g. "use a loop instead of sum" is not a solution). Linux, Python 3.8.

Answer: Looking at builtin_sum_impl in bltinmodule.c, int and float are optimized to perform the sum without releasing the Global Interpreter Lock (GIL). Python only allows byte code to execute in a single thread at a time. Every so often the byte code executor will release the GIL so that other threads can run. C code can release the GIL, letting python level code run in parallel. But sum doesn't do that. It holds the GIL while summing int so that it doesn't have to worry about other threads changing the thing being summed while in progress. That also means that no python code gets to run in any thread until it completes. One option is to do the work in a different process. For a forking operating system like linux, you could use a separate process to sum. The child can read parent memory so even if there is a large list in memory, it works. This adds the overhead of creating the child process, so not free. Its even better if you can generate the data in the child processs, especially if you want to do multiple sums in parallel. In your range example, you would to just that. Alternately, if you have a large set of integers to work with, you could move to numpy arrays where some operations like array.sum release the GIL. It wouldn't work in your example because range(2**100) is too big. But I assume that's just an example. You could scale this to multiple processes running in parallel, perhaps using multiprocessing.Pool. The best way to do this depends on several factors. For instance, If you have a large set of data in the parent, you don't want to pass it as a parameter in .map() (or other pool methods) because python has to copy the parameters to the workers. Instead you'd use some global variable that the worker just happens to know about.
Question: When debugging my code, my breakpoints are ignored by visual studio code, once I use these lines of code: from brainflow.board_shim import BoardShim, BrainFlowInputParams, BoardIds from brainflow.data_filter import DataFilter, FilterTypes, DetrendOperations, AggOperations  No error Message is given. I edited my launch.json in accordance to this post: Debugger Not Stopping at Breakpoints in VS Code for Python And tried several others, but nothing solves my issue. Probably my import is wrong/has an error? (if anybody knows something brainflow specific, please let me know. First time using it so.. yeah...) Here my Minimal Code example: #Bt_Connection #python -m pip install brainflow #pip install pybluez #python.exe -m pip install --upgrade pip import argparse import time import socket import os from brainflow.board_shim import BoardShim, BrainFlowInputParams, BoardIds from brainflow.data_filter import DataFilter, FilterTypes, DetrendOperations, AggOperations msg="X" print(msg) test= "test" print(test)  And some screenshots since the error really only becomes visible in the IDE. (Video would be better, but you have to believe me that the breakpoints are not hit.)   In the second picture the code only stops at the first line, since that is specified in my launch.json. But once I hit continue it just goes through the code without stoping and writes "X" and "test" in the terminal. launch.json for completions sake: { "version": "0.2.0", "configurations": [ { "name": "Python: Debug Current File", "type": "python", "request": "launch", "program": "${file}", "console": "integratedTerminal", "stopOnEntry": true, "justMyCode": false } ] }  Python Version: 3.11.2 64 bit And for Visual Studio Code: 

Answer: It has something to do with your python version. Using the python 3.10 version you will get the desired effect. Report it on GitHub: https://github.com/microsoft/debugpy/issues/1284
Question: I am trying to run a python script inside a dockerized miniconda environment. The issue I am facing is that when I docker run interactively(-it) and run the script manually inside, it works great. But when I docker run non-interactively, the modules that come with miniconda installation like cryptography, lxml, are not found. My dockerfile: ARG REGISTRY=harbor-west.reg.com/ci ARG FROM_TAG=master FROM harbor-west.reg.com/base-os/ubuntu:20.04 USER root ENV CONDA_DIR $HOME/miniconda3 RUN apt update && \ DEBIAN_FRONTEND=noninteractive apt install -y \ python3 \ python3-pip \ wget RUN pip install --upgrade \ google-api-python-client \ grpcio \ matplotlib \ numpy \ opencv-python \ pandas \ scikit-learn RUN mkdir /abc #Download a conda package under /abc/bin - steps removed for simplicity #install miniconda RUN wget https://repo.anaconda.com/miniconda/Miniconda3-py38_23.3.1-0-Linux-x86_64.sh RUN chmod 755 Miniconda3-py38_23.3.1-0-Linux-x86_64.sh RUN /bin/bash -c "./Miniconda3-py38_23.3.1-0-Linux-x86_64.sh -b" ENV PATH=$CONDA_DIR/bin:$PATH RUN /root/miniconda3/condabin/conda init WORKDIR /abc/bin CMD ["/bin/bash", "-c", "/abc/bin/start-prediction.sh"] #ENTRYPOINT ["/abc/bin/start-prediction.sh"]  Output with non-interactive docker run(unexpected): Traceback (most recent call last): File "prediction_server.py", line 2, in <module> from abc.learn import prepare_data, SuperResolution File "/abc/bin/abc/__init__.py", line 3, in <module> from abc.auth.tools import LazyLoader File "/abc/bin/abc/auth/__init__.py", line 1, in <module> from .api import RegSession File "/abc/bin/abc/auth/api.py", line 30, in <module> from ._auth import ( File "/abc/bin/abc/auth/_auth/__init__.py", line 2, in <module> from ._pki import PKIAuth File "/abc/bin/abc/auth/_auth/_pki.py", line 4, in <module> from ..tools._lazy import LazyLoader File "/abc/bin/abc/auth/tools/__init__.py", line 1, in <module> from .certificate import pfx_to_pem File "/abc/bin/abc/auth/tools/certificate.py", line 6, in <module> import cryptography ModuleNotFoundError: No module named 'cryptography'  Output with interactive docker run(as expected): (base)root@dcc788e0a8c5:/abc/bin# ./start-prediction.sh server listening on 0.0.0.0:50443  I tried echoing the PATH inside the container and looks ok: /root/miniconda3/bin:/root/miniconda3/condabin:/miniconda3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin  I'm not able to figure out what I'm missing. I would think if it works interactively, it should work non-interactively as well. Any pointers would be appreciated.

Answer: I struggled with the same issue. The key is when you perform conda init, it adds some lines in your shell rc, in your case .bashrc. source. Thus the solution is straightforward: source your .bashrc. Although, you have another solution that does not involve to edit your rc file. You can change your SHELL to be in your conda environment. When you are unsing this shell, install the packages requirements and run your code. SHELL ["conda", "run", "-n", "base", "/bin/bash", "-c"]
Question: names=[['Pat','Sam', np.nan, 'Tom', ''], ["Angela", np.nan, "James", ".", "Jackie"]] values=[[1, 9, 1, 2, 1], [1, 3, 1, 5, 10]]  I have 2 lists: names and values. Each value goes with a name, i.e., Pat corresponds to the value 1 and Sam corresponds to the value 9. I would like to remove the nan from names and the corresponding values from values. That is, I want a new_names list that looks like this: [['Pat','Sam', 'Tom', ''], ["Angela", "James", ".", "Jackie"]]  and a new_values list that looks like this: [[1, 9, 2, 1], [1, 1, 5, 10]]  My attempt was to first find the indices of these nan entries: all_nan_idx = [] for idx, name in enumerate(names): if pd.isnull(name): all_nan_idx.append(idx)  However, the above does not account nested lists.

Answer: Simply this? import numpy as np import pandas as pd names=[['Pat','Sam', np.nan, 'Tom', ''], ["Angela", np.nan, "James", ".", "Jackie"]] values=[[1, 9, 1, 2, 1], [1, 3, 1, 5, 10]] new_names = [] new_values = [] for names_, values_ in zip(names, values): n = [] v = [] for name, value in zip(names_, values_): if not pd.isnull(name): n.append(name) v.append(value) new_names.append(n) new_values.append(v)
Question: I'm trying to have RegEx match all the words in a dialogue that are said by a specific character. Every line is formatted as "[NAME]: [DIALOGUE]", so there's a consistent tag at the start of each line to check with, but I can't work out how to do that. For example, if I was looking Romeo's Dialogue in Romeo and Juliet, it would match each word in "Romeo: I love you Juliet", but wouldn't match anything in "Juliet: I love you Romeo". The only thing I've thought of as a possible solution is using lookbehind assertions, for which I have (?<=NAME:[.*])\w+, but that doesn't return any matches. Through some debugging and looking at the other answers, I've worked out that the issue is with adding the [.*], specifically the square brackets. This lead me to (?<=^NAME:).*\w+, which almost worked, but it matched the entire line of dialogue instead of the individual words. After looking through the review questions when making this post, I came across this question, which had the code \Aframe.*width\s(?<width>\d+)\sheight\s(?<height>\d+)\z. I tried modifying it to be \ANAME:.*\w+\s(?<\w+>\d+)\s\z and then to \ANAME:.*\w+\s(?\w+\d+)\s\z, but both returned errors about the second \w+, citing "bad escape". I then looked at this question, which had the code (^@property|(?!^)\G)(.*? )\K([^-\n]\w+), but even the base code without any modifcation returned the same "bad escape" error.

Answer: Simply iterate over the lines and match all words if the line starts with the given character name: import re def get_character_words(character_name, dialogue): result = [] for line in dialogue.splitlines(): if line.startswith(character_name): result += re.findall(fr'\w+', line[len(character_name) + 2:]) return result  Try it: text = ''' Romeo: Shall I hear more, or shall I speak at this? Juliet: ’Tis but thy name that is my enemy. Thou art thyself, though not a Montague. What’s Montague? It is nor hand, nor foot, Nor arm, nor face. O, be some other name Belonging to a man. What’s in a name? That which we call a rose Romeo: Call me but love, and I’ll be new baptized. Henceforth I never will be Romeo. ''' print(get_character_words('Romeo', text)) ''' [ 'Shall', 'I', 'hear', 'more', 'or', 'shall', 'I', 'speak', 'at', 'this', 'Call', 'me', 'but', 'love', 'and', 'I', 'll', 'be', 'new', 'baptized', 'Henceforth', 'I', 'never', 'will', 'be', 'Romeo' ] '''
Question: Is there an interface to access the information in pyproject.toml from Python? In particular, I'd like to access the dependencies. It doesn't seem hard to do toml.load("pyproject.toml")['project']['dependencies']  and then parse that list. But that would be rewriting logic that must already be available somewhere. Apparently there is importlib.metadata.requires(), which would be better than the line above, but that seems to give a similar list of strings that still requires manual parsing: >>> metadata.requires("scipy") ['numpy<1.27.0,>=1.19.5', 'pytest; extra == "test"', ...  (As for the XY problem, I was thinking about writing a test that tries to install a package with the lowest versions of its dependencies to see whether the package still works.)

Answer: In both cases, whether project metadata (such as dependency requirements) is obtained from pyproject.toml's standardized [project] section or via importlib.metadata the values should be the same. At least, as far as I know, the dependency requirements have the same format. Probably I would use the 3rd party packaging library, in particular its packaging.requirements.Requirement(), to parse the dependency requirements. This library can handle other values of project metadata as well, read the documentation. As far as I know this is the library that pip itself uses internally (so it should be always reliable and up-to-date). The issue with obtaining project metadata from pyproject.toml, is that some fields can be declared dynamic. Which means their value is not set in pyproject.toml. If you are fine with that, then the TOML parser from Python's standard library tomllib could be good enough. You could also give pyproject-parser a try, it seems like a correct and reliable library.
Question: Consider the following brief numpy session showcasing uint64 data type import numpy as np a = np.zeros(1,np.uint64) a # array([0], dtype=uint64) a[0] -= 1 a # array([18446744073709551615], dtype=uint64) # this is 0xffff ffff ffff ffff, as expected a[0] -= 1 a # array([0], dtype=uint64) # what the heck?  I'm utterly confused by this last output. I would expect 0xFFFF'FFFF'FFFF'FFFE. What exactly is going on here? My setup: >>> sys.platform 'linux' >>> sys.version '3.10.5 (main, Jul 20 2022, 08:58:47) [GCC 7.5.0]' >>> np.version.version '1.23.1' 

Answer: By default, NumPy converts Python int objects to numpy.int_, a signed integer dtype corresponding to C long. (This decision was made back in the early days when Python int also corresponded to C long.) There is no integer dtype big enough to hold all values of numpy.uint64 dtype and numpy.int_ dtype, so operations between numpy.uint64 scalars and Python int objects produce float64 results instead of integer results. (Operations between uint64 arrays and Python ints may behave differently, as the int is converted to a dtype based on its value in such operations, but a[0] is a scalar.) Your first subtraction produces a float64 with value -1, and your second subtraction produces a float64 with value 2**64 (since float64 doesn't have enough precision to perform the subtraction exactly). Both of these values are out of range for uint64 dtype, so converting back to uint64 for the assignment to a[0] produces undefined behavior (inherited from C - NumPy just uses a C cast). On your machine, this happened to produce wraparound behavior, so -1 wrapped around to 18446744073709551615 and 2**64 wrapped around to 0, but that's not a guarantee. You might see different behavior on other setups. People in the comments did see different behavior.
Question: I have the following code from pydantic import BaseModel class User(BaseModel): id: int name = "Jane Doe" def add_user(user: User): a = set() a.add(user) return a add_user(User(id=1))  When I run this, I get the following error: TypeError: unhashable type: 'User'  Is there a way this issue can be solved?

Answer: You just need to implement an hash function for the class. You can easily do that by hashing the User's id or name and return it as the hash... like this: from pydantic import BaseModel class User(BaseModel): id: int name = "Jane Doe" def __hash__(self) -> int: return self.name.__hash__() # or self.id.__hash__() def add_user(user: User): a = set() a.add(user) return a add_user(User(id=1))
Question: I keep getting the error 'Flask' object has no attribute 'session_cookie_name' right at initialization on an app that used to work. I built a smaller test app to test it and noticed that if I remove the line app.config["SESSION_TYPE"] = "filesystem" the error goes away. Problem is I need to use sessions in my Flask app and, since it is for testing, I would prefer to use the filesystem to store session data. here is my mini program for testing... I don't know what I am missing, this used to work fine and code examples online do it exactly this way but I keep getting the same error. This is the error I receive from flask import Flask, render_template,redirect, url_for, request, session from flask_session import Session app = Flask(__name__) app.config["SESSION_PERMANENT"] = False app.config["SESSION_TYPE"] = "filesystem" Session(app) @app.route("/") def index(): session["test"] = "test" return render_template ('index.html')  This worked before. It is all of the sudden not working. I have tried installing a different version of Flask_Session. I have tried adding the "SESSION_COOKIE_NAME" to app.config before Session(app) even though I understand its default is session. Nothing so far has worked or changed the error message.

Answer: This error happens when the currently used version of Flask does not have the session_cookie_name attribute. This attribute was added in Flask 2.0, so if you are using an older version of Flask, you will need to upgrade to a newer version. Just upgrade Flask by executing the command pip install --upgrade Flask Note: If you are using a virtual environment, be sure to activate it before upgrading.
Question: I have the following strings: 4563_1_some_data  The general pattern is r'\d{1,5}_[1-4]_some_data  Note, that numbers before first underscore may be the same for different some_data  So the question is: how to get all possible variants of replacement using regex?  Desired output: [4563_1_some_data, 4563_2_some_data, 4563_3_some_data, 4563_4_some_data]  My attempt: [re.sub(r'_\d_', r'_[1-4]_', row).group() for row in data]  But it has result: 4563_[1-4]_some_data  Seems like ordinary replacement. How should I activate pattern replacement and get the list?

Answer: You need to iterate over a range object to create your list.  Create that range object: To do that you need a pattern to get the [1-4] part from your pattern. You'll need another pattern to replace the number in the actual data with the variable from range object.  import re text = "4563_1_some_data" original_pattern = r"\d{1,5}_[1-4]_some_data" # A regex to get the `[1-4]` part. find_numbers_pattern = r"(?<=_)\[(\d)\-(\d)\](?=_)" # Get `1` and `4` to create a range object a, b = map(int, re.search(find_numbers_pattern, original_pattern).groups()) # Use `count=` to only change the first match. lst = [re.sub(r"(?<=_)\d(?=_)", str(i), text, count=1) for i in range(a, b + 1)] print(lst)  output: ['4563_1_some_data', '4563_2_some_data', '4563_3_some_data', '4563_4_some_data']  (?<=_)\[(\d)\-(\d)\](?=_) explanation: (?<=_): A positive lookbehind assertion to match _. \[(\d)\-(\d)\]: to get the [1-4] for example. (?=_): A positive lookahead assertion to find the _.