eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I'm new to HAProxy. I'm trying to run my Python Service's Docker Image in multiple instances and use HAProxy to Balance the Load. (python service is running at 8090 via fastapi) Firstly, I've created docker image of my python service. Then I've created docker-compose.yml: version : '3' services: elb: image: haproxy ports: - "8100:8100" volumes: - ./haproxy:/usr/local/etc/haproxy pythonapp1: image: my-python-service:0.0.1 ports: - "8090:8090"  Then I've created HAProxy folder and inside that folder I've created haproxy.cfg: # haproxy.cfg frontend http bind *:8100 mode http timeout client 10s use_backend all backend all mode http server s1 pythonapp1:8090  Then I run docker compose up and sent request to http://localhost:8100 - it worked perfectly fine. But when I added more containers it gave me error. version : '3' services: elb: image: haproxy ports: - "8100:8100" volumes: - ./haproxy:/usr/local/etc/haproxy pythonapp1: image: craft-text-detection-python-service:0.0.1 ports: - "8090:8090" pythonapp2: image: craft-text-detection-python-service:0.0.1 ports: - "8091:8090"  # haproxy.cfg frontend http bind *:8100 mode http timeout client 10s use_backend all backend all mode http server s1 pythonapp1:8090 server s2 pythonapp2:8091  Now when I run docker compose up and hit request for the first time at http://localhost:8100 it works fine. But when hit again at http://localhost:8100 it gives me 503 Service Unavailable. Guide me is it problem of port setting or something else?

Answer: There is no need to specify ports for pythonapp1 and pythonapp2 in docker compose Your docker compose should look like: version : '3' services: elb: image: haproxy ports: - "8100:8100" volumes: - ./haproxy:/usr/local/etc/haproxy pythonapp1: image: craft-text-detection-python-service:0.0.1 pythonapp2: image: craft-text-detection-python-service:0.0.1  And update your haproxy.cfg as well, which should look like: # haproxy.cfg frontend http bind *:8100 mode http timeout client 10s use_backend all backend all mode http server s1 pythonapp1:8090 server s2 pythonapp2:8090
Question: I am using the library signxml to sign XML signatures for SAML authentication. One of our implementer partners requires that we send the signature in SHA1. The base configuration of XMLSigner does not support SHA1 because it has been deprecated because SHA1 is not secure. Unfortunately I still have to send it as SHA1 because the other implementer won't change their code base. I have read the library documentation and unsure how to force SHA1 support. If you call this code below, it errors out at this point in the code: https://github.com/XML-Security/signxml/blob/9f06f4314f1a0480e22992bbb8209a71bc581e05/signxml/signer.py#L120 signed_saml_root = XMLSigner(method=signxml.methods.enveloped, signature_algorithm="rsa-sha1", digest_algorithm="sha1", c14n_algorithm="http://www.w3.org/2001/10/xml-exc-c14n#")\ .sign(saml_root, key=self.key, cert=self.cert, always_add_key_value=True) verified_data = XMLVerifier().verify(signed_saml_root, x509_cert=self.cert).signed_xml  The documentation mentions doing the following for SHA1 deprecation: SHA1 based algorithms are not secure for use in digital signatures. They are included for legacy compatibility only and disabled by default. To verify SHA1 based signatures, use: XMLVerifier().verify( expect_config=SignatureConfiguration( signature_methods=..., digest_algorithms=... ) )  But that looks for verification only, unsure how to make it work on signature. Can someone provide some advice on how to get SHA1 working with the signxml library.

Answer: You can overwrite function check_deprecated_methods in source to pass the error. from signxml import XMLSigner class XMLSignerWithSHA1(XMLSigner): def check_deprecated_methods(self): pass  Now, you can use class XMLSignerWithSHA1 to sign: signer = XMLSignerWithSHA1(signature_algorithm=SignatureMethod.RSA_SHA1, digest_algorithm=DigestAlgorithm.SHA1) signed = signer.sign(data, cert=cert, key=key)
Question: I've defined a fixture that accepts arguments to be used in an integration style test that requires some teardown upon completion. It looks something like this: @pytest.fixture def create_user_in_third_party(): from clients import third_party_client def _create_user_in_third_party(user: User): third_party_client.create_user(user) return _create_user_in_third_party  However, we then would like to clean up the created user once the test's lifecycle is complete. In a standard fixture, that may look like: @pytest.fixture def create_user(): user = User(name="Jim") user.save() yield user user.delete()  When applying that same paradigm to the fixture that takes args above, I am not seeing the fixture being called at all (returns a generator that isn't exercising the inner method code). That code (not working) looks like this: def create_user_in_third_party(): from clients import third_party_client def _create_user_in_third_party(user: User): third_party_client.create_user(user) yield # give back control to caller third_party_client.delete_user(user) return _create_user_in_third_party  is it possible to achieve the above without having to break up creation and deletion into two different fixtures ?

Answer: Using the docs that Andrej linked above: https://docs.pytest.org/en/7.1.x/how-to/fixtures.html#factories-as-fixtures , we can use the factory pattern approach to accrue created entities and delete them later. So it ends up looking like this: @pytest.fixture def create_user_in_third_party(): from clients import third_party_client created_users = [] def _create_user_in_third_party(user: User): third_party_client.create_user(user) created_users.append(user) yield _create_user_in_third_party for user in created_users: third_party_client.delete(user)
Question: I am trying to calculate a rolling sum or any other statistic (e.g. mean), within each group. Below I am giving an example where the window is 2 and the statistic is sum. df = pd.DataFrame.from_dict({'class': ['a', 'b', 'b', 'c', 'c', 'c', 'b', 'a', 'b'], 'val': [1, 2, 3, 4, 5, 6, 7, 8, 9]}) df['sum2_per_class'] = [1, 2, 5, 4, 9, 11, 10, 9, 16] # I want to compute this column # df['sum2_per_class'] = df[['class', 'val']].groupby('class').rolling(2).sum() # what I tried class val sum2_per_class 0 a 1 1 1 b 2 2 2 b 3 5 3 c 4 4 4 c 5 9 5 c 6 11 6 b 7 10 7 a 8 9 8 b 9 16  Here's what I tried and the corresponding error: df['sum2_per_class'] = df[['class', 'val']].groupby('class').rolling(2).sum() TypeError: incompatible index of inserted column with frame index 

Answer: As the error message conveys, the rolling sum operation returns a pandas Series with a MultiIndex, which can't be directly assigned to a single column in a dataframe. A possible fix is to use reset_index() to convert the MultiIndex to a normal index like the following: df['sum2_per_class'] = df[['class', 'val']].groupby('class').rolling(2).sum().reset_index(level=0, drop=True)  However, after running the above code a few of the I was getting unexpected NaN values in the 'sum2_per_class' column as follows: [NaN, NaN, 5, NaN, 9, 11, 10, 9, 16] while other values are as expected. After investigating the NaN issues I came to the following conclusion:  The Rolling Sum operation requires at least two CONSECUTIVE rows within each group to calculate the sum. for example for the first group 'a' we have: 1) Row 0 with val1=1 and Row 7 with val=8 you expect the rolling sum to be 1 + 8 = 9 while these rows are not consecutive and will result in NaN. For other groups where we got the expected rolling sum the grouped rows are consecutive. For example for group 'c' we have: Row 3, Row 4, and Row 5.  Update: To solve the NaN issues you can specify min_periods=1 in the Rolling function like below: df['sum2_per_class'] = df[['class', 'val']].groupby('class').rolling(2, min_periods=1).sum().reset_index(level=0, drop=True)
Question: I'm trying to create a class to represent a list of named tuples and I'm having trouble with accessing elements by name. Here's an example: from typing import NamedTuple class Record(NamedTuple): id: int name: str age: int class NamedTupleList: def __init__(self, data): self._data = data def attempt_access(self, row, column): print(f'{self._data[row].age=}') r = self._data[row] print(f'{type(column)=}') print(f'{column=}') print(f'{r[column]=}') data = [Record(1, 'Bob', 30), Record(2, 'Carol', 25), Record(3, 'Ted', 29), Record(4, 'Alice', 28), ] class_data = NamedTupleList(data) print('show data') print(f'{data[0]=}') print(f'{type(data[0])=}') print(f'{data[0].age=}') print('\nshow class_data') print(f'{type(class_data)=}') print('\nattempt_access by index') class_data.attempt_access(0, 2) print('\nattempt_access by name') class_data.attempt_access(0, Record.age) # why can't I do this?  Produces: data[0]=Record(id=1, name='Bob', age=30) type(data[0])=<class '__main__.Record'> data[0].age=30 show class_data type(class_data)=<class '__main__.NamedTupleList'> attempt_access by index self._data[row].age=30 type(column)=<class 'int'> column=2 r[column]=30 attempt_access by name self._data[row].age=30 type(column)=<class '_collections._tuplegetter'> column=_tuplegetter(2, 'Alias for field number 2') print(f'{r[column]=}') ~^^^^^^^^ TypeError: tuple indices must be integers or slices, not _collections._tuplegetter  So I can successfully access 'rows' and 'columns' by index, but if I want to access a column (i.e. namedtuple attribute) through a method call I get an error. What's interesting is that the column value is _tuplegetter(2, 'Alias for field number 2') so the index is known in my method but I can't get to it. Does anyone know how I can access this value so that I can pass a name to the method? I'm trying to avoid passing the name as a string - I'd really like to take advantage of the namespace since that's one of the advantages of a namedtuple after all.

Answer: Interesting question. The field is a descriptor, so you can invoke it: class NamedTupleList: def __init__(self, data): self._data = data def attempt_access(self, row, column): r = self._data[row] try: val = r[column] except TypeError: val = column.__get__(r) return val  Demo: >>> class_data.attempt_access(0, 2) 30 >>> class_data.attempt_access(0, Record.age) 30
Question: I've been observing the results when I apply cv2.erode() with different kernel values. In the code below, it is (3, 3), but it is changed to various ways such as (1, 3) or (5, 1). The reason for this observation is to understand kernel. I understand it in theory. And through practice, I can see what kind of results I get. But I want to go a little deeper. I would like to see what happens every time the pixel targeted by kernel changes. It's okay if you have thousands of images stored. How can I observe the intermediate process of cv2.erode()? Am I asking too much? image = cv2.imread(file_path, cv2.IMREAD_GRAYSCALE) _, thresholded_image = cv2.threshold(image, 127, 255, cv2.THRESH_BINARY) inverted_image = cv2.bitwise_not(thresholded_image) kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (3, 3)) eroded_image = cv2.erode(inverted_image, kernel, iterations=5) cv2.imshow('image', eroded_image) cv2.waitKey(0) cv2.destroyAllWindows() 

Answer: When you call cv2.erode from Python it eventually gets down to one C++ API call of cv::erode. As you can see in the documentation, this API does not support inspecting intermediate result from the process. This means it is also not available from the Python wrapper. The only way you can achieve what you want is to download the C++ source code for opencv (as it is open-source), change it to support inspecting intermediate result (e.g. by adding callbacks, or additional output images), compile it to a library and wrap for Python. Keep in mind however that doing so is far from being trivial.
Question: work = pd.DataFrame({"JOB" : ['JOB01', 'JOB01', 'JOB02', 'JOB02', 'JOB03', 'JOB03'], "STATUS" : ['ON_ORDER', 'ACTIVE','TO_BE_ALLOCATED', 'ON_ORDER', 'ACTIVE','TO_BE_ALLOCATED'], "PART" : ['PART01', 'PART02','PART03','PART04','PART05','PART06']})  How can I use Pandas to groupby the JOB, split Status into columns based on the values and concatenate the Part field based on the JOB. Desired Output: JOB | ON_ORDER | ACTIVE | TO_BE_ALLOCATED | PART_CON JOB01 | True | True | False | Part01\nPart02 JOB02 | True | False | True | Part03\nPart04 JOB03 | False | True | True | Part05\nPart06 

Answer: Try: x = df.groupby("JOB")["PART"].agg(", ".join).rename("PART_CON") y = pd.crosstab(df["JOB"], df["STATUS"]).astype(bool) print(pd.concat([y, x], axis=1).reset_index())  Prints:  JOB ACTIVE ON_ORDER TO_BE_ALLOCATED PART_CON 0 JOB01 True True False PART01, PART02 1 JOB02 False True True PART03, PART04 2 JOB03 True False True PART05, PART06
Question: Suppose I have the following dataframe: the_df = pl.DataFrame({'x1': [1,1,1], 'x2': [2,2,2], 'y1': [1,1,1], 'y2': [2,2,2]}) ┌─────┬─────┬─────┬─────┐ │ x1 ┆ x2 ┆ y1 ┆ y2 │ │ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╪═════╡ │ 1 ┆ 2 ┆ 1 ┆ 2 │ │ 1 ┆ 2 ┆ 1 ┆ 2 │ │ 1 ┆ 2 ┆ 1 ┆ 2 │ └─────┴─────┴─────┴─────┘  And and two lists, xs = ['x1', 'x2'], ys = ['y1', 'y2']. Is there a good way to add the products between x1/y1 and x2/y2 using .select()? So the result should look like the following. Specifically, I want to use the lists rather than writing out z1=x1*y1, z2=x2*y2 (the real data has more terms I want to multiply). ┌─────┬─────┬─────┬─────┬─────┬─────┐ │ x1 ┆ x2 ┆ y1 ┆ y2 ┆ z1 ┆ z2 │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╪═════╪═════╪═════╡ │ 1 ┆ 2 ┆ 1 ┆ 2 ┆ 1 ┆ 4 │ │ 1 ┆ 2 ┆ 1 ┆ 2 ┆ 1 ┆ 4 │ │ 1 ┆ 2 ┆ 1 ┆ 2 ┆ 1 ┆ 4 │ └─────┴─────┴─────┴─────┴─────┴─────┘ 

Answer: you can do something like this: zs = ['z1','z2'] df.with_columns( (pl.col(xc) * pl.col(yc)).alias(zc) for xc, yc, zc in zip(xs, ys, zs) ) ┌─────┬─────┬─────┬─────┬─────┬─────┐ │ x1 ┆ x2 ┆ y1 ┆ y2 ┆ z1 ┆ z2 │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╪═════╪═════╪═════╡ │ 1 ┆ 2 ┆ 1 ┆ 2 ┆ 1 ┆ 4 │ │ 1 ┆ 2 ┆ 1 ┆ 2 ┆ 1 ┆ 4 │ │ 1 ┆ 2 ┆ 1 ┆ 2 ┆ 1 ┆ 4 │ └─────┴─────┴─────┴─────┴─────┴─────┘
Question: I am trying to sum the hours user spent per day. I wanted to convert this into whole number or decimal to represent HOURS(e.g. 2, 6.5), to easily plot this to my chart. But the result of below code is in this format HH:mm:ss. Any one can help me with this? day = Clocking.objects.filter(clockout__isnull=False, user=nid).annotate(date=TruncDate('clockin')) .values('date').annotate(total=Sum(F('clockout') - F('clockin'))).values('total')  Here is my models.py class Clocking(models.Model): user= models.ForeignKey('User', models.DO_NOTHING) clockin = models.DateTimeField() clockout = models.DateTimeField(null=True) 

Answer: Based on Func using the sql functions TIMEDIFF and then TIME_TO_SEC, will return the seconds: day = Clocking.objects.filter(clockout__isnull=False, user=nid).annotate(date=TruncDate('clockin')) .values('date').annotate( total=Sum( Func( Func( F('clockout'),F('clockin'), function='TIMEDIFF'), function='TIME_TO_SEC') )).values('total')
Question: noob here. I have a dockerised jupyter lab instance where I have a bunch of packages installed for the root user, and now I want to add an additional kernel that has no packages at all (yet). Here's how I've tried to do that: # SYSTEM SETUP FROM python:3.11.5-bookworm ADD requirements/pip /requirements/pip RUN pip install -r /requirements/pip RUN mkdir /venv/ \ && cd /venv/ \ && python -m venv blank \ && /venv/blank/bin/pip install ipykernel \ && /venv/blank/bin/python -m ipykernel install --user --name=blank-env  I expect the root python environment to have all the packages listed in my requirements/pip file and my blank-env environment to be empty. However, when I run jupyter lab in the docker container and select the blank-env kernel, it appears to have access to all of the packages from my requirements file. What am I doing wrong here?

Answer:  Check Kernel Configuration: Make sure that when you install the ipykernel in your blank-env, you are activating the virtual environment before installing it. FROM python:3.11.5-bookworm ADD requirements/pip /requirements/pip RUN pip install -r /requirements/pip RUN python -m venv /venv/blank RUN /venv/blank/bin/pip install ipykernel RUN /venv/blank/bin/python -m ipykernel install --user --name=blank-env   Restart Jupyter Lab Server  Verify Kernel Environment import sys sys.executable    This will show you the path to the Python executable that the kernel is using. If it's pointing to the global Python environment, then there might be an issue with how the kernel was installed or activated.
Question: When I started Anaconda Prompt (Anaconda3),the following error message floats and I could not figure out hhow to resolve it. "Error while loading conda entry point: conda-content-trust (OpenSSL 3.0's legacy provider failed to load. This is a fatal error by default, but cryptography supports running without legacy algorithms by setting the environment variable CRYPTOGRAPHY_OPENSSL_NO_LEGACY. If you did not expect this error, you have likely made a mistake with your OpenSSL configuration.)" I have tried many proposed solutions from (https://stackoverflow.com) and others and none of them resolve the issue.

Answer: conda install cryptography   If you are using linux / Mac then add this command to .bashrc or .zshrc file  export CRYPTOGRAPHY_OPENSSL_NO_LEGACY=1 and then run source ./bashrc or ./zshrc   If you are using Window Under "System variables", click "New..." and add CRYPTOGRAPHY_OPENSSL_NO_LEGACY as the variable name and 1 as the value. Click "OK" to save the changes.
Question: I'm processing a large dataset, particularly these two columns:    Genotype Iteration     10010110011011101101011000010011111011111000000111001001101111111101101111001011 0   00011100001011010000000110010010100101101011001010101110110111000101000110000000 0   00100100100100101000100101100110100101110000100111000000011001011001101111000011 0   10001010101100000101110001011111000110101100101010111100110011011101010011111110 0   11010101010010001110100110110001001010101001111000111011110110101101010100011110 0    I want to create a new column that tells me how many 1s the column Genotype has. I've tried two methods:  With built-in str module  %%timeit total_df['Count_1'] = total_df['Genotype'].str.count('1') 10.9 s ± 183 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)   With apply():  %%timeit total_df['Count_1'] = total_df['Genotype'].apply(lambda x: x.count('1')) 2.63 s ± 13.8 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)  A significant improvement is obtained using the second approach but, as far as I knew, the apply() method was slower than using pandas built-in methods. What am I missing? I don't know if this is useful but pd.__version__ is 2.0.3.

Answer: The answer is that the str operations are not vectorize as @mozway and @roganjosh mentioned.
Question: This is not a pydantic question, but to explain why am I asking: pydantic.TypeAdapter() accepts (among many others) all the following type definitions as its argument and can create a working validator for them: int int|str list list[str|int] typing.Union[int,str] typing.Literal[10,20,30]  Example: >>> validator = pydantic.TypeAdapter(list[str|int]).validate_python >>> validator([10,20,"stop"]) [10, 20, 'stop'] >>> validator([10,20,None]) (traceback deleted) pydantic_core._pydantic_core.ValidationError: ...  I want to make a test if an argument is a such type defition. How do I write such test?  I started with isinstance(arg, type) for simple types like int or list then I added isinstance(arg, types.GenericAlias for list[str] etc. then I realized it does not recognize int|str (which itself behaves differently than typing.Union[int,str]). Also the Literal[] is not recognized ... I'm probably on a wrong track. 

Answer: I think you can use typing.get_origin combined with isinstance(x, type) like you already mentioned: import typing def defines_type(x): return isinstance(x, type) or typing.get_origin(x) is not None  Testing it out we see our defines_type function returns True for all these: int int|str list list[str|int] typing.Union[int,str] typing.Literal[10,20,30] type(None) Point # here Point = collections.namedtuple('Point', ['x', 'y']) Vector # here Vector = list[float] VectorExplicit # VectorExplicit: typing.TypeAlias = list[float] typing.Optional[int] collections.OrderedDict  and False for all these 1 'a' [10,20,'stop'] None [10,20,None] Point(10, -2) # here Point = collections.namedtuple('Point', ['x', 'y']) typing.Union # Interesting case! not a valid type or type annotation (I assume this behaviour is what you want) {'a', 'b'}
Question: I am trying to draw polygons on a map at arbitrary locations, including places that span the pole and the dateline. Conceptually, consider drawing instrument footprints for orbital measurements, where the corners of the footprints are known in lat/long (or cartesian). I have been able to get mid-latitude polygons to draw, but shapes spanning the pole come up blank. Here is some code to show where I have gotten: from matplotlib.patches import Polygon import matplotlib.pyplot as plt import cartopy.crs as ccrs # https://stackoverflow.com/questions/73689586/ geodetic = ccrs.Geodetic() # Define a polygon over the pole (long, lat) points_north = [ (0, 75), (270, 75), (180, 75), (90, 75), ] # Mid-latitude polygon (long, lat) points_mid = [ (10, 70), (30, 20), (60, 20), (80, 70), ] # Create a PlateCarree projection proj0 = ccrs.PlateCarree() proj0._threshold /= 5. # https://stackoverflow.com/questions/59020032/ ax = plt.axes( projection = proj0 ) # Add the polygons to the plot ax.add_patch( Polygon( points_north, alpha = 0.2, transform = geodetic ) ) ax.add_patch( Polygon( points_mid, alpha = 0.2, transform = geodetic ) ) # Some window dressing longlocs = list( range( -180, 181, 30 ) ) latlocs = [ -75, -60, -30, 0, 30, 60, 75 ] ax.gridlines( xlocs = longlocs, ylocs = latlocs, draw_labels = True, crs = proj0 ) ax.set_global() ax.set_xlabel('Longitude') ax.set_ylabel('Latitude') # Show the plot plt.show()  This produces the following map:  The mid-latitude polygon draws as expected, but the north pole square does not. I would expect that polygon to be smeared across the entire north with scalloped edges, similar to what is shown in the rotated pole boxes example. I have played around with RotatedPole but I really haven't come close to a general solution that lets me plot arbitrary footprints.

Answer: I think that the following is a general solution that is sufficient for my purposes: from matplotlib.patches import Polygon import matplotlib.pyplot as plt import cartopy.crs as ccrs # Test patches are boxes with regular angular width qSz = 40/2 poly = [ ( -qSz, qSz ), ( qSz, qSz ), ( qSz, -qSz ), ( -qSz, -qSz ) ] fig = plt.figure() ax = plt.axes( projection=ccrs.PlateCarree() ) # Patch locations are defined by their centers centerLats = range(-75, 90, 15 ) centerLons = range( 30, 360, 30 ) for centerLat, centerLon in zip( centerLats, centerLons ): rotated_pole = ccrs.RotatedPole( pole_latitude = centerLat - 90, pole_longitude = centerLon ) ax.add_patch( Polygon( poly, transform = rotated_pole, alpha = 0.3 ) ) ax.gridlines( draw_labels = True, crs = ccrs.PlateCarree() ) ax.set_global() plt.show()  This results in the following plot:  The gotcha is that the patches now have to be defined in the rotated frame, so for each of my patches (which I have in cartesian coordinates, including the center) I'll need to compute the corners in that rotated frame, which means that they'll all be small angles bracketing (0,0) like the example. But at least the rotated pole handles the pole and dateline when rotated back to PlateCarree.