eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have this workflow in place which works. I get a token from keycloak with admin username/password to this endpoint auth/realms/master/protocol/openid-connect/token With this token I request about client-secret of a specific client which is connected to another realms, not master. So, I request to this endpoint providing my brand new token in the header as bearer to this endpoint auth/admin/realms/realm_name/clients/client_name/client-secret And I can have client-secret With this client-secret I can get a client token requesting with client credentials to this endpoint auth/realms/realm_name/protocol/openid-connect/token And finally I use this client token to my stuff. I can use python-keycloak to get admin token keycloak_admin = KeycloakAdmin(server_url=url, username='user', password='password', verify=True)  But once I'm here, I cannot have client secret, due to my client is not in admin realm. Using browser with my regular admin, I can change between realms and accessing to other realm clients. And as I said, using a set of request to specific endpoints I can have what I want working, but I don't know how to do it using python-keycloak. Thanks a lot for your help. I guess I've made my self clear enough. Regards

Answer: Maybe I am wrong, but I would expect that the following would work: keycloak_admin = KeycloakAdmin(server_url=serv_url, username='user', password='pass', realm_name='{realm_name}', user_realm_name='master', verify=True)  Get the client client_id = keycloak_admin.get_client_id("{client_name}")  Get the Secret: secret = keycloak_admin.get_client_secrets(client_id)
Question: Does all() return False right after finding a False in a sequence? Try to run this code: def return_true(): print('I have just been printed') return True print(all((False, return_true())))  As you can see, I have just been printed is printed even though there is False before it. Another example: def return_false(): print('I have just been printed') return False print(any((True, return_false())))  In this case, I have just been printed is printed in this code even though there is True before.

Answer: Yes, all() and any() both short circuit the way you describe. all() will return early if any item is false-y, and any() will if any item is truthy. The reason you're seeing the printouts is because return_true() and return_false() are being called before all and any are even invoked. They must be. A function's arguments have to be evaluated before the function is called, after all. This:  print(all((False, return_true())))   is equivalent to: x = return_true() print(all((False, x)))  i.e. return_true() is evaluated unconditionally. To get the desired short-circuiting behavior, the sequence itself needs to be evaluated lazily. One simple way to do this is to make an iterable, not of the values we want to test, but of things we can call to get those values; and then use a generator expression to create a sequence that lazily calls them. See also. Here, that might look like: print(all( x() for x in (lambda: False, return_true) )) print(any( x() for x in (lambda: True, return_false) ))
Question: I uploaded a python package on Pypi, but I'd also like to upload it to Github, so it can be opensource and anyone can contribute. Is is possible to link the github repo with the already uploaded package on Pypi, so whenever I push something to the master branch, it also updates on Pypi?

Answer: In your Github repository there is a tab called Actions (next to Pull requests) where there are several actions like "Publish Python Package".  Selecting it will automatically add the relevant Code to your repository. You then only need to store your credentials, like username & password. You can do so under Settings > Secrets > Actions
Question: Based on the example from the Aggregation section of the User Guide import polars as pl from datetime import date def compute_age() -> pl.Expr: return date(2021, 1, 1).year - pl.col("birthday").dt.year() def avg_birthday(gender: str) -> pl.Expr: return compute_age().filter( pl.col("gender") == gender ).mean().alias(f"avg {gender} birthday") dataset = pl.read_csv(b""" state,gender,birthday GA,M,1861-06-06 MD,F,1920-09-17 PA,M,1778-10-13 KS,M,1926-02-23 CO,M,1959-02-16 IL,F,1937-08-15 NY,M,1803-04-30 TX,F,1935-12-03 MD,M,1756-06-03 OH,M,1786-11-15 """.strip(), try_parse_dates=True).lazy() q = ( dataset .group_by("state") .agg( avg_birthday("M"), avg_birthday("F"), (pl.col("gender") == "M").count().alias("# male"), (pl.col("gender") == "F").sum().alias("# female"), ) )  The result is inconsistent. For example, the first time I run q.collect().head() shape: (5, 5) ┌───────┬────────────────┬────────────────┬────────┬──────────┐ │ state ┆ avg M birthday ┆ avg F birthday ┆ # male ┆ # female │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ f64 ┆ f64 ┆ u32 ┆ u32 │ ╞═══════╪════════════════╪════════════════╪════════╪══════════╡ │ GA ┆ 160.0 ┆ null ┆ 1 ┆ 0 │ │ OH ┆ 235.0 ┆ null ┆ 1 ┆ 0 │ │ CO ┆ 62.0 ┆ null ┆ 1 ┆ 0 │ │ KS ┆ 95.0 ┆ null ┆ 1 ┆ 0 │ │ TX ┆ null ┆ 86.0 ┆ 1 ┆ 1 │ └───────┴────────────────┴────────────────┴────────┴──────────┘  The second time: shape: (5, 5) ┌───────┬────────────────┬────────────────┬────────┬──────────┐ │ state ┆ avg M birthday ┆ avg F birthday ┆ # male ┆ # female │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ str ┆ f64 ┆ f64 ┆ u32 ┆ u32 │ ╞═══════╪════════════════╪════════════════╪════════╪══════════╡ │ TX ┆ null ┆ 86.0 ┆ 1 ┆ 1 │ │ OH ┆ 235.0 ┆ null ┆ 1 ┆ 0 │ │ CO ┆ 62.0 ┆ null ┆ 1 ┆ 0 │ │ NY ┆ 218.0 ┆ null ┆ 1 ┆ 0 │ │ GA ┆ 160.0 ┆ null ┆ 1 ┆ 0 │ └───────┴────────────────┴────────────────┴────────┴──────────┘  I guess it may be caused by parallelization? Is this a bug or a feature? How to keep the result consistent?

Answer: Use maintain_order=True on group_by.  maintain_order: Ensure that the order of the groups is consistent with the input data. This is slower than a default group by. Setting this to True blocks the possibility to run on the streaming engine.
Question: I have made a small space invader game using pygame and I was wondering if I could play it on the browser using pyscript. Is this even possible ? Do I have to rewrite everything ?

Answer: No, Pygame is not supported in PyScript at this time. I'm not sure what is the best way to find out what packages are supported, but I have managed to piece together the following:  PyScript uses Pyodide to load packages, so only packages supported by Pyodide will be loadable in PyScript. That means either packages built with Pyodide or pure Python packages with wheels available on PyPI or from other URLs. Pygame is not yet supported by Pyodide. Edit: Pygame has been merged in Pyodide and will be available in a future release.  You can use the following test script to see if a package is supported: <html> <head> <link rel="stylesheet" href="https://pyscript.net/alpha/pyscript.css" /> <script defer src="https://pyscript.net/alpha/pyscript.js"></script> <py-env> - pygame </py-env> </head> <body> <h1>PyScript import test script</h1> <py-script> import pygame </py-script> </body> </html>  This is basically just a "try it and see what happens" script that will throw an error in the console if a package is not supported. The error you'll see will be  ValueError: Couldn't find a pure Python 3 wheel for 'pygame'. You can use micropip.install(..., keep_going=True) to get a list of all packages with missing wheels.
Question: I would like to have my Python script run a Linux shell command and store the output in a variable, without the command's output being shown to the user. I have tried this with os.system, subprocess.check_output, subprocess.run, subprocess.Popen, and os.popen with no luck. My current method is running os.system("ls -l &> /tmp/test_file") so the command stdout and stderr are piped to /tmp/test_file, and then I have my Python code read the file into a variable and then delete it. Is there a better way of doing this so that I can have the command output sent directly into the variable without having to create and delete a file, but keep it hidden from the user?

Answer: You can use the subprocess.run function.  One update as @Ashley Kleynhans say  "The results of stdout and stderr are bytes objects so you will need to decode them if you want to handle them as strings"  For this, you do not need to decode stdout or stderr because in the run method you can pass one more argument to get the return data as a string, which is text=True  "If used it must be a byte sequence, or a string if encoding or errors is specified or text is true" - from Documentation   from subprocess import run data = run("ANY COMMAND HERE", capture_output=True, shell=True, text=True) print(data.stdout) # If you want you can save it to a variable print(data.stderr) # ^^
Question: I have a column in my Used cars price prediction dataset named "Owner_Type". It has four unique values which are ['First', 'Second', 'Third', 'Fourth']. Now the order that makes the most sense is First > Second > Third > Fourth as the price decreases with respect to this order. How can I give this order to the values with OrdinalEncoder()? Please help me, thank you!

Answer: OrdinalEncoder has a categories parameter which accepts a list of arrays of categories. Here is a code example: from sklearn.preprocessing import OrdinalEncoder enc = OrdinalEncoder(categories=[['first','second','third','forth']]) X = [['third'], ['second'], ['first']] enc.fit(X) print(enc.transform([['second'], ['first'], ['third'],['forth']]))
Question: I am building a Python module with a structure like: mypackage/ mypackage/ __init__.py etc.py setup.py setup.cfg pyproject.toml  To build it, I am running $ python -m build. I noticed that version numbers weren't available (e.g. mypackage.__version__ is undefined after installing), and currently I am just setting it manually like: setup.py setup(..., version='0.0.1' )  pyproject.toml [project] version = '0.0.1'  I am new to Python package development and there are a few posts on this, but there does not seem to be a standard way of doing it. The package is quite small and ideally I'd like to just update one thing like __version__ = '0.0.1' inside __init__.py, and then have this parsed automatically in setup.py and pyproject.toml.

Answer: Do you really need/want a __version__ attribute at all? Keeping a __version__ attribute available in the module namespace is a popular convention, but it's possibly falling out of fashion these days because stdlib importlib.metadata is no longer provisional. The one obvious place for a version string is in the package metadata, duplicating that same information in a module attribute may be considered unnecessary and redundant. It also presents some conundrums for users when the version string exists in two different places - where should we look for it first, in the package metadata or in a module top-level namespace? And which one should we trust if the version information found in each of these places is different? So, there is some benefit to only storing it in one place, and that place must be the package's metadata. This is because the Version field is a required field in the Core metadata specifications, but packages which opt-in to providing a __version__ attribute are just following a convention. Getting/setting the version in package metadata If you're using a modern build system, then you would specify the version string directly in pyproject.toml as described in PEP 621 – Storing project metadata in pyproject.toml. The way already shown in the question is correct: [project] name = "mypkg" version = "0.0.1"  Users of mypkg could retrieve the version like so: from importlib.metadata import version version("mypkg")  Note that unlike accessing a __version__ attribute, this version is retrieved from the package metadata only, and the actual package doesn't even need to be imported. That's useful in some cases, e.g. packages which have import side-effects such as numpy, or the ability to retrieve versions of packages even if they have unsatisfied dependencies / complicated environment setup requirements. What if you want access to the version number within the package itself? Sometimes this is useful, for example to add a --version option to your command-line interface. But this doesn't imply you need a __version__ attribute hanging around, you can just retrieve the version from your own package metadata the same way: parser = argparse.ArgumentParser(...) ... parser.add_argument( "--version", action="version", version=importlib.metadata.version("mypkg"), )
Question: I don't understand the difference when hinting Iterable and Sequence. What is the main difference between those two and when to use which? I think set is an Iterable but not Sequence, are there any built-in data type that is Sequence but not Iterable? def foo(baz: Sequence[float]): ... # What is the difference? def bar(baz: Iterable[float]): ... 

Answer: The Sequence and Iterable abstract base classes (can also be used as type annotations) mostly* follow Python's definition of sequence and iterable. To be specific:  Iterable is any object that defines __iter__ or __getitem__. Sequence is any object that defines __getitem__ and __len__. By definition, any sequence is an iterable. The Sequence class also defines other methods such as __contains__, __reversed__ that calls the two required methods.  Some examples:  list, tuple, str are the most common sequences. Some built-in iterables are not sequences. For example, reversed returns a reversed object (or list_reverseiterator for lists) that cannot be subscripted.   * Iterable does not exactly conform to Python's definition of iterables — it only checks if the object defines __iter__, and does not work for objects that's only iterable via __getitem__ (see this table for details). The gold standard of checking if an object is iterable is using the iter builtin.
Question: I've been figuring out how to parse tkinter events via event.state to reduce the number of times that I have to call root.bind() (e.g., I can avoid binding both "<ButtonPress-1>" and "<Shift-ButtonPress-1>" by finding if shift was pressed via event.state). Of course, I've relied heavily on the tkinter source code (specifically the definition for __repr__, starting on line 234) to convert the integer of event.state to something I can understand: def getStatefromInt(state_int): # from https://github.com/python/cpython/blob/3.8/Lib/tkinter/__init__.py if isinstance(state_int, int): state = state_int mods = ('Shift', 'Lock', 'Control', 'Mod1', 'Mod2', 'Mod3', 'Mod4', 'Mod5', 'Button1', 'Button2', 'Button3', 'Button4', 'Button5') s = [] for i, n in enumerate(mods): if state & (1 << i): s.append(n) state = state & ~((1<< len(mods)) - 1) if state or not s: s.append(hex(state)) return s  One of the things that keeps coming up out of state when events occur is Mod1. What do Mod1 and the other ModX states represent? I thought the number might correspond to the type of button press, but all types of mouse clicks cause only Mod1. I have not been able to find information on what this means online, and I'm having a hard time seeing from the source code what it might mean.

Answer: ModX represents a modification, a Modifier Key.  In computing, a modifier key is a special key (or combination) on a computer keyboard that temporarily modifies the normal action of another key when pressed together. By themselves, modifier keys usually do nothing; that is, pressing any of the ⇧ Shift, Alt, or Ctrl keys alone does not (generally) trigger any action from the computer.  Tkinter is a crossplatform GUI-Toolkit and uses system specific facilities. Different OS using different methods, for example a PC that runs Linux signals Alt Gr as Mod 5 while a PC running Windows signals the same keystroke as Control-Alt. You can look at the Modifier Keys on tcl-lang.ord. Event.state is a bit mask indicating which of certain modifier keys and mouse buttons were down or active when the event triggered and is not reliable because of the system specific facilities as pointed out here.
Question: How can I skip the page and automatically logged in users, when they clicked on Login in With Google. 

Answer: You need to set up SOCIALACCOUNT_LOGIN_ON_GET=True in your configuration (by default it's False). according to https://django-allauth.readthedocs.io/en/latest/configuration.html:  SOCIALACCOUNT_LOGIN_ON_GET (=False) Controls whether or not the endpoints for initiating a social login (for example, “/accounts/google/login/”) require a POST request to initiate the handshake. For security considerations, it is strongly recommended to require POST requests.
Question: When we are coding python code, we typically use packages and modules that we import. For example, when we are coding we may write: import numpy import requests from bs4 import BeautifulSoup  When we are trying to integrate python with html with Pyscript (https://pyscript.net/), it just says that it doesn’t have the package installed. However, when this happens in normal python we use PiP and import it from there. However, what should we do when we need a package in Pyscript? Thank you!

Answer: At this time, bs4 is not supported. You will receive an error  ValueError: Couldn't find a pure Python 3 wheel for 'bs4'  You will also have problems using the requests package in pyscript. Usepyfetch instead of requests.get. To import numpy and requests, use <py-env> before <py-script>. Example: <body> <py-env> - numpy </py-env> <py-script> import numpy as np print(np.random.randn(10, 4)) </py-script> </body>  Pyscript also supports package versions:  <py-env> - numpy==1.22.3 </py-env>
Question: Given the scan of a fabric (snippet included here, it could easily be A4 size at 600 dpi) what would be the best method for finding the repetition pattern in the scan? I have tried:  splitting the image in 4 quarters and trying to find points via SIFT and OpenCV FFT as suggested here  I am aware of other answers on stackoverflow and other sites (this and this and this but they tend to be a bit too terse for an OpenCV beginner. I am thinking of eyeballing an area and optimizing via row-by-row and column-by-column comparison of pixels, but I am wondering if there is a another better path. 

Answer: The image's 2D autocorrelation is a good generic way to find repeating structures, as others have commented. There are some details to doing this effectively:  For this analysis, it is often fine to just convert the image to grayscale; that's what the code snippet below does. To extend this to a color-aware analysis, you could compute the autocorrelation for each color channel and aggregate the results. It helps to apply a window function first, to avoid boundary artifacts. For efficient computation, it helps to compute the autocorrelation through FFTs. Rather than use the whole spectrum in computing the autocorrelation, it helps to zero out very low frequencies, since these are irrelevant for texture. Rather than the plain autocorrelation, it helps to partially "equalize" or "whiten" the spectrum, as suggested by the Generalized Cross Correlation with Phase Transform (GCC-PHAT) technique.  Python code: # Copyright 2022 Google LLC. # SPDX-License-Identifier: Apache-2.0 from PIL import Image import matplotlib.pyplot as plt import numpy as np import scipy.signal image = np.array(Image.open('texture.jpg').convert('L'), float) # Window the image. window_x = np.hanning(image.shape[1]) window_y = np.hanning(image.shape[0]) image *= np.outer(window_y, window_x) # Transform to frequency domain. spectrum = np.fft.rfft2(image) # Partially whiten the spectrum. This tends to make the autocorrelation sharper, # but it also amplifies noise. The -0.6 exponent is the strength of the # whitening normalization, where -1.0 would be full normalization and 0.0 would # be the usual unnormalized autocorrelation. spectrum *= (1e-12 + np.abs(spectrum))**-0.6 # Exclude some very low frequencies, since these are irrelevant to the texture. fx = np.arange(spectrum.shape[1]) fy = np.fft.fftshift(np.arange(spectrum.shape[0]) - spectrum.shape[0] // 2) fx, fy = np.meshgrid(fx, fy) spectrum[np.sqrt(fx**2 + fy**2) < 10] = 0 # Compute the autocorrelation and inverse transform. acorr = np.real(np.fft.irfft2(np.abs(spectrum)**2)) plt.figure(figsize=(10, 10)) plt.imshow(acorr, cmap='Blues', vmin=0, vmax=np.percentile(acorr, 99.5)) plt.xlim(0, image.shape[1] / 2) plt.ylim(0, image.shape[0] / 2) plt.title('2D autocorrelation', fontsize=18) plt.xlabel('Horizontal lag (px)', fontsize=15) plt.ylabel('Vertical lag (px)', fontsize=15) plt.show()  Output:  The period of the flannel texture is visible at the circled point at 282 px horizontally and 290 px vertically.
Question: I am trying to insert data into a mssql database. I needed as fast method for this so I set the fast_executemany param to true. The upload works fine for most part but if one of the column is a datetime with timezone it crashes raising: (pyodbc.Error) ('HY104', '[HY104] [Microsoft][ODBC Driver 17 for SQL Server]Invalid precision value (0) (SQLBindParameter)') If I insert the same data with fast_executemany as False then everythhing works perfectly. Did anyone came across a similar problem or know what might be the issue? Sample code from sqlalchemy import create_engine, engine import pytz import pandas as pd from sqlalchemy.dialects.mssql import DATETIMEOFFSET import datetime engine_url = engine.URL.create( drivername='mssql', username='admin', password='**', host='**', port='1433', database='mytestdb1', query={'driver': "ODBC Driver 17 for SQL Server"} ) mssql_engine = create_engine(engine_url, echo=False, fast_executemany=True) base = datetime.datetime.today().replace(tzinfo=pytz.utc) date_list = [base - datetime.timedelta(days=x) for x in range(20)] df = pd.DataFrame(date_list, columns = ['date_time']) df.to_sql('test_insert', mssql_engine, schema='testschema1', if_exists='replace', dtype = {'date_time':DATETIMEOFFSET})  response: DBAPIError: (pyodbc.Error) ('HY104', '[HY104] [Microsoft][ODBC Driver 17 for SQL Server]Invalid precision value (0) (SQLBindParameter)') [SQL: INSERT INTO testschema1.test_datetime ([index], date_time) VALUES (?, ?)] [parameters: ((0, '2022-05-06 16:40:05.434984 +00:00'), (1, '2022-05-05 16:40:05.434984 +00:00'), (2, '2022-05-04 16:40:05.434984 +00:00'), (3, '2022-05-03 16:40:05.434984 +00:00'), (4, '2022-05-02 16:40:05.434984 +00:00'), (5, '2022-05-01 16:40:05.434984 +00:00'), (6, '2022-04-30 16:40:05.434984 +00:00'), (7, '2022-04-29 16:40:05.434984 +00:00') ... displaying 10 of 20 total bound parameter sets ... (18, '2022-04-18 16:40:05.434984 +00:00'), (19, '2022-04-17 16:40:05.434984 +00:00'))] (Background on this error at: http://sqlalche.me/e/14/dbapi)  sqlalchemy==1.4, pyodbc==4.0.32 and pandas==1.2.0 As I said the code works perfectly if I dont use fast_executemany.

Answer: This issue can be reproduced using SQLAlchemy 1.4.0. It was fixed in SQLAlchemy 1.4.1.