eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have a string like below in python testing_abc  I want to split string based on _ and extract the 2 element I have done like below split_string = string.split('_')[1]  I am getting the correct output as expected abc  Now I want this to work for below strings 1) xyz  When I use split_string = string.split('_')[1]  I get below error list index out of range  expected output I want is xyz 2) testing_abc_bbc  When I use split_string = string.split('_')[1]  I get abc as output expected output I want is abc_bbc Basically What I want is 1) If string contains `_` then print everything after the first `_` as variable 2) If string doesn't contain `_` then print the string as variable  How can I achieve what I want

Answer: Set the maxsplit argument of split to 1 and then take the last element of the resulting list. >>> "testing_abc".split("_", 1)[-1] 'abc' >>> "xyz".split("_", 1)[-1] 'xyz' >>> "testing_abc_bbc".split("_", 1)[-1] 'abc_bbc'
Question: If I do this for example: cv2.getTextSize('blahblah', cv2.FONT_HERSHEY_SIMPLEX, 2, 2) it returns ((262, 43), 19) so the width and height of the text in pixels are 262 and 43, but what is the 19? Here it says it "corresponds to the y coordinate of the baseline relative to the bottom of the text" but this still doesn't make it clear to me, as I'm not sure what the "baseline" is here?

Answer: The baseline here is the yellow line in the figure on page 124. It is the line on which the letters sit. That is according to  Pay attention to how the three little points (red,cyan, and green) are drawn and also to how the yellow baseline is shown.
Question: Here is my (simplified) Dockerfile # https://docs.aws.amazon.com/lambda/latest/dg/images-create.html#images-create-from-base FROM public.ecr.aws/lambda/python:3.8 # get the amazon linux extras RUN yum install -y amazon-linux-extras RUN amazon-linux-extras install epel -y  When it reaches the RUN amazon-linux-extras install epel -y line during the build, it gets Step 6/8 : RUN amazon-linux-extras install epel -y ---> Running in dbb44f57111a /var/lang/bin/python: No module named amazon_linux_extras The command '/bin/sh -c amazon-linux-extras install epel -y' returned a non-zero code: 1  I think that has to do with some python 2 vs. 3 stuff, but I'm not sure

Answer: You're correct, it's because amazon-linux-extras only works with Python 2. You can modify the RUN instruction to RUN PYTHON=python2 amazon-linux-extras install epel -y
Question: I have two lists: A: [[0, 1], [2, [3]], 4] B: [5, 6, 7, 8, 9] I wish list B could have the same shape with list A: [5, 6, 7, 8, 9] => [[5, 6], [7, [8]], 9] So list A and list B have the same dimension/shape: A: [[0, 1], [2, [3]], 4] B: [[5, 6], [7, [8]], 9] Consider about time complexity, I hope there is a way of O(n) if possible.

Answer: Assuming the number of items is identical, you could use a recursive function and an iterator: A = [[0, 1], [2, [3]], 4] B = [5, 6, 7, 8, 9] def copy_shape(l, other): if isinstance(other, list): other = iter(other) if isinstance(l, list): return [copy_shape(x, other) for x in l] else: return next(other) out = copy_shape(A, B)  output: [[5, 6], [7, [8]], 9] NB. the complexity is O(n). You can also use if hasattr(other, '__len__') or if not hasattr(other, '__next__') in place of if isinstance(other, list) to generalize to other iterables (except iterator).
Question: I have a dataframe as shown below: df = A col_1 col_45 col_3 1.0 4.0 45.0 [1, 9] 2.0 4.0 NaN [9, 10] 3.0 49.2 10.8 [1, 10]  The values in col_1 are of type float and the values in col_3 are in a list. For every row, I want to extract the values in col_1 and col_3 and put it together in a list. I tried the following: df[['col_1','col_3']].astype(float).values.tolist()  But it threw me a Value error: ValueError: setting an array element with a sequence.. I would like to have a list as follows: [[4.0,1.0,9.0], [4.0,9.0,10.0], [49.2,1.0,10.0]]  Is there a way to do this? Thanks.

Answer: Convert one element in col_1 to list then use merge two list like list_1 + list_2, You can use pandas.apply with axis=1 for iterate over each row: >>> df.apply(lambda row: [row['col_1']] + row['col_3'], axis=1) 0 [4.0, 1, 9] 1 [4.0, 9, 10] 2 [49.2, 1, 10] dtype: object >>> df.apply(lambda row: [row['col_1']] + row['col_3'], axis=1).to_list() [ [4.0, 1, 9], [4.0, 9, 10], [49.2, 1, 10] ]
Question: I have a data set which has a column that looks like this Badge Number 1 3 23 / gold 22 / silver 483  I need only the numbers. Here's my code: df = pd.read_excel('badges.xlsx') df['Badge Number'] = df['Badge Number'].str.extract('(\d+)') print(df)  I was expecting an output like: Badge Number 1 3 23 22 483  but I got Badge Number Nan Nan 23 22 Nan  Just to test, I dumped the dataframe to a .csv and read it back with pd.read_csv(). That gave me just the numbers, as I need (though of course that's not a solution) I also tried df['Badge Number'] = np.where(df['Badge Number'].str.isnumeric(), df['Badge Number'], df['Badge Number'].str.extract('(\d+)'))  but that just gave me all 1s. I know I am trying things I don't even remotely understand, but am hoping there's a straightforward solution.

Answer: Another option is while reading the XLS it self, specify your column to string. use dtype={'Badge Number': str} df = pd.read_excel('badges.xlsx',dtype={'Badge Number': str}) df['Badge Number'] = df['Badge Number'].str.extract('(\\d+)')
Question: I am dealing with a large dataframe (198,619 rows x 19,110 columns) and so am using the polars package to read in the tsv file. Pandas just takes too long. However, I now face an issue as I want to transform each cell's value x raising it by base 2 as follows: 2^x. I run the following line as an example: df_copy = df df_copy[:,1] = 2**df[:,1]  But I get this error: --------------------------------------------------------------------------- ValueError Traceback (most recent call last) /var/tmp/pbs.98503.hn-10-03/ipykernel_196334/3484346087.py in <module> 1 df_copy = df ----> 2 df_copy[:,1] = 2**df[:,1] ~/.local/lib/python3.9/site-packages/polars/internals/frame.py in __setitem__(self, key, value) 1845 1846 # dispatch to __setitem__ of Series to do modification -> 1847 s[row_selection] = value 1848 1849 # now find the location to place series ~/.local/lib/python3.9/site-packages/polars/internals/series.py in __setitem__(self, key, value) 512 self.__setitem__([key], value) 513 else: --> 514 raise ValueError(f'cannot use "{key}" for indexing') 515 516 def estimated_size(self) -> int: ValueError: cannot use "slice(None, None, None)" for indexing  This should be simple but I can't figure it out as I'm new to Polars.

Answer: The secret to harnessing the speed and flexibility of Polars is to learn to use Expressions. As such, you'll want to avoid Pandas-style indexing methods. Let's start with this data: import polars as pl nbr_rows = 4 nbr_cols = 5 df = pl.DataFrame({ "col_" + str(col_nbr): pl.int_range(col_nbr, nbr_rows + col_nbr, eager=True) for col_nbr in range(0, nbr_cols) }) df  shape: (4, 5) ┌───────┬───────┬───────┬───────┬───────┐ │ col_0 ┆ col_1 ┆ col_2 ┆ col_3 ┆ col_4 │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 │ ╞═══════╪═══════╪═══════╪═══════╪═══════╡ │ 0 ┆ 1 ┆ 2 ┆ 3 ┆ 4 │ │ 1 ┆ 2 ┆ 3 ┆ 4 ┆ 5 │ │ 2 ┆ 3 ┆ 4 ┆ 5 ┆ 6 │ │ 3 ┆ 4 ┆ 5 ┆ 6 ┆ 7 │ └───────┴───────┴───────┴───────┴───────┘  In Polars we would express your calculations as: df_copy = df.select(pl.lit(2).pow(pl.all()).name.keep()) print(df_copy)  shape: (4, 5) ┌───────┬───────┬───────┬───────┬───────┐ │ col_0 ┆ col_1 ┆ col_2 ┆ col_3 ┆ col_4 │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ f64 ┆ f64 ┆ f64 │ ╞═══════╪═══════╪═══════╪═══════╪═══════╡ │ 1.0 ┆ 2.0 ┆ 4.0 ┆ 8.0 ┆ 16.0 │ │ 2.0 ┆ 4.0 ┆ 8.0 ┆ 16.0 ┆ 32.0 │ │ 4.0 ┆ 8.0 ┆ 16.0 ┆ 32.0 ┆ 64.0 │ │ 8.0 ┆ 16.0 ┆ 32.0 ┆ 64.0 ┆ 128.0 │ └───────┴───────┴───────┴───────┴───────┘
Question: It's unclear from the docs what you actually do to verify the jwsRepresentation string from a StoreKit 2 transaction on the server side. Also "signedPayload" from the Apple App Store Notifications V2 seems to be the same, but there is also no documentation around actually validating that either outside of validating it client side on device. What gives? What do we do with this JWS/JWT?

Answer: Apple now provides an App Store Server Library available in a few different languages (including Python). You still need to manage loading the root certificates on your own, but here is an example: from functools import from appstoreserverlibrary.models.Environment import Environment from appstoreserverlibrary.signed_data_verifier import VerificationException, SignedDataVerifier @lru_cache(maxsize=None) def _load_apple_root_certificates(): # https://www.apple.com/certificateauthority/ certs_urls = [ "https://www.apple.com/appleca/AppleIncRootCertificate.cer", "https://www.apple.com/certificateauthority/AppleComputerRootCertificate.cer", "https://www.apple.com/certificateauthority/AppleRootCA-G2.cer", "https://www.apple.com/certificateauthority/AppleRootCA-G3.cer", ] return [requests.get(cert_url).content for cert_url in certs_urls] root_certificates = _load_apple_root_certificates() enable_online_checks = True bundle_id = "com.example" environment = Environment.SANDBOX app_apple_id = None # appAppleId must be provided for the Production environment signed_data_verifier = SignedDataVerifier(root_certificates, enable_online_checks, environment, bundle_id, app_apple_id) try: signed_notification = "ey.." payload = signed_data_verifier.verify_and_decode_notification(signed_notification) print(payload) except VerificationException as e: print(e)
Question: I've run into a problem after upgrades of my pip packages and my bigquery connector that returns query results suddenly stopped working with following error message from google.cloud import bigquery from google.oauth2 import service_account credentials = service_account.Credentials.from_service_account_file('path/to/file', scopes=['https://www.googleapis.com/auth/cloud-platform', 'https://www.googleapis.com/auth/drive', 'https://www.googleapis.com/auth/bigquery' ]) client = bigquery.Client(credentials=credentials) data = client.query('select * from dataset.table').to_dataframe()   PermissionDenied: 403 request failed: the user does not have bigquery.readsessions.create' permission  But! If you switched the code to data = client.query('select * from dataset.table').result()  (dataframe -> result) you received the data in RowIterator format and were able to properly read them. The same script using to_dataframe with the same credentials was working on the server. Therefore I set my bigquery package to the same version 2.28.0, which still did not help. I could not find any advices on this error / topic anywhere, so I just want to share if any of you faced the same thing.

Answer: There are different ways of receiving data from bigquery. Using the BQ Storage API is considered more efficient for larger result sets compared to the other options:  The BigQuery Storage Read API provides a third option that represents an improvement over prior options. When you use the Storage Read API, structured data is sent over the wire in a binary serialization format. This allows for additional parallelism among multiple consumers for a set of results  The Python BQ library internally determines whether it can use the BQ Storage API or not. For the result method, it uses the tradtional tabledata.list method internally, whereas the to_dataframe method uses the BQ Storage API if the according package is installed. However, using the BQ Storage API requires you to have the bigquery.readSessionUser Role respectively the readsessions.create right which in your case seems to be lacking. By uninstalling the google-cloud-bigquery-storage, the google-cloud-bigquery package was falling back to the list method. Hence, by de-installing this package, you were working around the lack of rights. See the BQ Python Libary Documentation for details.
Question: I'm a Java Spring boot developer and I develop 3-tier crud applications. I talked to a guy who seemed knowledgeable on the subject, but I didn't get his contact details. He was advocating for Python's FastAPI, because horizontally it scales better than Spring boot. One of the reasons he mentioned is that FastAPI is single-threaded. When the thread encounters a database lookup (or other work the can be done asyncly), it picks up other work to later return to the current work when the database results have come in. In Java, when you have many requests pending, the thread pool may get exhausted. I don't understand this reasoning a 100%. Let me play the devil's advocate. When the Python program encounters an async call, it must somehow store the program pointer somewhere, to remember where it needs to continue later. I know that that place where the program pointer is stored is not at all a thread, but I have to give it some name, so let's call it a "logical thread". In Python , you can have many logical threads that are waiting. In Java, you can have a thread pool with many real threads that are waiting. To me, the only difference seems to be that Java's threads are managed at the operating system level, whereas Python's "logical threads" are managed by Python or FastAPI. Why are real threads that are waiting in a thread pool so much more expensive than logical threads that are waiting? If most of my threads are waiting, why can't I just increase the thread pool size to avoid exhaustion?

Answer: The issues with Java threads in the question are addressed by the project Loom, which is now included in Jdk21. It is very well explained here https://www.baeldung.com/openjdk-project-loom :  Presently, Java relies on OS implementations for both the continuation [of threads] and the scheduler [for threads]. Now, in order to suspend a continuation, it's required to store the entire call-stack. And similarly, retrieve the call-stack on resumption. Since the OS implementation of continuations includes the native call stack along with Java's call stack, it results in a heavy footprint. A bigger problem, though, is the use of OS scheduler. Since the scheduler runs in kernel mode, there's no differentiation between threads. And it treats every CPU request in the same manner. (...) For example, consider an application thread which performs some action on the requests and then passes on the data to another thread for further processing. Here, it would be better to schedule both these threads on the same CPU. But since the [OS] scheduler is agnostic to the thread requesting the CPU, this is impossible to guarantee.  The question really boils down to Why are OS threads considered expensive?
Question: I'm working with FastAPI and Python on the backend to make external calls to a public API. After authentication, the public API gives an access token that grants access to a specific user's data. Where would be the best place to store/save this access token? I want to easily access it for all my future API calls with the public API service. I don't want a DB or long term storage as it only needs to last the session for the user. Appreciate all help!

Answer: Almost a year later, but I found a clean solution I was pleased with. I used Starlette's SessionMiddleware to store the access_token and user session data in the backend. Example: from fastapi import Request ... @router.get("/callback") async def callback(request: Request): ... request.session["access_token"] = access_token  Then later, in any endpoints where I need to use the token or get session data: @router.get("/top_artists") async def get_top_songs(request: Request): ... access_token = request.session.get("access_token")  This stores access_token and any other session data you want on the backend. Then, a cookie, 'session_id', is stored client-side and passed through Request to retrieve the session data from the server.
Question: I'm confused as why OpenTelemetry documentaion has OpenTelemetry Python API and OpenTelemetry Python SDK. Like when using the specification in python when we should consider pip install opentelemetry-api over pip install opentelemetry-sdk

Answer: There's no need to specify both the API and SDK dependencies in Python as the SDK has a dependency on the API: https://github.com/open-telemetry/opentelemetry-python/blob/main/opentelemetry-sdk/pyproject.toml#L29 The short answer for when to use each is that the API should be specified as a dependency for packages/libraries and the SDK for applications. The reason for this is that API contains the interface for instrumenting code with traces, metrics and logs. The SDK contains the additional functionality to collect, process and export that data. For example, if I wanted to include instrumentation in a PyPI package that I've created, I would add the API as a dependency and use that to instrument my code. Then if someone were to use my PyPI package in their application, they could include the instrumentation data that my package generates. However, they would have to install the SDK in order to configure their application to do anything with that telemetry data other than simply generate it, e.g. sending it to an OpenTelemetry collector or third party.
Question: I can't find the difference between the mlrose (https://pypi.org/project/mlrose/) project and mlrose-hiive (https://pypi.org/project/mlrose-hiive/). I know hiive has some kind of extensions compared to the original mlrose but I can't find some documentation or anything that explains the new features.

Answer: Accordign private training forum info where I have access, it seems mlrose-hiive:  mostly are improvements and fixes some dependency bugs, as in the six package: Python import error: cannot import name 'six' from 'sklearn.externals', is backwards-compatible, so, mlrose readthedocs should be OK, they even recommend that in the project repo readme (https://github.com/hiive/mlrose).  Also: https://pypi.org/project/mlrose-hiive/ Therefore, documentation in https://mlrose.readthedocs.io/en/stable/
Question: I am working with postgres. I want to fetch multiple fields using array_agg in sqlalchemy. But I couldn't find examples of such use anywhere. I made my request. But I can't process the result of array_agg. I'd like to get a list of strings, or better yet a list of tuples. It would also be nice to get rid of func.distinct, it's only needed because I can't write it like this: func.array_agg((Task.id, Task.user_id)) My query: data = session.query( Status.id, func.array_agg(func.distinct(Task.id, Task.user_id), type_=TEXT) ).join(Task).group_by(Status.id).limit(5).all()  I got: (100, '{"(91,1)","(92,1)","(93,1)","(94,1)"}') (200, '{"(95,1)","(96,1)","(97,1)","(98,1)","(99,1)"}')  But I want: (100, ["(91,1)","(92,1)","(93,1)","(94,1)"]) (200, ["(95,1)","(96,1)","(97,1)","(98,1)","(99,1)"])  Or better: (100, [(91,1),(92,1),(93,1),(94,1)]) (200, [(95,1),(96,1),(97,1),(98,1),(99,1)])  I try also: func.array_agg(func.distinct(Task.id, Task.user_id), type_=ARRAY(TEXT))  I got: (100, ['{', '"', '(', '9', '1', ',', '1', ')', '"', ',', '"', '(', '9', '2', ',', '1', ')', '"', ',', '"', '(', '9', '3', ',', '1', ')', '"', ',', '"', '(', '9', '4', ',', '1', ')', '"', '}']) (200, ['{', '"', '(', '9', '5', ',', '1', ')', '"', ',', '"', '(', '9', '6', ',', '1', ')', '"', ',', '"', '(', '9', '7', ',', '1', ')', '"', ',', '"', '(', '9', '8', ',', '1', ')', '"', ',', '"', '(', '9', '9', ',', '1', ')', '"', '}']) 

Answer: The problem here is that Postgresql's array_agg function is returning an array of unknown type; the default behaviour of the psycopg2 connector in this situation is to simply return the array literal as-is. This bug report exists from 2016. SQLAlchemy's maintainer, SO user zzzeek proposed creating a custom type to handle this case. I have modified the solution slightly to convert the tuple elements to integers, and to work with v1.4: import re from sqlalchemy.types import TypeDecorator class ArrayOfRecord(TypeDecorator): impl = sa.String # cache_ok = True seems to work, but I haven't tested extensively cache_ok = True def process_result_value(self, value, dialect): elems = re.match(r"^\{(\".+?\")*\}$", value).group(1) elems = [e for e in re.split(r'"(.*?)",?', elems) if e] return [tuple( map(int, re.findall(r'[^\(\),]+', e)) ) for e in elems]  Using it like this: with Session() as session: data = ( session.query( Status.id, sa.func.array_agg( sa.func.ROW(Task.id, Task.user_id), type_=ArrayOfRecord ).label('agg') ) .join(Task) .group_by(Status.id) ) print() for row in data: print(row) print()  outputs (100, [(91, 1), (92, 1), (93, 1), (94, 1)]) (200, [(95, 1), (96, 1), (97, 1), (98, 1)])
Question: this image is the diagram for GNS3 of routers want to configureTrying to Backup the configuration of a Cisco Router. but the connection is not opening.  from napalm import * import napalm drivers = napalm.get_network_driver('ios') device_detail = {'hostname':'192.168.1.2','username':'wahid','password':'wahid'} router = drivers(**device_detail) router.open() #The problem is here <- Exception has occurred: ValueError #Failed to enter enable mode. Please ensure you pass the 'secret' argument to #ConnectHandler. print('Connection is Opened with ->{}'.format(device_detail['hostname'])) config = router.get_config() print('Configuratin on this {} router ->'.format(device_detail['hostname'])) 

Answer: Can you try as follows: from napalm import get_network_driver from getpass import getpass hostname = input("IP address of router: ") username = input(f"Username of {hostname}: ") password = getpass(f"Password of {hostname}") secret = getpass(f"Enable password of {hostname}: ") driver = get_network_driver("ios") device_detail = { "hostname": hostname, "username": username, "password": password, "optional_args": { "secret": secret } } with driver(**device_detail) as router: print(router.get_facts())
Question: I have problem with importing tensorflow_text I tried importing like below two methods but none of them worked import tensorflow_text as text import tensorflow_text as tf_text  My tensorflow version is 2.9.1 and python version is Python 3.7.13. I tried installing tensorflow_text using below two methods but none of them is working. !pip install tensorflow-text !pip install -U tensorflow-text==2.9.0  I am using colab, I also tried reinstalling tensorflow but it still generates below error. NotFoundError: /usr/local/lib/python3.7/distpackages/tensorflow_text/python/ops/_sentencepiece_tokenizer.so: undefined symbol: _ZNSt7__cxx1119basic_ostringstreamIcSt11char_traitsIcESaIcEEC1Ev 

Answer: Update, Sometimes you need to reinstall and update tensorflow then install tensorflow_text. (Because you need your tensorflow.__version__ and tensorflow_text.__version__ to have the same version) !pip install -U tensorflow !pip install -U tensorflow-text import tensorflow as tf import tensorflow_text as text # Or install with a specific Version !pip install -U "tensorflow==2.8.*" !pip install -U "tensorflow-text==2.8.*" import tensorflow as tf import tensorflow_text as text  Old First, install tensorflow-text version 2.8.* like below: !pip install -q -U "tensorflow-text==2.8.*"  Then import tensorflow-text like below: import tensorflow_text as text
Question: I'm using VSCode v1.69.0. My OS is MacOS v10.15.7 (Catalina). I have a Python code that I'm usually debugging with VSCode Python debugger (using the launch.json file). For no apparent reason the debugger recently stopped working, i.e. when I click on "Run Debug", the debug icons (stop, resume, step over etc.) show up for 1 second, then disappear and nothing else happens. There is no error message in the terminal or anything (no command is launched apparently). I tried to uninstall VSCode and reinstall it => no success. I deleted the Python extension in /userprofile/.vscode/extensions and then reinstalled it (both the current v2022.10.1 and pre-release v2022.11.12011103) => no success. The python program runs properly on another IDE. Any idea?

Answer: If you are using python3.6 version then latest version of debugger no longer supports it. You can use the historical version 2022.08.*. Or use a new python version.