eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have a recursive function in the following form: def f(): if cond1: ... f() elif cond2: ...  I've "mechanically" converted it to an iterative function like this: def f(): while True: if cond1: ... elif cond2: ... break else: break  I believe this conversion is valid, but is there a more elegant way to do it? For example, one that doesn't need multiple break statements?

Answer: Since the loop effectively continues only when cond1 is met, you can make it the condition that the while loop runs on, and run the code specific to cond2 after the loop if it's met: def f(): while cond1: ... if cond2: ...
Question: Is it possible in python polars to transform the root_names of expression meta data? E.g. if I have an expression like expr = pl.col("A").dot(pl.col("B")).alias("AdotB")  to add suffixes to the root_names, e.g. transforming the expression to pl.col("A_suffix").dot(pl.col("B_suffix")).alias("AdotB_suffix")  I know that expr.meta.root_names() gives back a list of the column names, but I could not find a way to transform them.

Answer: There is an example in the tests that does query plan node rewriting in Python with callbacks:  https://github.com/pola-rs/polars/blob/main/py-polars/tests/unit/lazyframe/cuda/test_node_visitor.py  But I can't see any equivalent API for rewriting expressions? Out of interest, there is .serialize() which can dump to JSON. expr.meta.serialize(format="json")  # '{"Alias":[{"Agg":{"Sum":{"BinaryExpr":{"left":{"Column":"A"},"op":"Multiply","right":{"Column":"B"}}}}},"AdotB"]}' # ^^^^^ ^^^^^^^^^^ ^^^^^^^^^^ ^^^^^  Technically, you could modify the Alias and Column values, and .deserialize() back into an expression. def suffix_all(expr, suffix): def _add_suffix(obj): if "Column" in obj: obj["Column"] = obj["Column"] + suffix if "Alias" in obj: obj["Alias"][-1] += suffix return obj ast = expr.meta.serialize(format="json") new_ast = json.loads(ast, object_hook=_add_suffix) return pl.Expr.deserialize(json.dumps(new_ast).encode(), format="json")  df = pl.DataFrame({"A_suffix": [2, 7, 3], "B_suffix": [10, 7, 1]}) expr = pl.col("A").dot(pl.col("B")).alias("AdotB") df.with_columns(expr.pipe(suffix_all, "_suffix"))  shape: (3, 3) ┌──────────┬──────────┬──────────────┐ │ A_suffix ┆ B_suffix ┆ AdotB_suffix │ │ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 │ ╞══════════╪══════════╪══════════════╡ │ 2 ┆ 10 ┆ 72 │ │ 7 ┆ 7 ┆ 72 │ │ 3 ┆ 1 ┆ 72 │ └──────────┴──────────┴──────────────┘  Which does seem to "work" in this case, but the serialize docs do contain a warning:  Serialization is not stable across Polars versions  And it's probably just not a recommended approach in general.
Question: Reading through PEP 3101, it discusses how to subclass string.Formatter to define your own formats, but then you use it via myformatter.format(string). Is there a way to just make it work with f-strings? E.g. I'm looking to do something like f"height = {height:.2f}" but I want my own float formatter that handles certain special cases.

Answer: As I mentioned in comments F-strings are actually processed at compile time, not runtime. you can't override the default f-string formatter is due to how Python implements f-strings at a fundamental level. For example if you run f"height = {height:.2f}"  It effectively converts "height = {}".format(format(height, '.2f'))  conversion happens during compilation, there's no way to "hook into" or modify this process at runtime. What you can do is. use decimal Module for Precision Control. Reference - https://realpython.com/how-to-python-f-string-format-float/ from decimal import Decimal value = Decimal('1.23456789') print(f"Value: {value:.2f}")  Output: Value: 1.23  Decimal implements the format method and handles the standard format specifiers like .2f. Side Note: We aren't really "overriding" the default formatter here - it's implementing the existing format specification protocol. As your goal is to handle special cases for float formatting, using Decimal could solve your problem...
Question: I need to draw random samples without replacement from a 1D NumPy array. However, performance is critical since this operation will be repeated many times. Here’s the code I’m currently using: import numpy as np # Example array array = np.array([10, 20, 30, 40, 50]) # Number of samples to draw num_samples = 3 # Draw samples without replacement samples = np.random.choice(array, size=num_samples, replace=False) print("Samples:", samples)  While this works for one sample, it requires a loop to generate multiple samples, and I believe there could be a way to optimize or vectorize this operation to improve performance when sampling multiple times.  Is there a way to vectorize or otherwise optimize this operation? Would another library (e.g., TensorFlow, PyTorch) provide better performance for this task? Are there specific techniques for bulk sampling that avoid looping in Python? 

Answer: The code below generates random samples of a list without replacement in a vectorized manner. This solution is particularly useful when the number of simulations is large and the number of samples per simulation is low. import numpy as np def draw_random_samples(len_deck, n_simulations, n_cards): """ Draw random samples from the deck. Parameters ---------- len_deck : int Length of the deck. n_simulations : int How many combinations of cards are generated. (Doubles could occur.) n_cards : int How many cards to draw from the deck per simulation. (All cards are unique.) Returns ------- indices : array-like Random indices of the deck. """ indices = np.random.randint(0, len_deck, (1, n_simulations)) for i in range(1, n_cards): new_indices = np.random.randint(0, len_deck-i, n_simulations) new_indices += np.sum(new_indices >= indices - np.arange(i)[:,None], axis=0) indices = np.vstack((indices, new_indices)) indices = np.sort(indices, axis=0) return indices.T
Question: Google has a page describing how to use one of their Gemini-1.5 models to transcribe audio. They include a sample script (see below). The script grabs the audio file from Google Storage via the Part.from_uri() command. I would, instead, like to use a local file. Setting the URI to "file:///..." does not work. How can I do that? Google's (working, cloud-based) code is here: import vertexai from vertexai.generative_models import GenerativeModel, GenerationConfig, Part # TODO(developer): Update and un-comment below line # PROJECT_ID = "your-project-id" vertexai.init(project=PROJECT_ID, location="us-central1") model = GenerativeModel("gemini-1.5-flash-002") prompt = """ Can you transcribe this interview, in the format of timecode, speaker, caption. Use speaker A, speaker B, etc. to identify speakers. """ audio_file_uri = "gs://cloud-samples-data/generative-ai/audio/pixel.mp3" audio_file = Part.from_uri(audio_file_uri, mime_type="audio/mpeg") contents = [audio_file, prompt] response = model.generate_content(contents, generation_config=GenerationConfig(audio_timestamp=True)) print(response.text) # Example response: # [00:00:00] Speaker A: Your devices are getting better over time... # [00:00:16] Speaker B: Welcome to the Made by Google podcast, ... # [00:01:00] Speaker A: So many features. I am a singer. ... # [00:01:33] Speaker B: Amazing. DeCarlos, same question to you, ... 

Answer: Although one might expect the following base-64 encoding and decoding to cancel, the following code appears to work. (The code is a slightly modified version from this page.) ... encoded_audio = base64.b64encode(open(audio_path, "rb").read()).decode("utf-8") mime_type = "audio/mpeg" audio_content = Part.from_data( data=base64.b64decode(encoded_audio), mime_type=mime_type ) contents = [audio_content, prompt] ...
Question: I’m new to pyo3, and I would like to achieve the following thing. My main code is in Python, but I would like to call a Rust function from this Python code. The trick is, the Rust function should be able to call a Python function provided as an argument. Let’s say that I have a Python function, that cannot be embedded inside of the Rust module: def double(x): return 2*x  I want to create a Rust function taking this function as an argument: use pyo3::prelude::*; fn my_rust_function(a: usize, f: ...) -> PyResult<usize> { ... }  Finally, this Rust function would be used somewhere in the same Python code as above: import rust_module def double(x): return 2*x def main(): return rust_module.my_rust_function(3, double)  Would such a behavior be possible? I read parts of the documentation about how to call Rust from Python and Python from Rust, but never such a double-way use.

Answer: In my_rust_function(), take Bound<'_, PyAny> (or any variant, such as Borrowed<'_, '_, PyAny> or Py<PyAny>, but prefer Bound unless you have reasons not to), and use the call0() method on it to call it without arguments, or call1() to call it with positional arguments only, or call() to call it with both positional and keyword arguments. Exporting the Rust function to Python should be done as usual. For example: use pyo3::prelude::*; #[pyfunction] fn my_rust_function(a: usize, f: Bound<'_, PyAny>) -> PyResult<usize> { f.call1((a,))?.extract::<usize>() } #[pymodule] fn rust_module(m: &Bound<'_, PyModule>) -> PyResult<()> { m.add_function(wrap_pyfunction!(my_rust_function, m)?)?; Ok(()) }
Question: I'm using the below to skip a group of records when a certain condition is met: if (condition met): ... [next(it) for x in range(19)]  Where it is an itertuples object created to speed up looping through a large dataframe (yes, the loop is necessary). it = df.itertuples() for row in it: ...  What's the idiomatic way of dealing with a StopIteration return from the next call (presumably due to reaching the end of the dataframe)?

Answer: So, in general, the solution to a potential exception being raised is to use exception handling. But in the case of next, you can simply use the second argument (which will make it return a default value in case the iterator is exhausted). But you shouldn't use a list comprehension like this anyway, so the most basic solution: if condition: for _ in range(19): next(it, None)  But perhaps more elegantly (using the approach from the consume recipe in the itertools docs): import itertools def advance(it, n): next(itertools.islice(it, n, n), None) ... if condition: advance(it, 19)  But again, a perfectly idiomatic solution would have been: try: [next(it) for x in range(19)] except StopIteration: pass  leaving aside the use of a list comprehension for side-effects
Question: I'm just starting out with Python on my Windows computer. I wrote a simple script to print "Hello, World!" like this: print("Hello, World!")  When I double-click the .py file to run it, a command prompt window briefly appears and then closes right away. I can't see the "Hello, World!" message. How can I make the window stay open after the script finishes so I can read the output? Thanks in advance!

Answer: you can use input function print("Hello, World!") input("Press Enter to exit...")
Question: Please help me filter my dataframe for a condition, that should be fulfilled in any two columns. Imagine a list of students with grades in different sports. I want to filter the list so the new list passed_students shows only those who have scored a 4 or greater in at least two different sports. Students = { "Names": ["Tom", "Rick", "Sally","Sarah"], "Football": [4, 5, 2,1], "Basketball": [4, 2, 4,2], "Volleyball": [6, 1, 6,1], "Foosball": [4, 3, 4,3], }  The Code should return this: passed_Students = { "Names": ["Tom", "Sally"], "Football": [4,2], "Basketball": [4,4], "Volleyball": [6,6], "Foosball": [4,4], }  I can make it work if one grade above 4 is sufficient: import numpy as np import pandas as pd Students = { "Names": ["Tom", "Rick", "Sally","Sarah"], "Football": [4, 5, 2,1], "Basketball": [4, 2, 4,2], "Volleyball": [6, 1, 6,1], "Foosball": [4, 3, 4,3], } Students = pd.DataFrame(Students) passed_Students= Students[(Students["Football"]>3) |(Students["Basketball"]>3)|(Students["Volleyball"]>3)|(Students["Foosball"]>3) ] print(passed_Students)  This returns: Students = { "Names": ["Tom", "Rick", "Sally"], "Football": [4, 5, 2], "Basketball": [4, 2, 4], "Volleyball": [6, 1, 6], "Foosball": [4, 3, 4], }  But how can I make it such, that any two grades of 4 or above qualify for passed_students, thereby returning only this? passed_Students = { "Names": ["Tom", "Sally"], "Football": [4,2], "Basketball": [4,4], "Volleyball": [6,6], "Foosball": [4,4], } 

Answer: drop the "Names", then compare to 4 with ge, sum to count the number of True per row and filter with boolean indexing: passed_Students = Students[Students.drop(columns=['Names']) .ge(4).sum(axis=1).ge(2)]  Output:  Names Football Basketball Volleyball Foosball 0 Tom 4 4 6 4 2 Sally 2 4 6 4  Intermediates: # Students.drop(columns=['Names']).ge(4) Football Basketball Volleyball Foosball 0 True True True True 1 True False False False 2 False True True True 3 False False False False # Students.drop(columns=['Names']).ge(4).sum(axis=1) 0 4 1 1 2 3 3 0 dtype: int64 # Students.drop(columns=['Names']).ge(4).sum(axis=1).ge(2) 0 True 1 False 2 True 3 False dtype: bool
Question: Say I build a BERTopic model using from bertopic import BERTopic topic_model = BERTopic(n_gram_range=(1, 1), nr_topics=20) topics, probs = topic_model.fit_transform(docs)  Inspecting probs gives me just a single value for each item in docs. probs array([0.51914467, 0. , 0. , ..., 1. , 1. , 1. ])  I would like the entire probability vector across all topics (so in this case, where nr_topics=20, I want a vector of 20 probabilities for each item in docs). In other words, if I have N items in docs and K topics, I would like an NxK output.

Answer: For individual topic probability across each document you need to add one more argument. topic_model = BERTopic(n_gram_range=(1, 1), nr_topics=20, calculate_probabilities=True)  Note: This calculate_probabilities = True will only work if you are using HDBSCAN clustering embedding model. And Bertopic by default uses all-MiniLM-L6-v2. Official documentation: https://maartengr.github.io/BERTopic/api/bertopic.html They have mentioned the same in document as well.
Question: All, I got the following error when trying to import states.geojson file as described in this page https://www.twilio.com/en-us/blog/geospatial-analysis-python-geojson-geopandas-html. I think that this file is among the pre-installed files with the geopands. I am using geopandas version 0.14.4 import geopandas as gpd states = gpd.read_file('states.geojson')  Here is the error Traceback (most recent call last): File fiona/ogrext.pyx:130 in fiona.ogrext.gdal_open_vector File fiona/ogrext.pyx:134 in fiona.ogrext.gdal_open_vector File fiona/_err.pyx:375 in fiona._err.StackChecker.exc_wrap_pointer CPLE_OpenFailedError: states.geojson: No such file or directory The above exception was the direct cause of the following exception: Traceback (most recent call last): Cell In[1], line 2 states = gpd.read_file('states.geojson') # built in-file File ~/anaconda3/lib/python3.11/site-packages/geopandas/io/file.py:289 in _read_file return _read_file_fiona( File ~/anaconda3/lib/python3.11/site-packages/geopandas/io/file.py:315 in _read_file_fiona with reader(path_or_bytes, **kwargs) as features: File ~/anaconda3/lib/python3.11/site-packages/fiona/env.py:457 in wrapper return f(*args, **kwds) File ~/anaconda3/lib/python3.11/site-packages/fiona/__init__.py:342 in open colxn = Collection( File ~/anaconda3/lib/python3.11/site-packages/fiona/collection.py:226 in __init__ self.session.start(self, **kwargs) File fiona/ogrext.pyx:876 in fiona.ogrext.Session.start File fiona/ogrext.pyx:136 in fiona.ogrext.gdal_open_vector DriverError: Failed to open dataset (flags=68): states.geojson  Thanks

Answer: The read_file method of geopandas expect a file adress as input as can be seen here in the documentation https://geopandas.org/en/stable/docs/reference/api/geopandas.read_file.html import geopandas as gpd gpd.read_file("./directory/fileName.json")  it seems that the geojson that you are seeking is a geojson file for the US states. You could find this here https://github.com/PublicaMundi/MappingAPI/blob/master/data/geojson/us-states.json?short_path=1c1ebe5 download the file and then use the function to read it and store it as a geodataframe
Question: I have the following PyTorch code value_tensor = torch.sparse_coo_tensor(indices=query_indices.t(), values=values, size=(num_lines, img_size, img_size)).to(device=device) value_tensor = value_tensor.to_dense() indices = torch.arange(0, img_size * img_size).repeat(len(lines)).to(device=device) line_tensor_flat = value_tensor.flatten() img, _ = scatter_max(line_tensor_flat, indices, dim=0) img = torch.reshape(img, (img_size, img_size))  Note the line: value_tensor = value_tensor.to_dense(), this is unsurprisingly slow. However, I cannot figure out how to obtain the same results with a sparse tensor. The function in question calls reshape which is not available on sparse tensors. I'm using Scatter Max but opened to using anything that works.

Answer: You should be able to directly use scatter_max on the sparse tensor if you keep the indices that you pass to scatter_max also sparse (i.e, only the non-zero ones). Consider this example query_indices = torch.tensor([ [0, 0, 0, 1, 1, 1], [0, 1, 2, 0, 1, 2], [0, 1, 0, 0, 1, 0] ]) values = torch.tensor([1, 2, 3, 4, 5, 6]) num_lines = 2 img_size = 3 value_tensor = torch.sparse_coo_tensor( indices=query_indices, values=values, size=(num_lines, img_size, img_size) ) # need to coalesce because for some reason sparse_coo_tensor doesn't guarantee uniqueness of indices value_tensor = value_tensor.coalesce()  Then, compute flat_indices as a sparse tensor containing just the non-zero 1-d indices (2-d indices are converted to 1-d indices similar to your arange) indices = value_tensor.indices() values = value_tensor.values() batch_indices = indices[0] # "line" (in your terminology) indices row_indices = indices[1] col_indices = indices[2] flat_indices = row_indices * img_size + col_indices  You can use flat_indices to scatter_max flattened_result, _ = scatter_max( values, flat_indices, dim=0, dim_size=img_size * img_size ) per_line_max = flattened_result.reshape(img_size, img_size)  indices tensor([[0, 0, 0, 1, 1, 1], [0, 1, 2, 0, 1, 2], [0, 1, 0, 0, 1, 0]]) values tensor([1, 2, 3, 4, 5, 6]) flat_indices tensor([0, 4, 6, 0, 4, 6]) per_line_max tensor([[4, 0, 0], [0, 5, 0], [6, 0, 0]])  The output I get is the same as what I get from your code.
Question: I have an array a of increasing indexes, e.g. [2 5 9 10], which indicates positions of value change. Assuming the output values are 0 and 1, I want to get array b: [0 0 1 1 1 0 0 0 0 1 0]  Is there a NumPy magic to transform a into b?

Answer: One way among many others a=np.array([2,5,9,10]) x=np.zeros((a.max()+1,), dtype=np.uint8) x[a]=1 b=x.cumsum()%2  Some explanation (but I guess code, in this rare case, is its own explanation, since it is quite easy, once you see it) x (after x[a]=1) contains 1 at each given position in a. So x.cumsum() contains a value that increments for each of those values: 0 for the 2 first, then 1 for 3 next then 2, then 3, then 4... So x.cumsum()%2 alternates between 1 and 0. Note that I use np.uint8 type because I am cheap, and I can't help thinking "why shoud I pay 32 bits when 8 are enough for a size 11 array". But in reality, since 256 is even, it wouldn't really matter even if a had billions of values. Just x.cumsum() would roll back from 255 to 0 because of overflow. And then x.cumsum()%2 would have the same value.
Question: I have the following dataframe:    Account ID Subscription type Cancellation flag     123 Basic 1   222 Basic 0   234 Hybrid 1   345 Hybrid 1    Now I would like to calculate the percentage of cancellations, but grouped by the subscription type. I would like to get it in a format so that I can easily create a bar chart out of the percentages grouped by the subscription type.

Answer: Use a groupby.mean: out = df.groupby('Subscription type')['Cancellation flag'].mean().mul(100)  Output: Subscription type Basic 50.0 Hybrid 100.0 Name: Cancellation flag, dtype: float64  Then plot.bar: out.plot.bar()   Or directly with seaborn.barplot: import seaborn as sns sns.barplot(df, x='Subscription type', y='Cancellation flag', estimator='mean', errorbar=None)  Output:
Question: I have a Django project, everything goes well, but when I tried to handle the expiration of the session I got confused In a test view print(request.session.get_session_cookie_age()) give me this error SessionStore' object has no attribute get_session_cookie_age According to the documentation it should returns the value of the setting SESSION_COOKIE_AGE Trying debugging with this code : from django.contrib.sessions.models import Session sk = request.session.session_key s = Session.objects.get(pk=sk) pprint(s.get_decoded())  >>> {'_auth_user_backend': 'django.contrib.auth.backends.ModelBackend', '_auth_user_hash': '055c9b751ffcc6f3530337321e98f9e5e4b8a623', '_auth_user_id': '6', '_session_expiry': 3600}  Here is the config : Version Django = 2.0 Python 3.6.9 INSTALLED_APPS = [ 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'django.contrib.postgres', 'corsheaders', 'rest_framework', 'rest_framework.authtoken', 'drf_yasg', 'wkhtmltopdf', ] MIDDLEWARE = [ 'django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'corsheaders.middleware.CorsMiddleware', ]  Thank's in advance

Answer: That functionality is not in Django 2.0, see here.
Question: I am trying to plot a heatmap with gridlines. This is my code (adapted from this post): # Plot a heatmap with gridlines import numpy as np import pandas as pd import matplotlib.pyplot as plt import seaborn as sns from functional import seq arr = np.random.randn(3, 20) plt.tight_layout() ax = plt.subplot(111) ax.imshow(arr, cmap='viridis') xr = ax.get_xlim() yr = ax.get_ylim() ax.set_xticks(np.arange(max(xr))-0.5, minor=True) ax.set_yticks(np.arange(max(yr))-0.5, minor=True) ax.grid(which='minor', snap=False, color='k', linestyle='-', linewidth=1) ax.tick_params(which='major', bottom=False, left=False) ax.tick_params(which='minor', bottom=False, left=False) ax.set_xticklabels([]) ax.set_yticklabels([]) for spine in ax.spines.values(): spine.set_visible(False) plt.show()  I get the following plot (cropped to content):  There are no vertical grid lines after the 3rd, 8th, and third to last columns. As this script uses minor ticks to plot grid lines, I also thought to print the xticks and make the same plot without hiding the labels: print('xticks:', np.arange(max(xr))-0.5) xticks: [-0.5 0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5 12.5 13.5 14.5 15.5 16.5 17.5 18.5]   which show that all the necessary ticks are there. What could be the issue here?

Answer: I have no idea why this happens. Using the major grid instead fixes this issue: import numpy as np import pandas as pd import matplotlib.pyplot as plt arr = np.random.randn(3, 20) plt.tight_layout() ax = plt.subplot(111) ax.imshow(arr, cmap='viridis') xr = ax.get_xlim() yr = ax.get_ylim() ax.set_xticks(np.arange(max(xr))-0.5, minor=False) ax.set_yticks(np.arange(max(yr))-0.5, minor=False) ax.grid(which='major', snap=False, color='k', linestyle='-', linewidth=1) ax.tick_params(which='major', bottom=False, left=False) ax.tick_params(which='minor', bottom=False, left=False) plt.show()