eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: The following pandas code removes all the .0 decimal precision if I have a float column with 1.0, 2.0, 3.0 values: import pandas as pd df = pd.DataFrame({ "date": ["2025-01-01", "2025-01-02"], "a": [1.0, 2.0], "c": [1.0, 2.1], }) print(df) columns = df.columns.difference(["date"]) df[columns] = df[columns].map(lambda x: int(x) if x.is_integer() else x) print(df)  The output of the above code:  date a c 0 2025-01-01 1.0 1.0 1 2025-01-02 2.0 2.1 date a c 0 2025-01-01 1 1.0 1 2025-01-02 2 2.1  How can I do it using Polars?

Answer: Something like this does the trick. Note that it is not typically advised to have the schema depend on the data itself. We can, however, avoid any for-by-row iteration and used a vectorised UDF with map_batches def maybe_cast_int(s: pl.Series) -> pl.Series: """Cast the Series to an Int64 type if all values are whole numbers.""" s2 = s.cast(pl.Int64) return s2 if (s2 == s).all() else s df = pl.DataFrame({ "date": ["2025-01-01", "2025-01-02"], "a": [1.0, 2.0], "c": [1.0, 2.1], }) df.with_columns(pl.col("a", "c").map_batches(maybe_cast_int))  shape: (2, 3) ┌────────────┬─────┬─────┐ │ date ┆ a ┆ c │ │ --- ┆ --- ┆ --- │ │ str ┆ i64 ┆ f64 │ ╞════════════╪═════╪═════╡ │ 2025-01-01 ┆ 1 ┆ 1.0 │ │ 2025-01-02 ┆ 2 ┆ 2.1 │ └────────────┴─────┴─────┘  This example shows it a bit better by not overwriting original columns df.select( "a", pl.col("a").map_batches(maybe_cast_int).alias("b"), "c", pl.col("c").map_batches(maybe_cast_int).alias("d"), )  shape: (2, 4) ┌─────┬─────┬─────┬─────┐ │ a ┆ b ┆ c ┆ d │ │ --- ┆ --- ┆ --- ┆ --- │ │ f64 ┆ i64 ┆ f64 ┆ f64 │ ╞═════╪═════╪═════╪═════╡ │ 1.0 ┆ 1 ┆ 1.0 ┆ 1.0 │ │ 2.0 ┆ 2 ┆ 2.1 ┆ 2.1 │ └─────┴─────┴─────┴─────┘
Question: I have a Django project where I'm working on an e-commerce application. I'm using SQLite as my database, and I'm trying to add a creation_date field to my VendorProduct model so that it records when a product was created. What I Did I added the creation_date field to my VendorProduct model like this: models.py (VendorProduct Model) from django.db import models from django.utils.timezone import now from userauths.models import CustomUser class VendorProduct(models.Model): user = models.ForeignKey(CustomUser, on_delete=models.SET_NULL, null=True, blank=True) creation_date = models.DateTimeField(auto_now_add=True, blank=True, null=True) def __str__(self): return self.title  Migrations I ran the following commands to apply the migration: python manage.py makemigrations vendorpannel python manage.py migrate vendorpannel  Then, I verified that the creation_date column exists in my database using: from django.db import connection with connection.cursor() as cursor: cursor.execute("PRAGMA table_info(vendorpannel_vendorproduct);") columns = cursor.fetchall() for column in columns: print(column)  The output confirms that creation_date exists in my database: (7, 'creation_date', 'datetime', 0, None, 0)  Admin Panel Configuration I updated my admin panel to display creation_date: admin.py from django.contrib import admin from .models import VendorProduct class VendorProductAdmin(admin.ModelAdmin): list_display = ('creation_date') readonly_fields = ('creation_date',) admin.site.register(VendorProduct, VendorProductAdmin)  Problem: creation_date Not Showing in Django Admin Even after making these changes and applying migrations, the creation_date field is not appearing in the Django admin panel under list_display. What I Tried  Checked if the field exists in the database using SQL queries (confirmed ✅). Cleared Django cache and restarted the server: python manage.py collectstatic python manage.py runserver   Checked if the field name is correct in list_display (confirmed ✅). Verified that list_display works for other fields (other fields display correctly, just creation_date is missing ❌).  Expected Outcome  I expect to see creation_date displayed in the Django admin panel under VendorProduct. The field should be read-only and should not be manually editable.  Question Why is creation_date not appearing in the Django admin panel, even though:  It exists in the database. It's correctly added in list_display. Migrations were applied successfully.  What am I missing here?  Thank you in advance  

Answer: list_display field is for List view only. It's the admin page listing instances of VendorProduct in your case. On your screenshot you have Add view which shows only editable fields. Since your creation_date field has auto_now_add option, it makes it non-editable. If you want to manually provide creation_date, but still keep default now behavior then instead of auto_now_add use default option. creation_date = models.DateTimeField(default=datetime.now, blank=True)  Then this field will show up in add form. Maybe try to remove it from read-only field if this is not enough In all other cases this field should show up for existing instances
Question: I have a regex in Python which uses \b to split words. When I use it on Devanagiri text, I notice that not all characters in the Unicode block are defined as word characters. Certain punctuation marks appear to be defined as non-word characters. This is fundamentally wrong as words in this script can end with these characters. Is it possible to tell regex to treat the entire block from 0x900 to 0x97f as word characters? See for example the following regex. '(?<!\.)(a(?:bc|de)|zip|चाय|पानी)\b'  Here, the first four words abc, ade, zip and चाय are detected at proper word boundaries. The word पानी however, ends with a vowel ी and regex does not treat it as a valid word boundary when ideally it should be. >>> import re >>> re.findall(r"(?<!\.)(a(?:bc|de)|zip|चाय|पानी)\b", 'This is abc, ade, चाय, पानी and abca') ['abc', 'ade', 'चाय']  Can I change this regex behavior and if yes, how?

Answer: The problem with the pattern is that \b detects U+093E (DEVANAGARI VOWEL SIGN AA) and 0940 (DEVANAGARI VOWEL SIGN II) as non-word characaters, so the boundaries in the word पानी occur after each consonant and before the dependent vowels. It is critical to understand when working with Python regular expressions, with text in Devanagari Script, that the definitions of the re modules \w and \b are fundamentally different from Unicode's definitions. The easiest fix is to use the regex module instead. This regex engine has Unicode support unlike the re module. import regex as re re.findall(r"(?<!\.)(a(?:bc|de)|zip|चाय|पानी)\b", 'This is abc, ade, चाय, पानी and abca') # ['abc', 'ade', 'चाय', 'पानी']
Question: I am not trying to achieve anything -- apart from learning how generator functions and coroutines work on a brick level, which I am not really getting yet, despite lots of reading.... $cat test.py #No integer def eee(): num = yield print(f"First num: {num}") num = yield print(f"Second num: {num}") num = yield print(f"Third num: {num}") #integer def ddd(): yield 100 num = yield print(f"First num: {num}") num = yield print(f"Second num: {num}") num = yield print(f"Third num: {num}") e=eee() e.send(None) e.send(1) e.send(2) try: e.send(3) except StopIteration as e: print(f'Done with e: {e}\n') d=ddd() print(d.send(None)) d.send(1) d.send(2) d.send(3) $python3 test.py First num: 1 Second num: 2 Third num: 3 Done with e: 100 First num: 2 Second num: 3  why does d swallow the num 1?

Answer: d "swallows" the 1 you sent it because you added this line yield 100  to ddd, that wasn't in eee. That line receives the 1. The 2 goes to the next yield.
Question: Here, column "AB" is just being created and at the same time is being used as input to create column "ABC". This fails. df = df.with_columns( (pl.col("A")+pl.col("B")).alias("AB"), (pl.col("AB")+pl.col("C")).alias("ABC") )  The only way to achieve the desired result is a second call to with_columns. df1 = df.with_columns( (pl.col("A")+pl.col("B")).alias("AB") ) df2 = df1.with_columns( (pl.col("AB")+pl.col("C")).alias("ABC") ) 

Answer: Underlying Problem In general, all expressions within a (with_columns, select, filter, group_by) context are evaluated in parallel. Especially, there are no columns previously created within the same context. Solution Still, you can avoid writing large expressions multiple times, by saving the expression to a variable. import polars as pl df = pl.DataFrame({ "a": [1], "b": [2], "c": [3], }) ab_expr = pl.col("a") + pl.col("b") df.with_columns( ab_expr.alias("ab"), (ab_expr + pl.col("c")).alias("abc"), )  shape: (1, 5) ┌─────┬─────┬─────┬─────┬─────┐ │ a ┆ b ┆ c ┆ ab ┆ abc │ │ --- ┆ --- ┆ --- ┆ --- ┆ --- │ │ i64 ┆ i64 ┆ i64 ┆ i64 ┆ i64 │ ╞═════╪═════╪═════╪═════╪═════╡ │ 1 ┆ 2 ┆ 3 ┆ 3 ┆ 6 │ └─────┴─────┴─────┴─────┴─────┘  Note that polar's query plan optimization accounts for the joint sub-plan and the computation doesn't necessarily happen twice. This can be checked as follows. ab_expr = pl.col("a") + pl.col("b") ( df .lazy() .with_columns( ab_expr.alias("ab"), (ab_expr + pl.col("c")).alias("abc"), ) .explain() )  simple π 5/6 ["a", "b", "c", "ab", "abc"] WITH_COLUMNS: [col("__POLARS_CSER_0xd4acad4332698399").alias("ab"), [(col("__POLARS_CSER_0xd4acad4332698399")) + (col("c"))].alias("abc")] WITH_COLUMNS: [[(col("a")) + (col("b"))].alias("__POLARS_CSER_0xd4acad4332698399")] DF ["a", "b", "c"]; PROJECT */3 COLUMNS  Especially, polars is aware of the sub-plan __POLARS_CSER_0xd4acad4332698399 shared between expressions. Syntacic Sugar (?) Moreover, the walrus operation might be used to do the variable assignment within the context. df.with_columns( (ab_expr := pl.col("a") + pl.col("b")).alias("ab"), (ab_expr + pl.col("c")).alias("abc"), )
Question: I am working on a GUI where a user can specify the both the min and max x limit. When the value is left blank I would like it to be automatically calculated. One limit can be set while the other is automatically calculated by setting it to None. But after setting one limit and then setting it to None does not automatically update the limit. Both values can be automatically calculated by using set_xlim(auto=True) but this forces both values to be automatically calculated and does not allow only one value to be automatically calculated. Is there a way of having just one limit automatically re-calculated? For example in MATLAB xlim[-Inf, 4] would automatically calculate the first limit. Below is an example without a GUI import matplotlib.pyplot as plt x_data = [1, 2, 3, 4, 5] y_data = [1, 2, 3, 4, 5] fig, ax = plt.subplots(1) ax.plot(x_data, y_data, '.') # Initially limits are automatically calculated ax.set_xlim(auto=True) print(ax.get_xlim()) # prints: (np.float64(0.8), np.float64(5.2)) # User sets value with GUI ax.set_xlim(2, 4) print(ax.get_xlim()) # prints: (np.float64(2.0), np.float64(4.0)) # Later user changes their mind and leaves the first limit blank ax.set_xlim(None, 4) # print(ax.get_xlim()) # prints: (np.float64(2.0), np.float64(4.0)) plt.show() 

Answer: As noted in the comment, we can "reset" to automatic axis limits using ax.set_xlim(auto=True) again followed by ax.autoscale_view(). # Later user changes their mind and leaves the first limit blank ax.set_xlim(auto=True) ax.autoscale_view() ax.set_xlim(None, 4) print(ax.get_xlim()) # prints: (0.8, 4.0)
Question: I'm using matplotlib to plot my data in a 4x2 grid of subplots. The matplotlib.pyplot.tight_layout automatically fits the subplots, legend, and text labels into a figure that I can save as png. However, when the legend is extremely long, tight_layout seems to add extra horizontal space to some subplots. What is the most efficient way to avoid this extra space? The subplots_adjust function looks promising, but there's a lot of trial-and-error to adjust everything and I'm hoping to find a quicker automated solution using tight_layout. My MWE is: import numpy as np import matplotlib.pyplot as plt t = np.linspace(-5,5,100) x1 = np.sin(t) x2 = np.cos(t) x3 = np.sin(2*t) x4 = np.cos(2*t) x5 = 2*np.sin(t) x6 = 2*np.cos(t) x7 = np.sin(0.5*t) x8 = np.cos(0.5*t) fig, ax = plt.subplots(nrows=4, ncols=2, figsize=(10, 7)) for r in range(4): for c in range(2): ax[r,c].plot(t,x1,label='preliminary 1') ax[r,c].plot(t,x2,label='preliminary 2') ax[r,c].plot(t,x3,label='trial 1, result 1') ax[r,c].plot(t,x4,label='trial 1, result 2') ax[r,c].plot(t,x5,label='trial 1, result 6') ax[r,c].plot(t,x6,label='trial 4, result 1') ax[r,c].plot(t,x7,label='trial 12, result 2') ax[r,c].plot(t,x8,label='trial 15, result 2') ax[0,1].legend(loc='best', bbox_to_anchor = (0.3, -1.1, 1.2, 2)) plt.tight_layout() plt.savefig('myfig.png') plt.show()  

Answer: If you use the newer Constrained Layout rather than tight_layout then you can easily add a figure legend at the top right. Changes are:  Create the figure with constrained layout fig, ax = plt.subplots(nrows=4, ncols=2, figsize=(10, 7), layout='constrained')  Get hold of the legend handles and labels from a single axes handles, labels = ax[0, 1].get_legend_handles_labels()  Use these handles and labels in a figure legend fig.legend(handles, labels, loc='outside right upper')  Remove the calls to ax.legend and tight_layout.
Question: This is my first time developing a multi-tenant SaaS application in Django. In this SaaS each company has its own PostgreSQL database, and these databases are created dynamically when a company registers. I cannot predefine all databases in settings.DATABASES, as companies can register at any time without requiring a server restart. My current solution uses a Middleware to detect the company from the subdomain or a JWT token and then modify connections.databases at runtime to configure the connection to the company's database: import redis from django.db import connections from django.core.exceptions import ImproperlyConfigured from django.utils.connection import ConnectionDoesNotExist from rest_framework_simplejwt.authentication import JWTAuthentication from myapp.models import Company # Company model stored in the global database class CompanyDBMiddleware: def __init__(self, get_response): self.get_response = get_response self.jwt_authenticator = JWTAuthentication() self.cache = redis.Redis(host='localhost', port=6379, db=0) def __call__(self, request): company_db = self.get_database_for_company(request) if not company_db: raise ImproperlyConfigured("Could not determine the company's database.") # Register connection only if it does not exist in `connections.databases` if company_db not in connections.databases: connections.databases[company_db] = { 'ENGINE': 'django.db.backends.postgresql', 'NAME': company_db, 'USER': 'postgres', 'PASSWORD': 'your_password', 'HOST': 'localhost', 'PORT': '5432', 'CONN_MAX_AGE': 60, # To avoid opening and closing connections on each request } request.company_db = company_db response = self.get_response(request) # Close connection after the response try: connections[company_db].close() except ConnectionDoesNotExist: pass return response def get_database_for_company(self, request): subdomain = request.get_host().split('.')[0] company_db = None cache_key = f"company_db_{subdomain}" company_db = self.cache.get(cache_key) if company_db: return company_db.decode("utf-8") try: company = Company.objects.using('default').get(subdomain=subdomain, active=True) company_db = company.db_name self.cache.setex(cache_key, 300, company_db) # Cache the database name for 5 minutes return company_db except Company.DoesNotExist: return None  My questions are:  Is it correct to modify connections.databases dynamically on each request to handle multiple databases? Is there a better way to do this in Django without restarting the application when registering new databases? How does this practice affect performance in environments with load balancing and multiple Django instances? Would it be better to deploy a separate API per client in its own Django container? I am considering giving each client their own portal on a separate domain and only deploying their frontend in a container while keeping a centralized API. Is this approach more efficient?  I appreciate any recommendations on best practices or potential issues with this approach.

Answer: You can use django-tenants library. Its designed for this purpose. https://django-tenants.readthedocs.io/en/latest/
Question: I am plotting a graph with date on the x axis and data on the y axis. However the graph is completly wrong and I don't understand why... df['Date_TrailClean'] = pd.to_datetime(df['Date_TrailClean']) # x axis values x = df['Date_TrailClean'] # corresponding y axis values y = df['AdjTotItems'] fig, ax = plt.subplots() # plotting the points ax.plot(x, y) ax.xaxis.set_major_locator(mdates.MonthLocator(bymonth=(1, 7))) ax.xaxis.set_minor_locator(mdates.MonthLocator()) ax.set_ylabel(r'Adjusted Total Items') # function to show the plot plt.show()  Which produces a graph like this, as if the data is plotted on the wrong axes?  Data can be accessed here: https://docs.google.com/spreadsheets/d/10bf0dEUz5lvBjh4kWH8XXfX0yOzdU8NYjSzQBwNV4bk/edit?usp=sharing

Answer: import pandas as pd import matplotlib.pyplot as plt import matplotlib.dates as mdates df = pd.read_excel('sample_data.xlsx') df['Date_TrailClean'] = pd.to_datetime(df['Date_TrailClean']) # Sort x-axis df = df.sort_values('Date_TrailClean') x = df['Date_TrailClean'] y = df['AdjTotItems'] fig, ax = plt.subplots(figsize=(10, 6)) ax.plot(x, y, color='blue', marker='o', label='Adjusted Total Items') ax.xaxis.set_major_locator(mdates.MonthLocator(bymonth=(1, 7))) ax.xaxis.set_minor_locator(mdates.MonthLocator()) ax.xaxis.set_major_formatter(mdates.DateFormatter('%b %Y')) # e.g., Jan 2025, Jul 2025 plt.xticks(rotation=45) ax.set_xlabel('Date') ax.set_ylabel(r'Adjusted Total Items') ax.set_title('Adjusted Total Items Over Time') ax.grid(True) plt.tight_layout() plt.show()   As others pointed out, sorting the x-axis data gives you above plot.
Question: I'm trying to detect text that has a coloured background in a MS Word docx, to separate it from the "normal" text. from docx import Document ... # Load the document doc = Document(docx_path) highlighted_text = [] normal_text = [] # Iterate through all paragraphs for para in doc.paragraphs: # Iterate through all runs in the paragraph for run in para.runs: print(run.text + " - " + str(run.font.highlight_color)) # Check if the run has a highlight color set if run.font.highlight_color is not None: highlighted_text.append(run.text) print(f"Found highlighted text: '{run.text}' with highlight color: {run.font.highlight_color}") return highlighted_text  However, in my test document it's only found grey highlights:  This is the results from the print statement: Text (normal) - None Text in grey - GRAY_25 (16) Found highlighted text: 'Text in grey ' with highlight color: GRAY_25 (16) Text in yellow - None Text in green - None So not sure where I'm going wrong. I don't think the text has been been shaded as that is across a whole line. Addendum: It only works for grey for me - which I have highlighted in MS Office - however the other highlights, which are getting missed have been done by someone else. This might have been done with an old copy of Office, or docx compatible software or some other method of highlighting he text that isn't "highlighting" Any ideas?

Answer: This script performs well for me: from docx import Document def extract_highlighted_text(docx_path): doc = Document(docx_path) highlighted_texts = [] for para in doc.paragraphs: for run in para.runs: if run.font.highlight_color is not None: highlighted_texts.append(run.text) return highlighted_texts docx_file = "text.docx" highlighted_texts = extract_highlighted_text(docx_file) print("Highlighted Texts:") for text in highlighted_texts: print(text)  Highlighted text in Docx:  Result:
Question: I've been trying to get a Sonic The Hedgehog game working in Pygame and it works for the most part. Sonic can run fast, jump across platforms like a normal platformer, gets affected by gravity, collects rings, etc. (check attached video) https://imgur.com/a/q1YrAXO However, I cannot for the life of me get Sonic to run at different angles and go up slopes and loops like in the real games. I am aware such a project doesn't exist in Pygame (for the public to access anyway) and making the precise, mathematically accurate scripts can be hard to do. Problematic code: (from levels.py)  def update(self): self.sonic.update() self.camera.update(self.sonic) # Follow Sonic # Keep Sonic grounded self.sonic.grounded = False for tile in self.tile_group: if self.sonic.mask.overlap(tile.mask, (tile.rect.x - self.sonic.hitbox.x, tile.rect.y - self.sonic.hitbox.y)): self.sonic.Yvel = 0 self.sonic.grounded = True self.sonic.jumped = False self.sonic.angle = tile.angle break  (from characters.py)  if not self.grounded: self.Yvel += self.gravityforce # Apply gravity self.Yvel = min(self.Yvel, 10) # Cap max fall speed self.Xvel = self.groundSpeed else: # Adjusting speed with some trigonometry self.Yvel = self.groundSpeed * math.sin(self.angle * -1) self.Xvel = self.groundSpeed * math.cos(self.angle) self.x += self.Xvel self.rect.x = self.x # Update self.rect with the new x coordinate self.y += self.Yvel self.rect.y = self.y self.hitbox.x = self.x self.hitbox.y = self.y  I am using Tiled for mapmaking by the way, I'm not sure if that helps in my situation. I've manually added an angle attribute to the tiles currently in use and given them unique angles such as 20, 45, 90, etc. (which is why Sonic suddenly changes angle and jumps/phases through floor at times). With all that being said, can anyone help or offer some advice? Literally anything will be appreciated.

Answer: You can use vector math so that the character follows the surface angle. Create a horizontal movement vector and then rotate it by the negative of the surface angle. That way when the character position is updated the movement should be aligned along the slope. e.g. if not self.grounded: # Apply gravity normally when in the air. self.Yvel += self.gravityforce self.Yvel = min(self.Yvel, 10) else: # On the ground, create a horizontal movement vector… speed = self.groundSpeed * move_direction # move_direction = -1, 0, or 1 speed_vector = pygame.math.Vector2(speed, 0) # ...then rotate it to follow the surface. # (Note: the minus sign adjusts for your coordinate system.) speed_vector = speed_vector.rotate(-self.angle) self.Xvel = speed_vector.x self.Yvel = speed_vector.y  Example output of pygame script I wrote to show this working.