eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Here is a code to demo my question: from multiprocessing import Process def worker(): print("Worker running") if __name__ == "__main__": p = Process(target=worker) p.start() input("1...") input("2...") p.join()  Note, ran on Python 3.13, Windows x64. And the output I got is (after inputting Enter twice): 1... 2... Worker running Process finished with exit code 0  From the output, we can see the process actually initialized and started to run after the 2nd input. While I thought start() should block and guarantee the child process is fully initialized. Is this a normal behavior of Python multiprocessing? Because if Threading is used here instead, this issue seldom occur. I always get the thread run before the line input("1..."). May I ask, if Process.start() doesn't guarantee the process is fully-started, how should we code to ensure the child process is actually running before proceeding in the parent?

Answer: This is normal behaviour, and it's usually exactly what you want when you choose multiprocessing over, say, threading, i.e., the processes continue in parallel and do not block each other. As mentioned in the comments, here's an example how you can make sure the worker is running before proceeding: import time from multiprocessing import Process, Event def worker(start_event): print("Worker started") start_event.set() print("Worker is doing some work") time.sleep(2) if __name__ == "__main__": start_event = Event() p = Process(target=worker, args=(start_event,)) p.start() start_event.wait() print("Worker has started. Continuing main process.") print("Waiting for worker to finish") p.join()  A common pattern, however, is to communicate with the worker via a work queue and a stop event (or some other means) to tell it to shut down.
Question: Python Sql Alchemy generates table with VARCHAR for sql lite instead of INTEGER so select for sql lite ordered by with alphabet number Given City Table in Sql lite generated from sql alchemy: class City(Base): __tablename__ = "city" id: Mapped[int] = mapped_column(Integer, primary_key=True, index=True) city_name: Mapped[str] = mapped_column(String, index=True)  It generates:  Data: 15470 Paris 100567 Paris  Query: select(City).where(City.city_name == city_name.upper()).order_by(City.id).limit(1) SELECT city.id, city.city_name FROM city WHERE city.city_name = :city_name_1 ORDER BY city.id LIMIT :param_1  returns city id with 100567 instead of 15470. Do I need to provide example with sql insert as well?

Answer: In SQLite, if the id column is stored as TEXT instead of INTEGER, the ORDER BY will sort alphabetically, not numerically. That's why '100567' can come before '15470'. To fix it, you should cast id to an integer when ordering: SELECT id, city_name FROM city WHERE city_name = 'Paris' ORDER BY CAST(id AS INTEGER) LIMIT 1;
Question: I know of two ways to call a superclass's constructor from a derived class: super().__init__() and base.__init__(self). Why does the second require me to explicitly supply self as an argument? Here's a snippet that demonstrates the difference: class base: def __init__(self,/): print("initializing") class der(base): def __init__(self): super().__init__() #no error base.__init__(self) #no error base.__init__() #error, self required obj = der()  The third constructor call throws this error: File "demo.py", line 11, in <module> obj = der() ^^^^^ File "demo.py", line 9, in __init__ base.__init__() #error, should be: base.__init__(self) ^^^^^^^^^^^^^^^ TypeError: base.__init__() missing 1 required positional argument: 'self'  I expect base.__init__() to work since super().__init__() doesn't require an explicit self.

Answer: TL;DR: super() is an instance. der is a class. To undersand this properly, you have to understand how instance methods in Python work. When you write an instance method, you have to include self as the first parameter like this: class Foo: def bar(self): ...  This is because when you call an instance method, Python automatically supplies the instance itself as the method's first argument under the hood. These two lines will do the same thing: x.bar() Foo.bar(x)  The only difference is that the first line calls bar from an instance and the second calls bar from a class. In your example, base is a class (the equivalent to Foo) instead of an instance (the equivalent of x). Thus, if you want to call the base constructor, you need to manually supply self - Python won't do it for you. The super version doesn't need that because super isn't the class base - it's a proxy for the instance self that pretends it's of type der instead of type base. That means super() is still an instance, not a class, so super().__init__() will automatically supply self as a parameter to base.__init__ and you don't need to do it manually.
Question: i have dataframe, in which we are given x and y columns and between them i want to sketch regression model, main idea is that i should use dash framework, as according chow test , there could be difference between two regression model at different instance value,based on following link : dash models i wrote following code : import pandas as pd from dash import Dash,html,dcc,callback,Output,Input from sklearn.linear_model import LinearRegression import plotly.express as px data =pd.read_csv("regression.csv") model =LinearRegression() print(data) app = Dash() # Requires Dash 2.17.0 or later app.layout = [ html.H1(children='Our regression Model', style={'textAlign':'center'}), dcc.Dropdown(data.Year.unique(), '2004', id='dropdown-selection'), dcc.Graph(id='graph-content') ] @callback( Output('graph-content', 'figure'), Input('dropdown-selection', 'value') ) def scatter_graph(value): selected =data[data.Year==value] return px.scatter(selected,x='x',y='y') @callback( Output('graph-content', 'figure'), Input('dropdown-selection', 'value') ) def Regression_graph(value): selected =data[data.Year==value] X =selected['x'].values X =X.reshape(-1,1) y =selected['y'].values model.fit(X,y) y_predicted =model.predict(X) return px.line(selected,x='x',y=y_predicted) if __name__ =='__main__': app.run(debug=True)  this part works fine : @callback( Output('graph-content', 'figure'), Input('dropdown-selection', 'value') ) def scatter_graph(value): selected =data[data.Year==value] return px.scatter(selected,x='x',y='y')  but second decorator for regression plot does not work, here is example :  please help me how to fix it?

Answer: It may need to add all code in one function and return both figures  fig = px.scatter(selected, x=x, y=y) fig.add_traces(px.line(selected, x=x, y=y_predicted).data) return fig   Full working code with random data. import pandas as pd from dash import Dash,html,dcc,callback,Output,Input from sklearn.linear_model import LinearRegression import plotly.express as px from dash.exceptions import PreventUpdate import random random.seed(0) # to get always the same random values data = pd.DataFrame({ 'x': random.choices(range(0, 100), k=100), 'y': random.choices(range(0, 100), k=100), 'Year': ['2025']*50 + ['2024']*50, }) #data = pd.read_csv("regression.csv") model = LinearRegression() print(data) app = Dash() # Requires Dash 2.17.0 or later app.layout = [ html.H1(children='Our regression Model', style={'textAlign':'center'}), dcc.Dropdown(data.Year.unique(), '2004', id='dropdown-selection'), dcc.Graph(id='graph-content') ] @callback( Output('graph-content', 'figure'), Input('dropdown-selection', 'value') ) def update_graph(value): if value is None: raise PreventUpdate print(f'{value = }') selected = data[data.Year==value] x = selected['x'].values print(f'{x = }') X = x.reshape(-1,1) print(f'{X = }') y = selected['y'].values print(f'{y = }') model.fit(X, y) y_predicted = model.predict(X) # fig = px.line(selected, x=x, y=y_predicted) # fig.add_scatter(x=X, y=y) # doesn't show it fig = px.scatter(selected, x=x, y=y) fig.add_traces(px.line(selected, x=x, y=y_predicted).data) return fig if __name__ =='__main__': app.run(debug=True)
Question: I am trying to use selenium to click the Accept all or Reject all button on a cookie pop up for the the website autotrader.co.uk, but I cannot get it to make the pop up disappear for some reason. This is the pop up:  and here is the html: <button title="Reject All" aria-label="Reject All" class="message-component message-button no-children focusable sp_choice_type_13" style="opacity: 1; padding: 10px 5px; margin: 10px 5px; border-width: 2px; border-color: rgb(5, 52, 255); border-radius: 5px; border-style: solid; font-size: 14px; font-weight: 400; color: rgb(255, 255, 255); font-family: arial, helvetica, sans-serif; width: calc(35% - 20px); background: rgb(5, 52, 255);">Reject All</button>   The code I have tried is the following: from selenium import webdriver from selenium.webdriver.chrome.service import Service from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC import time path_to_driver = r"C:\path_to_project\chromedriver.exe" service = Service(executable_path=path_to_driver) driver = webdriver.Chrome(service=service) driver.get("https://www.autotrader.co.uk") time.sleep(5) WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.CLASS_NAME, 'message-component message-button no-children focusable sp_choice_type_13'))).click() time.sleep(10) driver.quit()  Can anyone help here?

Answer: As you can see, the pop-up window is embedded within an <iframe>Selenium must first switch the driver's context to that iframe before attempting to locate or interact with any elements contained within it. wait for the desired iframe element to be available to switch to it: iframe = wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, 'iframe[id^="sp_message_iframe_"]')))  Note: Since the id attribute of the iframe appears to be dynamically generated, it is recommended to locate the iframe using a partial match strategy, such as CSS selector or XPath with the contains() function or partial match strategy ( https://stackoverflow.com/a/56844649/11179336) This is how you can do: import time from selenium.webdriver import Chrome, ChromeOptions from selenium.webdriver.common.by import By from selenium.webdriver.support.wait import WebDriverWait from selenium.webdriver.support import expected_conditions as EC chrome_options = ChromeOptions() chrome_options.add_experimental_option("excludeSwitches", ['enable-automation']) driver = Chrome(options=chrome_options) driver.get("https://www.autotrader.co.uk") wait = WebDriverWait(driver, 10) # wait for the target iframe to get loaded in order to switch to it wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, 'iframe[id^="sp_message_iframe_"]'))) # click to 'Reject All' wait.until(EC.element_to_be_clickable((By.XPATH, '//button[@title="Reject All"]'))).click() # Switch back to the main page content driver.switch_to.default_content() # Now you can continue interacting with the main page here time.sleep(5)
Question: Given a polars dataframe, I want to extract all duplicated rows while also applying an additional filter condition, for example: import polars as pl df = pl.DataFrame({ "name": ["Alice", "Bob", "Alice", "David", "Eve", "Bob", "Frank"], "city": ["NY", "LA", "NY", "SF", "LA", "LA", "NY"], "age": [25, 30, 25, 35, 28, 30, 40] }) # Trying this: df.filter((df.is_duplicated()) & (pl.col("city") == "NY")) # error  However, this results in an error:  SchemaError: cannot unpack series of type object into bool  Which alludes that df.is_duplicated() returns a series of type object, but in reality, it's a Boolean Series. Surprisingly, reordering the predicates by placing the expression first makes it work (but why?): df.filter((pl.col("city") == "NY") & (df.is_duplicated())) # works! correctly outputs: shape: (2, 3) ┌───────┬──────┬─────┐ │ name ┆ city ┆ age │ │ --- ┆ --- ┆ --- │ │ str ┆ str ┆ i64 │ ╞═══════╪══════╪═════╡ │ Alice ┆ NY ┆ 25 │ │ Alice ┆ NY ┆ 25 │ └───────┴──────┴─────┘  I understand that the optimal approach when filtering for duplicates based on a subset of columns is to use pl.struct, like: df.filter((pl.struct(df.columns).is_duplicated()) & (pl.col("city") == "NY")) # worksWhich works fine with the additional filter condition. However, I'm intentionally not using pl.struct because my real dataframe has 40 columns, and I want to check for duplicated rows based on all the columns except three, so I did the following: df.filter(df.drop("col1", "col2", "col3").is_duplicated()) Which works fine and is much more convenient than writing all 37 columns in a pl.struct. However, this breaks when adding an additional filter condition to the right, but not to the left: df.filter( (df.drop("col1", "col2", "col3").is_duplicated()) & (pl.col("col5") == "something") ) # breaks! df.filter( (pl.col("col5") == "something") & (df.drop("col1", "col2", "col3").is_duplicated()) ) # works!  Why does the ordering of predicates (Series & Expression vs Expression & Series) matter inside .filter() in this case? Is this intended behavior in Polars, or a bug?

Answer: The error is not .filter() specific. and I don't think it's a bug. Expressions allow you to use Series on the RHS, and it will return an expression. pl.lit(True) & pl.Series([1, 2]) # <Expr ['[(true) & (Series)]'] at 0x134D05F90>  But the other way round doesn't make sense, and errors. pl.Series([1, 2]) & pl.lit(True) # ComputeError: cannot cast 'Object' type  As for using a struct, you can wrap .exclude() in a struct. pl.struct(pl.exclude("col1", "col2", "col3")).is_duplicated()
Question: I recently learned that Discord has updated with a feature called "User Install" that allows users to install bots to their personal accounts, not just servers. As I understand it, this enables users to use certain bot commands in servers where the bot isn't officially present. I'm developing a Discord bot using py-cord and want to implement this new feature, but I can't find any information about it in the documentation. My questions are:  Does py-cord already support Discord's User Install Commands feature? If supported, how can I define and implement a command in py-cord that can be installed by users and used in any server? Are there any relevant decorators or special configurations to mark these types of commands?  My current code structure: import discord from discord.ext import commands bot = commands.Bot() # Regular slash command @bot.slash_command(name="hello", description="Say hello") async def hello(ctx): await ctx.respond(f"Hello, {ctx.author.name}!") # I want to know how to modify this to be a user-installable command # @bot.???_command(name="usercommand") # async def user_command(ctx): # await ctx.respond("This is a user-installed command") bot.run("TOKEN")  Thanks for any guidance or suggestions!

Answer: As of Pycord v2.6, this is supported. if you want your command to be available in both guilds and to users who have installed your bot, you can specify both guild_install and user_install in an integration types parameter inside a @bot.slash_command() decorator: @bot.slash_command( name="hello", description="say hello", integration_types={ discord.IntegrationType.guild_install, discord.IntegrationType.user_install, }, ) async def hello(ctx: discord.ApplicationContext): await ctx.respond("hello!")
Question: Background information below the question. In Python 3, I can define a class with an abstract method and implement it in a derived class using a more specialized signature. I know this works, but like many things work in many programming languages, it may not be good practice. So is it? from abc import ABC, abstractmethod class Base(ABC): @abstractmethod def foo(self, *args, **kwargs): raise NotImplementedError() class Derived(Base): def foo(self, a, b, *args, **kwargs): print(f"Derived.foo(a={a}, b={b}, args={args}, kwargs={kwargs})") d = Derived() d.foo(1, 2, 3, "bar", baz="baz") # output: # Derived.foo(a=1, b=2, args=(3, 'bar'), kwargs={'baz': 'baz'})  Is this good or bad practice?  More information as promised. I have an interface that defines an abstract method. It returns some sort of handle. Specialized implementations must always be able to return a sort of default handle if the method is called without any extra arguments. However, they may define certain flags to tweak the handle to the use case of the caller. In this case, the caller is also the one that instantiated the specialized implementation and knows about these flags. Generic code operating only on the interface or the handles does not know about these flags but does not need to. from abc import ABC, abstractmethod class Manager(ABC): @abstractmethod def connect(self, *args, **kwargs): raise NotImplementedError() class DefaultManager(Manager): def connect(self, *, thread_safe: bool = False): if thread_safe: return ThreadSafeHandle() else: return DefaultHandle()  It is specific to my use case that a Manager implementation may want to issue different implementations of handles specific to the use case of the caller. Managers are defined in one place in my code and callers may or may not have specialized needs, such as thread safety in the example, for the managers they use.

Answer: Your first example would be completely fine, as everything the abstract parent accepts is also accepted by the derived class, the argument types are identical (
a
, 
b
 untyped). See comment for why the first snippet isn't quite type-correct, only an override with def foo(self, a:Any=default, b:Any=default, *args, **kwargs): would be correct.  To be type-correct, the type of the accepted arguments has to get broader, not narrower. So I would take issue with the concrete example, since using a DefaultManager as a Manager would imply one could pass any amount of positional arguments to it, and any value into thread_safe. More concretely, IMO, the best practice here is this: from abc import ABC, abstractmethod class Manager(ABC): @abstractmethod def connect(self): raise NotImplementedError() class DefaultManager(Manager): def connect(self, *, thread_safe: bool = False): if thread_safe: return ThreadSafeHandle() else: return DefaultHandle()  because everything Manager.connect accepts is also accepted by DefaultManager.connect
Question: I'm currently using ABC's to ensure that my child classes implement a specific method (or property, in this particular case). I want to make it impossible to create children of Entity without implementing similar_entities: class Entity(ABC): @property @abstractmethod def similar_entities(self) -> list[str]: return [] class SubEntity1(Entity): @property def similar_entities(self): return ["a", "b", "c"] SubEntity1() # Can be instantiated class SubEntity2(Entity): pass SubEntity2() # TypeError  However, I don't actually want to prohibit users from creating instances of the base class Entity itself; creating a "generic" entity is meaningful in this case, and I would like to use the implementation of similar_entities defined on the Entity class itself. I'm only using @abstractmethod to "idiot-check" myself so that I don't accidentally forget to implement this property on my subclasses. I know full well that the "correct" way to do this would be to create a stub implementation of Entity and use that for generic instances instead: class GenericEntity(Entity): @property def similar_entities(self) -> list[str]: return super().similar_entities GenericEntity() # ...  But to me this essentially violates DRY, and since I'm working with such a simple and limited context I'm wondering if there's a way to have my cake and eat it too. Is there any way to massage @abstractmethod to only complain on child-instances? If not, is there some better, more flexible way to establish this contract between my classes? I'm also not opposed to "rolling-my-own" decorator if this is more appropriate in this case.

Answer: Here is my code that fits:  Implementing similar_entities on subclasses is allowed. Entity can be instantiated. Subclasses defined without similar_entities property raise TypeError.  class MyMeta(type): def __init__(self, classname, superclasses, attributedict): super().__init__(classname, superclasses, attributedict) if 'similar_entities' not in self.__dict__: raise TypeError class Entity(metaclass=MyMeta): @property def similar_entities(self): return []  Example 1: class SubEntity1(Entity): @property def similar_entities(self): return ["a", "b", "c"] print(SubEntity1().similar_entities) # ['a', 'b', 'c']  Example 2: class SubEntity2(Entity): pass print(SubEntity2().similar_entities) # TypeError  Example 3: print(Entity().similar_entities) # []
Question: I have below Python code using polars, and I do not want Python to auto parse values as dates or integers unless explicitly stated. schema_overrides doesn't prevent auto conversion either. import polars as pl # Read the CSV file with all columns as strings using schema_overrides file_path = "./xyz.csv" df = pl.read_csv(file_path, schema_overrides={'*': pl.Utf8}) # Display the DataFrame print(df)  I get below error:  polars.exceptions.ComputeError: could not parse p35038 as dtype i64 at column 'Employee ID' (column number 3) 

Answer: This is what infer_schema=False is for.  When False, the schema is not inferred and will be pl.String if not specified in schema or schema_overrides.  pl.read_csv(b"""a,b,c 1,2,3""") # shape: (1, 3) # ┌─────┬─────┬─────┐ # │ a ┆ b ┆ c │ # │ --- ┆ --- ┆ --- │ # │ i64 ┆ i64 ┆ i64 │ # ╞═════╪═════╪═════╡ # │ 1 ┆ 2 ┆ 3 │ # └─────┴─────┴─────┘  pl.read_csv(b"""a,b,c 1,2,3""", infer_schema=False) # shape: (1, 3) # ┌─────┬─────┬─────┐ # │ a ┆ b ┆ c │ # │ --- ┆ --- ┆ --- │ # │ str ┆ str ┆ str │ # ╞═════╪═════╪═════╡ # │ 1 ┆ 2 ┆ 3 │ # └─────┴─────┴─────┘  "*" in your example is taken literally, it is not treated as a "Wildcard".
Question: Recently I trained two MLP model and saved weights for future work. I develop one module for loading model and use these model in another module. Load model module contain this code to load models: def creat_model_extractor(model_path, feature_count): """ This function create model and set weights :param model_path: address of weights files :param feature_count: Number of nodes in input layer """ try: tf.keras.backend.clear_session() node_list = [1024, 512, 256, 128, 64, 32] model = Sequential() model.add(Input(shape=(feature_count,))) for node in node_list: model.add(Dense(node, activation='relu')) model.add(Dropout(0.2)) model.add(LayerNormalization()) model.add(Dense(16, activation='relu')) model.add(LayerNormalization()) model.add(Dense(1, activation='sigmoid')) @tf.function def inference_step(inputs): return tf.stop_gradient(model(inputs, training=False)) model.inference_step = inference_step model.load_weights(model_path) model.trainable = False for layer in model.layers: layer.trainable = False except Exception as error: logger.warning(error, exc_info=True) return None return model  And this is predict function SMALL_MODEL = creat_model_extractor(MODEL_PATH_SMALL, small_blocks_count) (SMALL_MODEL.inference_step(small_blocks_normal) > 0.5).numpy().astype(int)  Problem: After predict label 'SMALL_MODEL' size change. It become bigger and And after a while, the RAM fills up. What should I do to prevent RAM from filling up? This problem happen even in one module

Answer: Every time you call model.inference_step(), TensorFlow creates a new computation graph, because your @tf.function is dynamically bound inside your model object. TensorFlow is trying to trace and cache the @tf.function, but it can't re-use the existing trace properly, because model.inference_step is reattached dynamically and behaves non-standardly. You should not dynamically attach inference_step inside the function. Instead, move the @tf.function outside the model and use the model directly for prediction. @tf.function def inference(model, inputs): return tf.stop_gradient(model(inputs, training=False))  and call: SMALL_MODEL = creat_model_extractor(MODEL_PATH_SMALL, small_blocks_count) predictions = inference(SMALL_MODEL, small_blocks_normal) labels = (predictions > 0.5).numpy().astype(int)