eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I downloaded chromium version 136 and im using it on macOS 64 ARM. I opened it manually and added a chrome web extension and it works. When I close chrome, quit, and open it the extension is always active. But when I run my code with NoDriver from a python script, the chromium tab thats opened does not have the extension and its inside an incognito mode. How can I enable the extension for this way? Chromium setup code async def worker(emailStr): browser = None PassProvided = None password = None if ".com:" in emailStr: emailStr, password = emailStr.split(":", 1) PassProvided = password try: # Select a random proxy if enabled proxy = None if USE_PROXIES and proxies: proxy = random.choice(proxies) else: print(f"{red_text}Not using proxy {reset}") # Randomize window size and position for unique fingerprint window_width = random.randint(1000, 1500) window_height = random.randint(750, 950) x_position = random.randint(0, 500) y_position = random.randint(0, 500) # Build Chrome arguments args = [ f"--window-size={window_width},{window_height}", f"--window-position={x_position},{y_position}", "--disable-sync", "--no-first-run", "--no-default-browser-check", "--disable-backgrounding-occluded-windows", "--disable-renderer-backgrounding", "--disable-background-timer-throttling", "--disable-breakpad", "--disable-extensions", "--incognito", "--disable-dev-shm-usage", ] # Inject proxy if used if proxy: host, port, username, proxyPass = parse_proxy(proxy) proxy_creds= [username,proxyPass] proxy_url = f"http://{host}:{port}" args.append(f"--proxy-server={proxy_url}") # Start nodriver browser browser = await nd.start( browser_executable_path=CHROMIUM_PATH, headless=HEADLESS_MODE, stealth=True, browser_args=args ) # Set up proxy authentication main_tab = await browser.get("draft:,") await setup_proxy(proxy_creds[0], proxy_creds[1], main_tab) # Navigate to Target homepage tab = await browser.get("https://www.target.com/") # Clear browser storage for clean session await tab.evaluate(""" () => { localStorage.clear(); sessionStorage.clear(); } """) 

Answer: Try: args = [ f"--window-size={window_width},{window_height}", f"--window-position={x_position},{y_position}", "--disable-sync", "--no-first-run", "--no-default-browser-check", "--disable-backgrounding-occluded-windows", "--disable-renderer-backgrounding", "--disable-background-timer-throttling", "--disable-breakpad", # "--disable-extensions", # Remove this line! # "--incognito", # Remove this if you want to reuse a profile with extensions "--disable-dev-shm-usage", "--load-extension=/Users/your_user/extension_folder", # "--user-data-dir=/Users/your_user/Library/Application Support/Chromium", # Optional ]   Incognito mode, where extensions are usually disabled by default.  Headless or automated Chrome sessions typically do not load extensions in incognito sessions unless you explicitly allow it.  The argument --disable-extensions disables all extensions.
Question: Below code tokenises the text and identifies the grammar of each tokenised word. import nltk from nltk.tokenize import sent_tokenize, word_tokenize from nltk.corpus import wordnet as wn #nltk.download() text = "Natural language processing is fascinating" # tokenise the sentence words = word_tokenize(text) print(words) # identify noun, verb, etc grammatically in the sentence for w in words: tmp = wn.synsets(w)[0].pos() print (w, ":", tmp)  The output is; ['Natural', 'language', 'processing', 'is', 'fascinating'] Natural : n language : n processing : n is : v fascinating : v  Where n is noun and v is verb Can some Python code expert please advises me how to format the output so it will look like below; nouns = ["natural", "language", "processing"] verbs = ["is", "fascinating"]  I need assistance to change the result output format. I think it needs some relevant python code to perform this requirement.

Answer: You can achieve it this way : # Lists to store parts of speech nouns = [] verbs = [] for w in words: synsets = wn.synsets(w) if synsets: pos = synsets[0].pos() if pos == 'n': nouns.append(w.lower()) elif pos == 'v': verbs.append(w.lower())  full solution: import nltk from nltk.tokenize import word_tokenize from nltk.corpus import wordnet as wn # Make sure the necessary NLTK data is available nltk.download('punkt') nltk.download('wordnet') nltk.download('punkt_tab') text = "Natural language processing is fascinating" # Tokenize the text words = word_tokenize(text) # Lists to store parts of speech nouns = [] verbs = [] for w in words: synsets = wn.synsets(w) if synsets: pos = synsets[0].pos() if pos == 'n': nouns.append(w.lower()) elif pos == 'v': verbs.append(w.lower()) print(f"nouns = {nouns}") print(f"verbs = {verbs}")  output: nouns = ['natural', 'language', 'processing'] verbs = ['is', 'fascinating']
Question: i am making todo app or program on python in which output is not working according to the code while True: user_action = input("type add, show, edit, remove or exit ") user_action = user_action.strip() if 'add'in user_action: todo = user_action[4:] with open('todos.txt', 'r') as file: todos = file.readlines() todos.append(todo) with open('todos.txt', 'w') as file : file.writelines(todos) elif 'show'in user_action: with open('todos.txt', 'r') as file : todos = file.readlines() #new_todos = [item.strip("\n") for item in todos] for index, items in enumerate(todos): items = items.strip("\n") index = index + 1 row = f"{index}-{items}" print(row) elif 'edit' in user_action: number = int(user_action[5:]) print(number) number = number - 1 with open('todos.txt', 'r') as file: todos = file.readlines() new_todo = input("Type the new todo:") todos[number] = new_todo + "\n" with open('todos.txt', 'w') as file : file.writelines(todos) elif 'remove' in user_action: number = int(user_action[6:]) with open('todos.txt', 'r') as file: todos = file.readlines() index = number - 1 todo_to_remove = todos[index].strip("\n") todos.pop(index) with open('todos.txt', 'w') as file : file.writelines(todos) message = f"todo {todo_to_remove} was removed from the list" print(message) elif 'exit' in user_action: break else: print("Command is not valid ") print("bye!")  ` hey this is the code for my program but the problem is when i am adding input "add anything" and after that i am asking the program to show my todos then it is not showing me the correct output type add, show, edit, remove or exit add bro type add, show, edit, remove or exit show 1-ab 2-cd 3-hihellohikrrish 4-hi 5- 6-hiibro type add, show, edit, remove or exit add broski type add, show, edit, remove or exit show 1-ab 2-cd 3-hihellohikrrish 4-hi 5- 6-hiibrobroski  this is the output i received , the output of show should be 1-ab 2-cd 3-hihellohikrrish 4-hi 5- 6-hii 7-bro 8-broski 

Answer: As OldBoy said in the comment, you are currently removing the newline characters with the strip() Python method. The strip method removes leading and trailing characters. This means you append all the inputs with the add functionality in a single line. I think I fixxed it by adding the '\n' which is the new line character, as the todo items print with show in different lines  if 'add'in user_action: todo = user_action[4:] with open('todos.txt', 'r') as file: todos = file.readlines() todos.append(todo + '\n')
Question: I have these arrays in Awkward Array: my_indices = ak.Array([[0,1],[0],[1]]) my_dummy_arr = ak.Array([[1,1],[1,1],[1,1]])  I want to generate this result: [[1,1],[1,0],[0,1]]  Basically, for each sub-array, I want to keep 1 at the positions in my_indices and set 0 elsewhere, matching the structure of my_dummy_arr. How can I achieve this with Awkward Array? (Note that this is a dummy example and the actual data is much larger, so I do not want to use slow operations like python loops or list comprehension)

Answer: You can achieve that via a local index for the dummy array: local_index = ak.local_index(my_dummy_arr) mask = ak.any(local_index[:, :, None] == my_indices[:, None, :], axis=-1) result = ak.where(mask, 1, 0)
Question: I have an image img with 1000 rows and columns each. Now I would like to consider each pixel as x- and y-coordinates and extract the respective value. An illustrated example of what I want to achieve:  In theory, the code snippet below should work. But it is super slow (I stopped the execution after some time). img = np.random.rand(1000,1000) xy = np.array([(x, y) for x in range(img.shape[1]) for y in range(img.shape[0])]) xy = np.c_[xy, np.zeros(xy.shape[0])] for i in range(img.shape[0]): for j in range(img.shape[1]): xy[np.logical_and(xy[:,1] == i, xy[:,0] == j),2] = img[i,j]  Is there a faster way (e.g. some numpy magic) to go from one table to the other? Thanks in advance!

Answer: A possible solution: pd.DataFrame(img).stack().reset_index().to_numpy()  Method pd.DataFrame creates a dataframe where rows represent y-coordinates and columns represent x-coordinates. stack then compresses the dataframe's columns into a single column, turning the x-values into part of a hierarchical index alongside the y-values. reset_index flattens this multi-level index into columns, resulting in a dataframe with columns [y, x, val]. Lastly, to_numpy converts the dataframe into a numpy array.  Alternatively, we can use only numpy, through np.meshgrid and np.hstack (to horizontally concatenate the vertical vectors): h, w = img.shape y, x = np.meshgrid(np.arange(w), np.arange(h)) np.hstack([ x.reshape(-1, 1), y.reshape(-1, 1), img.reshape(-1, 1) ])
Question: Is it possible to write a docstring test for a function that prints out the colored text into the command line? I want to test only the content ignoring the color or to add somehow the information on color into the docstring. In the example below the test has failed, but it should not. Example from colorama import Fore def print_result(): """Prints out the result. >>> print_result() Hello world """ print('Hello world') def print_result_in_color(): """Prints out the result. >>> print_result_in_color() Hello world """ print(Fore.GREEN + 'Hello world' + Fore.RESET) if __name__ == '__main__': import doctest doctest.testmod()  Output Failed example: print_result_in_color() Expected: Hello world Got: Hello world 

Answer: Yes, it is possible to encode the color information into the docstring using wrapper that alters the docstrings. Below is the example code that passes the test from colorama import Fore def wrapper(func): # wrapper func that alters the docstring func.__doc__ = func.__doc__.format(**{"GREEN": Fore.GREEN, "RESET": Fore.RESET}) return func def print_result(): """Prints out the result. >>> print_result() Hello world """ print("Hello world") @wrapper def print_result_in_color(): """Prints out the result. >>> print_result_in_color() {GREEN}Hello world{RESET} """ print(Fore.GREEN + "Hello world" + Fore.RESET) if __name__ == "__main__": import doctest doctest.testmod()  Note: You can also look into these answers for a more in-depth view on this
Question: I'm using Beam SqlTransform in python, trying to define/pass nullable fields. This code works just fine: with beam.Pipeline(options=options) as p: # ... # Use beam.Row to create a schema-aware PCollection | "Create beam Row" >> beam.Map(lambda x: beam.Row( user_id=int(x['user_id']), user_name=str(x['user_name]) )) | 'SQL' >> SqlTransform("SELECT user_id, COUNT(*) AS msg_count FROM PCOLLECTION GROUP BY user_id")  However, I am not able to create nullable fields with this approach. Without the direct cast, I'm getting a decoding Field error. user_id = json.get('user_id')  throws: Failed to decode Schema due to an error decoding Field proto: name: "user_id" type { nullable: true logical_type { urn: "beam:logical:pythonsdk_any:v1" } }  Without using beam.Row, any other object, throws a missing schema error. Cannot call getSchema when there is no schema  What is the proper way to define nullable fields?

Answer: When working with Apache Beam's SqlTransform in Python, you need to properly define nullable fields in your schema. Here are the correct approaches: Option 1: Using beam.Row with Optional Types The most straightforward way is to use Python's Optional type hint with beam.Row: from typing import Optional with beam.Pipeline(options=options) as p: (p | "Create beam Row" >> beam.Map(lambda x: beam.Row( user_id=Optional[int](x.get('user_id')), # This makes the field nullable user_name=Optional[str](x.get('user_name')) )) | 'SQL' >> SqlTransform("SELECT user_id, COUNT(*) AS msg_count FROM PCOLLECTION GROUP BY user_id") )  Option 2: Using None for Null Values Alternatively, you can explicitly pass None for null values: with beam.Pipeline(options=options) as p: (p | "Create beam Row" >> beam.Map(lambda x: beam.Row( user_id=int(x['user_id']) if x['user_id'] is not None else None, user_name=str(x['user_name']) if x['user_name'] is not None else None )) | 'SQL' >> SqlTransform("SELECT user_id, COUNT(*) AS msg_count FROM PCOLLECTION GROUP BY user_id") )  Option 3: Using Schema Definition For more complex schemas, you can define the schema explicitly: from apache_beam.typehints import RowTypeConstraint from apache_beam.typehints.schemas import Field, FieldType from apache_beam.typehints.schemas import LogicalType schema = { 'user_id': FieldType(LogicalType('beam:logical:pythonsdk_any:v1'), nullable=True), 'user_name': FieldType(LogicalType('beam:logical:pythonsdk_any:v1'), nullable=True) } with beam.Pipeline(options=options) as p: (p | "Create beam Row" >> beam.Map(lambda x: beam.Row( user_id=x.get('user_id'), user_name=x.get('user_name') )).with_output_types(RowTypeConstraint.from_fields(schema)) | 'SQL' >> SqlTransform("SELECT user_id, COUNT(*) AS msg_count FROM PCOLLECTION GROUP BY user_id") )
Question: I'm working on Python project that involves processing a very large graph - it has millions of nodes and edges. The goal is to perform a breadth-first search (BFS) or depth-first search (DFS) from a given start node to compute shortest paths or reachability. Here's the challenge :  The graph is too large to fit comfortably into memory if stored natively. The traversal needs to be fast, ideally leveraging multiple CPU cores. I want to avoid race conditions and ensure thread-safe updates to shared data structures. The graph data is stored as adjacency lists in files, and I want to process it efficiently without loading the entire graph at once.  Currently, I have a basic BFS implementation using a Python dictionary for adjacency, but it runs slowly and hits memory limits: from collections import deque def bfs(graph, start): visited = set() queue = deque([start]) while queue: node = queue.popleft() if node not in visited: visited.add(node) queue.extend(graph.get(node, [])) return visited  Question  How can I parallelize BFS/DFS in Python to speed up traversal? Should I use multiprocessing, concurrent.features, or another approach?  I'm open to suggestions involving alternative algorithms (like bidirectional BFS), external databases, or memory-mapped files. I’ve tried the following:  A basic BFS using Python’s built-in set, deque, and dict, as shown in the code.  Storing the entire adjacency list in memory using a dictionary of lists, but this doesn’t scale well for millions of nodes.  I attempted to use the multiprocessing module to run multiple BFS instances in parallel from different starting points, but coordinating shared state and combining results without race conditions became complex.  Looked into NetworkX, but found it quite slow and memory-heavy for very large graphs.  I also tried reading adjacency data from a file line-by-line and processing it lazily, but traversal logic became messy and error-prone.   I expected to be able to:  Traverse the graph quickly (within a few seconds) even with millions of nodes.  Use multicore processing to speed up traversal.  Efficiently manage memory without crashing due to RAM limitations.  Ideally, stream data or keep only parts of the graph in memory at any given time.  I know Python isn’t the fastest language for this, but I’d like to push it as far as reasonably possible before considering rewriting in C++ or Rust.  

Answer: You can use combination of "pyspark + graphframes" to achieve this. Sample Code from pyspark.sql import SparkSession from graphframes import GraphFrame # Create Spark session spark = SparkSession.builder \ .appName("BFS") \ .config("spark.jars.packages", "graphframes:graphframes:0.8.2-spark3.0-s_2.12") \ .getOrCreate() # Define vertices and edges as DataFrames vertices = spark.createDataFrame([ ("a", "Alice"), ("b", "Bob"), ("c", "Charlie"), ("d", "David"), ("e", "Esther") ], ["id", "name"]) edges = spark.createDataFrame([ ("a", "b"), ("b", "c"), ("c", "d"), ("d", "e") ], ["src", "dst"]) # Create GraphFrame g = GraphFrame(vertices, edges) # Run BFS from "a" to "e" results = g.bfs(fromExpr="id = 'a'", toExpr="id = 'e'") results.show(truncate=False)
Question: Good evening, Based on tutorial here : Flet tutorial i am following the codes , everything and i would like to ask one question, here is my code : import flet as ft from flet import * def main(page:ft.Page): greeting =ft.Ref[ft.Column]() def hello_here(e): greeting.current.controls.append(ft.Text(f'hello {first_name.current.value} {last_name.current.value}')) first_name.current.value ="" last_name.current.value ="" first_name.current.focus() page.update() page.title="My greatings" first_name =ft.Ref[ft.TextField]() last_name =ft.Ref[ft.TextField]() page.add( ft.TextField(ref=first_name,label="Enter First Name",autofocus=True), ft.TextField(ref=last_name,label="Enter Last Name"), ft.Column(ref=greeting), ft.ElevatedButton(text='Say Hello', on_click=hello_here) ) ft.app(target=main)  if i run it :flet flet_ref_example.py i got this result :  as you see, every time, we are adding elements, it is printed on page one following to another, is it possible to make a simple correction and old text should be replaced by new one? i think this line should be corrected  greeting.current.controls.append(ft.Text(f'hello {first_name.current.value} {last_name.current.value}'))  but how?

Answer: Simply assign new list instead of appending to old one def hello_here(e): greeting.current.controls = [ft.Text(f'hello {first_name.current.value} {last_name.current.value}')] # ... rest ...   EDIT: If flet would make some problem with memory leak when you assign new list then you can also use .clear() to remove old elements without creating new list in memory. def hello_here(e): greeting.current.controls.clear() greeting.current.controls.append(ft.Text(f'hello {first_name.current.value} {last_name.current.value}')) # ... rest ...
Question: I've read the documentation and simply cannot understand why I can't seem to achieve my objective. All I want to do is output integers with a thousands separator where appropriate. I'm loading a spreadsheet from my local machine that is in the public domain here Here's my MRE: import pandas as pd WORKBOOK = "/Volumes/Spare/Downloads/prize-june-2025.xlsx" def my_formatter(v): return f"{v:,d}" if isinstance(v, int) else v df = pd.read_excel(WORKBOOK, header=2, usecols="B,C,E:H") print(df.dtypes) df.style.format(my_formatter) print(df.head())  Output: Prize Value int64 Winning Bond NO. object Total V of Holding int64 Area object Val of Bond int64 Dt of Pur datetime64[ns] dtype: object Prize Value Winning Bond NO. Total V of Holding Area Val of Bond Dt of Pur 0 1000000 103FE583469 50000 Stockport 5000 2005-11-29 1 1000000 352AC359547 50000 Edinburgh, City Of 5000 2019-02-11 2 100000 581WF624503 50000 Birmingham 20000 2024-06-03 3 100000 265SM364866 50000 Hertfordshire 32500 2016-01-31 4 100000 570HE759643 11000 Hertfordshire 11000 2024-02-22  I have determined that my_formatter() is never called and I have no idea why.

Answer: Your approach works fine, however style does not modify the DataFrame in place. Instead it returns a special object that can be displayed (for instance in a notebook) or exported to a file. You could see the HTML version in jupyter with: df.style.format(my_formatter)  (this should be the last statement of the current cell!) Or a text version with: print(df.style.format(my_formatter).to_string())    Note that your approach is however quite slow. If you have homogeneous dtypes, you could take advantage of the builtin thousands parameter: df.style.format(thousands=',')  Or if you want to use a custom format per column, pass a dictionary: df.style.format({c: '{:,d}' for c in df.select_dtypes('number')})  And, finally, if you want to change the data to strings and return a DataFrame, you would need to use map: out = df.map(my_formatter)
Question: I've got a matplotlib graph with labelled X-ticks:  The labels repeat (in case that's relevant). In the real graph, there is a multi-level X-axis with more clarification in the lower layers. That works fine, but I want to be able to hover the mouse and see the X-coordinate in the top-right of the graph. Whenever I set xticks to labels, I just get a blank X-coordinate:  If I use ax.xaxis.set_major_formatter('{x:g}'), it gets rid of my labels but the cursor coordinate starts working:  Is there any way to make the cursor location still show the X coordinate even when I have a labelled X axis? This also affects mplcursors: it shows the X value as empty if I click on a line between points or with the label if I click exactly on a point (whereas I'd like to see the underlying numerical X-coordinate as "A" is a bit meaningless without the context from the secondary axis):  Source code: import matplotlib.pyplot as plt import numpy as np import mplcursors x = np.arange(0, 9) y = np.random.rand(*x.shape) labels = ['A', 'B', 'C']*3 fig, ax = plt.subplots() ax.plot(x, y, 'bx-', label='random') ax.set_xticks(x, labels=labels) # This makes the coordinate display work, but gets rid of the labels: # ax.xaxis.set_major_formatter('{x:g}') mplcursors.cursor(multiple=True) plt.show() 

Answer: From [ Matplotlib mouse cursor coordinates not shown for empty tick labels ] adding ax.format_coord = lambda x, y: 'x={:g}, y={:g}'.format(x, y) anywhere before plt.show() makes it show the right coordinates. If you want to display A, B or C as x coordinates, you can make a custom coordinates format string function in which you round the current x position to an integer to get an index and get the corresponding label from labels def custom_format_coord(x_val, y_val): ix = int(round(x_val)) if 0 <= ix < len(labels): x_label = labels[ix] else: x_label = f"{x_val}" return f"x={x_label}, y={y_val:g}" ax.format_coord = custom_format_coord
Question: I'm using streamlit's st.data_editor, and I have this DataFrame: import streamlit as st import pandas as pd df = pd.DataFrame( [ {"command": "test 1", "rating": 4, "is_widget": True}, {"command": "test 2", "rating": 5, "is_widget": False}, {"command": "test 3", "rating": 3, "is_widget": True}, ] ) edited_df = st.data_editor(df)  is there a way to color a specific cell based on a condition? I want to colorize in yellow the cell (in the rating col) when the value is less then 2.

Answer: According to docs, you can't color cell if it's not in disabled column.  Styles from pandas.Styler will only be applied to non-editable columns.  If disabling rating column is acceptable to you, you can use pandas.Styler like this: import streamlit as st import pandas as pd df = pd.DataFrame( [ {"command": "test 1", "rating": 4, "is_widget": True}, {"command": "test 2", "rating": 5, "is_widget": False}, {"command": "test 3", "rating": 1, "is_widget": True}, ] ) # you can use .applymap(), but it's deprecated df = df.style.map( lambda val: 'background-color: yellow' if val < 2 else '', subset=['rating'] ) edited_df = st.data_editor(df, disabled=["rating"])