eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Can ANSI escape code SGR 38 - Set foreground color with argument 2;r;g;b be used with print function? Example of use with code 33 is of course OKBLUE = '\033[94m'  I would like to use 038 instead to be able to use any RGB color. Is that posible? I tried GREEN = '\038[2;0;153;0m' ENDC = '\033[0m' print(f"{GREEN} some text {ENDC}")  Expected to change the color of "some text" in green

Answer: To use an RGB color space within the terminal* the following escape sequence can be used: # Print Hello! in lime green text. print('\033[38;2;146;255;12mHello!\033[0m') # ^ # | # \ The 38 goes here, to indicate a foreground colour. # Print Hello! in white text on a fuschia background. print('\033[48;2;246;45;112mHello!\033[0m')  Explanation: \033[38;2;146;255;12mHello!\033[0m ^ ^ ^ ^ ^ ^ ^ ^ ^ | | | R G B | | | | | | | | | \ Reset the colour to default | | | | | | | | | | | \ Escape character | | | | | | | | \ R;G;B \ Text to print | | | | | \ Indicate the following sequence is RGB | | | \ Code to instruct the setting of an 8 or 24-bit foreground (text) colour | \ Escape character  The use of 38;2 indicates an RGB (foreground) sequence is to follow. However, the use of 38;5 indicates the following (foreground) value comes from the 256-colour table. To clarify what appears to be a misconception, \033 (octal) or \x1b (hexidecimal) corresponds to the ASCII table's ESC character, which is used here to introduce an escape sequence of terminal text colouring. Whereas the 38 is used to instruct the following 8 or 24-bit colour to be set as foreground, (after the escape sequence has been introduced). Additionally, 48 can be used to set the background colour, as demonstrated in the code example above. *Providing the terminal emulator supports 24-bit colour sequences. (e.g. Xterm, GNOME terminal, etc.) Link to the Wikipedia article which explains this topic of 24-colour (RGB) in greater depth.
Question: I'm using Selenium in Python (3.11) with a Firefox (107) driver. With the driver I navigate to a page which, after several actions, triggers an OS alert (prompting me to launch a program). When this alert pops up, the driver hangs, and only once it is closed manually does my script continue to run. I have tried driver.quit(), as well as using os.system("taskkill /F /pid " + str(process.ProcessId))  with the driver's PID, with no luck. I have managed to prevent the pop-up from popping up with options.set_preference("security.external_protocol_requires_permission", False)  but the code still hangs the same way at the point where the popup would have popped up. I don't care whether the program launches or not, I just need my code to not require human intervention at this key point. here is a minimal example of what I currently have: from selenium.webdriver import ActionChains, Keys from selenium.webdriver.firefox.options import Options from seleniumwire import webdriver options = Options() options.binary_location = r'C:\Program Files\Mozilla Firefox\firefox.exe' options.set_preference("security.external_protocol_requires_permission", False) driver = webdriver.Firefox(options=options) # Go to the page driver.get(url) user_field = driver.find_element("id", "UserName") user_field.send_keys(username) pass_field = driver.find_element("id", "Password") pass_field.send_keys(password) pass_field.send_keys(Keys.ENTER) #this is the point where the pop up appears reqs = driver.requests print("Success!") driver.quit() 

Answer: There are some prefs you can try profile = webdriver.FirefoxProfile() profile.set_preference('dom.push.enabled', False) # or profile = webdriver.FirefoxProfile() profile.set_preference('dom.webnotifications.enabled', False) profile.set_preference('dom.webnotifications.serviceworker.enabled', False)
Question: In my python file I always start with the following lines import sys import matplotlib as mpl sys.append('C:\\MyPackages') rc_fonts = { "text.usetex": True, 'font.size': 20, 'text.latex.preamble': r"\usepackage{bm}", } mpl.rcParams.update(rc_fonts)  Is there a way to indicate to VScode that each time I create a new file.py, it will start with the previous lines ? For now, I copy/paste a 'template.py' but this is not really convenient. And because I work with Windows, I also tried to add 'C:\MyPackages' to the user variables Path but it didn't work.

Answer: For Doing this kind of repetitive task we can use snippets in VSCode. Step 1 : Hit > shift+ctrl+p open command palette. Step 2 : Select Snippets: Configure User Snippets Step 3 : Select Python Step 4 : paste below code in python.json file. change prefix value. like "prefix": "hedwin" so now when you type hedwin vscode will paste our code snippet "": { "prefix": "", "body": [ "import sys", "import matplotlib as mpl", "sys.append('C:\\\\MyPackages')", "", "rc_fonts = {", " \"text.usetex\": True,", " 'font.size': 20,", " 'text.latex.preamble': r\"\\usepackage{bm}\",", "}", "mpl.rcParams.update(rc_fonts)" ], "description": "" }  For making snippet : snippet generator
Question: I'm trying to merge two dictionaries based on key value. However, I'm not able to achieve it. Below is the way I tried solving. dict1 = {4: [741, 114, 306, 70], 2: [77, 325, 505, 144], 3: [937, 339, 612, 100], 1: [52, 811, 1593, 350]} dict2 = {1: 'A', 2: 'B', 3: 'C', 4: 'D'}  My resultant dictionary should be output = {'D': [741, 114, 306, 70], 'B': [77, 325, 505, 144], 'C': [937, 339, 612, 100], 'A': [52, 811, 1593, 350]}  My code def mergeDictionary(dict_obj1, dict_obj2): dict_obj3 = {**dict_obj1, **dict_obj2} for key, value in dict_obj3.items(): if key in dict_obj1 and key in dict_obj2: dict_obj3[key] = [value , dict_obj1[key]] return dict_obj3 dict_3 = mergeDictionary(dict1, dict2)  But I'm getting this as output dict_3={4: ['D', [741, 114, 306, 70]], 2: ['B', [77, 325, 505, 144]], 3: ['C', [937, 339, 612, 100]], 1: ['A', [52, 811, 1593, 350]]} 

Answer: Use a simple dictionary comprehension: output = {dict2[k]: v for k,v in dict1.items()}  Output: {'D': [741, 114, 306, 70], 'B': [77, 325, 505, 144], 'C': [937, 339, 612, 100], 'A': [52, 811, 1593, 350]}
Question: I'm trying to use pandas.DataFrame.assign in Pandas 1.5.2. Let's consider this code, for instance: df = pd.DataFrame({"col1":[1,2,3], "col2": [4,5,6]}) df.assign( test1="hello", test2=df.test1 + " world" )  I'm facing this error:  AttributeError: 'DataFrame' object has no attribute 'test1'  However, it's explicitly stated in the documentation that:  Assigning multiple columns within the same assign is possible. Later items in **kwargs may refer to newly created or modified columns in df; items are computed and assigned into df in order.  So I don't understand: how can I refer to newly created or modified columns in df when calling assign?

Answer: You can pass a callable to assign. Here use a lambda to reference the DataFrame.  Parameters **kwargsdict of {str: callable or Series} The column names are keywords. If the values are callable, they are computed on the DataFrame and assigned to the new columns. The callable must not change input DataFrame (though pandas doesn’t check it). If the values are not callable, (e.g. a Series, scalar, or array), they are simply assigned.  df = pd.DataFrame({"col1":[1,2,3], "col2": [4,5,6]}) df.assign( test1="hello", test2=lambda d: d.test1 + " world" )  Output:  col1 col2 test1 test2 0 1 4 hello hello world 1 2 5 hello hello world 2 3 6 hello hello world
Question: I'm trying to send HTTPS requests as quickly as possible. I know this would have to be concurrent requests due to my goal being 150 to 500+ requests a second. I've searched everywhere, but get no Python 3.11+ answer or one that doesn't give me errors. I'm trying to avoid AIOHTTP as the rigmarole of setting it up was a pain, which didn't even work. The input should be an array or URLs and the output an array of the html string.

Answer: This works, getting around 250+ requests a second. This solution does work on Windows 10. You may have to pip install for concurrent and requests. import time import requests import concurrent.futures start = int(time.time()) # get time before the requests are sent urls = [] # input URLs/IPs array responses = [] # output content of each request as string in an array # create an list of 5000 sites to test with for y in range(5000):urls.append("https://example.com") def send(url):responses.append(requests.get(url).content) with concurrent.futures.ThreadPoolExecutor(max_workers=10000) as executor: futures = [] for url in urls:futures.append(executor.submit(send, url)) end = int(time.time()) # get time after stuff finishes print(str(round(len(urls)/(end - start),0))+"/sec") # get average requests per second  Output: 286.0/sec Note: If your code requires something extremely time dependent, replace the middle part with this: with concurrent.futures.ThreadPoolExecutor(max_workers=10000) as executor: futures = [] for url in urls: futures.append(executor.submit(send, url)) for future in concurrent.futures.as_completed(futures): responses.append(future.result())  This is a modified version of what this site showed in an example. The secret sauce is the max_workers=10000. Otherwise, it would average about 80/sec. Although, when setting it to beyond 1000, there wasn't any boost in speed.
Question: I have the following input data: df = pd.DataFrame({"ID" : [1, 1, 1, 2, 2, 2, 2], "length" : [0.7, 0.7, 0.7, 0.8, 0.6, 0.6, 0.7], "height" : [7, 9, np.nan, 4, 8, np.nan, 5]}) df ID length height 0 1 0.7 7 1 1 0.7 9 2 1 0.7 np.nan 3 2 0.8 4 4 2 0.6 8 5 2 0.6 np.nan 6 2 0.7 5  I want to be able to fill the NaN if a group of "ID" all have the same "length", fill with the maximum "height" in that group of "ID", else fill with the "height" that correspond to the maximum length in that group. Required Output:  ID length height 0 1 0.7 7 1 1 0.7 9 2 1 0.7 9 3 2 0.8 4 4 2 0.6 8 5 2 0.6 4 6 2 0.7 5  Thanks.

Answer: You could try with sort_value then we use groupby find the last #last will find the last not NaN value df.height.fillna(df.sort_values(['length','height']).groupby(['ID'])['height'].transform('last'),inplace=True) df Out[296]: ID length height 0 1 0.7 7.0 1 1 0.7 9.0 2 1 0.7 9.0 3 2 0.8 4.0 4 2 0.6 8.0 5 2 0.6 4.0 6 2 0.7 5.0
Question: I'm on Windows 10 using VS Code 1.73.1 and am retrofitting my program with the Python logging module. My program is generally functioning. The main thing I did is change all the print statements to logger.debug and I know the variable formatting needs to be changed from {} to %s. I also added the encoding flag to my file handler. A couple more things:  When I run it from the VS Code command line, it does create a file with debug statements but does not display any output to the Terminal, Debug Console or Output windows. When I use the F5 function to run it, it does NOT create a file or display any console output anywhere.  print('something') works and displays in either the Terminal or Debug Console depending on the launch.json setting, but logger.debug('something') does not display in either console.  My request/question: Using logger.debug, why is nothing printing to console, and no file is even created? I also tried running the code below in its own separate .py file with the same problem:  print displays to the Debug Console, but logger.debug does not. Logging code block: import logging # Create a custom logger logger = logging.getLogger(__name__) logger.setLevel(logging.DEBUG) # Streaming Handler c_handler = logging.StreamHandler() c_handler.setLevel(logging.INFO) c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s') c_handler.setFormatter(c_format) logger.addHandler(c_handler) # File Handler f_handler = logging.FileHandler('download_shelf.log', encoding='utf-8') f_handler.setLevel(logging.DEBUG) f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') f_handler.setFormatter(f_format) logger.addHandler(f_handler) logger.debug(f"Log messages:") ### `print` statement displays in either the Terminal ### or Debug Console depending on `launch.json` setting  I made sure my launch.json file was set up in the local .vscode folder (see bottom code section). However, I did have to manually create this file, then restarted VS Code. Information I found said there was a link in the Debug panel, which I know I've seen before. It wasn't there for this file. My directory structure: toplevel/ .vscode/ launch.json src/ subdirectory/ myfile.py  My launch.json file contains "console": "internalConsole" { "configurations": [ {"name":"Python: Current File", "type":"python", "request":"launch", "program":"${file}", // "console":"integratedTerminal", "console": "internalConsole", "justMyCode":true }, ] } 

Answer: After using a different logger example, I realized the problem was the Level setting in the first example I used had "INFO" and not "DEBUG" so nothing was showing up. Oops... import logging logger = logging.getLogger('simple_example') logger.setLevel(logging.DEBUG) console = logging.StreamHandler() console.setLevel(level=logging.DEBUG) formatter = logging.Formatter('%(levelname)s : %(message)s') console.setFormatter(formatter) logger.addHandler(console) logger.debug('simple message')
Question: I wrote scripts performing the same computations with dict and hashmap in Rust and in Python. Somehow the Python version is more than 10x faster. How is that happens? Rust script: ` use std::collections::HashMap; use std::time::Instant; fn main() { let now = Instant::now(); let mut h = HashMap::new(); for i in 0..1000000 { h.insert(i, i); } let elapsed = now.elapsed(); println!("Elapsed: {:.2?}", elapsed); }  Output: Elapsed: 828.73ms Python script: import time start = time.time() d = dict() for i in range(1000000): d[i] = i print(f"Elapsed: {(time.time() - start) * 1000:.2f}ms")  Output: Elapsed: 64.93ms The same is for string keys. I searched for different workarounds about different hashers for HashMap but none of them is giving more than 10x speed up

Answer: Rust by default doesn't have optimizations enabled, because it's annoying for debugging. That makes it, in debug mode, slower than Python. Enabling optimizations should fix that problem. This behaviour is not specific to Rust, it's the default for most compilers (like gcc/g++ or clang for C/C++, both require the -O3 flag for maximum performance). You can enable optimizations in Rust by adding the --release flag to the respective cargo commands, like: cargo run --release  Python doesn't distinguish between with or without optimizations, because Python is an interpreted language and doesn't have an (explicit) compilation step. This is the behavior of your code on my machine:  Python: ~180ms Rust: ~1750 ms Rust (with --release): ~160ms  Python's dict implementation is most likely written in C and heavily optimized, so it should be similar in performance to Rust's HashMap, which is exactly what I see on my machine.
Question: After pip3 installing tensorflow and the transformers library, I'm receiving the titular error when I try loading this from transformers import pipeline classifier = pipeline("text-classification",model='bhadresh-savani/distilbert-base-uncased-emotion')  The error traceback looks like: RuntimeError: Failed to import transformers.models.distilbert.modeling_tf_distilbert because of the following error (look up to see its traceback): No module named 'keras.saving.hdf5_format'  I have ensured keras got installed with transformers, so I'm not sure why it isn't working

Answer: If you are using the latest version of TensorFlow and Keras then you have to try this code and you have got this error as shown below RuntimeError: Failed to import transformers.models.distilbert.modeling_tf_distilbert because of the following error (look up to see its traceback): No module named 'keras.saving.hdf5_format'  Now, expand this error traces as I have shown below  Now click on the 14 frames and select as shown below  Now comment this line as shown in the picture below  Now, try this and your error will gone. The problem is that this is in the older version of keras and you are using the latest version of keras. So, you can skip all these steps and go back to the older version and it will work eventually.
Question: I'm trying to use pyFirmata, but I can't get it to work. Even the most basic of the library does not work. I guess there is something wrong with the library code. from pyfirmata import Arduino,util import time port = 'COM5' board = Arduino(port)  I get this error: Traceback (most recent call last): File "c:\Users\Public\pythonpublic\arduino.py", line 5, in <module> board = Arduino(port) ^^^^^^^^^^^^^ File "C:\Users\marce\AppData\Roaming\Python\Python311\site-packages\pyfirmata\__init__.py", line 19, in __init__ super(Arduino, self).__init__(*args, **kwargs) File "C:\Users\marce\AppData\Roaming\Python\Python311\site-packages\pyfirmata\pyfirmata.py", line 101, in __init__ self.setup_layout(layout) File "C:\Users\marce\AppData\Roaming\Python\Python311\site-packages\pyfirmata\pyfirmata.py", line 157, in setup_layout self._set_default_handlers() File "C:\Users\marce\AppData\Roaming\Python\Python311\site-packages\pyfirmata\pyfirmata.py", line 161, in _set_default_handlers self.add_cmd_handler(ANALOG_MESSAGE, self._handle_analog_message) File "C:\Users\marce\AppData\Roaming\Python\Python311\site-packages\pyfirmata\pyfirmata.py", line 185, in add_cmd_handler len_args = len(inspect.getargspec(func)[0]) ^^^^^^^^^^^^^^^^^^ AttributeError: module 'inspect' has no attribute 'getargspec'. Did you mean: 'getargs'? 

Answer: According to the first line of pyFirmata docs:  It runs on Python 2.7, 3.6 and 3.7  You are using Python 3.11. The inspect (core library module) has changed since Python 3.7.
Question: Python 3.6.5 Using this answer as a guide, I attempted to see whether some modules, such as math were imported. But Python tells me they are all imported when they are not. >>> import sys >>> 'math' in sys.modules True >>> 'math' not in sys.modules False >>> math.pi Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'math' is not defined >>> import math >>> 'math' in sys.modules True >>> math.pi 3.141592653589793 

Answer: to explain this, let's define this function: def import_math(): import math import_math()  the above function will import the module math, but only in its local scope, anyone that tries to reference math outside of it will get a name error, because math is not defined in the global scope. any module that is imported is saved into sys.modules so a call to check import_math() print("math" in sys.modules)  will print True, because sys.modules caches any module that is loaded anywhere, whether or not it was available in the global scope, a very simple way to define math in the global scope would then to import_math() math = sys.modules["math"]  which will convert it from being only in sys.modules to being in the global scope, this is just equivalent to import math  which defines a variable math in the global scope that points to the module math. now if you want to see whether "math" exists in the global scope is to check if it is in the global scope directly. print("math" in globals()) print("math" in locals())  which will print false if "math" wasn't imported into the global or local scope and is therefore inaccessable.
Question: I have the following list: lst = ['L38A', '38', 'L', 'A', '-6.7742', '-3.5671', '0.00226028', '0.4888', 'L38C', '38', 'L', 'C', '-7.7904', '-6.6306', '0.0', '0.4888', 'L38D', '38', 'L', 'D', '-6.3475', '-3.0068', '0.00398551', '0.4888', 'L38E', '38', 'L', 'E', '-6.4752', '-3.4645', '0.00250913', '0.4888'] I'm looking to extract the first element (posiiton 0) in the list ('L38A') and the 5th element (position 4) (-6.7742) multiple times: Desired output [('L38A','-6.7742'), ('L38C','-7.7904'), ('L38D','-6.3475')...('L38E','-6.4752')] I have tried: lst[::5]

Answer: We can handle this via a zip operation and list comprehension: lst = ['L38A', '38', 'L', 'A', '-6.7742', '-3.5671', '0.00226028', '0.4888', 'L38C', '38', 'L', 'C', '-7.7904', '-6.6306', '0.0', '0.4888', 'L38D', '38', 'L', 'D', '-6.3475', '-3.0068', '0.00398551', '0.4888', 'L38E', '38', 'L', 'E', '-6.4752', '-3.4645', '0.00250913', '0.4888'] it = [iter(lst)] * 8 output = [(x[0], x[4]) for x in zip(*it)] print(output)  This prints: [('L38A', '-6.7742'), ('L38C', '-7.7904'), ('L38D', '-6.3475'), ('L38E', '-6.4752')]  The first zip generates a list of 8 tuples, with 8 being the number of values in each cycle. The comprehension then generates a list of 2-tuples containing the first and fifth elements.
Question: I have a graph that looks like this:  I want to sort the color combinations for the dots on this, to achieve something like one color for all the versions that start with 17, different one for 18 and lastly the 20. I don't know if I can do this in plotly since it is very specific and found no information on this. Is it possible to also add different colors for the sub versions, like for 17 we have different categories like 17.2.3, 17.2.2 and so on. here is my data:  Days Difference commitDate Year-Month 18538 1291 2021-01-25 11:15:48 2020-01 18539 1135 2020-11-30 05:11:41 2020-11 18540 1100 2020-08-17 07:22:54 2020-08 18541 900 2020-08-17 07:12:05 2020-01 18542 340 2020-01-09 06:21:03 2020-01 18543 203 2019-11-20 06:03:28 2019-11 18544 120 2019-11-15 02:50:28 2019-11  This is the code I have written so far: data1= final_api.query("info_title=='Avi CertificateManagementProfile Object API'") data1['commitDate'] = pd.to_datetime(final_api['commitDate']) import plotly.graph_objects as go fig = go.Figure() fig.add_trace(go.Scatter(mode='lines', x=data1["commitDate"], y=data1["Days_difference"], line_color='black', line_width=1, line_shape='vh', showlegend=False ) ) fig.add_trace(go.Scatter(mode='markers', x=data1["commitDate"], y=data1["Days_difference"], marker=dict(color=data1['day'], colorscale='plasma', size=10), showlegend=False ) ) for _,row in data1.iterrows(): fig.add_annotation( go.layout.Annotation( x=row["commitDate"], y=row["Days_difference"], text=row['info_version'], showarrow=False, align='center', yanchor='bottom', yshift=10, textangle=-90) ) fig.update_layout(template='plotly_white',title_text=' Version Change in Avi CertificateManagementProfile Object API over its Age',title_x=0.5, xaxis_title='Year-Month', yaxis_title='Age of the API (in days)', xaxis_tickformat = '%d %B (%a)<br>%Y', height=700, width=1300) fig.update_xaxes(showline=True, linewidth=1, linecolor='black', mirror=True) fig.update_yaxes(showline=True, linewidth=1, linecolor='black', mirror=True) fig.show()  Any help or guidance will be appreciated.

Answer: Currently you are assigning marker color based on the 'day' column in your argument marker=dict(color=data1['day'], colorscale='plasma', size=10), but it sounds like you want to assign the color based on the major version. You can extract the major version from the info_version column, and store it in a new column called "major_version": data1['major_version'] = data1['info_version'].str.split('.').str.get(0)  Then I would suggest using px.scatter which makes it easier to pass a categorical column as a color (see the documentation here). Then you would restructure your code to plot the markers first, and then the lines connecting the markers: import plotly.express as px fig = px.scatter(data1, x='commitDate', y='Days_difference', color='major_version') fig.add_trace(go.Scatter(mode='lines', x=data1["commitDate"], y=data1["Days_difference"], line_color='black', line_width=1, line_shape='vh', showlegend=False ) ) fig.update_layout(showlegend=False)  And keep the rest of your code the same.