eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I'm attempting to parse HTML time strings in Python: from datetime import datetime input = "2025-03-24T07:01:53+00:00" output = datetime.strptime(input, "%Y-%m-%d%X%:z") print(output)  Running this with Python 3.13 returns the following error: Traceback (most recent call last): File "/Users/dread_pirate_roberts/html_time_converter/main.py", line 69, in <module> output = datetime.strptime(input, "%Y-%m-%d%X%:z") File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/_strptime.py", line 674, in _strptime_datetime tt, fraction, gmtoff_fraction = _strptime(data_string, format) ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/_strptime.py", line 445, in _strptime raise ValueError("'%s' is a bad directive in format '%s'" % (bad_directive, format)) from None ValueError: ':' is a bad directive in format '%Y-%m-%d%X%:z'  This doesn't make sense to me because %:z was added in Python 3.12. Edit: I discovered that my code actually has 2 bugs. For one, the format string should include a T. Also, InSync's answer, so the code should be: from datetime import datetime input = "2025-03-24T07:01:53+00:00" output = datetime.strptime(input, "%Y-%m-%dT%X%z") print(output) 

Answer: This is a known CPython bug. For now (3.13), %:z only works in .strftime(): >>> import datetime >>> d = datetime.datetime.now(tz = datetime.UTC) >>> d datetime.datetime(2025, 1, 1, 1, 1, 1, 111111, tzinfo=datetime.timezone.utc) >>> d.strftime('%z') '+0000' >>> d.strftime('%:z') '+00:00'  It should also be noted that %z already parses +00:00 and similar inputs: >>> datetime.datetime.strptime('+00:30', '%z') datetime.datetime(1900, 1, 1, 0, 0, tzinfo=datetime.timezone(datetime.timedelta(seconds=1800)))
Question: I have several images of animals in the same directory as the script. How can I modify the following script to process an image but force the output to only be a single selection from a list: from pathlib import Path import base64 import requests def encode_image_to_base64(image_path): """Convert an image file to base64 string.""" return base64.b64encode(image_path.read_bytes()).decode('utf-8') def extract_text_from_image(image_path): """Send image to local Llama API and get text description.""" base64_image = encode_image_to_base64(image_path) payload = { "model": "llama3.2-vision", "stream": False, "messages": [ { "role": "user", "content": ( "With just one word, classify this image into one of these exact categories:\n" "- dog\n" "- cat\n" "- butterfly\n" ), "images": [base64_image] } ] } response = requests.post( "http://localhost:11434/api/chat", json=payload, headers={"Content-Type": "application/json"} ) return response.json().get('message', {}).get('content', 'No text extracted') def process_directory(): """Process all images in current directory and create text files.""" for image_path in Path('.').glob('*'): if image_path.suffix.lower() in {'.png', '.jpg', '.jpeg', '.gif', '.bmp', '.webp'}: print(f"\nProcessing {image_path}...") text = extract_text_from_image(image_path) image_path.with_suffix('.txt').write_text(text, encoding='utf-8') print(f"Created {image_path.with_suffix('.txt')}") process_directory()  However, despite different prompt engineering, I get some answers that will do more than just select from a list. For example, it may occassionally output "From the image, there is a winged insect, therefore my guess is "butterfly." ANSWER: Butterfly." If I define the list as allowed_options = ['dog', 'cat', 'butterfly'] I only want it to output a single string from that list and nothing else.

Answer: Llama3.2-vision supports structured outputs, so you can specify the schema of the response you want, and the model should follow it. Your request should look like this: payload = { "model": "llama3.2-vision", "stream": False, "messages": [ { "role": "user", "content": ( "Classify this image into one of these exact categories:\n" "- dog\n" "- cat\n" "- butterfly\n" ), "images": [base64_image] } ], "format": { "type": "object", "properties": { "animal": { "enum": [ "dog", "cat", "butterfly" ], "type": "string" } }, "required": [ "animal" ] } }  And now the output should be a JSON with an attribute animal. For example: { "animal": "dog" }
Question: I want to add aliases to consecutive occurrences of the same gene name in column gene_id. If the gene_id value is unique, it should be unchanged. Here is my example input: df_genes_data = {'gene_id': ['g0', 'g1', 'g1', 'g2', 'g3', 'g4', 'g4', 'g4']} df_genes = pd.DataFrame.from_dict(df_genes_data) print(df_genes.to_string()) gene_id 0 g0 1 g1 2 g1 3 g2 4 g3 5 g4 6 g4 7 g4  and there is the desired output:  gene_id 0 g0 1 g1_TE1 2 g1_TE2 3 g2 4 g3 5 g4_TE1 6 g4_TE2 7 g4_TE3  Any ideas on how to perform it? I've been looking for solutions but found only ways to count consecutive occurrences, not to label them with aliases. EDIT: I've tried to find gene_id values which occur more than once in my data: rep = [] gene_list = df_genes['gene_id'] for idx in range(0, len(gene_list) - 1): if gene_list[idx] == gene_list[idx + 1]: rep.append(gene_list[idx]) rep = list(set(rep)) print("Consecutive identical gene names are : " + str(rep))  but I have no idea how to add desired aliases to them.

Answer: Use shift+ne+cumsum to group the consecutive values, then groupby.transform('size') to identify the groups of more than 2 values, and groupby.cumcount to increment the name: # Series as name for shorter reference s = df_genes['gene_id'] # group consecutive occurrences group = s.ne(s.shift()).cumsum() # form group and save as "g" for efficiency g = s.groupby(group) # identify groups with more than 1 value m = g.transform('size').gt(1) # increment values df_genes.loc[m, 'gene_id'] += '_TE'+g.cumcount().add(1).astype(str)  Output:  gene_id 0 g0 1 g1_TE1 2 g1_TE2 3 g2 4 g3 5 g4_TE1 6 g4_TE2 7 g4_TE3  Intermediates:  gene_id group m cumcount+1 suffix 0 g0 1 False 1 1 g1 2 True 1 _TE1 2 g1 2 True 2 _TE2 3 g2 3 False 1 4 g3 4 False 1 5 g4 5 True 1 _TE1 6 g4 5 True 2 _TE2 7 g4 5 True 3 _TE3
Question: I was doing this question on Codeforces (2057B - Gorilla and the Exam) where I had to use the frequencies of different numbers in a list. First I tried calculating it with a dictionary but I got TLE verdict (Time limit exceeded). After reading the editorial I implemented it by sorting the input list and then using comparisons with the previous element to generate the frequency array. This was faster than the approach using the dictionary but sorting is an O(nlogn) operation which should take more time than creating a dictionary which only requires O(n) time. Dictionary approach (slower) t= int(input()) for _ in range(t): n, k = map(int, input().split()) arr = map(int, input().split()) frequence_table = {} for num in arr: try: frequence_table[num] += 1 except: frequence_table[num] = 1 freq = list(frequence_table.values()) freqs = sorted(freq, reverse=True) while k>0: if k>=freqs[-1]: k -= freqs.pop() else: break print(max(len(freqs), 1))  Sorting based approach (faster) t= int(input()) for _ in range(t): n, k = map(int, input().split()) arr = map(int, input().split()) arr = sorted(arr) freq = [1] for i in range(1, len(arr)): if arr[i] == arr[i-1]: freq[-1] += 1 else: freq.append(1) freqs = sorted(freq, reverse=True) while k>0: if k>=freqs[-1]: k -= freqs.pop() else: break print(max(len(freqs), 1)) 

Answer:  Why is using a dictionary slower than sorting a list to generate frequency array?  Apparently because one input is specially crafted to hack Python's dict implementation, so that building your dict doesn't take the expected linear time. It can take up to quadratic time, and likely they did that. See Python's TimeComplexity page (showing O(n) worst case time for each dict access) and Tutorial: How to hack hash table in Python 3 on CodeForces itself. One way to defeat it is to simply add 1 to all the numbers, i.e., change  for num in arr:  to this:  for num in arr: num += 1  Then the solution doesn't get stopped and rejected at the 1 second time limit but gets accepted with around 0.18 seconds. Despite doing more work, and without really changing the data like randomizing or sorting would. The effect is simply that Python's collision resolution takes different paths then, which is enough to thwart the brittle attack. Another way, also accepted in around 0.18 seconds, is to simply not convert to ints, instead counting the strings. I.e., change  arr = map(int, input().split())  to this:  arr = input().split()  That's also safer, as Python deliberately and unpredictably randomizes string hashes exactly to defeat such attacks:  the __hash__() values of str and bytes objects are “salted” with an unpredictable random value. [...] This is intended to provide protection against a denial-of-service caused by carefully chosen inputs that exploit the worst case performance of a dict insertion, O(n2) complexity.
Question: 1.000000000000001 ** float('inf') == float('inf')  while 1.0000000000000001 ** float('inf') == 1.0  What determines the exact threshold here? It seems to be an issue of float precision, where numbers below a certain threshold are considered the same as 1.0. Is there a way to get better precision in Python?

Answer: As others have said, this is due to IEEE754 floats not being able to represent your constants with the precision you're expecting. Python provides some useful tools that lets you see what's going on, specifically the math.nextafter method and and decimal module. The decimal module is useful to see the decimal expansion of a float (i.e. how us humans read/write numbers), e.g.: from decimal import Decimal as D a = 1.000000000000001 b = 1.0000000000000001 print(D(a), D(b))  which outputs: 1.0000000000000011102230246251565404236316680908203125 1  these are the actual values seen by your computer when you use those constants. As others have said, you can see that b is actually just 1 so it's "lost" the tailing digit you entered. You just have to know that it's going to do that as a programmer. To see nearby values the nextafter method is very useful, e.g.: import math c = math.nextafter(1.0, math.inf) print(c, D(c))  which outputs: 1.0000000000000002 1.0000000000000002220446049250313080847263336181640625  the first number is what Python prints by default and only includes enough precision to be able to unambiguously distinguish the value, the second uses the decimal module and is exact. Having a play with these is a great way to further understand what you're actually telling your computer to do. Chux also talked about hex representations of floats, in my experience these take a bit longer for people to understand but are another tool to know about when your code seems to be misbehaving. In Python you use float.hex() to see the actual bits that make up a float, using the format defined by C99, e.g.: one = 1.0 print(one.hex(), c.hex(), math.nextafter(c, math.inf).hex())  which will output: 0x1.0000000000000p+0 0x1.0000000000001p+0 0x1.0000000000002p+0  This output is closer to how your computer represents floats. The hexadecimal fractional digits can make these values somewhat awkward to interpret, but you can get there with some more magic numbers (i.e. these come from the spec). The second is saying something like 0x10000000000001 * 2**(0-52), the -52 is needed to "shift" the all but the first digit right into the fraction.
Question: from pydantic import BaseModel class User2(BaseModel): name:str age: int ''' def __setattr__(self, key, value): super().__setattr__(key, value) ''' user2 = User2(name="Alice", age=1) >>> user2.foo = 1 # Should report: Cannot assign to attribute "foo" for class "User2" \ Attribute "foo" is unknown  if I uncomment the __setattr__ function, user2.foo = 1, coc-pyright doesn't report an error. I want to use a __setattr__ method but also have it report an error. How can I do this?

Answer: The presence of __setattr__ does not report reportAttributeAccess issue for pyright as well as mypy. The trick is to not reveal that such a method exists. This can be done by defining it in an if not TYPE_CHECKING: block. Linters and checkers might mark the code there as unreachable which can be undesired, therefore you can redirect the call to a normal private method that is executed instead: from pydantic import BaseModel from typing import TYPE_CHECKING class User2(BaseModel): name: str age: int if not TYPE_CHECKING: # hide from type-checker to report AttributeAccessIssue def __setattr__(self, key, value): self._setattr(key, value) # Use a different method to not create "unreachable code" def _setattr(self, key, value): super().__setattr__(key, value) user2 = User2(name="Alice", age=1) user2.foo = 1 # reportAttributeAccess
Question: I was playing around with Path objects and found an interesting behaviour. When testing with python3.11, Path.glob returns a generator object. >>> from pathlib import Path >>> >>> cwd = Path.cwd() >>> cwd.glob("*") <generator object Path.glob at 0x100b5e680> >>> import sys;sys.version '3.11.6 (v3.11.6:8b6ee5ba3b, Oct 2 2023, 11:18:21) [Clang 13.0.0 (clang-1300.0.29.30)]'  I tested the same code with Python3.13, but this time it returns a map object. >>> from pathlib import Path >>> >>> cwd = Path.cwd() >>> cwd.glob("*") <map object at 0x101867940> >>> import sys;sys.version '3.13.0 (v3.13.0:60403a5409f, Oct 7 2024, 00:37:40) [Clang 15.0.0 (clang-1500.3.9.4)]'  I know the result can be obtained by wrapping it in a list, but I am curios to understand why this change was made. My initial thought was that it might be because of the performance reasons. But a quick test shows that python3.13 version slower than python3.11 on my system. Here is the benchmark result: $ python3.11 -m timeit -n 20000 'from pathlib import Path;cwd=Path.cwd();r=list(cwd.glob("*.py"))' 20000 loops, best of 5: 72.9 usec per loop $ python3.13 -m timeit -n 20000 'from pathlib import Path;cwd=Path.cwd();r=list(cwd.glob("*.py"))' 20000 loops, best of 5: 75.1 usec per loop 

Answer: pathlib.Path.glob learned a bunch of new things in Python 3.13, and I suspect the change in return value type from one generator type to another generator was part of that:  Changed in version 3.13: The recurse_symlinks parameter was added. Changed in version 3.13: The pattern parameter accepts a path-like object. Changed in version 3.13: Any OSError exceptions raised from scanning the filesystem are suppressed. In previous versions, such exceptions are suppressed in many cases, but not all.  The particular change that made glob() return a map is in this PR (found via git log -S "map(self._from_parsed_string") – you can read the description and commit message there for all the details.
Question: I have recently purchased a MAX30101 breakout board / sensor and have been using it with a Raspberry Pi 4 and Python. I have been using the sparkfun-qwiic-max3010x library provided by Sparkfun to communicate with the sensor and collect data. The GitHub page for the Python sparkfun-qwiic-max3010x library includes an example for calculating heart rate but not SpO2. The Arduino library includes an example for calculating heart rate and SpO2. I was wondering if there is an algorithm available or if it is possible to calculate SpO2 accurately in Python on the Raspberry Pi 4. I have seen an algorithm in the user guide of the MAX30101 sensor but have not been able to accurately get an SpO2 reading with it in Python in the following code. The SpO2 value sometimes goes in the negatives and will give extremely different consecutive values. import time import numpy as np import qwiic_max3010x # Initialize the MAX30101 sensor sensor = qwiic_max3010x.QwiicMax3010x() sensor.begin() sensor.setup() sensor.setPulseAmplitudeGreen(0) # Function to read raw values from the sensor def read_raw_values(num_samples=100): red_values = [] ir_values = [] for _ in range(num_samples): red = sensor.getRed() ir = sensor.getIR() red_values.append(red) ir_values.append(ir) time.sleep(0.01) # Adjust the delay as needed sensor.shutDown() return np.array(red_values), np.array(ir_values) # Function to calculate AC and DC components def calculate_ac_dc(raw_values): dc = np.mean(raw_values) # DC component ac = raw_values - dc # AC component return ac, dc def calculate_spo2(red_ac, red_dc, ir_ac, ir_dc): r = (np.mean(red_ac) / red_dc) / (np.mean(ir_ac) / ir_dc) spo2 = 104 - (17 * r) return spo2 # Main loop try: while True: # Read raw values red_raw, ir_raw = read_raw_values(num_samples=100) # Calculate AC and DC components red_ac, red_dc = calculate_ac_dc(red_raw) ir_ac, ir_dc = calculate_ac_dc(ir_raw) spo2 = calculate_spo2(red_ac, red_dc, ir_ac, ir_dc) print("Spo2: ", spo2) # Wait for a bit before the next reading time.sleep(1) # Adjust the sleep time as needed except KeyboardInterrupt: print("Program stopped.") 

Answer: Calculating r the way you do is incorrect. By the very nature of red_ac its mean is almost zero (it is not exactly zero because of rounding errors). Same for ir_ic. Dividing two such values results in a huge meaningless error, precisely what you observe. Notice that in the Maxim paper you linked they do not divide averages. They divide instant readings, normalized by the averages. So the first thing you shall try is to calculate instant values of r[i] = (red_ac[i]/red_dc) / (ir_ac[i]/ir_dc) and average them over the observation window. In the ideal case you'll get reasonable results. Then you'd have to deal with the noise. The signals you sample are far from ideal, and it is not coincidental that the Arduino code does not resemble the simple formula. Most of the SpO2 computation is fighting noise.
Question: The problem It's similar to the problem of finding the minimal set of vertices in a directed graph from which all vertices can be reached, except that a node must have all its predecessors reached in order to be reached itself. More formally, let S be a set of nodes belonging to a directed graph G. A node n of G is said to be reachable from S if and only if n belongs to S or if all predecessors of n are reachable from S. We'll say that S is a generator of G if all nodes of G are reachable from S. We can be sure that such a generator always exists for every graph (i.e. the set containing all nodes will always work). The question is to design an algorithm that returns a minimum-length generator of G, given a graph G. Example Let's take this graph. {1,3,6} is a generator because {1} unlocks 2, {2,3} unlocks 5 and {1,3,6} unlocks 4. Now, if you take S = {1,5}, nodes 3, 4 and 6 will remain unreached. S is therefore not a generator of this graph. In fact, we can see that, if S is a generator, every cycle of the graph must contain at least one node in S.

Answer: This problem is NP hard. That can be seen by reducing the minimum feedback vertex set problem to your reachability problem. Start with any directed graph G. Construct a new graph H by adding a single vertex v with an edge connecting it to every vertex in G. Let S be any solution to your reachability problem on H. It is obvious that v must be in S because it has no predecessor, and is a predecessor of everything else. It is also clear that if you take S away from G, it cannot contain any cycles. And therefore S must contain a vertex feedback set for G. Conversely, any subset S of H that contains both v and a vertex feedback set for G will solve your reachability problem. (You can prove this by induction on the size of G minus S. Just prove that there is always an element with no predecessor in G, remove that element, then apply the induction hypothesis.) Therefore a solver for your minimum reachability problem allows us to solve the minimum feedback vertex problem for arbitrary graphs. Since that problem is NP hard, so is your problem.
Question: Working with a single series instead of a dataframe, how can I materialize an expression on that series? For instance, when I run time = pl.Series([pl.datetime(2025, 3, 27)]) (time + pl.duration(minutes=5)).to_numpy()  I get AttributeError Traceback (most recent call last) Cell In[46], line 2 1 time = pl.Series([pl.datetime(2025, 3, 27)]) ----> 2 (time + pl.duration(minutes=5)).to_numpy() AttributeError: 'Expr' object has no attribute 'to_numpy' 

Answer: You need a "DataFrame" to run expressions. Your example is "wrong" because passing an Expr creates a Series with dtype object. pl.Series([pl.datetime(2025, 3, 27)]) # shape: (1,) # Series: '' [o][object] # [ # 2025-03-27 00:00:00.alias("datetime") # ] #  pl.select() is shorthand for creating an empty frame. s = pl.select(pl.datetime(2025, 3, 27)).to_series()  # shape: (1,) # Series: 'datetime' [datetime[μs]] # [ # 2025-03-27 00:00:00 # ]  pl.select(s + pl.duration(minutes=5))  shape: (1, 1) ┌─────────────────────┐ │ datetime │ │ --- │ │ datetime[μs] │ ╞═════════════════════╡ │ 2025-03-27 00:05:00 │ └─────────────────────┘  You can add .to_series() if you want to go back to a Series. There is also a Series method to do this without expressions.  .dt.offset_by()  s.dt.offset_by("5m")  # shape: (1,) # Series: 'datetime' [datetime[μs]] # [ # 2025-03-27 00:05:00 # ]
Question: What I want to Achieve My goal is to extract a date that has at least a day and a Month, but could also have a minute, hour, and year. I want to avoid the parser finding integers and thinking that implies a day of the current month. Furthermore, I also want the parser to find a date that is only a small part of a larger string. Think: 'Today is the most wonderful 27th of March 2025' = datetime.datetime(2025, 3, 27, 0, 0) '2gb of ram' != datetime.datetime(2025, 3, 2, 0, 0) (Assuming we are currently in March) What I tried so far Using the fuzzy=True argument in dateutils from dateutil import parser #OK: Correct Datetime is returned: datetime.datetime(2025, 3, 30, 0, 0) parser.parse('Today is the most wonderful 30th March 2025', fuzzy=True) #NOT OK: Integer is not ignored, Datetime is returned: datetime.datetime(2025, 3, 2, 0, 0) parser.parse('2 is my lucky number', fuzzy=True)  Using the REQUIRED_PARTS setting in dateparser frome dateparser import parse # OK: Returns correct datetime: datetime.datetime(2025, 3, 30, 0, 0 parse('30 March', settings={'REQUIRE_PARTS': ['month', 'day']}) #OK: Integer is Ignored, no datetime returned parse('30', settings={'REQUIRE_PARTS': ['month', 'day']}) #NOT OK: Datetime Should be Found parse('Today is the most wonderful 30th of March', settings={'REQUIRE_PARTS': ['month', 'day']})  It would be great if I could combine the fuzzy=True argument from the dateutils module with the settings argument from the dateparser module, but seeing as they are separate modules, that is not feasible. Is there another way to achieve the same functionality?

Answer: Use from dateparser.search import search_dates Here's a quick function: from dateparser import parse from dateparser.search import search_dates def extract_date(text:str, exclusions:list=['now', 'today', 'tomorrow', 'yesterday', 'hour', 'minute', 'seconds', 'month', 'months','year', 'years'], required:list=['month', 'day']): ''' Check if the text contains at least Day and Month to parse date off If yes, return datetime object. If not, return None. - Inputs: * text: string to extract date from * exclusions: list of words to exclude from parsing (e.g. ['today', 'tomorrow']) * required: list of required date components (e.g. ['day', 'month']) ''' # Parse the date # It will return only the first result, if found try: return search_dates(text.lower(), settings={'REQUIRE_PARTS': required, 'SKIP_TOKENS': exclusions})[0][1] # Error, return None except (IndexError, TypeError): return None  Testing here, it worked ok.  Text | Date Extracted ----------------------------- TEXT: Today is the most wonderful 30th March 2025 || ** DATE PARSED: 2025-03-30 TEXT: 2 is my lucky number || ** DATE PARSED: None TEXT: I was born on 1990-01-01 || ** DATE PARSED: 1990-01-01 00:00:00 TEXT: I will go to Paris on 2025-01-01 || ** DATE PARSED: 2025-01-01 00:00:00 TEXT: I will go to Paris on 2040-09 missing day || ** DATE PARSED: None TEXT: 25 thousand days || ** DATE PARSED: None TEXT: It costs 25 dollars || ** DATE PARSED: None TEXT: I will go to NYC in 25 days || ** DATE PARSED: 2025-04-21 16:47:31.137955 TEXT: I will go to Rome in 1 month || ** DATE PARSED: None