eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am trying to create a small program that works with OpenAI's Whisper. I then build the Python script to the .exe file using PyInstaller (auto-py-to-exe). When I run the .exe file, I get the following error: Traceback (most recent call last): File "transformers\utils\versions.py", line 108, in require_version File "importlib\metadata\__init__.py", line 996, in version File "importlib\metadata\__init__.py", line 969, in distribution File "importlib\metadata\__init__.py", line 548, in from_name importlib.metadata.PackageNotFoundError: No package metadata was found for tqdm During handling of the above exception, another exception occurred: Traceback (most recent call last): File "Whisper.py", line 2, in <module> File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module File "whisper\__init__.py", line 12, in <module> File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module File "whisper\decoding.py", line 11, in <module> File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module File "whisper\tokenizer.py", line 8, in <module> File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module File "transformers\__init__.py", line 30, in <module> File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module File "transformers\dependency_versions_check.py", line 41, in <module> File "transformers\utils\versions.py", line 123, in require_version_core File "transformers\utils\versions.py", line 110, in require_version importlib.metadata.PackageNotFoundError: No package metadata was found for The 'tqdm>=4.27' distribution was not found and is required by this application. Try: pip install transformers -U or pip install -e '.[dev]' if you're working with git main [10092] Failed to execute script 'Whisper' due to unhandled exception!  This is the source code of my program: import whisper import os from timeit import default_timer as timer start = timer() audio_file = "MyAudioFile.wav" model = whisper.load_model("small") options = {"language": "de"} res = model.transcribe(f"C:/Users/Username/Desktop/Transcribe/{audio_file}", **options) end = timer() os.system("cls") print(res["text"]) print("\nFinished in " + str(end-start) + " seconds!") input("Press ENTER to close the program.")  Everything works without problems when I run the Python script from the console or from VS code. However, unfortunately not when I start it as .exe. I have already tried to reinstall the packages such as tqdm with pip, without success. I am grateful for any help and hints!

Answer: My problem was solved by adding --recursive-copy-metadata "openai-whisper"  to the PyInstaller command.
Question: Autopep8 don't working at all. Here's my setting: { "editor.defaultFormatter": "ms-python.autopep8", "editor.formatOnSave": true, "editor.formatOnPaste": true, "editor.formatOnType": true, "autopep8.showNotifications": "always", "indentRainbow.colorOnWhiteSpaceOnly": true, "[python]": { "editor.formatOnType": true }, "workbench.colorTheme": "Monokai Dimmed", "editor.codeActionsOnSave": { }, "autopep8.args": [ "--in-place --aggressive --aggressive" ] }  For exaple I using this code: import math, sys; def example1(): ####This is a long comment. This should be wrapped to fit within 72 characters. some_tuple=( 1,2, 3,'a' ); some_variable={'long':'Long code lines should be wrapped within 79 characters.', 'other':[math.pi, 100,200,300,9876543210,'This is a long string that goes on'], 'more':{'inner':'This whole logical line should be wrapped.',some_tuple:[1, 20,300,40000,500000000,60000000000000000]}} return (some_tuple, some_variable) def example2(): return {'has_key() is deprecated':True}.has_key({'f':2}.has_key('')); class Example3( object ): def __init__ ( self, bar ): #Comments should have a space after the hash. if bar : bar+=1; bar=bar* bar ; return bar else: some_string = """ Indentation in multiline strings should not be touched. Only actual code should be reindented. """ return (sys.path, some_string)  The only way I can use autopep8 is import it as module and run autopep8.fix_code('''code example''') and I can't get result that I want. That am i doing wrong? Thanks! P. S. I'm using VS Code 1.75.1, Python 3.9.16, autopep8 v2022.2.0. Excepted result: import math import sys def example1(): # This is a long comment. This should be wrapped to fit within 72 # characters. some_tuple = (1, 2, 3, 'a') some_variable = { 'long': 'Long code lines should be wrapped within 79 characters.', 'other': [ math.pi, 100, 200, 300, 9876543210, 'This is a long string that goes on'], 'more': { 'inner': 'This whole logical line should be wrapped.', some_tuple: [ 1, 20, 300, 40000, 500000000, 60000000000000000]}} return (some_tuple, some_variable) def example2(): return ('' in {'f': 2}) in {'has_key() is deprecated': True} class Example3(object): def __init__(self, bar): # Comments should have a space after the hash. if bar: bar += 1 bar = bar * bar return bar else: some_string = """ Indentation in multiline strings should not be touched. Only actual code should be reindented. """ return (sys.path, some_string) 

Answer: Add "python.formatting.provider": "autopep8", to your settings.json. Read document for more information.
Question: I have a list of lists, where each list contains a dictionary and integer. Sometimes duplicate lists occur, and I wish to remove these from the parent list directly. Currently, I am creating a new list and iterating over the old list to ensure only unique values are appended, but I feel this is bad practice. Can this be rewritten to a one-liner with list comprehension, or can the original list be filtered directly instead, for performance enhancement? TRIAL=[[{'http': '46.101.160.223:80', 'https': '46.101.160.223:80'}, 0], [{'http': '66.70.178.214:9300', 'https': '66.70.178.214:9300'}, 0], [{'http': '130.61.100.135:80', 'https': '130.61.100.135:80'}, 0], [{'http': '157.245.27.9:3128', 'https': '157.245.27.9:3128'}, 0], [{'http': '185.246.84.7:8080', 'https': '185.246.84.7:8080'}, 0], [{'http': '185.246.84.7:8080', 'https': '185.246.84.7:8080'}, 0], [{'http': '130.61.100.135:80', 'https': '130.61.100.135:80'}, 1]] #We have some duplicates which we want to filter out if there with function temporary_list=[] for i in TRIAL: if i[0] not in [item[0] for item in temporary_list]: temporary_list.append(i) temporary_list (desired outcome) [[{'http': '46.101.160.223:80', 'https': '46.101.160.223:80'}, 0], [{'http': '66.70.178.214:9300', 'https': '66.70.178.214:9300'}, 0], [{'http': '130.61.100.135:80', 'https': '130.61.100.135:80'}, 0], [{'http': '157.245.27.9:3128', 'https': '157.245.27.9:3128'}, 0], [{'http': '185.246.84.7:8080', 'https': '185.246.84.7:8080'}, 0]] 

Answer: If you can use pandas: import pandas as pd df = pd.DataFrame(TRIAL, columns=['d', 'i']) out = df.groupby(df['d'].apply(dict.values).apply(tuple)).last().values.tolist() print(out) [[{'http': '130.61.100.135:80', 'https': '130.61.100.135:80'}, 1], [{'http': '157.245.27.9:3128', 'https': '157.245.27.9:3128'}, 0], [{'http': '185.246.84.7:8080', 'https': '185.246.84.7:8080'}, 0], [{'http': '46.101.160.223:80', 'https': '46.101.160.223:80'}, 0], [{'http': '66.70.178.214:9300', 'https': '66.70.178.214:9300'}, 0]]
Question: I am learning how to use VSCode for Python development by building a simple Flask app. I use Poetry for dependency management. I have a simple app.py where I'm importing flask and defining a basic route like so: from flask import Flask app = Flask(__name__) @app.route("/") def home(): return "Hello, Flask!"  In the VSCode terminal, I run poetry shell and then python -m flask run, and the server starts up no problem. I'm able to hit the endpoint and get a response. However, my VSCode editor is telling me that it's unable to import flask. I'm unable to run the VSCode debugger for this reason.  I'm missing some step. Does anybody have any idea? Thank you in advance!

Answer: Re-starting VSCode did the trick. I wasn't seeing the Poetry interpreter when I looked for it via cmd+shift+p and Python: Select Interpreter prior to re-starting, and now I do. Selecting the Poetry interpreter it fixes the issue!
Question: sp.simplify doesn't seem to work when your expression is given by sp.sympify. How can I change that? import sympy as sp r = sp.Symbol('r', real = True) f_str = 'sqrt(1/r**4)' f1 = sp.sympify( f_str ) f2 = sp.sqrt(1/r**4) for f in f1,f2: sp.pprint(sp.simplify(f))  which outputs  ____ ╱ 1 ╱ ── # f1 ╱ 4 ╲╱ r 1 ── # f2 2 r  I was expecting that given a real value (r), a sympify expression could get simplified

Answer: The r in f1 isn't the same as the r symbol you defined: >>> f1.free_symbols == f2.free_symbols False  In particular, this means that the assumption that r is real doesn't carry through, which is necessary for the simplification you want. You can remedy this postmortem by substituting the r in f1 with your r symbol: >>> f1 # old r, no assumptions sqrt(r**(-4)) >>> f1.subs("r", r) # your r, with real assumption r**(-2)  In general, you can specify assumptions for string inputs at construction time by passing a dictionary mapping string symbols to your desired SymPy Symbols: >>> f3 = sp.sympify(f_str, {"r": r}) >>> f3 r**(-2) >>> f3.free_symbols == f2.free_symbols True
Question: key: Literal['all', 'a', 'b'] = 'a' def do_a(): pass def do_b(): pass match key: case 'a': do_a() case 'b': do_b() case 'all': do_a() do_b()  Is there any way to simplify the code so I can remove the case 'all'? Something like match key: case 'a' | 'all': do_a() case 'b' | 'all': do_b() 

Answer: Example Setup: from typing import Literal key: Literal['all', 'a', 'b'] def do_a(): print('do_a') def do_b(): print('do_b')   Solution I: You can just use if: if key in ('a', 'all'): do_a() if key in ('b', 'all'): do_b()   Solution II: You could use a function mapper, like so: function_mapper = { 'all': (do_a, do_b), 'a': (do_a,), 'b': (do_a,), } key = 'all' for func in function_mapper[key]: func()
Question: Lets say I have a polars dataframe like this in python  newdata = pl.DataFrame({ 'A': [1, 2, 3, 4], 'B': [5, 6, 7, 8], 'C': [9, 10, 11, 12], 'D': [13, 14, 15, 16] })  And I want to subtract from every value in every column the corresponding value from another frame baseline = pl.DataFrame({ 'A': [1], 'B': [2], 'C': [3], 'D': [4] })  In pandas, and numpy, the baseline frame is automagically broadcasted to the size of the newdata, and I can just do; data=newdata-baseline  But that doesn't work in polars. So what is the cleanest way to achieve this in polars?

Answer: Here loop by column names can be used: newdata.with_columns( [pl.col(c) - baseline[c] for c in newdata.columns] )
Question: What does it mean when Kivy gives this warning message? What can cause it? [WARNING] [Label ] pop style stack without push 

Answer: I don't see any documentation about this message specifically, but here's the relevant bit from the source code that handles BBCode-style text markup: def _pop_style(self, k): if k not in self._style_stack or len(self._style_stack[k]) == 0: Logger.warning('Label: pop style stack without push') return v = self._style_stack[k].pop() self.options[k] = v  The BBCode format lets you, for instance, make a word italic like [i]this[/i]. As the documentation puts it:  A tag is defined as [tag], and should have a corresponding [/tag] closing tag.  The wording of the error message is probably not the most user-friendly. It refers to the fact that the tags are handled internally as a stack; an opening tag is pushed onto the stack, and a closing tag is popped off of it. The error means you're trying to pop a style that's not already pushed onto the stack—in other words, you've closed a tag that's not already open.
Question: I am a new user to Apache Airflow. I am building a DAG like the following to schedule tasks: def add(): return 1 + 1 def multiply(a): return a * 999 dag_args = { 'owner': 'me', 'depends_on_past': False, 'start_date': datetime(2023, 2, 27), 'email': ['me@myhome.com'], 'email_on_failure': True, 'email_on_retry': True, 'retries': 1, 'retry_delay': timedelta(minutes=3)} with DAG( dag_id='dag', start_date=datetime(2023, 2, 27), default_args=dag_args, schedule_interval='@once', end_date=None,) as dag: t1 = PythonOperator(task_id="t1", python_callable=add, dag=dag ) t2 = PythonOperator(task_id="t2", python_callable=multiply, dag=dag)  As you can see, t2 is dependent on the result of t1. I wonder that is there any way for me to pass the return result from t1 directly to t2. I am using Apache Airflow 2.5.1 version and Python 3.9. I did some research on xcom, and found that all results of Airflow tasks are stored there, which can be accessed via code task_instance = kwargs['t1'] task_instance.xcom_pull(task_ids='t1')

Answer: Your DAG can be simplified using taskflow API. It will handle the Xcom and simplify the code. import pendulum from airflow.decorators import dag, task @dag( schedule_interval=None, start_date=pendulum.datetime(2023, 1, 1, tz="UTC"), catchup=False, ) def taskflow_api_etl(): @task() def add(): return 1+1 @task() def multiply(a: int): return a * 99 order_data = add() multiply(order_data) # multiply uses the Xcom genreated by add() etl_dag = taskflow_api_etl()  This code will generate the DAG:  When executing, add() task will generate Xcom with value 2:  The downstream multiply will read the xcom and do 2*99:
Question: I am building a TensorFlow model and have a ton of functions and modules that have proper docstrings. I installed mkdocs due to popular demand and the documentation does appear to be very easy to write. Nevertheless, I don't want to manually write up the entire API reference of all my modules inside this package. I am using mkdocstrings but I am unable to find a way to automate all of these and store them in the references section in mkdocs as you see with any documentation sites like numpy/pandas. ​ I tried pdoc3, but it only solves 1 problem for me. mkgendocs was something I was hoping it would work, but this requires another config file! I followed this post but it was not working for me. ​ Any suggestions/resources on how I can autogenerate all my API docstrings into an api references page in mkdocs? Sphinx is too advanced to work with sorry. I am trying to get my team to document more so I need something easy to use and MkDocs looks like the best option currently.

Answer: The solution is described in this recipe for mkdocstrings: https://mkdocstrings.github.io/recipes/#automatic-code-reference-pages
Question: I'm using Pynamodb for interacting with dynamodb. However, last_evaluated_key always returns null even if there are multiple items. When I run this query results = RecruiterProfileModel.profile_index.query( hash_key=UserEnum.RECRUITER, scan_index_forward=False, limit=1, )  If I try getting this value results.last_evaluated_key  it always returns null. However, if I dump the results object, it returns { "page_iter": { "_operation": {}, "_args": [ "recruiter" ], "_kwargs": { "range_key_condition": null, "filter_condition": null, "index_name": "PROFILE_INDEX", "exclusive_start_key": { "sk": { "S": "PROFILE#6ab0f5bc-7283-4236-9e37-ea746901d19e" }, "user_type": { "S": "recruiter" }, "created_at": { "N": "1677398017.685749" }, "pk": { "S": "RECRUITER#6ab0f5bc-7283-4236-9e37-ea746901d19e" } }, "consistent_read": false, "scan_index_forward": false, "limit": 1, "attributes_to_get": null }, "_last_evaluated_key": { "sk": { "S": "PROFILE#95a4f201-2475-45a7-b096-5167f6a4d639" }, "user_type": { "S": "recruiter" }, "created_at": { "N": "1677398017.68518" }, "pk": { "S": "RECRUITER#95a4f201-2475-45a7-b096-5167f6a4d639" } }, "_is_last_page": false, "_total_scanned_count": 2, "_rate_limiter": null, "_settings": { "extra_headers": null } }, "_map_fn": {}, "_limit": 0, "_total_count": 1, "_index": 1, "_count": 1, "_items": [ { "company_name": { "S": "fletcher-rodriguez" }, "created_at": { "N": "1677398017.685749" }, "last_name": { "S": "craig" }, "first_name": { "S": "tyler" }, "designation": { "S": "manager" }, "verification_status": { "S": "pending" }, "sk": { "S": "PROFILE#6ab0f5bc-7283-4236-9e37-ea746901d19e" }, "user_type": { "S": "recruiter" }, "email": { "S": "fowlerkathy@hernandez.com" }, "pk": { "S": "RECRUITER#6ab0f5bc-7283-4236-9e37-ea746901d19e" } } ] }  You can clearly see that last_evaluated_key exists over there. I'm getting confused here. Please help. I've deadlines to meet. Thank you in advance. Edit: Here is a video attachment of trying to get the value of last_eveluated_key in different ways and they all return null pynamodb last_evaluated_key issue Gdrive video

Answer: The result returned by the query() is the ResultIterator object, which is an iterator. Its last_evaluated_key is the key of the last item you pulled from the iterator, not DynamoDB. Because your code have not yet asked to retrieve any items, the last_evaluated_key is not set. You need to process some or all of the items in the result: for item in results: print(item) return results.last_evaluated_key  then you will get the key of the last item you processed.
Question: I am not the best coder, but I am trying to figure out how to calculate the number of possible combinations and actually generate every combination, but with some rules. I have two sets of "things," primaries (P) and secondaries (S). In this case I have P = 16 and S = 7. So a valid combination needs at least one P value in it, but does not need an S value to be valid, thus: P1, S1, S2 is Valid P1, P2, P3 is Valid P1, P2, P3, P4, S1, S2 is Valid But, S1, S2, S3 is NOT Valid. Also P1, S1 is the same as S1, P1. I wrote a program, which I think does the trick, but it is terrible and takes like two days to run. I created this code, which outputs the correct results (I think), but which is very resource intensive: import itertools P_num = 16 S_num = 7 R = P_num + S_num P = list(range(1,P_num+1)) S = list(range(1,S_num+1)) P = ["P" + str(suit) for suit in P] S = ["S" + str(suit) for suit in S] stuff = P + S totalarray = {new_list: [] for new_list in range(1,R+1)} for L in range(len(stuff) + 1): print(L) for subset in itertools.combinations(stuff, L): sublist = sorted(subset) if any(x in sublist for x in P): if sublist not in totalarray[L]: totalarray[L].append(sublist) run = 0 for each in totalarray.keys(): print(each, len(totalarray[each])) run += len(totalarray[each]) print(run)  I could really use some advise on ways to optimize this problem, I am sure there is a better way to do this without so many nested operations. I am hoping to get the same results, but just more optimized.

Answer: Printing to a terminal is relatively slow. According to your rules, there will be 8,388,480 valid combinations. Writing the valid combinations to a file will be much faster than sending output to a terminal. Try this: from itertools import combinations from time import perf_counter OUTPUT_FILE = '/Volumes/G-Drive/combos.txt' def isvalid(c): for v in c: if v[0] == 'P': return True return False start = perf_counter() P_num = 16 S_num = 7 stuff = [f'P{n}' for n in range(1, P_num+1)] + [f'S{n}' for n in range(1, S_num+1)] count = 0 with open(OUTPUT_FILE, 'w') as cfile: for k in range(1, len(stuff)+1): for combo in combinations(stuff, k): if isvalid(combo): cfile.write(f'{combo}\n') count += 1 print(f'Count={count:,}, Duration={perf_counter()-start:.2f}s')  Output: Count=8,388,480, Duration=11.07s
Question: While teaching python to a friend i tried this statement : val = "hi" if (val=="hello") or ("w" in val): print("hello") elif(val=="hi"): print("hi")  And to my great surprise it worked. I always tought in Python you couldn't do an elif without else. Has it been always like that or the syntax has changed since a particular version?

Answer: else is optional, and follows any number of elif statements. From the specification of version 1.6: if_stmt: "if" expression ":" suite ("elif" expression ":" suite)* ["else" ":" suite]  The * in this syntax means zero or more elements, and [ and ] means an optional element. Python 1.6 was the first version released as open source. That said, I'm almost certain it has always been like that, because it is standard among most, if not all, programming languages.
Question: This works, but not algorithmically optimal since I dont need the min value to be stored while the function is parsing the array: def is_non_negative(m): return np.min(m) >= 0  Edit: Depending on the data an optimal function could indeed save a lot because it will terminate at the first encounter of a negative value. If only one negative value is expected, time will be cut by a factor of two in average. However building the optimal algorithm outside numpy library will be at a huge cost (Python code vs C++ code).

Answer: One pure-Numpy solution is to use a chunk based strategy: def is_non_negative(m): chunkSize = max(min(65536, m.size/8), 4096) # Auto-tunning for i in range(0, m.size, chunkSize): if np.min(m[i:i+chunkSize]) < 0: return False return True  This solution is only efficient if the arrays are big, and chunks are big enough for the Numpy call overhead to be small and small enough to split the global array in many parts (so to benefit from the early cut). The chunk size needs to be pretty big so to balance the relatively big overhead of np.min on small arrays.  Here is a Numba solution: import numba as nb # Eagerly compiled funciton for some mainstream data-types. @nb.njit(['(float32[::1],)', '(float64[::1],)', '(int_[::1],)']) def is_non_negative_nb(m): for e in m: if e < 0: return False return True  It turns out this is faster than using np.min on my machine although the code is not well auto-vectorized (ie. do not use SIMD instruction) by LLVM-Lite (the JIT of Numba). For an even faster code, you need to use a C/C++ code and use a chunk-based SIMD-friendly code, and possibly use SIMD intrinsics if the compiler does not generate an efficient code which is unfortunately rather frequent in this case.
Question: I am cleaning a csv for data analysis and I'm new to python, so I am trying my best to make this as straightforward as possible in case anyone wants to go back into this later. I want to perform a straightforward operation on four columns and add a new column with the result, then efficiently repeat that for 10 other sets of columns. My dataframe looks like this: df = pd.DataFrame({'A1' : [10, 20, 30, 10], 'A2' : [10,20,30,40], 'A3' : [30, 0, 40, 10], 'A4' : [75, 0, 0, 25], 'B1' : [10, 20, 30, 40], 'B2' : [30, 0, 20, 40], 'B3' : [10, 10, 20, 30], 'B4' : [40, 30, 20, 10]}) # A1 A2 A3 A4 B1 B2 B3 B4 # 10 10 30 75 10 30 10 40 # 20 20 0 0 20 0 10 30 # 30 30 40 0 30 20 20 20 # 10 40 10 25 40 40 30 10  I want to create a new column (A_dif) with the value of (A1+A2+A3)-A4. I've able to do that as follows: df['A_dif'] = df.loc[:, A1:A3].sum(numeric_only=True, axis=1) - df.loc[:,'A4']  However, I need to do that for the B columns (and about 10 similar groups of columns). I can do that manually, but I would like an efficient function that accomplishes this. I tried to create following function (and then make a loop with it) but can't get it to work: def difference(df, a: str, b: str, c: str) : df.loc[:, a:b].sum(numeric_only=True, axis=1) - df.loc[:,c] test = difference(df, 'A1', 'A3', 'A4') print(test) # returns None  Thank you for any help you can offer!

Answer: You can group columns by a suffix (here the first letter of column name) and compute your function: def difference(df): return df.iloc[:, :3].sum(numeric_only=True, axis=1) - (df.iloc[:, 3]) df1 = df.groupby(df.columns.str[0], axis=1).apply(difference).add_suffix('_diff') out = pd.concat([df, df1], axis=1) print(out) # Output A1 A2 A3 A4 B1 B2 B3 B4 A_diff B_diff 0 10 10 30 75 10 30 10 40 -25 10 1 20 20 0 0 20 0 10 30 40 0 2 30 30 40 0 30 20 20 20 100 50 3 10 40 10 25 40 40 30 10 35 100  You can also group columns by position. If you need to iterate over 4 columns each time: df1 = (df.groupby(np.arange(len(df.columns)) // 4, axis=1) .apply(difference).add_suffix('_diff'))