eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: # A function that returns the frequency of each value: def myFunc(e): return cars.count(e) cars = ['Ford', 'Ford', 'Ford', 'Mitsubishi','Mitsubishi', 'BMW', 'VW'] cars.sort(key=myFunc) print(cars)  Output: ['Ford', 'Ford', 'Ford', 'Mitsubishi', 'Mitsubishi', 'BMW', 'VW']  What I expect: ['BMW', 'VM', 'Mitsubishi', 'Mitsubishi', 'Ford', 'Ford', 'Ford']  Counts: Ford - 3 Mitsubishi - 2 BMW - 1 VM - 1  It should sort in ascending order of count in the list.

Answer: The problem is that you are using cars inside the key function, but .sort is in-place. This causes cars to be unreliable in intermediate calls to the key function. We can see the problem if we print cars inside the key function: def myFunc(e): print(cars) return cars.count(e) cars = ['Ford', 'Ford', 'Ford', 'Mitsubishi', 'Mitsubishi', 'BMW', 'VW'] cars.sort(key=myFunc)  This outputs [] [] [] [] [] [] []  so cars.count will return 0 regardless what element is passed and the list will retain its original order. Use sorted(...) which is not in-place: def myFunc(e): return cars.count(e) cars = ['Ford', 'Ford', 'Ford', 'Mitsubishi', 'Mitsubishi', 'BMW', 'VW'] cars = sorted(cars, key=myFunc) print(cars)  This outputs ['BMW', 'VW', 'Mitsubishi', 'Mitsubishi', 'Ford', 'Ford', 'Ford']  As a side-note, in this case you can use cars.count directly, without defining the wrapper function: cars = sorted(cars, key=cars.count)
Question: I am trying to create a canary resource that uses a script that needs a secret. I'm trying to add a policy statement to the canary role (which I'm creating as part of the cdk). To do this I need to get the secrets full arn, I can get the partial arn with secret_from_name = secretsmanager.Secret.from_secret_name_v2  then use it like resources = [secret_from_name.secret_arn]  but that doesn't give me the full arn and the permissions don't work. .....because no identity-based policy allows the secretsmanager:GetSecretValue action  Thought I would get around this by doing resources = [secret_from_name.secret_full_arn]  But because this is derived by name, it doesn't get the full arn and you get 'undefined' I also tried getting it from attribute using the partial arn, no joy there either. So is there any way around this? As what I don't want to do is pass around full arn's or is there another way I can grant access to this reousece?

Answer: Secret ARNs have a dash and 6 random characters at the end. Define the IAM policy statement's resource with a -?????? wildcard suffix to grant your role access to all versions of the secret name. { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "secretsmanager:GetSecretValue", "Resource": [ "arn:aws:secretsmanager:<region>:<account-id-number>:secret:<secret-name>-??????" ] } ] }  In a CDK context you can simply use string concatenation to assemble the policy statement's resource ARN from the secret's name. Or use a CDK ARN utility (Arn.format or Stack.format_arn).
Question: I want to get the app instance in my router file, what should I do ? My main.py is as follows: # ... app = FastAPI() app.machine_learning_model = joblib.load(some_path) app.include_router(some_router) # ...  Now I want to use app.machine_learning_model in some_router's file , what should I do ?

Answer: Since FastAPI is actually Starlette underneath, you could store the model on the application instance using the generic app.state attribute, as described in Starlette's documentation (see State class implementation too). Example: app.state.ml_model = joblib.load(some_path)  As for accessing the app instance (and subsequently, the model) from outside the main file, you can use the Request object. As per Starlette's documentation, where a request is available (i.e., endpoints and middleware), the app is available on request.app. Example: from fastapi import Request @router.get('/') def some_router_function(request: Request): model = request.app.state.ml_model  Alternatively, one could now use request.state to store global variables/objects, by initializing them within the lifespan handler, as explained in this answer.
Question: I am trying to figure out how to install python2 on centos9 stream. I am getting the errors below. Any suggestions? sudo dnf install python2 Last metadata expiration check: 0:04:48 ago on Thu 24 Feb 2022 01:43:10 PM EST. No match for argument: python2 Error: Unable to find a match: python2 

Answer:  how to install python2 on centos9 stream  You cannot do that from any repo. No available repo / no packages. Reason: python2.7 had End Of Life January 1, 2020. https://endoflife.date/python
Question: I did try uncompyle6, decompyle3, and others, but none of them worked with Python 3.10. Is it even possible to do this right now?

Answer: Use pycdc. Github: https://github.com/zrax/pycdc git clone https://github.com/zrax/pycdc cd pycdc cmake . make make check pycdc C:\Users\Bobby\example.pyc
Question: I'm on Windows using PowerShell and WSL 'Ubuntu 20.04 LTS'. I have no native Linux Distro, and I cant use virtualisation because of nested device reasons. My purpose is to use a Windows Python script in PowerShell to call WSL to decrypt some avd-snapshots into raw-images. I already tried os.popen, subprocess.Popen/run/call, win32com.client, multiprocessing, etc. I can boot the WSL shell, but no further commands are getting passed to it. Does somebody know how to get the shell into focus and prepared for more instructions? Code Example: from multiprocessing import Process import win32com.client import time, os, subprocess def wsl_shell(): shell = win32com.client.Dispatch("wscript.shell") shell.SendKeys("Start-Process -FilePath C:\\Programme\\WindowsApps\\CanonicalGroupLimited.Ubuntu20.04onWindows_2004.2021.825.0_x64__79rhkp1fndgsc\\ubuntu2004.exe {ENTER}") time.sleep(5) os.popen("ls -l") if __name__ == '__main__': ps = Process(target = wsl_shell) ps.start() 

Answer: There are a few ways of running WSL scripts/commands from Windows Python, but a SendKeys-based approach is usually the last resort, IMHO, since it's:  Often non-deterministic Lacks any control logic  Also, avoid the ubuntu2004.exe (or, for other users who find this, the deprecated bash.exe command). The much more capable wsl.exe command is what you are looking for. It has a lot of options for running commands that the <distroname>.exe versions lack. With that in mind, here are a few simplified examples:  Using os.system import os os.system('wsl ~ -e sh -c "ls -l > filelist.txt"')  After running this code in Windows Python, go into your Ubuntu WSL instance and you should find filelist.txt in your home directory. This works because:  os.system can be used to launch the wsl command The ~ tells WSL to start in the user's home directory (more deterministic, while being able to avoid specifying each path in this case) wsl -e sh runs the POSIX shell in WSL (you could also use bash for this) Passing -c "<command(s)>" to the shell runs those commands in the WSL shell  Given that, you can pretty much run any Linux command(s) from Windows Python. For multiple commands:  Either separate them with a semicolon. E.g.: os.system('wsl ~ -e sh -c "ls -l > filelist.txt; gzip filelist.txt')   Or better, just put them all in a script in WSL (with a shebang line), set it executable, and run the script via: wsl -e /path/to/script.sh  That could even be a Linux Python script (assuming the correct shebang line in the script): wsl -e /path/to/script.py  So if needed, you can even call Linux Python from Windows Python this way.    Using subprocess.run The os.system syntax is great for "fire and forget" scripts where you don't need to process the results in Python, but often you'll want to capture the output of the WSL/Linux commands for processing in Python. For that, use subprocess.run: import subprocess cp = subprocess.run(["wsl", "~", "-e", "ls", "-l"], capture_output=True) print(cp.stdout)  As before, the -e argument can be any type of Linux script you want. Note that subprocess.run also gives you the exit status of the command.
Question: I don't understand optional query parameters in FastAPI. How is it different from default query parameters with a default value of None? What is the difference between arg1 and arg2 in the example below where arg2 is made an optional query parameter as described in the above link? @app.get("/info/") async def info(arg1: int = None, arg2: int | None = None): return {"arg1": arg1, "arg2": arg2} 

Answer: This is covered in the FastAPI reference manual, albeit just as a small note: async def read_items(q: Optional[str] = None):   FastAPI will know that the value of q is not required because of the default value = None. The Optional in Optional[str] is not used by FastAPI, but will allow your editor to give you better support and detect errors.  (Optional[str] is the same as str | None pre 3.10 for other readers) Since your editor might not be aware of the context in which the parameter is populated and used by FastAPI, it might have trouble understanding the actual signature of the function when the parameter is not marked as Optional. You may or may not care about this distinction.
Question: Since the pre-commit hook does not allow even warnings and commits issued by bandit, I need to find a way to execute bash commands from python scripts without bandit complaining.  Using the subprocess python package, bandit has always complained so far, no matter what I did. I used ".run()", ".check_call()", ".Popen()", .. all without shell=True and yet there's no avail. If there is a secure alternative to subprocess, I'd also be interested, but I'm sure it must work somehow with subprocess as well.  Example which is not accepted by bandit: import shlex import subprocess ... bash_command = ( f'aws s3 cp {source_dir} s3://{target_bucket_name} --recursive' f' --profile {profile_name}') subprocess.check_call(shlex.split(bash_command), text=True) 

Answer: In order for the code to be secure, you need to know that source_dir target_bucket_name profile_name aren't malicious: e.g. can an untrusted user pass .ssh as the value to be copied? Once you know the subprocess line is secure, you can add # nosec comment to tell bandit not to give a warning about the line: subprocess.check_call(shlex.split(bash_command), text=True) # nosec   (The command aws s3 ... running in subprocess.check_call isn't running in a bash shell, which might confuse people reading the question. Python will directly start the aws process, passing arguments.)
Question: I am trying to use the lux library in python to get visualization recommendations. It shows warnings like NumExpr defaulting to 8 threads.. import pandas as pd import numpy as np import opendatasets as od pip install lux-api import lux import matplotlib  And then: link = "https://www.kaggle.com/noordeen/insurance-premium-prediction" od.download(link) df = pd.read_csv("./insurance-premium-prediction/insurance.csv")  But, everything is working fine. Is there any problem or should I ignore it? Warning shows like this: 

Answer: This is not really something to worry about in most cases. The warning comes from this function, here the most important part: ... env_configured = False n_cores = detect_number_of_cores() if 'NUMEXPR_MAX_THREADS' in os.environ: # The user has configured NumExpr in the expected way, so suppress logs. env_configured = True n_cores = MAX_THREADS ... if 'NUMEXPR_NUM_THREADS' in os.environ: requested_threads = int(os.environ['NUMEXPR_NUM_THREADS']) elif 'OMP_NUM_THREADS' in os.environ: requested_threads = int(os.environ['OMP_NUM_THREADS']) else: requested_threads = n_cores if not env_configured: log.info('NumExpr defaulting to %d threads.'%n_cores)  So if neither NUMEXPR_MAX_THREADS nor NUMEXPR_NUM_THREADS nor OMP_NUM_THREADS are set, NumExpr uses so many threads as there are cores (even if the documentation says "at most 8", yet this is not what I see in the code). You might want to use another number of threads, e.g. while really huge matrices are calculated and one could profit from it or to use less threads, because there is no improvement. Set the environment variables either in the shell or prior to importing numexpr, e.g. import os os.environ['NUMEXPR_MAX_THREADS'] = '4' os.environ['NUMEXPR_NUM_THREADS'] = '2' import numexpr as ne
Question: In pandas 1.4.0: append() was deprecated, and the docs say to use concat() instead.  FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.  Codeblock in question: def generate_features(data, num_samples, mask): """ The main function for generating features to train or evaluate on. Returns a pd.DataFrame() """ logger.debug("Generating features, number of samples", num_samples) features = pd.DataFrame() for count in range(num_samples): row, col = get_pixel_within_mask(data, mask) input_vars = get_pixel_data(data, row, col) features = features.append(input_vars) print_progress(count, num_samples) return features  These are the two options I've tried, but did not work: features = pd.concat([features],[input_vars])  and pd.concat([features],[input_vars])  This is the line that is deprecated and throwing the error: features = features.append(input_vars) 

Answer: You can store the DataFrames generated in the loop in a list and concatenate them with features once you finish the loop. In other words, replace the loop: for count in range(num_samples): # .... code to produce `input_vars` features = features.append(input_vars) # remove this `DataFrame.append`  with the one below: tmp = [] # initialize list for count in range(num_samples): # .... code to produce `input_vars` tmp.append(input_vars) # append to the list, (not DF) features = pd.concat(tmp) # concatenate after loop  You can certainly concatenate in the loop but it's more efficient to do it only once.
Question: FROM python:3.8 WORKDIR /app COPY requirements.txt / RUN pip install --requirement /requirements.txt COPY ./app /app EXPOSE 8000 CMD ["uvicorn", "app.main:app", "--host=0.0.0.0" , "--reload" , "--port", "8000"]  when i used  docker-compose up -d ModuleNotFoundError: No module named 'app'   the folders in Fastapi framework:  fastapi  app -main.py      language_detector.py   Dockerfile  docker-compose  

Answer: CMD ["uvicorn", "main:app", "--host=0.0.0.0" , "--reload" , "--port", "8000"]  Your work directory is /app and the main.py file is already there. So you don't need to call app.main module. Just call main.py script directly in CMD.
Question: For example, the "Show" event in the example below is tied to clicking the "Show" button. Is there a way to programmatically fire off the "Show" event without actually clicking the button? The goal is to automate clicking a series of buttons and filling text boxes by just clicking one other button instead, like a browser autofill. import PySimpleGUI as sg sg.theme("BluePurple") layout = [ [sg.Text("Your typed chars appear here:"), sg.Text(size=(15, 1), key="-OUTPUT-")], [sg.Input(key="-IN-")], [sg.Button("Show"), sg.Button("Exit")], ] window = sg.Window("Pattern 2B", layout) while True: # Event Loop event, values = window.read() print(event, values) if event == sg.WIN_CLOSED or event == "Exit": break if event == "Show": # Update the "output" text element to be the value of "input" element window["-OUTPUT-"].update(values["-IN-"]) window.close() 

Answer: From martineau's comment above:  You can generate a click of the button as if the user clicked on it by calling its click() method. See the docs.  Aditionally, you can fire a specific event with: write_event_value(key, value)
Question: I am trying to create a while loop which will iterate between 2 time objects, while datetime.datetime.now().time() <= datetime.datetime.now() +relativedelta(hour=1): but on every n minutes or second interval. So if the starting time was 1:00 AM, the next iteration should begin at 1:05 AM with n being 5 mins. So the iteration should begin after 5 mins of the start time and not for example, from the end of an iteration, which is the case when using sleep. Could you please advise how this could be accomplished? A possible solution to this was from here: write python script that is executed every 5 minutes import schedule import time def func(): print("this is python") schedule.every(5).minutes.do(func) while True: schedule.run_pending() time.sleep(1)  With this, the start time has to be 1 am. Secondly, what if the program needs to run say 5 min + 1. In that case a 6 min interval wont work.

Answer: I believe this could be considered an object-oriented "canonical" solution which creates a Thread subclass instance that will call a specified function repeatedly every datetime.timedelta units until canceled. The starting and how long it's left running are not details the class concerns itself with, and are left to the code making use of the class to determine. Since most of the action occurs in a separate thread, the main thread could be doing other things concurrently, if desired. import datetime from threading import Thread, Event import time from typing import Callable class TimedCalls(Thread): """Call function again every `interval` time duration after it's first run.""" def __init__(self, func: Callable, interval: datetime.timedelta) -> None: super().__init__() self.func = func self.interval = interval self.stopped = Event() def cancel(self): self.stopped.set() def run(self): next_call = time.time() while not self.stopped.is_set(): self.func() # Target activity. next_call = next_call + self.interval # Block until beginning of next interval (unless canceled). self.stopped.wait(next_call - time.time()) def my_function(): print(f"this is python: {time.strftime('%H:%M:%S', time.localtime())}") # Start test a few secs from now. start_time = datetime.datetime.now() + datetime.timedelta(seconds=5) run_time = datetime.timedelta(minutes=2) # How long to iterate function. end_time = start_time + run_time assert start_time > datetime.datetime.now(), 'Start time must be in future' timed_calls = TimedCalls(my_function, 10) # Thread to call function every 10 secs. print(f'waiting until {start_time.strftime("%H:%M:%S")} to begin...') wait_time = start_time - datetime.datetime.now() time.sleep(wait_time.total_seconds()) print('starting') timed_calls.start() # Start thread. while datetime.datetime.now() < end_time: time.sleep(1) # Twiddle thumbs while waiting. print('done') timed_calls.cancel()  Sample run: waiting until 11:58:30 to begin... starting this is python: 11:58:30 this is python: 11:58:40 this is python: 11:58:50 this is python: 11:59:00 this is python: 11:59:10 this is python: 11:59:20 this is python: 11:59:30 this is python: 11:59:40 this is python: 11:59:50 this is python: 12:00:00 this is python: 12:00:10 this is python: 12:00:20 done
Question: I can't find anyone who explains to a layman how to load an onnx model into a python script, then use that model to make a prediction when fed an image. All I could find were these lines of code: sess = rt.InferenceSession("onnx_model.onnx") input_name = sess.get_inputs()[0].name label_name = sess.get_outputs()[0].name pred = sess.run([label_name], {input_name: X.astype(np.float32)})[0]  But I don't know what any of that means. And everywhere I look, everybody already seems to know what they mean, so nobody's explaining it. That would be one thing if I could just run this code, but I can't. It gives me this error: onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid rank for input: Input3 Got: 2 Expected: 4 Please fix either the inputs or the model.  So I need to actually know what those things mean so I can figure out how to fix the error. Will someone knowledgeable please explain?

Answer: Let's first start by going over the code you provided, to make everything clear. sess = ort.InferenceSession("onnx_model.onnx")  This line loads the model into a session object. This means that the layers, functions and weights used in the model are made ready to perform inferences. input_name = sess.get_inputs()[0].name label_name = sess.get_outputs()[0].name  The two methods get_inputs and get_outputs each retrieve some meta information about the model, that being what inputs the model expects, and what outputs it can provide. Off of this meta information in these lines, only the first input & output is actually used, and off of these, only the name is being gotten, and saved into variables. For the last line, let's tackle that part by part. pred = sess.run(...)[0]  This performs a inference on the model, we'll go over the inputs to this method after this, but for now, the output is a list of different outputs. These outputs are each numpy arrays. In this case only the first output in this list is being used, and saved to the pred variable ([label_name], {input_name: X.astype(np.float32)})  These are the inputs to sess.run. The fist is a list of names of outputs that you want to be computed by the session. The second argument is a dict, where each input's name maps to numpy arrays. These arrays are are expected to be of the same dimension as the ones supplied during creation of the model. Similarly the types of these arrays should also match the types used during creation of the model. The error you encountered seems to indicate that the supplied array doesn't have the expected dimensions. These intended amount of dimensions seems to be 4. To gain clarity about what the exact shape and data type of the input array should be, there are visualization tools, like Netron
Question: Say x_1, x_2, ..., x_n are n objects and one wants to pick one of them so that the probability of choosing x_i is proportional to some number u_i. Numpy provides a function for that: x, u = np.array([x_1, x_2, ..., x_n]), np.array([u_1, ..., u_n]) np.random.choice(x, p = u/np.sum(u))  However, I have observed that this code sometimes throws a ValueError saying "probabilities do not sum to 1.". This is probably due to the round-off errors of finite precision arithmetic. What should one do to make this function work properly?

Answer: After reading the answer https://stackoverflow.com/a/60386427/6087087 to the question pointed by @Pychopath, I have found the following solution, inspired by the documentation of numpy.random.multinomial https://docs.scipy.org/doc/numpy-1.15.0/reference/generated/numpy.random.multinomial.html Say p is the array of probabilities which may not be exactly 1 due to roundoff errors, even if we normalized it with p = p/np.sum(p). This is not rare, see the comment by @pd shah at the answer https://stackoverflow.com/a/46539921/6087087. Just do p[-1] = 1 - np.sum(p[0:-1]) np.random.choice(x, p = p)  And the problem is solved! The roundoff errors due to subtraction will be much smaller than roundoff errors due to normalization. Moreover, one need not worry about the changes in p, they are of the order of roundoff errors.