Myashka/SO-Python_QA-filtered-2023-tanh_score-after_2023_02 huggingface dataset:
Title: Declare symbols local to functions in SymPy
Tags: python,sympy,subclassing

Question: I have a custom Sympy cSymbol class for the purpose of adding properties to declared symbols. This is done as follows:
class cSymbol(sy.Symbol):
    def __init__(self,name,x,**assumptions):
        self.x = x 
        sy.Symbol.__init__(name,**assumptions)

The thing is that when I declare a cSymbol within a function (say, it affects the property x of a cSymbol declared outside the function if the names are the same (here "a"):
def some_function():
    dummy = cSymbol("a",x=2)

a = cSymbol("a",x=1)
print(a.x)  # >> 1
some_function()
print(a.x)  # >> 2, but should be 1

Is there a way to prevent this (other than passing distinct names) ? Actually I am not sure to understand why it behaves like this, I thougt that everything declared within the function would stay local to this function.
Full code below:
import sympy as sy

class cSymbol(sy.Symbol):
    def __init__(self,name,x,**assumptions):
        self.x = x 
        sy.Symbol.__init__(name,**assumptions)
        
def some_function():
    a = cSymbol("a",x=2)


if __name__ == "__main__":
    a = cSymbol("a",x=1)
    print(a.x)  # >> 1
    some_function()
    print(a.x)  # >> 2, but should be 1

Answer: You aren't creating a local Python variable in the subroutine, you are create a SymPy Symbol object and all Symbol objects with the same name and assumptions are the same. It doesn't matter where they are created. It sounds like you are blurring together the Python variable and the SymPy variable which, though both bearing the name "variable", are not the same.
Title: Is there a way for a Python program to "cd" to a folder that has a space in it?
Tags: python,cmd

Question: I am creating a code editor, and I am trying to create a run feature. Right now I see that the problems come when I encounter a folder with a space in its name. It works on the command line, but not with os.system().
def run(event):
    if open_status_name != False:
        directory_split = open_status_name.split("/")
        for directory in directory_split:
            if directory_split.index(directory) > 2:
                true_directory = directory.replace(" ", "\s")
                print(true_directory)
                data = os.system("cd " + directory.replace(" ", "\s"))
                print(data)

I tried to replace the space with the regex character "\s" but that also didn't work.

Answer: os.system runs the command in a shell. You'd have to add quotes to get the value though: os.system(f'cd "{directory}"'). But the cd would only be valid for that subshell for the brief time it exists - it would not change the directory of your python program. Use os.chdir(directory) instead.
Note - os.chdir can be risky as any relative paths you have in your code suddenly become invalid once you've done that. It may be better to manage your editor's "current path" on your own.
Title: Will Google Cloud run this type of application?
Tags: python,google-cloud-platform

Question: I have a python script which run 24 hours on my local system and my script uses different third party libraries that are installed using pip in python
Libraries
BeautifulSoup
requests
m3u8

My python script is recording some live stream videos from a website and is storing on system. How google cloud will help me to run this script 24/hours daily and 7days a week.I am very new to clouds. Please help me i want to host my script on google cloud so i want to make sure that my script will work there same as it is working on local system so my money will not lost .

Answer: If you want to run 24/7 application on the cloud, whatever the cloud, you must not use solution with timeout (like Cloud Run or Cloud Functions).
You can imagine using App Engine flex, but it won't be my best advice.
The most efficient for me (low maintenance, cost efficient), is to use GKE autopilot. A Kubernetes cluster managed for you, you pay only the CPU/Memory that your workloads use.
You have to containerize your app to do that.
Title: How to creat connection websocket qxbroker in python
Tags: python-3.x,websocket,cloudflare

Question: how to bypass HTTP/1.1 403 Forbidden in connect to wss://ws2.qxbroker.com/socket.io/EIO=3&transport=websocket, i try change user-agent and try use proxy and  add cookis but not work
class WebsocketClient(object):


    def __init__(self, api):
        websocket.enableTrace(True)
        Origin = 'Origin: https://qxbroker.com'
        Extensions = 'Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits'
        Host = 'Host: ws2.qxbroker.com'
        Agent = 'User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36 OPR/94.0.0.0'
       
        self.api = api
        self.wss=websocket.WebSocketApp(('wss://ws2.qxbroker.com/socket.io/EIO=3&transport=websocket'), on_message=(self.on_message),
          on_error=(self.on_error),
          on_close=(self.on_close),
          on_open=(self.on_open),
          header=[Origin,Extensions,Agent])


request and response header this site protect with cloudflare
--- request header ---
GET /socket.io/?EIO=3&transport=websocket HTTP/1.1
Upgrade: websocket
Host: ws2.qxbroker.com
Sec-WebSocket-Key: 7DgEjWxUp8N8PVY7N7vyDw==
Sec-WebSocket-Version: 13
Connection: Upgrade
Origin: https://qxbroker.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36
-----------------------
--- response header ---
HTTP/1.1 403 Forbidden
Date: Sat, 11 Feb 2023 23:33:11 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: close
Permissions-Policy: accelerometer=(),autoplay=(),camera=(),clipboard-read=(),clipboard-write=(),fullscreen=(),geolocation=(),gyroscope=(),hid=(),interest-cohort=(),magnetometer=(),microphone=(),payment=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),sync-xhr=(),usb=()
Referrer-Policy: same-origin
X-Frame-Options: SAMEORIGIN
Cache-Control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Thu, 01 Jan 1970 00:00:01 GMT
Set-Cookie: __cf_bm=7TD4hk4.bntJRdP6w9K.AjXF5MsV9LERTJV00jL2Uww-1676158391-0-AZFOKw90ZYdyy4RxX1xJ4jZQMt74+3UkQDZpDrdXE8BxGJULfe8j0T8EZnpUNXr2W3YHd/FxRoO/bPhKA2Dc0E0=; path=/; expires=Sun, 12-Feb-23 00:03:11 GMT; domain=.qxbroker.com; HttpOnly; Secure; SameSite=None
Server-Timing: cf-q-config;dur=6.9999950937927e-06
Server: cloudflare
CF-RAY: 7980e3583b6a0785-MRS

Answer: Sending cookies in websocketapp argument?
"__cf_bm=7TD4hk4.bntJRdP6w9K.AjXF5MsV9LERTJV00jL2Uww-1676158391-0-AZFOKw90ZYdyy4RxX1xJ4jZQMt74+3UkQDZpDrdXE8BxGJULfe8j0T8EZnpUNXr2W3YHd/FxRoO/bPhKA2Dc0E0=; path=/; expires=Sun, 12-Feb-23 00:03:11 GMT; domain=.qxbroker.com; HttpOnly; Secure; SameSite=None"
Title: How to creat connection websocket qxbroker in python
Tags: python-3.x,websocket,cloudflare

Question: how to bypass HTTP/1.1 403 Forbidden in connect to wss://ws2.qxbroker.com/socket.io/EIO=3&transport=websocket, i try change user-agent and try use proxy and  add cookis but not work
class WebsocketClient(object):


    def __init__(self, api):
        websocket.enableTrace(True)
        Origin = 'Origin: https://qxbroker.com'
        Extensions = 'Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits'
        Host = 'Host: ws2.qxbroker.com'
        Agent = 'User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36 OPR/94.0.0.0'
       
        self.api = api
        self.wss=websocket.WebSocketApp(('wss://ws2.qxbroker.com/socket.io/EIO=3&transport=websocket'), on_message=(self.on_message),
          on_error=(self.on_error),
          on_close=(self.on_close),
          on_open=(self.on_open),
          header=[Origin,Extensions,Agent])


request and response header this site protect with cloudflare
--- request header ---
GET /socket.io/?EIO=3&transport=websocket HTTP/1.1
Upgrade: websocket
Host: ws2.qxbroker.com
Sec-WebSocket-Key: 7DgEjWxUp8N8PVY7N7vyDw==
Sec-WebSocket-Version: 13
Connection: Upgrade
Origin: https://qxbroker.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36
-----------------------
--- response header ---
HTTP/1.1 403 Forbidden
Date: Sat, 11 Feb 2023 23:33:11 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: close
Permissions-Policy: accelerometer=(),autoplay=(),camera=(),clipboard-read=(),clipboard-write=(),fullscreen=(),geolocation=(),gyroscope=(),hid=(),interest-cohort=(),magnetometer=(),microphone=(),payment=(),publickey-credentials-get=(),screen-wake-lock=(),serial=(),sync-xhr=(),usb=()
Referrer-Policy: same-origin
X-Frame-Options: SAMEORIGIN
Cache-Control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Thu, 01 Jan 1970 00:00:01 GMT
Set-Cookie: __cf_bm=7TD4hk4.bntJRdP6w9K.AjXF5MsV9LERTJV00jL2Uww-1676158391-0-AZFOKw90ZYdyy4RxX1xJ4jZQMt74+3UkQDZpDrdXE8BxGJULfe8j0T8EZnpUNXr2W3YHd/FxRoO/bPhKA2Dc0E0=; path=/; expires=Sun, 12-Feb-23 00:03:11 GMT; domain=.qxbroker.com; HttpOnly; Secure; SameSite=None
Server-Timing: cf-q-config;dur=6.9999950937927e-06
Server: cloudflare
CF-RAY: 7980e3583b6a0785-MRS

Answer: i resolved the problem sending "header" parameter = {
"User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko)"
}
Title: pip install of pandas
Tags: python,pandas,dataframe,machine-learning,pycharm

Question: I have recently attempted to install pandas through pip. It appears to go through the process of installing pandas and all dependencies properly. After I update to the latest version through cmd as well and everything appears to work; typing in pip show pandas gives back information as expected with the pandas version showing as 1.5.3
However, it appears that when attempting to import pandas to a project in PyCharm (I am wondering if this is where the issue lies) it gives an error stating that it can't be found. I looked through the folders to make sure the paths were correct and that pip didn't install pandas anywhere odd; it did not.
I uninstalled python and installed the latest version; before proceeding I would like to know if there is any reason this issue has presented itself. I looked into installing Anaconda instead but that is only compatible with python version 3.9 or 3.1 where as I am using the newest version, 3.11.2

Answer: When this happens to me

I reload the environment variables by running the command
source ~/.bashrc
right in the pycharm terminal.

I make sure the I have activated the correct venv (where the package installations go) by cd to path_with_venv then running
source ~/pathtovenv/venv/bin/activate

If that does not work, hit CMD+, to open your project settings and and under Python Interpreter select the one with the venv that you have activated. Also check if pandas appears on the list of packages that appear below the selected interpreter, if not you may search for it and install it using this way and not the pip install way
Title: Normalize -1 ~ 1
Tags: python,machine-learning,deep-learning,data-preprocessing

Question: there are many ways about normalize skils for ml and dl. It is known to provide only normalization for 0 to 1.
I want to know that is some ways to normalize -1 between 1.

Answer: You can use the min-max scalar or the z-score normalization here is what u can do in sklearn
from sklearn.preprocessing import MinMaxScaler
from sklearn.preprocessing import StandardScaler
or hard code it like this
x_scaled = (x - min(x)) / (max(x) - min(x)) * 2 - 1 -> this one for minmaxscaler
x_scaled = (x - mean(x)) / std(x) -> this one for standardscaler
Title: Normalize -1 ~ 1
Tags: python,machine-learning,deep-learning,data-preprocessing

Question: there are many ways about normalize skils for ml and dl. It is known to provide only normalization for 0 to 1.
I want to know that is some ways to normalize -1 between 1.

Answer: Yes, there are ways to normalize data to the range between -1 and 1. One common method is called Min-Max normalization. It works by transforming the data to a new range, such that the minimum value is mapped to -1 and the maximum value is mapped to 1. The formula for this normalization is:
x_norm = (x - x_min) / (x_max - x_min) * 2 - 1
Where x_norm is the normalized value, x is the original value, x_min is the minimum value in the data and x_max is the maximum value in the data.
Another method for normalizing data to the range between -1 and 1 is called Z-score normalization, also known as standard score normalization. This method normalizes the data by subtracting the mean and dividing by the standard deviation. The formula for this normalization is:
x_norm = (x - mean) / standard deviation
Where x_norm is the normalized value, x is the original value, mean is the mean of the data and standard deviation is the standard deviation of the data.
Title: Normalize -1 ~ 1
Tags: python,machine-learning,deep-learning,data-preprocessing

Question: there are many ways about normalize skils for ml and dl. It is known to provide only normalization for 0 to 1.
I want to know that is some ways to normalize -1 between 1.

Answer: Consider re-scale the normalized value. e.g. normalize to 0..1, then multiply by 2 and minus 1 to have the value fall into the range of -1..1
Title: How to run allure generate command while using aws code build
Tags: python,amazon-web-services,pytest,allure

Question: I am using aws codebuild to execute my testsuite. It says 'permission denied' when I try to run allure genrate in aws code build.
Pleas share the solution if anyone knows on how to generate allure report while working with aws code build.
I am using pytest and the scenario is working fine in local. but failes in aws as aws build is not allowing me to run allure generate command.
on successful dev deployment -- > tetssuite execution -- > generate allure repors --> uploade them to s3 --> send the report via email using aws SNS with lambda.
all above steps are working fine, but the 3rd step.(allure generate).
Please share the solution if anyone knows how to do it.

Answer: I am able to fix this is by downloading allure package freshly outside of the $CODEBUILD_SRC_DIR and set the path for the same location .
(Initially I made this part of test repository itself and add that location to PATH, which was not working)
Title: using isintance on a pydantic model
Tags: python,pydantic

Question: I am expecting multiple data types as input to a function & want to take a specific action if its a pydantic model (pydantic model here means class StartReturnModel(BaseModel)).
In case of model instance I can check it, using isinstance(model, StartReturnModel) or isinstance(model, BaseModel) to identify its a pydantic model instance.
Based on the below test program I can see that type(StartReturnModel) returns as  ModelMetaclass.  Can I use this to identify a pydantic model? or is there any better way to do it?
from pydantic.main import ModelMetaclass
from typing import Optional

class StartReturnModel(BaseModel):
    result: bool
    pid: Optional[int]

print(type(StartReturnModel))
print(f"is base model: {bool(isinstance(StartReturnModel, BaseModel))}")
print(f"is meta model: {bool(isinstance(StartReturnModel, ModelMetaclass))}")

res = StartReturnModel(result=True, pid=500045)
print(f"\n{type(res)}")
print(f"is start model(res): {bool(isinstance(res, StartReturnModel))}")
print(f"is base model(res): {bool(isinstance(res, BaseModel))}")
print(f"is meta model(res): {bool(isinstance(res, ModelMetaclass))}")

*****Output****
<class 'pydantic.main.ModelMetaclass'>
is base model: False
is meta model: True

<class '__main__.StartReturnModel'>
is start model(res): True
is base model(res): True
is meta model(res): False

Answer: Yes you can use it, but why not use isinstance or issubclass.
Title: module 'keras.utils.generic_utils' has no attribute 'get_custom_objects' when importing segmentation_models
Tags: python,tensorflow,keras,image-segmentation

Question: I am working on google colab with the segmentation_models library. It worked perfectly the first week using it, but now it seems that I can't import the library anymore. Here is the error message, when I execute import segmentation_models as sm :
---------------------------------------------------------------------------

AttributeError                            Traceback (most recent call last)

<ipython-input-3-6f48ce46383f> in <module>
      1 import tensorflow as tf
----> 2 import segmentation_models as sm

            3 frames

/usr/local/lib/python3.8/dist-packages/efficientnet/__init__.py in init_keras_custom_objects()
     69     }
     70 
---> 71     keras.utils.generic_utils.get_custom_objects().update(custom_objects)
     72 
     73 

AttributeError: module 'keras.utils.generic_utils' has no attribute 'get_custom_objects'

Colab uses tensorflow version 2.11.0.
I did not find any information about this particular error message. Does anyone know where the problem may come from ?

Answer: Encountered the same issue sometimes. How I solved it:

open the file keras.py, change all the 'init_keras_custom_objects' to 'init_tfkeras_custom_objects'.

the location of the keras.py is in the error message. In your case, it should be in /usr/local/lib/python3.8/dist-packages/efficientnet/
Title: Cannot copy/move file from remote SFTP server to local machine by Paramiko code running on remote SSH server
Tags: python,ssh,sftp,paramiko

Question: I want to copy a file from my SFTP server to local computer. However, when I run my code, it didn't show any error while I still cannot find my file on local computer. My code like that:
import paramiko
host_name ='10.110.100.8'
user_name = 'abc'
password ='xyz'
port = 22
remote_dir_name ='/data/.../PMC1087887_00003.jpg' 
local_dir_name = 'D:\..\pred.jpg'

t = paramiko.Transport((host_name, port))
t.connect(username=user_name, password=password)
sftp = paramiko.SFTPClient.from_transport(t)
sftp.get(remote_dir_name,local_dir_name)

I have found the main problem. If I run my code in local in VS Code, it works. But when I login in my server by SSH in VS Code, and run my code on server, I found that my file appeared in current code folder (for example /home/.../D:\..\pred.jpg) and its name is D:\..\pred.jpg. How to solve this problem if I want to run code on server and download file to local?

Answer: If you call SFTPClient.get on the server, it will, as any other file manipulation API, work with files on the server.
There's no way to make remote Python script directly work with files on your local machine.
You would have to use some API to push the files to your local machine. But for that, your local machine would have to implement the API. For example, you can run an SFTP server on the local machine and "upload" the files to it.
Title: python/regex: match letter only or letter followed by number
Tags: python,regex

Question: I want to split this string 'AB4F2D' in ['A', 'B4', 'F2', 'D'].
Essentially, if character is a letter, return the letter, if character is a number return previous character plus present character (luckily there is no number >9 so there is never a X12).
I have tried several combinations but I am not able to find the correct one:
def get_elements(input_string):

    patterns = [
        r'[A-Z][A-Z0-9]',
        r'[A-Z][A-Z0-9]|[A-Z]',
        r'\D|\D\d',
        r'[A-Z]|[A-Z][0-9]',
        r'[A-Z]{1}|[A-Z0-9]{1,2}'
        ]

    for p in patterns:
        elements = re.findall(p, input_string)
        print(elements)

results:
['AB', 'F2']
['AB', 'F2', 'D']
['A', 'B', 'F', 'D']
['A', 'B', 'F', 'D']
['A', 'B', '4F', '2D']

Can anyone help? Thanks

Answer: \D\d?
One problem with yours is that you put the shorter alternative first, so the longer one never gets a chance. For example, the correct version of your \D|\D\d is \D\d|\D. But just use \D\d?.
Title: Detect the speed of the vehicle using MPU6050
Tags: python,raspberry-pi,gyroscope,mpu6050

Question: i have been trying to get speed of the vehicle using MPU-6050 but couldn't find my way to do it so,
in the end i am stuck here
def stateCondition():
while True:
    acc_x = read_raw_data(ACCEL_XOUT_H)
    acc_y = read_raw_data(ACCEL_YOUT_H)
    acc_z = read_raw_data(ACCEL_ZOUT_H)
    gyro_x = read_raw_data(GYRO_XOUT_H)
    gyro_y = read_raw_data(GYRO_YOUT_H)
    gyro_z = read_raw_data(GYRO_ZOUT_H)
    # Full scale range +/- 250 degree/C as per sensitivity scale factor
    Ax = acc_x/16384.0
    Ay = acc_y/16384.0
    Az = acc_z/16384.0
    Gx = gyro_x/131.0
    Gy = gyro_y/131.0
    Gz = gyro_z/131.0

can some one please write the rest of it so that it returns the speed of the vehicle in km/hr or whatever it is!!!!!
Thank you

Answer: An MPU6050 will provide you with information about changes in motion (acceleration or decelleration mostly, but also curves). It will not provide you with absolute values. That can only be achieved by integrating over time, but this requires a known start position/speed. Also, it is very inexact, particularly with cheap motion sensors such as this one.
To get the speed of a vehicle, it is much easier to use a GNSS module instead.
Title: Django - after sign-in template don't know that user is authenticated
Tags: python,django,django-views,django-templates,django-authentication

Question: Below code probably works (no errors present):
views.pl
class SignInView(View):

    def get(self, request):
        return render(request, "signin.html")

    def post(self, request):
        user = request.POST.get('username', '')
        pass = request.POST.get('password', '')

        user = authenticate(username=user, password=pass)

        if user is not None:
            if user.is_active:
                login(request, user)
                return HttpResponseRedirect('/')
            else:
                return HttpResponse("Bad user.")
        else:
            return HttpResponseRedirect('/')

....but in template:
{% user.is_authenticated %}

is not True. So I don't see any functionality for authenticated user.
What is the problem?

Answer: You should do like {% if request.user.is_authenticated %} or {% if user.is_authenticated %}
Title: 'RobertaTokenizerFast' object has no attribute '_in_target_context_manager' error while loading data into custom NER model
Tags: python,spacy,named-entity-recognition

Question: I am trying to load data into a custom NER model using spacy, I am getting an error:-
'RobertaTokenizerFast' object has no attribute '_in_target_context_manager'
however, it works fine with the other models.
Thank you for your time!!

Answer: I faced the same issue after upgrading my environment from {Python 3.9 + Spacy 3.3} to {Python 3.10 + Space 3.5}. Resolved this by upgrading and re-packaging the model.
Title: Python file won't run in vs code using play button
Tags: python-3.x,visual-studio-code

Question: i wrote a basic python program and tried running it using the play button but nothing happens,
i look through the interpreters and the one for python isnt detected
can someone guide me
tried looking online for answers but most are confusing since i can't seem to find some of the settings they are recommending  i use

Answer: Hey, my suggestion would be :

First check the installation of python on your machine, and if it
doesn't help then,
Open keyboard shortcuts in VS Code 'CTRL + K and CTRL + S' or by
clicking settings button in bottom-left corner.
Search "Run Python File in Terminal".
You will get first option with the same title.
Double click the Key Binding area in front of title.
And set a keyboard shortcut for running Python {eg: 'ALT + Q' (My shortcut)}. This would be much
convenient.
Title: How to write to a CSV file with pandas while appending to the next empty row without writing the columns again?
Tags: python,pandas,csv

Question: I have a pandas data frame that looks like this:
# df1
 Id  A  B  C
  3  4  5  6

I wrote this to a csv and it works great the first time,
however when I append the CSV it rewrites the columns and the values again
like this:
 Id  A  B  C
  3  4  5  6
 Id  A  B  C
  3  4  5  6

Is there a method for the 2nd iteration afterwards to only write the value and not the columns when writing to a csv through pandas?
I have tried using the 'a' command for appending and to empty my dataframe so it's just the columns to use as a header to write to the csv and then the as a separate dataframe append the values however pandas does not allow for empty dataframes

Answer: Set header=False option for each next df.to_csv call to exclude column names from record.
Title: How would I go about finding the optimal way to split up an order
Tags: python,optimization,cvxpy,operations-research

Question: I have a problem (that I think I'm over complicating) but for the life of me I can't seem to solve it.
I have 2 dataframes. One containing a list of items with quantities that I want to buy. I have another dataframe with a list of suppliers, unit cost and quantity of items available. Along with this I have a dataframe with shipping cost for each supplier.
I want to find the optimal way to break up my order among the suppliers to minimise costs.
Some added points:

Suppliers won't always be able to fulfil the full order of an item so I want to also be able to split an individual item among suppliers if it is cheaper
Shipping only gets added once per supplier (2 items from a supplier means I still only pay shipping once for that supplier)

I have seen people mention cvxpy for a similar problem but I'm struggling to find a way to use it for my problem (never used it before).
Some advice would be great.
Note: You don't have to write all the code for me but giving a bit of guidance on how to break down the problem would be great.
TIA

Answer: Some advice too large for a comment:
As @Erwin Kalvelagen alludes to, this problem can be described as a math program, which is probably the most common-sense approach.
The generalized plan of attack is to figure out how to create an expression of the problem using some modeling package and then turn that problem over to a solver engine which uses diverse techniques to find the optimal answer.
cvxpy is certainly 1 of the options to do the first part with.  I'm partial to pyomo, and pulp is also viable.  pulp also installs with a solver (cbc) which is suitable for this type of problem.  In other cases, you may need to install separately.
If you take this approach, look through a text or some online examples on how to formulate a MIP (mixed integer program).  You'll have some sets (perhaps items, suppliers, etc.), data that form constraints or limits, some variables indexed by the sets, and an objective....likely to minimize cost.
Forget about the complexities of split-orders and combined shipping at first and just see if you can get something working with toy data, then build out from there.
Title: How to handle 300 parameters in Django Model / Form?
Tags: python,django,e-commerce

Question: I develop an app for creating products in online shop. Let's suppose I have 50 categories of products and each of these has some required parameters for product (like color, size, etc.).
Some parameters apper in all categories, and some are unique. That gives me around 300 parameters (fields) that should be defined in Django model.
I suppose it is not good idea to create one big database with 300 fields and add products that have 1-15 parameters there (leaving remaining fields empty). What would be the best way to handle it?
What would be the best way to display form that will ask only for parameters required in given category?

Answer: If you have to keep the Model structure as you have defined it here, I would create a "Product" "Category" "ProductCategory" tables.
Product table is as follows:




ProductID
ProductName




1
Shirt


2
Table


3
Vase




Category table is following




CategoryID
CategoryName




1
Size


2
Color


3
Material




ProductCategory




ID
ProductID
CategoryID
CategoryValue




1
1 (Shirt)
1 (Size)
Medium


2
2 (Table)
2 (Color)
Dark Oak


3
3 (Vase)
3 (Material)
Glass


3
3 (Vase)
3 (Material)
Plastic




This would be the easiest way, which wouldn't create 300 columns, would allow you to reuse categories across different types of products, but in the case of many products, would start to slowdown the database queries, as you would be joining 2 big tables. Product and ProductCategory
You could split it up in more major Categories such as "Plants", "Kitchenware" etc etc.
Title: Time complexity of recursion of multiplication
Tags: python,recursion,time-complexity,big-o

Question: What is the worst case time complexity (Big O notation) of the following function for positive integers?
def rec_mul(a:int, b:int) -> int:
        if b == 1:
            return a
        
        if a == 1:
            return b
        
        else:
            return a + rec_mul(a, b-1)

I think it's O(n) but my friend claims it's O(2^n)
My argument:
The function recurs at any case b times, therefor the complexity is O(b) = O(n)
His argument:
since there are n bits, a\b value can be no more than (2^n)-1,
therefor the max number of calls will be O(2^n)

Answer: Background
A unary encoding of the input uses an alphabet of size 1: think tally marks. If the input is the number a, you need O(a) bits.
A binary encoding uses an alphabet of size 2: you get 0s and 1s. If the number is a, you need O(log_2 a) bits.
A trinary encoding uses an alphabet of size 3: you get 0s, 1s, and 2s. If the number is a, you need O(log_3 a) bits.
In general, a k-ary encoding uses an alphabet of size k: you get 0s, 1s, 2s, ..., and k-1s. If the number is a, you  need O(log_k a) bits.
What does this have to do with complexity?
As you are aware, we ignore multiplicative constants inside big-oh notation. n, 2n, 3n, etc, are all O(n).
The same holds for logarithms. log_2 n, 2 log_2 n, 3 log_2 n, etc, are all O(log_2 n).
The key observation here is that the ratio log_k1 n / log_k2 n is a constant, no matter what k1 and k2 are... as long as they are greater than 1. That means f(log_k1 n) = O(log_k2 n) for all k1, k2 > 1.
This is important when comparing algorithms. As long as you use an "efficient" encoding (i.e., not a unary encoding), it doesn't matter what base you use: you can simply say f(n) = O(lg n) without specifying the base. This allows us to compare runtime of algorithms without worrying about the exact encoding you use.
So n = b (which implies a unary encoding) is typically never used. Binary encoding is simplest, and doesn't provide a non-constant speed-up over any other encoding, so we usually just assume binary encoding.
That means we almost always assume that n = lg a + lg b as the input size, not n = a + b. A unary encoding is the only one that suggests linear growth, rather than exponential growth, as the values of a and b increase.

One area, though, where unary encodings are used is in distinguishing between strong NP-completeness and weak NP-completeness. Without getting into the theory, if a problem is NP-complete, we don't expect any algorithm to have a polynomial running time, that is, one bounded by O(n**k) for some constant k when using an efficient encoring.
But some algorithms do become polynomial if we allow a unary encoding. If a problem that is otherwise NP-complete becomes polynomial when using an unary encoding, we call that a weakly NP-complete problem. It's still slow, but it is in some sense "faster" than an algorithm where the size of the numbers doesn't matter.
Title: Time complexity of recursion of multiplication
Tags: python,recursion,time-complexity,big-o

Question: What is the worst case time complexity (Big O notation) of the following function for positive integers?
def rec_mul(a:int, b:int) -> int:
        if b == 1:
            return a
        
        if a == 1:
            return b
        
        else:
            return a + rec_mul(a, b-1)

I think it's O(n) but my friend claims it's O(2^n)
My argument:
The function recurs at any case b times, therefor the complexity is O(b) = O(n)
His argument:
since there are n bits, a\b value can be no more than (2^n)-1,
therefor the max number of calls will be O(2^n)

Answer: Your friend and you can both be right, depending on what is n. Another way to say this is that your friend and you are both wrong, since you both forgot to specify what was n.
Your function takes an input that consists in two variables, a and b. These variables are numbers. If we express the complexity as a function of these numbers, it is really O(b log(ab)), because it consists in b iterations, and each iteration requires an addition of numbers of size up to ab, which takes log(ab) operations.
Now, you both chose to express the complexity in function of n rather than a or b. This is okay; we often do this; but an important question is: what is n?
Sometimes we think it's "obvious" what is n, so we forget to say it.

If you choose n = max(a, b) or n = a + b, then you are right, the complexity is O(n).
If you choose n to be the length of the input, then n is the number of bits needed to represent the two numbers a and b. In other words, n = log(a) + log(b). In that case, your friend is right, the complexity is O(2^n).

Since there is an ambiguity in the meaning of n, I would argue that it's meaningless to express the complexity as a function of n without specifying what n is. So, your friend and you are both wrong.
Title: Time complexity of recursion of multiplication
Tags: python,recursion,time-complexity,big-o

Question: What is the worst case time complexity (Big O notation) of the following function for positive integers?
def rec_mul(a:int, b:int) -> int:
        if b == 1:
            return a
        
        if a == 1:
            return b
        
        else:
            return a + rec_mul(a, b-1)

I think it's O(n) but my friend claims it's O(2^n)
My argument:
The function recurs at any case b times, therefor the complexity is O(b) = O(n)
His argument:
since there are n bits, a\b value can be no more than (2^n)-1,
therefor the max number of calls will be O(2^n)

Answer: You are both right.
If we disregard the time complexity of addition (and you might discuss whether you have reason to do so or not) and count only the number of iterations, then you are both right because you define:
n = b
and your friend defines
n = log_2(b)
so the complexity is O(b) = O(2^log_2(b)).
Both definitions are valid and both can be practical. You look at the input values, your friend at the lengths of the input, in bits.
This is a good demonstration why big-O expressions mean nothing if you don't define the variables used in those expressions.
Title: Django Correct Date / Time not PC date/time
Tags: python,django,python-datetime

Question: Is there a way to get the exact date/time from the web rather than taking the PC date/time?
I am creating a website where the answer is time relevant. But i don't want someone cheating by putting their pc clock back. When i do:
today = datetime.datetime.today()

or
now = datetime.datetime.utcnow().replace(tzinfo=utc)

I still get whatever time my pc is set to.
Is there a way to get the correct date/time.

Answer: datetime.today() takes its time information from the server your application is running on. If you currently run your application with python manage.py localhost:8000, the server is your local PC. In this scenario, you can tamper with the time setting of your PC and see different results.
But in production environment, your hosting server will provide the time information. Unless you have a security issue, no unauthorized user should be able to change that.
Title: jupyter console doesn't work on my computer anymore
Tags: python,archlinux,jupyter-console

Question: I sometimes use jupyter console to try out things in python.
I'm running arch linux and installed everything through the arch repos.
I hadn't ran jupyter console in quite some time, but while trying to launch it, i can't get it to work anymore.
Here is the error :
Jupyter console 6.5.1

Python 3.10.9 (main, Dec 19 2022, 17:35:49) [GCC 12.2.0]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.10.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: 
Task exception was never retrieved
future: <Task finished name='Task-7' coro=<ZMQTerminalInteractiveShell.handle_external_iopub() done, defined at /usr/lib/python3.10/site-packages/jupyter_console/ptshell.py:839> exception=TypeError("object int can't be used in 'await' expression")>
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/jupyter_console/ptshell.py", line 842, in handle_external_iopub
    poll_result = await self.client.iopub_channel.socket.poll(500)
TypeError: object int can't be used in 'await' expression
Shutting down kernel

I tried reinstalling everything through pacman in case I accidentally changed something I shouldn't, but it changed nothing.
Any tips on what could be wrong ?

Answer: I don't have enough rep to comment but I do not have the same issue. I can launch Jupyter QT Console just fine, and I have the same python version and IPython version. Just thought I would share, even though I don't use Jupyter Console. I do all my .ipynb in vscode and all other coding in neovim.  I don't know if there is a difference between the console you are talking about and QT console, but Jupyter QT Console works fine for me, just unbearably light theme :).
Title: Pandas plot, vars() argument must have __dict__ attribute?
Tags: python,pandas,matplotlib

Question: It was working perfectly earlier but for some reason now I am getting strange errors.
pandas version: 1.2.3
matplotlib version: 3.7.0
sample dataframe:
df
    cap       Date
0    1     2022-01-04
1    2     2022-01-06
2    3     2022-01-07
3    4     2022-01-08

df.plot(x='cap', y='Date')
plt.show()

df.dtypes
cap              int64
Date    datetime64[ns]
dtype: object

I get a traceback:
Traceback (most recent call last):
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
  File "/Volumes/coding/venv/lib/python3.8/site-packages/pandas/plotting/_core.py", line 955, in __call__
    return plot_backend.plot(data, kind=kind, **kwargs)
  File "/Volumes/coding/venv/lib/python3.8/site-packages/pandas/plotting/_matplotlib/__init__.py", line 61, in plot
    plot_obj.generate()
  File "/Volumes/coding/venv/lib/python3.8/site-packages/pandas/plotting/_matplotlib/core.py", line 279, in generate
    self._setup_subplots()
  File "/Volumes/coding/venv/lib/python3.8/site-packages/pandas/plotting/_matplotlib/core.py", line 337, in _setup_subplots
    fig = self.plt.figure(figsize=self.figsize)
  File "/Volumes/coding/venv/lib/python3.8/site-packages/matplotlib/_api/deprecation.py", line 454, in wrapper
    return func(*args, **kwargs)
  File "/Volumes/coding/venv/lib/python3.8/site-packages/matplotlib/pyplot.py", line 813, in figure
    manager = new_figure_manager(
  File "/Volumes/coding/venv/lib/python3.8/site-packages/matplotlib/pyplot.py", line 382, in new_figure_manager
    _warn_if_gui_out_of_main_thread()
  File "/Volumes/coding/venv/lib/python3.8/site-packages/matplotlib/pyplot.py", line 360, in _warn_if_gui_out_of_main_thread
    if _get_required_interactive_framework(_get_backend_mod()):
  File "/Volumes/coding/venv/lib/python3.8/site-packages/matplotlib/pyplot.py", line 208, in _get_backend_mod
    switch_backend(rcParams._get("backend"))
  File "/Volumes/coding/venv/lib/python3.8/site-packages/matplotlib/pyplot.py", line 331, in switch_backend
    manager_pyplot_show = vars(manager_class).get("pyplot_show")
TypeError: vars() argument must have __dict__ attribute

Answer: The solution by NEStenerus did not work for me, because I don't have tkinter installed and did not want to change my package configuration.
Alternative Fix
Instead, you can disable the "show plots in tool window" option, by going to
Settings | Tools | Python Scientific | Show plots in tool window and unchecking it.
Title: Reverse Index through a numPy ndarray
Tags: python,numpy,indexing

Question: I have a n x n dimensional numpy array of eigenvectors as columns, and want to return the last v of them as another array. However, they are currently in ascending order, and I wish to return them in descending order.
Currently, I'm attempting to index as follows
eigenvector_array[:,-1:-v]

But this doesn't seem to be working. Is there a more efficient way to do this?

Answer: Lets re-write this to make it a little less confusing.
eigenvector_array[:,-1:-v]
to:
eigenvector_array[:][-1:-v]
Now remember how slicing works in python:
[start:stop:step]
If you set step. to -1 it will return them in reverse, so:
eigenvector_array[:,-1:-v:-1] should be your answer.
Title: How can I make snakefile rule append the results to the input file of the rule file?
Tags: python,pipeline,snakemake

Question: I am building a snakmake pipeline, in the final rule i have an existing files that i want the snakefile to append to:
Here is the rule:
rule Amend: 
    input:
        Genome_stats = expand("global_temp_workspace/result/{sample}.Genome.stats.tsv", sample= sampleID),
        GenomeSNV = expand("global_temp_workspace/result/{sample}.Genome.SNVs.tsv", sample= sampleID),
        GenomesConsensus = expand("global_temp_workspace/analysis/{sample}.renamed.consensus.fasta", sample= sampleID),
    output: 
        Genome_stats="global_temp_workspace/result/Genome.stats.tsv",
        GenomeSNV="global_temp_workspace/result/Genome.SNVs.tsv",
        GenomesConsensus="global_temp_workspace/result/Genomes.consensus.fasta"
    threads: workflow.cores
    shell: 
        """
        cat {input.Genome_stats} | tail -n +2 >> {output.Genome_stats} ;\ 
        cat {input.GenomesConsensus} >>  {output.GenomesConsensus} ;\ 
        cat {input.GenomeSNV} | tail -n +2 >> {output.GenomeSNV} ;\ 
        """

how can i solve it?
Thank you
I tried to do the dynamic() in the output and adding the  touch {output.Genome_stats} {output.GenomesConsensus} {output.GenomeSNV} at the end of the shell. but did not work.
whenevr i run the snakemake i get:
$ time snakemake --snakefile  V2.5.smk  --cores all 
Building DAG of jobs...
Nothing to be done.
Complete log: .snakemake/log/2023-02-15T123050.937009.snakemake.log

real    0m1.022s
user    0m2.744s
sys     0m2.797s

Answer: This behaviour is not idempotent and is usually a recipe for trouble. What happens if the machine breaks down or the process is killed during the write stage? What happens if a rule is accidentally ran twice?
As advised by @Cornelius Roemer in the comment to the question, the safer way is to write to a new file. If the overwrite-like behaviour is desired, then the new file can be moved to the original file location, but some record/checkpoint file should be created to make sure that Snakemake knows not to re-process the file.
Title: Poetry shell command prompt: what gives the (base) part?
Tags: git,shell,python-venv,python-poetry

Question: I am developing python projects under git control using poetry to manage my venvs.
From my project's directory I issue a "poetry shell" command and my new shell command prompt becomes something like:
(isagog-ai-py3.10) (base) bob@Roberts-Mac-mini isagog-ai %

where the first part in bracket gives me the name pf the project and the python version I'm using, and the last part of the prompt is my current directory name.
But what is it that gives me the "(base)" part? I'm actually working on a "dev" branch.

Answer: This is base environment from conda.
Title: Unexpected behavior using if .. or .. Python
Tags: python,if-statement

Question: I'm reading in a list of samples from a text file and in that list every now and then there is a "channel n" checkpoint. The file is terminated with the text eof. The code that works until it hits the eof which it obviously cant cast as a float
log = open("mq_test.txt", 'r')
data = []
for count, sample in enumerate(log):
    if "channel" not in sample:
        data.append(float(sample))
        
print(count)
log.close()

So to get rid of the ValueError: could not convert string to float: 'eof\n' I added an or to my if as so,
log = open("mq_test.txt", 'r')
data = []
for count, sample in enumerate(log):
    if "channel" not in sample or "eof" not in sample:
        data.append(float(sample))
        
print(count)
log.close()

And now I get ValueError: could not convert string to float: 'channel 00\n'
So my solution has been to nest the ifs & that works.
Could somebody explain to me why the or condition failed though?

Answer: I think it's a logic issue which "and" might be used instead of "or"
Title: What does async actually do in FastAPI?
Tags: python-3.x,asynchronous,async-await,fastapi

Question: I have two scripts:
from fastapi import FastAPI
import asyncio

app = FastAPI()

@app.get("/")
async def root():
    a = await asyncio.sleep(10)
    return {'Hello': 'World',}

And second one:
from fastapi import FastAPI
import time
  
app = FastAPI()

@app.get("/")
def root():
    a = time.sleep(10)
    return {'Hello': 'World',}

Please note the second script doesn't use async. Both scripts do the same, at first I thought, the benefit of an async script is that it allows multiple connections at once, but when testing the second code, I was able to run multiple connections as well. The results are the same, performance is the same and I don't understand why would we use async method. Would appreciate your explanation.

Answer: FastAPI Docs:

You can mix def and async def in your path operation functions as much as you need and define each one using the best option for you. FastAPI will do the right thing with them.
Anyway, in any of the cases above, FastAPI will still work asynchronously and be extremely fast.

Both endpoints will be executed asynchronously, but if you define your endpoint function asynchronously, it will allow you to use await keyword and work with asynchronous third party libraries
Title: Why does python interpreter consider 2.0 and 2 to be the same in an when used as a dictionary key
Tags: python,function,dictionary

Question: I was going through twitter when i came across the function below
def func():
    d = {1: "I", 2.0: "love", 2: "Python"}
    return d[2.0]
print(func())

When i ran the code, i got Python as the output and i expected it to be love. I know that you cannot have multiple key in a dictionary. However what i want to know is why Python Interpreter considers 2.0 and 2 as the same and returns the value of 2

Answer: In your example, the keys 2.0 and 2 are considered the same because their hash values are equal. This is because in Python, float and integer objects can be equal even if they have different types and representations. In particular, the integer 2 and the floating-point number 2.0 have the same value, so they are considered equal.
That's why you should always use consistent types for keys in dictionaries. Always remember to use integers or floats.
Title: trying to open flake8 on vs code from command palette error on mac
Tags: python,visual-studio-code,flake8

Question: I've installed flake 8 in the terminal, but when i try and select python linter on vs code in the command palette i get the following error: "Command 'Python: Select Linter' resulted in an error (command 'python.setLinter' not found)". I'm on a mac, version 11.5.2.
I have seen other solutions for this problem for windows on stack but not sure how to proceed on mac, please advise

Answer: There are many possibilities. You can try the following methods:

Reinstall Python extension or use Pre-release version.
Start VsCode as administrator.
Try to delete the.vscode folder in the project.