Myashka/SO-Python_QA-filtered-2023-tanh_score-after_2023_02 huggingface dataset:
Title: "cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_'" on AWS Lambda using a layer
Tags: python,amazon-web-services,amazon-s3,aws-lambda,boto3

Question: What I want to achieve
To scrape an website using AWS Lambda and save the data on S3.
The issues I'm having
When I execute Lambda, the following error message appears.

{   "errorMessage": "Unable to import module 'lambda_function': cannot
import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_'
(/opt/python/urllib3/util/ssl_.py)",   "errorType":
"Runtime.ImportModuleError",   "requestId":
"fb66bea9-cbad-4bd3-bd4d-6125454e21be",   "stackTrace": [] }

Code
The minimum Lambda code is as follows.
import requests
import boto3 

def lambda_handler(event, context):
    s3 = boto3.client('s3')
    upload_res = s3.put_object(Bucket='horserace-dx', Key='/raw/a.html', Body='testtext')
    
    return event

An layer was added to the Lambda. Files were save in python folder using the commands below , frozen in a zip file, then uploaded to AWS Lambda as a layer.
!pip install requests -t ./python --no-user
!pip install pandas -t ./python --no-user
!pip install beautifulsoup4 -t ./python --no-user


The bucket horserace-dx exists
The folder raw exists
The role of the Lambda is properly set. It can read from and write to S3
The runtime of the Lambda is Python 3.9. The python version of the local computer is 3.9.13.

What I did so far
I google "cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_'" and found some suggestions. I made the layer with the following code and tried again in vain.
!pip install requests -t ./python --no-user
!pip install pandas -t ./python --no-user
!pip install beautifulsoup4 -t ./python --no-user
!pip install urllib3==1.26.15 -t ./python --no-user

So what should I do to achieve what I want to achieve? Any suggestions would be greatly appreciated.

Answer: Execute the following commands.
pip install requests==2.25.0 -t ./python --no-user
pip install beautifulsoup4 -t ./python --no-user
pip install pytz -t ./python --no-user

On PyPI, download the following whl files from the pages of numpy and pandas



numpy-1.24.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
pandas-2.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl


Unzip the files and move the contents to the python folder.

Zip the python folder and upload it to AWS Lambda Layer.

Set the layer to the Lambda.

Then the code runs without errors.
Title: "cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_'" on AWS Lambda using a layer
Tags: python,amazon-web-services,amazon-s3,aws-lambda,boto3

Question: What I want to achieve
To scrape an website using AWS Lambda and save the data on S3.
The issues I'm having
When I execute Lambda, the following error message appears.

{   "errorMessage": "Unable to import module 'lambda_function': cannot
import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_'
(/opt/python/urllib3/util/ssl_.py)",   "errorType":
"Runtime.ImportModuleError",   "requestId":
"fb66bea9-cbad-4bd3-bd4d-6125454e21be",   "stackTrace": [] }

Code
The minimum Lambda code is as follows.
import requests
import boto3 

def lambda_handler(event, context):
    s3 = boto3.client('s3')
    upload_res = s3.put_object(Bucket='horserace-dx', Key='/raw/a.html', Body='testtext')
    
    return event

An layer was added to the Lambda. Files were save in python folder using the commands below , frozen in a zip file, then uploaded to AWS Lambda as a layer.
!pip install requests -t ./python --no-user
!pip install pandas -t ./python --no-user
!pip install beautifulsoup4 -t ./python --no-user


The bucket horserace-dx exists
The folder raw exists
The role of the Lambda is properly set. It can read from and write to S3
The runtime of the Lambda is Python 3.9. The python version of the local computer is 3.9.13.

What I did so far
I google "cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_'" and found some suggestions. I made the layer with the following code and tried again in vain.
!pip install requests -t ./python --no-user
!pip install pandas -t ./python --no-user
!pip install beautifulsoup4 -t ./python --no-user
!pip install urllib3==1.26.15 -t ./python --no-user

So what should I do to achieve what I want to achieve? Any suggestions would be greatly appreciated.

Answer: The resolution that worked for me was to update the AWS Lambda runtime to 3.10 and if applicable any AWS Lambda layers to python 3.10 and then ensure you have packaged the latest release of requests. If you are using botocore or boto3 you will also need to ensure that you have included them in the code package or in an AWS Lambda Layer including versions boto3>=1.26.153 and botocore>=1.29.153.
Title: "cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_'" on AWS Lambda using a layer
Tags: python,amazon-web-services,amazon-s3,aws-lambda,boto3

Question: What I want to achieve
To scrape an website using AWS Lambda and save the data on S3.
The issues I'm having
When I execute Lambda, the following error message appears.

{   "errorMessage": "Unable to import module 'lambda_function': cannot
import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_'
(/opt/python/urllib3/util/ssl_.py)",   "errorType":
"Runtime.ImportModuleError",   "requestId":
"fb66bea9-cbad-4bd3-bd4d-6125454e21be",   "stackTrace": [] }

Code
The minimum Lambda code is as follows.
import requests
import boto3 

def lambda_handler(event, context):
    s3 = boto3.client('s3')
    upload_res = s3.put_object(Bucket='horserace-dx', Key='/raw/a.html', Body='testtext')
    
    return event

An layer was added to the Lambda. Files were save in python folder using the commands below , frozen in a zip file, then uploaded to AWS Lambda as a layer.
!pip install requests -t ./python --no-user
!pip install pandas -t ./python --no-user
!pip install beautifulsoup4 -t ./python --no-user


The bucket horserace-dx exists
The folder raw exists
The role of the Lambda is properly set. It can read from and write to S3
The runtime of the Lambda is Python 3.9. The python version of the local computer is 3.9.13.

What I did so far
I google "cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_'" and found some suggestions. I made the layer with the following code and tried again in vain.
!pip install requests -t ./python --no-user
!pip install pandas -t ./python --no-user
!pip install beautifulsoup4 -t ./python --no-user
!pip install urllib3==1.26.15 -t ./python --no-user

So what should I do to achieve what I want to achieve? Any suggestions would be greatly appreciated.

Answer: You can solve this adding in your deployment.txt:
urllib3==1.16
Title: Scheduling Jupyter Notebooks to run one after the other
Tags: python,anaconda,jupyter,jupyter-lab

Question: I have 9 jupyter notebooks in Anaconda that are parsing large txt files, validating their data and providing validation output in 9 different excel workbooks to end users. With my algo being now complete, I need to take all these notebooks into production and they need to be run in a particular order one after the other without having the user need to click the Run button anywhere (they have no idea of Anaconda or Jupyter).
My question is - is there an option of designing a workflow in Anaconda itself which will enable this? Or any other external option to make these Jupyters run?
I tried using Jupyter Labs but to no avail.

Answer: Hello have you tried to automate the execution of Jupyter notebooks throught Papermill? You can use it to set up your jobs in a specific order, one approach is to use a workflow management tool or create a script that executes the notebooks sequentially. You can try Papermill that is a Python library that allows you to parameterize and execute Jupyter notebooks.You can define the execution order of the notebooks by creating a script that uses Papermill to run each notebook one after the other. You can pass input parameters and output paths between the notebooks as needed.
regards
juliano
Title: Gekko not finding solution to non-linear program unless I predefine values of integer variables
Tags: python,nonlinear-optimization,gekko

Question: I'm currently using Gekko to solve a non-linear program. The program involves sending some input volume through multiple exchanges, and every exchange along the way requires users to pay a one-time fee. Although these exchange networks can get quite complex, the network I have set up to test my solution simply goes 1 --> 2 --> 3, with the arrows representing the exchanges that are used.
To model the use of the exchange, I have added a binary variable Q which is 1 if the exchange is used and 0 if the exchange is not used. Given that the volume has to flow from 1 to 3 in the network above, both exchanges must be used, however, Gekko fails to discover this fact on its own and says that the solution is not found.
As more complex networks will require Gekko to decide for itself whether it should use some exchanges, I would like it to be able to do this on its own.
from gekko import GEKKO

m = GEKKO(remote=False)
m.options.SOLVER = 1

# Constants
vol = m.Const(10, 'Vol')
x = m.Const(10, 'x')
y = m.Const(10, 'y')
k = m.Const(100, 'k')
g = m.Const(0.5, 'gas_fee')



# Inputs and Outputs
i121, i231, o231 = [m.Var(lb=0) for i in range(3)]

# Activation
q121, q231 = [m.Var(lb=0, ub=1, integer=True) for i in range(2)]



# Constraints
m.Equation((y - (k/(x+i121))) == i231)
m.Equation((y - (k/(x+i231))) == o231)

m.Equation(i121 == i121*q121)
m.Equation(i231 == i231*q231)

m.Equation(i121 == vol)


# Objective Function
m.Minimize(g*q121 + g*q231 - o231)
m.solve(disp=True)

I have tried initiating the binary variables with  values of 1, in which case the solution is found correctly - so for some reason it seems that Gekko does not try changing the binary variables to 1 on its own to find the solution.
q121, q231 = [m.Var(value = 1, lb=0, ub=1, integer=True) for i in range(2)]

FYI, the correct solution would be:
i121 = 10
q121 = 1
i231 = 5
o231 = 3.33333...
q231 = 1
objective function = -2.3333

Answer: Managed to get it working by editing the constraints for activating edges with the binary q variable.
Instead of using:
m.Equation(i121 == i121*q121)
I'm now using:
m.Equation(i121 <= H*q121)
Where H is some very large value (larger than the max value for i121)
This seems to be working even for more complex problems with more edges.
Title: Python class attribute access fails if attribute has no value
Tags: python,python-3.x,python-descriptors

Question: Earlier today i stumbled upon a behaviour with python classes which i could not fully comprehend.
If we take this piece of code:
class A:
    a: str

When I test out the attribute access for a like so A.a or even A().a, I get an AttributeError: type object 'A' has no attribute 'a'.
On the contrary, if i just give A.a a value:
class A:
    a: str = 'attribute a'

then i can access it with A.a. # returns "attribute a".
I do not fully understand why the first case raised an AttributeError.

Answer: In the first case, you are telling the type checker that there will be an attribute called a of type str, but you are not setting it.
In the second case, you are actually giving a a value
Title: Getting a ModuleNotFoundError with librosa
Tags: python,python-3.x,data-science,librosa,modulenotfounderror

Question: I am trying to load the audio files into the NumPy array using this code
#%%
import librosa
import matplotlib.pyplot as plt
import IPython.display as ipd
import os, os.path
import time
import joblib
import numpy as np

#%%
fname = 'archive\\Actor_01\\03-01-01-01-01-01-01.wav'

data, sampling_rate = librosa.load(fname)
plt.figure(figsize=(15, 5))
librosa.display.waveshow(data, sr=sampling_rate)

ipd.Audio(fname)

# %%
lst = []

for subdir, dirs, files in os.walk('archive'):
    for file in files:
        try:
            X, sample_rate = librosa.load(os.path.join(subdir, file), res_type='kaiser_fast')
            mfccs = np.mean(librosa.feature.mfcc(y=X, sr=sample_rate, n_mfcc=40).T, axis=0)
            file_class = int(file[7:8]) - 1
            arr = mfccs, file_class
            lst.append(arr)
        except ValueError as err:
            print(err)
            continue

Everything runs fine except I am getting an error at line 25
ModuleNotFoundError                       Traceback (most recent call last)
c:\Users\powellt1\Documents\COMP5500\NonGitSED\algorithm.py in line 7
      23 for file in files:
      24     try:
----> 25         X, sample_rate = librosa.load(os.path.join(subdir, file), res_type='kaiser_fast')
      26         mfccs = np.mean(librosa.feature.mfcc(y=X, sr=sample_rate, n_mfcc=40).T, axis=0)
      27         file_class = int(file[7:8]) - 1
File c:\Users\powellt1\AppData\Local\Programs\Python\Python311\Lib\site-packages\librosa\core\audio.py:193, in load(path, sr, mono, offset, duration, dtype, res_type)
    190     y = to_mono(y)
    192 if sr is not None:
--> 193     y = resample(y, orig_sr=sr_native, target_sr=sr, res_type=res_type)
    195 else:
    196     sr = sr_native

File c:\Users\powellt1\AppData\Local\Programs\Python\Python311\Lib\site-packages\librosa\core\audio.py:684, in resample(y, orig_sr, target_sr, res_type, fix, scale, axis, **kwargs)
    675     y_hat = np.apply_along_axis(
    676         soxr.resample,
    677         axis=axis,
   (...)
    681         quality=res_type,
    682     )
    683 else:
--> 684     y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis)

I know I have librosa installed properly and it works in the previous cells.
I've tried to reinstall and verify the installation of librosa and I've tried using
from librosa.core import load

but nothing seems to be working to fix the error.

Answer: I figured out after coming through the documentation that you have to install resampy package separately from the librosa package itself. After installing the package I was able to get it to work.
Title: Getting a ModuleNotFoundError with librosa
Tags: python,python-3.x,data-science,librosa,modulenotfounderror

Question: I am trying to load the audio files into the NumPy array using this code
#%%
import librosa
import matplotlib.pyplot as plt
import IPython.display as ipd
import os, os.path
import time
import joblib
import numpy as np

#%%
fname = 'archive\\Actor_01\\03-01-01-01-01-01-01.wav'

data, sampling_rate = librosa.load(fname)
plt.figure(figsize=(15, 5))
librosa.display.waveshow(data, sr=sampling_rate)

ipd.Audio(fname)

# %%
lst = []

for subdir, dirs, files in os.walk('archive'):
    for file in files:
        try:
            X, sample_rate = librosa.load(os.path.join(subdir, file), res_type='kaiser_fast')
            mfccs = np.mean(librosa.feature.mfcc(y=X, sr=sample_rate, n_mfcc=40).T, axis=0)
            file_class = int(file[7:8]) - 1
            arr = mfccs, file_class
            lst.append(arr)
        except ValueError as err:
            print(err)
            continue

Everything runs fine except I am getting an error at line 25
ModuleNotFoundError                       Traceback (most recent call last)
c:\Users\powellt1\Documents\COMP5500\NonGitSED\algorithm.py in line 7
      23 for file in files:
      24     try:
----> 25         X, sample_rate = librosa.load(os.path.join(subdir, file), res_type='kaiser_fast')
      26         mfccs = np.mean(librosa.feature.mfcc(y=X, sr=sample_rate, n_mfcc=40).T, axis=0)
      27         file_class = int(file[7:8]) - 1
File c:\Users\powellt1\AppData\Local\Programs\Python\Python311\Lib\site-packages\librosa\core\audio.py:193, in load(path, sr, mono, offset, duration, dtype, res_type)
    190     y = to_mono(y)
    192 if sr is not None:
--> 193     y = resample(y, orig_sr=sr_native, target_sr=sr, res_type=res_type)
    195 else:
    196     sr = sr_native

File c:\Users\powellt1\AppData\Local\Programs\Python\Python311\Lib\site-packages\librosa\core\audio.py:684, in resample(y, orig_sr, target_sr, res_type, fix, scale, axis, **kwargs)
    675     y_hat = np.apply_along_axis(
    676         soxr.resample,
    677         axis=axis,
   (...)
    681         quality=res_type,
    682     )
    683 else:
--> 684     y_hat = resampy.resample(y, orig_sr, target_sr, filter=res_type, axis=axis)

I know I have librosa installed properly and it works in the previous cells.
I've tried to reinstall and verify the installation of librosa and I've tried using
from librosa.core import load

but nothing seems to be working to fix the error.

Answer: As you can see , the ModuleNotFoundError is not for Librosa module at all. It is for a module that is used either within the librosa module or you are passing in the wrong dimensions (not likely , but check.)
Also, check if there is any filter names 'kaiser_fast' , I think that is what is causing error.
Title: Is it safe to run multiple `pip install` commands at the same time?
Tags: python,installation,pip

Question: Suppose I am running pip install for a large library, which is taking a long time to install. Is it safe to run additional pip installs at the same time? (in a different shell, but the same environment)

Answer: Technically, it is safe to run additional pip installs in a different shell or command prompt while another pip installation process is running in the background. This is because each shell or command prompt instance has its own interpreter session and environment, so they won't interfere with each other.
However, keep in mind that running multiple pip installations simultaneously can put additional load on your system's resources (CPU, memory, disk I/O) and may slow down all the installations. Additionally, if there are any shared dependencies between the packages being installed, this could lead to version conflicts or errors.
So, while it is theoretically safe to do so, it is generally not recommended to run multiple pip installs at the same time, especially if you're installing large libraries. It's better to wait for one installation to finish before starting another one to avoid potential issues or performance problems. If you need to install multiple libraries, you could create a requirements.txt file with all the dependencies and install them together using pip install -r requirements.txt.
Title: Does time.sleep stop subprocess in python?
Tags: python,c++,subprocess,signals,sleep

Question: We have C++ binary which will watch for some directories, and if any new file is added to those directories, C++ binary parses the file, creates new file and place it in some other directory.
We need to test the processed file. We used python to test this. Our python test case does below

Start the C++ binary in sub process.
Add some files in to a directory.
Wait for some time to give some processing time to C++ binary.
test the processed file content.

Below is the pseudo code I used.
file = 'SomeFile.txt'
prc = subprocess.Popen([cpppath, "-v", "-c", cfg], stdout=file)
# Add some files in to a directory
time.sleep(5) # To give time to C++ binary
# Test the content

But the issue I observed is, time.sleep() also stops C++ binary. I came to know this because we have a cron thread in C++ binary which will print I am Alive every second. This is getting printed till the time we call timer.sleep() in python. Once the sleep ends, Python test started testing the content. But as C++ binary didn't get chance to process the file in watch directory, test is failing.
My manager suggested to use signals. Python test case will create the files in watch directory and waits for sigusr1 and once the signal comes, test will proceed. C++ binary process the input files, place the processed file in a directory and signals parent process which is python test. But even for this, python test needs to use sleep() or pause() which will be the same issue again.
Can anyone please let me know if there is any way to solve the issue.

Answer: No, calling time.sleep() in Python does not affect the binary started with subprocess.Popen(). There must be some other reason the test doesn't work as expected.
Title: How can I use a variable or list item with xarray / rioxarray to identify a band for use in functions?
Tags: python,python-xarray

Question: I'm attempting to iterate through netcdf files, and would like the band name that I'm working with to be defined by matching a list of expected bands with what is available in the netcdf. This part is working ok.
What I'm having trouble with is using subsequent functions where I would ordinarily call the band by name, and substituting in listname[0].
For example:
Instead of using bandcount = up_sampled.band_name.shape[0] I would like to instead be able to use bandcount = up_sampled.common_key[0].shape[0] where common_key is the list.
The error is: ``'Dataset' object has no attribute 'common_key'```
I believe what I'm looking for is something similar to string substitution, but for substituting in the name of the band.
For the record, and anyone reading in future I'm matching the bands expected vs bands available in each iteration by using:
#Open the ds with rioxarray and call it ds_netcdf

#create list of keys w want to assess against
expected_keys = ['max_apples', 'max_oranges']

#print the variables in ds_netcdf
ds_keys = list(ds_netcdf.keys())

#find keys common to both lists, ds_keys and max_keys
common_key = list(set(ds_keys) & set(max_keys))

The trouble is when I try something like:
#determine the maximum number of bands in the raster
bandcount = up_sampled.common_key[0].shape[0]

Answer: The answer was to access like so:
up_sampled[common_key[]].shape[0]
Title: How to iterate through an HTML list in Jinja?
Tags: python,flask,jinja2

Question: I am using Jinja with flask and im trying to go over a list with a for statement but it shows nothing in the main view.
Here is the HTML code:
<div class="container mt-4">
    <div class="col d-flex justify-content-center">
        <div class="card-deck">
            {% for CleverHub in CleverHubs %}
            <div class="card border-dark mb-3" style="width: 18rem;">
                <div class="card-body">
                    <div class="card-header">
                        <h4>{{ CleverHub }}</h4>
                    </div>
                </div>
            </div>
            {% endfor %}
        </div>
    </div>
</div>
```html
and the script goes like this 
```js
<script>
    var CleverHubs = {{amountOfCleverHubs}}
    console.log(CleverHubs)

</script>

The console shows an array: [0]
For each device it adds a 0 the list.

Answer: You're passing the variable amountOfCleverHubs to your template, but the value is an array with only a single element, which is 0. That's why you're seeing [0] in the console.
Title: caused error while Importing opcua xml files
Tags: python,python-asyncio,opc-ua

Question: I have created testing_nodeset2.xml using siemens SiOME modelling editor , so first of all i imported opcua device , opcua machinery and opcua pump xml files into editor and then created own namespace using companion specification . finally i exported testing_noeset2.xml file. Now i want to import xml files.

is the correct way to create own namespace ?
do i need to import all xml files into python code according opcua companion specification ?
here is the code:

import asyncio
async def main():
    server = Server()
    await server.init()     
    server.set_endpoint("opc.tcp://192.168.1.216:4000")
    await server.import_xml("Opc.Ua.Nodeset2.xml")
    await server.import_xml("Opc.Ua.Di.Nodeset2.xml")
    await server.import_xml("Opc.Ua.Machinery.Nodeset2.xml")
    await server.import_xml("Opc.Ua.Pumps.Nodeset2.xml")
    await server.import_xml("testing_Nodeset.xml")
    await server.start()

if __name__ == "__main__":
    asyncio.run(main())```

here is the error i am getting : 

AddNodesItem: Requested NodeId NodeId(Identifier=3062, NamespaceIndex=0, NodeIdType=<NodeIdType.Numeric: 2>) already exists
failure adding node NodeData(nodeid:NodeId(Identifier=3062, NamespaceIndex=0, NodeIdType=<NodeIdType.Numeric: 2>)) "The requested node id is already used by another node."(BadNodeIdExists)
Traceback (most recent call last):
  File "c:\Users\user\OneDrive\Desktop\thesis\opcua\opc_server\opc_server\nodeset file testing\nodeset.py", line 50, in <module>
    asyncio.run(main())
  File "C:\Program Files\Python310\lib\asyncio\runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "C:\Program Files\Python310\lib\asyncio\base_events.py", line 646, in run_until_complete
    return future.result()
  File "c:\Users\user\OneDrive\Desktop\thesis\opcua\opc_server\opc_server\nodeset file testing\nodeset.py", line 40, in main
    await server.import_xml("Opc.Ua.Nodeset2.xml")
  File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\asyncua\server\server.py", li    node = await self._add_node_data(nodedata, no_namespace_migration=True)
  File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\asyncua\common\xmlimporter.py", line 228, in _add_node_data
    node = await self.add_object(nodedata, no_namespace_migration)
  File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\asyncua\common\xmlimporter.py", line 349, in add_object
    res[0].StatusCode.check()
  File "C:\Users\user\AppData\Roaming\Python\Python310\site-packages\asyncua\ua\uatypes.py", line 328, in check
    raise UaStatusCodeError(self.value)
asyncua.ua.uaerrors._auto.BadNodeIdExists: "The requested node id is already used by another node."(BadNodeIdExists)



does anybody has an idea , how can this error be solved? 
your advice and solution would be apreciated.

Answer: You should not import Opc.Ua.Nodeset2.xml. Because it is already import at the start of the server.
Title: Catch 22 in Python while trying to start a FLASK app with Gunicorn
Tags: python,flask,gunicorn

Question: I need to use Gunicorn to run a Flask app, yet Gunicorn doesn't read this part of the code if name == 'main':
import sys
#...

app = Flask(__name__)
CORS(app)  # Enable CORS for all routes


@app.route('/api/myapi', methods=['POST'])
def myapi():

#...


@app.after_request
def add_header(response):
response.headers['Access-Control-Allow-Origin'] = '*'
response.headers['Access-Control-Allow-Headers'] = 'Content-Type'
return response


#if __name__ == '__main__':
#    print('Starting APP server...')
#    context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
#    context.load_cert_chain('/etc/letsencrypt/live/mywebsite.com/cert.pem',
#                           '/etc/letsencrypt/live/mywebsite.com/privkey.pem')
#    app.run(host='mywebsite.com', port=5000, ssl_context=context)

If I uncommented the "if name == 'main':" then I have a development FLASK app and it works like a charm, but that's not the idea so I'm using Gunicorn with a separated file called wsgi.py
from vicky_server_ionos_8 import app

if __name__ == '__main__':
    app.run()

And then
gunicorn --bind 88.888.888.888:5000 wsgi:app

So here's the catch 22...to not run the app in a developement way...I need to use Gunicorn...yet Gunicorm seems to ignore this if name == 'main': so the script never hits app.run() so the app never runs.
I thought that Gunicorn does that automaticly, yep no...the app never starts because Guarnicon seems to ignore the very part that runs the app...
What could be the issue or any ideas how to use correctly Gunicorn?

Answer: Ok, I found the right way to do it.
Explanation: I was working with Gunicorn yet the best way to go is Gunicorn3 in my case. Then I just forgot to add the SSL certificates so the app was serving on http, so the client side would never allowed such connection.
Solution:
gunicorn3 --workers=3 --bind 88.888.888.888:5000 --certfile /etc/letsencrypt/live/mysite.com/cert.pem --keyfile /etc/letsencrypt/live/mysite.com/privkey.pem vicky_server_ionos_8:app
Thanks for the help and hope it helps as well.
Greetings.
Title: python code does not execute all lines of code in exe file
Tags: python

Question: I have used pyinstaller to convert it but the code does not execute all my lines code in the exe file and cmd ,  but to notice is the original lines code are running perfectly in vs code
i tried py installer  also another named "auto py to exe "
the only thing I want to make the code run in user's windows
import datetime

from dateutil.relativedelta import relativedelta

print("Welcome to the Recharge Assistant")

# Get today's date
today = datetime.date.today()

# Get the same day next year
same_day_next_year = datetime.date(today.year + 1, today.month, today.day)

# Get a valid date from the user
while True:
    user_input = input("Please enter your current validity (DD/MM/YYYY), or 'q' to quit: ")

    # Check if user wants to quit
    if user_input.lower() == 'q':
        break

    try:
        # Convert user's input to a date object
        user_date = datetime.datetime.strptime(user_input, "%d/%m/%Y").date()

        # Check if user's input is in the past or exceeds the same day next year
        if user_date < today:
            user_date = today
        elif user_date > same_day_next_year:
            user_date = same_day_next_year

        # Calculate the number of days between user_date and same_day_next_year
        DAYS_DIFF = (same_day_next_year - user_date).days
        if DAYS_DIFF == 0:
            print("You may have entered a wrong year or date!")

        # Handle leap year
        if DAYS_DIFF == 366:
            DAYS_DIFF = 365

        # Calculate the number of months and days in days_diff
        months = DAYS_DIFF // 30
        days = DAYS_DIFF % 30

        # Print the updated user date
        print("Current user Validity:", user_date)
        print("Max number of days that can be recharged until next year:", DAYS_DIFF)
        print(f"Number of months that can be recharged: {months} months and {days} days")

        while True:
            try:
                get_month = int(input("How many months do you need? "))
                get_days = get_month * 30
                new_date = user_date + relativedelta(months=get_month)
                print(f"You entered {get_month} months which is {get_days} days.")
                print("The resulting date will be:", new_date)
                break
            except ValueError:
                print("Invalid input. Please enter a valid integer.")

    except ValueError:
        print("Invalid date format. Please enter a valid date in the format DD/MM/YYYY.")

    # Add a break statement if the user entered a valid date to allow for quitting
    if user_input.lower() != 'q':
        break

Answer: Never had this problem before, but you should try reinstalling Py and Check if you have missing dependencies that can cause this error.
I suggest $ pip check and even if youre using it or not check your Conda eviroment.
I study your case and hear about you can try to shutdown the console or either set console ==true to false
Title: Is vars() same as __dict__?
Tags: python

Question: I read that vars() is a built-in that returns the __dict__ attribute of class, module or object. But when I checked for vars(Person) is Person.__dict__, it returned False(Person is the name of class).
class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

vars(Person) is Person.__dict__  # False

Answer: Classes create a new mappingproxy on every __dict__ access. You would see the exact same results from Person.__dict__ is Person.__dict__.
Title: Validation and getting everything to write to file
Tags: python,loops,validation,writetofile

Question: So I'm working on a password program that logs all incorrect attempts (just the date and time with an error message saying why)
The issue I'm having is when I input a password that is say, >10 and then input a password that is <6...all failed attempts will be logged as >10 even if that isn't the case.
I'm very much a beginner to Python so feel free to explain everything to me like I'm 5.
import datetime


def main():

    MIN_LENGTH = 6
    MAX_LENGTH = 10
    PASSWORD_LOG_FILE = "password_log.txt"
    password = input("Please enter your password for checking: ")
    password_length = len(password)


    while len(password) < 6 or len(password) > 10:
        print("Invalid")
        error_log = str(datetime.datetime.now())
        output_file = open(PASSWORD_LOG_FILE, "a")
        output_file.write(error_log)
        password = input("Please enter your password for checking: ")
        if password_length < MIN_LENGTH:
            output_file.write("{}, password <6".format(error_log))
        elif password_length > MAX_LENGTH:
            output_file.write("{}, password >10".format(error_log))
        output_file.write("\n")
        output_file.close()

    if password.isnumeric() is True:
        comment = "password is weak - contains only numbers."
    elif password.isalpha() is True:
        comment = "password is weak - contains only letters."
    else:
        comment = "password is strong."

    print("Your password is {:} characters long".format(len(password)))
    print("Your {:s}".format(comment))

    input_file = open(PASSWORD_LOG_FILE, "r")

    for line in input_file:
        print(line, end='')

    input_file.close()

main()

I realise an issue may be in the "While" loop and tried moving bits of code around and re-formatting but I either trigger an endless loop or the error message doesn't begin in a new line etc.
As mentioned above, What I would like to happen is that it tells me the password was invalid because it was either <6 or <10. Right now it logs all incorrect attempts as the same as the first attempt - I can't seem to locate the dodgy line of code that is causing this!

Answer: You store password_length once at the start, and use it in your if check. However, len(password) is not the same as password_length! You correctly use len(password) in your while loop, but you don't update password_length to the new length after you retype a password.
You have a few options of how to fix this, but I think the easiest one may be to add the line password_length = len(password) just after password = input("Please enter your password for checking: ").
Title: S3FS is not recognised in lambda when using torch
Tags: python,aws-lambda,pytorch,kedro

Question: I have a Kedro Pipeline Node that on AWS Lambda that accesses s3. It runs if I'm not using torch but fails with Install s3fs to access S3 if I add torch as a dependency.
I have a Kedro Pipeline I want to deploy on AWS Step Functions.
My requirements look like this:
Python3.9
Pillow==9.5.0
aws_lambda_powertools==2.15.0
fsspec==2023.5.0
kedro==0.18.8
numpy==1.24.3
pandas==2.0.1
pydantic==1.10.7
pytest==7.3.1
rasterio==1.3.6
rawpy==0.18.1
s3fs==2023.5.0

The lambda access some data on s3.
With this setup everything runs fine.
However if I add torch,
torch==2.0.1+cpu -f https://download.pytorch.org/whl/torch_stable.html
torchvision==0.15.2+cpu -f https://download.pytorch.org/whl/torch_stable.html

I get the following error:
{
  "errorMessage": "\nInstall s3fs to access S3.\nFailed to instantiate DataSet 'companies' of type 'kedro.extras.datasets.pandas.csv_dataset.CSVDataSet'.",
  "errorType": "DataSetError",
  "requestId": "3da771f3-af50-49a9-98de-0a6d924018f2",
  "stackTrace": [
    "  File \"/home/app/lambda_handler.py\", line 18, in handler\n    session.run(node_names=[node_to_run])\n",
    "  File \"/home/app/kedro/framework/session/session.py\", line 413, in run\n    catalog = context._get_catalog(\n",
    "  File \"/home/app/kedro/framework/context/context.py\", line 287, in _get_catalog\n    catalog = settings.DATA_CATALOG_CLASS.from_config(\n",
    "  File \"/home/app/kedro/io/data_catalog.py\", line 277, in from_config\n    data_sets[ds_name] = AbstractDataSet.from_config(\n",
    "  File \"/home/app/kedro/io/core.py\", line 162, in from_config\n    raise DataSetError(\n"
  ]
}

This error is also just appearing in the lambda. if I install all those requirements locally on my linux it runs fine.

Answer: I was able to solve this issue. Torch did install a version of urllib3 that was incompatible with s3fs. So what I had to do was install torch with "urllib3<2" and it worked.
Title: Getting TypeError: WebDriver.__init__() got an unexpected keyword argument ‘desired_capabilities’ when using Appium with Selenium 4.10
Tags: python,selenium-webdriver,appium,python-appium

Question: Error:
HOOK-ERROR in before_scenario: TypeError: WebDriver.__init__() got an unexpected keyword argument 'desired_capabilities' 
Hello, we currently cannot run our script together with the latest Selenium 4.10. Is this Appium error or Python error?
Here is the capabilities that we used. We're currently trying to get the capabilities for platformName by targetOS = self.driver.capabilities['platformName'] but we're hit with this error
capabilities = {
    "platformName": "Android",
    "appium:platformVersion": "11.0",
    "appium:deviceName": "emulator-5554",
    "appium:app": "/Users/faithberroya/Downloads/test.apk",
    "appium:automationName": "UiAutomator2",
    "appium:appPackage": "com.test.school.assignment.rc",
    "appium:appActivity": "com.test.school.assignment.ui.SplashActivity"
}

# launch app
context.driver = webdriver.Remote("http://0.0.0.0:4723/wd/hub", capabilities)
# add wait time
context.driver.implicitly_wait(20)
# app
context.app = Application(context.driver)

Current pip list
Appium-Python-Client     2.10.1
behave                   1.2.6
certifi                  2023.5.7
pip                      23.1.1
requests                 2.31.0
selenium                 4.9.0

Answer: Appium-Python-Client，The version is too high, try version Appium-Python-Client==1.2.0
Title: Getting TypeError: WebDriver.__init__() got an unexpected keyword argument ‘desired_capabilities’ when using Appium with Selenium 4.10
Tags: python,selenium-webdriver,appium,python-appium

Question: Error:
HOOK-ERROR in before_scenario: TypeError: WebDriver.__init__() got an unexpected keyword argument 'desired_capabilities' 
Hello, we currently cannot run our script together with the latest Selenium 4.10. Is this Appium error or Python error?
Here is the capabilities that we used. We're currently trying to get the capabilities for platformName by targetOS = self.driver.capabilities['platformName'] but we're hit with this error
capabilities = {
    "platformName": "Android",
    "appium:platformVersion": "11.0",
    "appium:deviceName": "emulator-5554",
    "appium:app": "/Users/faithberroya/Downloads/test.apk",
    "appium:automationName": "UiAutomator2",
    "appium:appPackage": "com.test.school.assignment.rc",
    "appium:appActivity": "com.test.school.assignment.ui.SplashActivity"
}

# launch app
context.driver = webdriver.Remote("http://0.0.0.0:4723/wd/hub", capabilities)
# add wait time
context.driver.implicitly_wait(20)
# app
context.app = Application(context.driver)

Current pip list
Appium-Python-Client     2.10.1
behave                   1.2.6
certifi                  2023.5.7
pip                      23.1.1
requests                 2.31.0
selenium                 4.9.0

Answer: I face similar issue. By default Appium python client uses latest selenium release(4.10.0). So i downgraded the version to 4.9.0 with Appium-python-client as 2.9.0 and it worked fine.
Title: A random word generator that has the user type the word to confirm
Tags: python,function,if-statement,while-loop

Question: Source code for a random word generator that has the user type the word to confirm
def word_list(): 
  import random #Import random function
  words = ['hello',"yes","no","I","look","where","can","he","with","see","do","we","a","you","play","am","what","one","the","this","have","on","want","go","like","and","hurt","to","under","day","is","or","of","it","are","said","big","up","that","little","down","there","my","she","out","good","her","all","yes","make","read","no","they","your\n"]
  question = random.choice(list(words)) #Generates a random word from the list
  print(question) #Displays the word that was generated
  answer = input('Retype the word and press enter:') #User retypes the 
  while True: #Loop that allows user to 
    if answer == question: #If statement that confirms the user spelled the word right
      print('Great job you spelled it correctly!')
      input('Press enter for another word')
    else: 
      while answer != question:
        print('You spelled that word wrong, please try again!')
        answer = input('Retype the word and press enter:')

Answer: while True starts an endless loop, so you have to take care, that it terminates.
The easiest solution is adding a break at the end of the then branch.
Title: How can I test Python virtual code environments programatically?
Tags: python,virtualenv,cicd,python-venv

Question: We manage a significant number of Python virtual code environments in our estate. We are moving to a more aggresive OS level patching schedule which means there is higher risk to break some Python virtual code environments due to some OS level dependencies changing or issues during installation. We are therefore looking at different options to try to mitigate this risk. Aside from recreating the virtual code environment after OS patching what else can we do to confirm the code environment it's working fine? Of course running the code/app/model that the code environment is meant to support would be ideal but this is not practical/possible for various reasons that are not relevant to the question.
I was thinking that perhaps I could look at the requirements.txt of each virtual code environment and attempt to import all packages that required to be installed in each virtual code environment. From previous experience the majority of virtual code environment issues happen either during the creation of the code environment (ie when packages are installed/code env created) or when the required packages are imported in code. So this approach will cover most of the risk leaving only potential run time issues which should be a very small proportion.
What's you view on this approach? Do you see any issues with it? One thing I know will be a challenge is that package names don't always match the "import" name used in code but I suspect we will be able to work around this problem building a dictionary of package names and import names.

Answer: Process
I believe this should be done in a two step procedure

Create environment for testing (OS / Python Version)
Run a set of integration tests to determine pass state

Step 1 ensures you can create a environment which accurately represents what will be used in production.
Step 2 ensures you can test the environment and make sure it does what you expect.
Then all you have to do is repeat this procedure for each environment, adding  additional tests as you see fit.
Creating an environment
This can be done in many ways:

Docker containers
Virtual machines using IaC provisioning
Manual config (not repeatable)

Manual config is not repeatable and would be slow.
Running Integration Tests
This is dependent on what you are testing but could be done with

Bash / Python Scripts
Web API calls
File discovery
Other low level system calls
Title: Reducing number of arguments in SQLAlchemy router function
Tags: python,rest,sqlalchemy,arguments,pylint

Question: I have a GET router function which searches for employee(s) and return complete information of employee(s) . It works fine but when I check code quality it shows too-many-arguments error in async function.
I don't wants to disable this pylint error since its not a good practice.
Here is my code snippet.
@router.get('/employee', response_model=List[schemas.Employee],
    responses={502: {"model": schemas.ErrorResponse},
    422: {"model": schemas.ErrorResponse}},  status_code=200
async def get_employee(request: Request,emp_name: str = Query(None, example="John"),
    search_string: str = Query(None, example="Canada"),
    skip: int = Query(0, ge=0),
    limit: int = Query(100, ge=0),
    db_: Session = Depends(get_db)):
    '''fetches all the employees with details
    * the optional query parameter can be used to filter the result set
    * skip=n: skips the first n objects in return list
    * limit=n: limits the no. of items to be returned to n'''
    return search_employee(db_, emp_name=emp_name,
        search_string = search_string, skip=skip, limit=limit)
        

When i use args/**kwargs
as below I could only see a single field in swagger named args/kwargs.
async def get_persons(request: Request, db_: Session = Depends(get_db), **kwargs):
    emp_name = kwargs.get('emp_name', None)
    search_string = kwargs.get('search_string', None)
    skip = kwargs.get('skip', 0)
    limit = kwargs.get('limit', 100)

    return search_employee(db_, emp_name=emp_name,
          search_string = search_string, skip=skip, limit=limit)

What I need in Swagger is different fields emp_name,search_string,skip,limit where I can enter values separately. Please help. TIA

Answer: If this was not a hard requirement of @router.get there's a maintenance benefit in refactoring the argument of your function. This is what pylint is warning about.
But if the API forces you to have a particular number of arguments for your code to work properly, then you should disable the pylint warning. Following the expected practice of @router.get is a lot more valuable than following this particular refactoring message.
You know better than pylint, think about the reason for the pylint message, then disable if it's too costly to fix, not valuable enough or if following the advice would make the code worse like in this case.
Title: PIP very slow when using it in a conda environment
Tags: python,pip,conda

Question: It has been some months since the problem appeared from nowhere, every time I install something using pip from a conda environment (fresh or old) it takes ages to start the process, i have been using -vvv for debugging the issue.
When i'm using the main conda environment the output look like the following:
Using pip 21.1.3 from /localdisk/<username>/anaconda3/lib/python3.9/site-packages/pip (python 3.9)
Non-user install because site-packages writeable
Created temporary directory: /tmp/pip-ephem-wheel-cache-79hpacew
Created temporary directory: /tmp/pip-req-tracker-2x18ocvp
...

while if I am in an environment the output looks like this:
Using pip 23.0.1 from /localdisk/lpuglia/anaconda3/envs/vpunn/lib/python3.8/site-packages/pip (python 3.8)
Non-user install because site-packages writeable
[*** HERE PIP HANGS FOR ABOUT 2 MINUTES ***]
Created temporary directory: /tmp/pip-build-tracker-xb9h_yb9
Initialized build tracking at /tmp/pip-build-tracker-xb9h_yb9
...

besides the obvious difference in version between the two pips, I really can't say what is the problem. Using -vvv doesn't shed any light on the source of the problem. I was expecting some hanging due to network problems but from the output it looks like that the hang happens before creating a folder.
Interestingly enough, if I call pip again just after it finishes (few seconds) it doesn't hang for 2 minutes. This time window is very small, waiting for 10 seconds will force another hang on the next pip installation.
I'm out of ideas on how to debug this issue, if anyone could suggest a way to understand better the problem or a solution it would be ideal.

Answer: To answer my own question: it looks like that synchronizing pip version across base environment and the working environment solves the problem. Not sure why this happens, but any other answer with a better explanation will be accepted as the correct one.