Myashka/SO-Python_QA-filtered-2023-tanh_score-after_2023_02 huggingface dataset:
Title: How to view-cast / reinterpret-cast in pythran / numpy?
Tags: python,numpy,reinterpret-cast,pythran

Question: I'm trying to do numpy view-casting (which I believe in C/C++ land would be called reinterpret-casting) in pythran:
The following silly made-up example takes an array of unsigned 8-byte integers, reinterprets them as twice as many unsigned 4-byte integers, slices off the first and last (this also does not touch the actual data; it only changes the "base pointer") and reinterprets as unsigned 8-byte again, the total effect being a frame shift. (We'll worry about endianness some other time.)
import numpy as np

A = np.arange(5,dtype="u8")
a = A.view("u4")
B = a[1:9].view("u8")
A
# array([0, 1, 2, 3, 4], dtype=uint64)
B
# array([ 4294967296,  8589934592, 12884901888, 17179869184], dtype=uint64)
np.shares_memory(A,B)
# True

I cannot have pythran translate this directly because it doesn't know the .view attribute.
Is there a way to reinterpret cast arrays in pythran?

Answer: As far as I can see there is no direct way to perform reinterpret casting of arrays in Pythran. Pythran doesn't support the numpy.view function, and there isn't a direct equivalent that can be used instead. Its just a limitation of Pythran as it only supports a subset of numpy's functionality.
Your best bet is probably to perform the casting in Python using numpy, then pass the result to the Pythran function. that could be feasible if the casting operation isn't a major bottleneck in your code.
Or you could use a different compiler such as Cython if you're familiar with one.
Title: Azure DevOps release pipeline AttributeError: type object 'Retry' has no attribute 'DEFAULT_METHOD_WHITELIST'
Tags: python,azure-devops,azure-databricks,azure-pipelines-release-pipeline,urllib3

Question: I have a release pipeline that has been working fine until 5/4/2023, when it started throwing this error and getting hung up in a retry loop upon trying to start a Databricks cluster. The log looks like this, and it does not exit until a user manually cancels it:
2023-05-04T15:31:48.9504235Z ##[section]Starting: Start cluster
2023-05-04T15:31:48.9507476Z ==============================================================================
2023-05-04T15:31:48.9507600Z Task         : Start a Databricks Cluster
2023-05-04T15:31:48.9507679Z Description  : Make sure a Databricks Cluster is started
2023-05-04T15:31:48.9507786Z Version      : 0.5.6
2023-05-04T15:31:48.9507851Z Author       : Microsoft DevLabs
2023-05-04T15:31:48.9507957Z Help         : 
2023-05-04T15:31:48.9508027Z ==============================================================================
2023-05-04T15:31:49.5839599Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:31:49.6142380Z Cluster *** not running, turning on...
2023-05-04T15:31:49.7846916Z Error: AttributeError: type object 'Retry' has no attribute 'DEFAULT_METHOD_WHITELIST'
2023-05-04T15:31:49.9842231Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:32:20.0262096Z Starting...
2023-05-04T15:32:20.2317560Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:32:50.2656819Z Starting...
2023-05-04T15:32:50.4489482Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:33:20.4791525Z Starting...
2023-05-04T15:33:20.6468387Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:33:50.6758000Z Starting...
2023-05-04T15:33:50.9257125Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:34:20.9617006Z Starting...
2023-05-04T15:34:21.1491705Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:34:51.1782850Z Starting...
2023-05-04T15:34:51.3835540Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:35:21.4213881Z Starting...
2023-05-04T15:35:21.6628904Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:35:51.6999474Z Starting...
2023-05-04T15:35:51.8763360Z parse error: Invalid numeric literal at line 1, column 6

This is happening for release pipelines to several different environments.
I tried restarting the Databricks cluster, but the same thing happens once the cluster starts again.
If the Start Cluster step is removed, the same happens in the next step, where it tries to deploy notebooks to a workspace.

Answer: I had a same problem, when starting the Databricks Cluster.
I tried the following:

Removed the 'Use Python version' task from the agent.

Added the task, 'Install Python on Windows'.

On the 'Agent Job', Select 'Agent Specification' and switched to 'windows-latest'.


The Release worked, hope it helps.
Title: Azure DevOps release pipeline AttributeError: type object 'Retry' has no attribute 'DEFAULT_METHOD_WHITELIST'
Tags: python,azure-devops,azure-databricks,azure-pipelines-release-pipeline,urllib3

Question: I have a release pipeline that has been working fine until 5/4/2023, when it started throwing this error and getting hung up in a retry loop upon trying to start a Databricks cluster. The log looks like this, and it does not exit until a user manually cancels it:
2023-05-04T15:31:48.9504235Z ##[section]Starting: Start cluster
2023-05-04T15:31:48.9507476Z ==============================================================================
2023-05-04T15:31:48.9507600Z Task         : Start a Databricks Cluster
2023-05-04T15:31:48.9507679Z Description  : Make sure a Databricks Cluster is started
2023-05-04T15:31:48.9507786Z Version      : 0.5.6
2023-05-04T15:31:48.9507851Z Author       : Microsoft DevLabs
2023-05-04T15:31:48.9507957Z Help         : 
2023-05-04T15:31:48.9508027Z ==============================================================================
2023-05-04T15:31:49.5839599Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:31:49.6142380Z Cluster *** not running, turning on...
2023-05-04T15:31:49.7846916Z Error: AttributeError: type object 'Retry' has no attribute 'DEFAULT_METHOD_WHITELIST'
2023-05-04T15:31:49.9842231Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:32:20.0262096Z Starting...
2023-05-04T15:32:20.2317560Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:32:50.2656819Z Starting...
2023-05-04T15:32:50.4489482Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:33:20.4791525Z Starting...
2023-05-04T15:33:20.6468387Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:33:50.6758000Z Starting...
2023-05-04T15:33:50.9257125Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:34:20.9617006Z Starting...
2023-05-04T15:34:21.1491705Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:34:51.1782850Z Starting...
2023-05-04T15:34:51.3835540Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:35:21.4213881Z Starting...
2023-05-04T15:35:21.6628904Z parse error: Invalid numeric literal at line 1, column 6
2023-05-04T15:35:51.6999474Z Starting...
2023-05-04T15:35:51.8763360Z parse error: Invalid numeric literal at line 1, column 6

This is happening for release pipelines to several different environments.
I tried restarting the Databricks cluster, but the same thing happens once the cluster starts again.
If the Start Cluster step is removed, the same happens in the next step, where it tries to deploy notebooks to a workspace.

Answer: Faced similar issue in Devops release while listing databricks clusters.
Enforce urllib3 version back to 1.26.1 in installation task
pip install databricks-cli==0.15
pip install urllib3==1.26.1
Title: RuntimeError: await wasn't used with future when using twisted, pytest_twisted plugin, and asyncio reactor
Tags: python,python-asyncio,twisted

Question: I'm running into the RuntimeError: await wasn't used with future in a simple pytest. I have the pytest_twisted plugin enabled with the argument --reactor asyncio. I can see that twisted is using asyncio and all my twisted tests run fine. However, this code gives me the above error.
async def _sleep():
    await asyncio.sleep(1.0)


@defer.inlineCallbacks
def test_sleep():
    yield defer.ensureDeferred(_sleep())

It's just a simple test case to see if I can mix asyncio and twisted code together. The full stack trace is as follows:
Traceback (most recent call last):
  File "test/test_simple.py", line 23, in test_sleep
    yield defer.ensureDeferred(_sleep())
  File "/usr/local/lib/python3.10/site-packages/twisted/internet/defer.py", line 1697, in _inlineCallbacks
    result = context.run(gen.send, result)
  File "test/test_cli.py", line 18, in _sleep
    await asyncio.sleep(1.0)
  File "/usr/local/lib/python3.10/asyncio/tasks.py", line 605, in sleep
    return await future
RuntimeError: await wasn't used with future

Anything obvious jump out?
Twisted: 22.10.0
Python: 3.10.11
pytest: 7.3.1
pytest_twisted: 1.14.0

Answer: When writing a test using pytest-twisted you should use pytest_twisted.inlineCallbacks instead of twisted.internet.defer.inlineCallbacks or you should write your tests using async/await instead of inlineCallbacks/yield and decorate them with twisted.internet.defer.ensureDeferred.
The reason for this is largely "obscure implementation details" that aren't generally interesting, just a consequence of the specific way that pytest-twisted integrates pytest and Twisted.
Title: Why is X staying = 1?
Tags: python,time,pyautogui,integer-arithmetic

Question: while 1:

    x=+1
    pic = pyautogui.screenshot()
    pic.save(str(x)+".png", "PNG")
    print(x)
    time.sleep(5)

X won't increase every time the loop, loops
I want the screenshot to save under a different name every time it screenshots, so I put x=+1 to constantly give a new name for the screen shot to save under but it only stays at one.

Answer: You should write x += 1 which means x = x + 1.
Here you just value the x to +1 (x = +1)
A little advice: is better to write mathematical syntax with space to don't get problems like this.
Title: Creating python virtualenv is failing
Tags: python,virtualenv

Question: Creating python3.10 virtualenv is failing
$python -v -m virtualenv   --python /usr/bin/python3.10 py310

# /home/test/.local/lib/python3.7/site-packages/virtualenv/activation/python/__pycache__/__init__.cpython-37.pyc matches /home/test/.local/lib/python3.7/site-packages/virtualenv/activation/python/__init__.py
# code object from '/home/test/.local/lib/python3.7/site-packages/virtualenv/activation/python/__pycache__/__init__.cpython-37.pyc'
import 'virtualenv.activation.python' # <_frozen_importlib_external.SourceFileLoader object at 0x7fc0a0849810>
# /home/test/.local/lib/python3.7/site-packages/virtualenv/activation/xonsh/__pycache__/__init__.cpython-37.pyc matches /home/test/.local/lib/python3.7/site-packages/virtualenv/activation/xonsh/__init__.py
# code object from '/home/test/.local/lib/python3.7/site-packages/virtualenv/activation/xonsh/__pycache__/__init__.cpython-37.pyc'
import 'virtualenv.activation.xonsh' # <_frozen_importlib_external.SourceFileLoader object at 0x7fc0a0849d90>
import 'virtualenv.activation' # <_frozen_importlib_external.SourceFileLoader object at 0x7fc0a0aa3c90>
KeyError: 'scripts'

Answer: Had the same issue. You need to locate which virtualenv it is using and upgrade it to latest version (at the time of this writing, the latest version is 20.23.0)
Title: Attempting to use CustomJS in Bokeh to create interactive plot
Tags: javascript,python,bokeh

Question: I am trying to create an interactive plot with Bokeh on a Jupyter Notebook. I am trying to add interactivity to the plot, making it change in real time. The goal is to simply change the position of some points on a plot. I attempted using the CustomJS method, since that is what is told in the documentation.
However, this is my first experience with JavaScript, and I am not exactly sure of how I must apply JavaScript with Bokeh.
I have attempted this code:
source = ColumnDataSource(data=dict(x=[1,0,1], y=[0,1,1]))

plot = figure(width=400, height=400)
plot.circle('x', 'y', source=source, line_width=3, line_alpha=0.6)

slider1 = Slider(start=0.1, end=6, value=1, step=.1, title="Escalar 1")
slider2 = Slider(start=0.1, end=6, value=1, step=.1, title="Escalar 2")

update_tamanho = CustomJS(args=dict(source=source, slider1=slider1, slider2=slider2), code="""
const f1 = cb_obj.value
const f2 = cb_obj.value
const x = source.data.x
const y = source.data.y
x[0] = f1
x[2] = f1
y[1] = f2
y[2] = f2
source.data = { x, y }
""")
slider2.js_on_change('value', update_tamanho)
slider1.js_on_change('value', update_tamanho)

show(column(slider1, slider2, plot))

Which is an adaptation of the code in the tutorial.
The plot is shown correctly, with three dots at [1,0], [0,1] and [1,1]. The sliders are also shown correctly. However, whenever I move the sliders, the plot doesn't update. What must I do to fix this and what would be the most relevant things for learning interactive plots with Bokeh?
Thank you.

Answer: Try source.change.emit(); in your js code after changing of your data
Title: ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with LibreSSL 2.8.3
Tags: python,openai-api,urllib3

Question: After pip install openai, when I try to import openai, it shows this error:
the 'ssl' module of urllib3 is compile with LibreSSL not OpenSSL

I just followed a tutorial on a project about using API of OpenAI. But when I get to the first step which is the install and import OpenAI, I got stuck. And I tried to find the solution for this error but I found nothing.
Here is the message after I try to import OpenAI:
Python 3.9.6 (default, Mar 10 2023, 20:16:38) 
[Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import openai
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/yule/Library/Python/3.9/lib/python/site-packages/openai/__init__.py", line 19, in <module>
    from openai.api_resources import (
  File "/Users/mic/Library/Python/3.9/lib/python/site-packages/openai/api_resources/__init__.py", line 1, in <module>
    from openai.api_resources.audio import Audio  # noqa: F401
  File "/Users/mic/Library/Python/3.9/lib/python/site-packages/openai/api_resources/audio.py", line 4, in <module>
    from openai import api_requestor, util
  File "/Users/mic/Library/Python/3.9/lib/python/site-packages/openai/api_requestor.py", line 22, in <module>
    import requests
  File "/Users/mic/Library/Python/3.9/lib/python/site-packages/requests/__init__.py", line 43, in <module>
    import urllib3
  File "/Users/mic/Library/Python/3.9/lib/python/site-packages/urllib3/__init__.py", line 38, in <module>
    raise ImportError(
ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with LibreSSL 2.8.3. See: https://github.com/urllib3/urllib3/issues/2168

I tried to --upgrade the urllib3 but still not working, the result is:
pip3 install --upgrade urllib3
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: urllib3 in ./Library/Python/3.9/lib/python/site-packages (2.0.2)

Answer: We ran into this problem and there were two problems:

The urllib3 version is not compatible.
-> We removed the current version and tried to install urllib3==1.26.15

Then we ran into the second problem we can't install this version. And we found out Mac Mini uses 'zsh' which didn't allow us to completely install this version of urllib3. We changed to 'bash' to install then came back to zsh.
Everything works.
Title: ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with LibreSSL 2.8.3
Tags: python,openai-api,urllib3

Question: After pip install openai, when I try to import openai, it shows this error:
the 'ssl' module of urllib3 is compile with LibreSSL not OpenSSL

I just followed a tutorial on a project about using API of OpenAI. But when I get to the first step which is the install and import OpenAI, I got stuck. And I tried to find the solution for this error but I found nothing.
Here is the message after I try to import OpenAI:
Python 3.9.6 (default, Mar 10 2023, 20:16:38) 
[Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import openai
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/yule/Library/Python/3.9/lib/python/site-packages/openai/__init__.py", line 19, in <module>
    from openai.api_resources import (
  File "/Users/mic/Library/Python/3.9/lib/python/site-packages/openai/api_resources/__init__.py", line 1, in <module>
    from openai.api_resources.audio import Audio  # noqa: F401
  File "/Users/mic/Library/Python/3.9/lib/python/site-packages/openai/api_resources/audio.py", line 4, in <module>
    from openai import api_requestor, util
  File "/Users/mic/Library/Python/3.9/lib/python/site-packages/openai/api_requestor.py", line 22, in <module>
    import requests
  File "/Users/mic/Library/Python/3.9/lib/python/site-packages/requests/__init__.py", line 43, in <module>
    import urllib3
  File "/Users/mic/Library/Python/3.9/lib/python/site-packages/urllib3/__init__.py", line 38, in <module>
    raise ImportError(
ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with LibreSSL 2.8.3. See: https://github.com/urllib3/urllib3/issues/2168

I tried to --upgrade the urllib3 but still not working, the result is:
pip3 install --upgrade urllib3
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: urllib3 in ./Library/Python/3.9/lib/python/site-packages (2.0.2)

Answer: You should upgrade your system's LibreSSL version . use brew upgrade openssl@1.1
Title: ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with LibreSSL 2.8.3
Tags: python,openai-api,urllib3

Question: After pip install openai, when I try to import openai, it shows this error:
the 'ssl' module of urllib3 is compile with LibreSSL not OpenSSL

I just followed a tutorial on a project about using API of OpenAI. But when I get to the first step which is the install and import OpenAI, I got stuck. And I tried to find the solution for this error but I found nothing.
Here is the message after I try to import OpenAI:
Python 3.9.6 (default, Mar 10 2023, 20:16:38) 
[Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import openai
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/yule/Library/Python/3.9/lib/python/site-packages/openai/__init__.py", line 19, in <module>
    from openai.api_resources import (
  File "/Users/mic/Library/Python/3.9/lib/python/site-packages/openai/api_resources/__init__.py", line 1, in <module>
    from openai.api_resources.audio import Audio  # noqa: F401
  File "/Users/mic/Library/Python/3.9/lib/python/site-packages/openai/api_resources/audio.py", line 4, in <module>
    from openai import api_requestor, util
  File "/Users/mic/Library/Python/3.9/lib/python/site-packages/openai/api_requestor.py", line 22, in <module>
    import requests
  File "/Users/mic/Library/Python/3.9/lib/python/site-packages/requests/__init__.py", line 43, in <module>
    import urllib3
  File "/Users/mic/Library/Python/3.9/lib/python/site-packages/urllib3/__init__.py", line 38, in <module>
    raise ImportError(
ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with LibreSSL 2.8.3. See: https://github.com/urllib3/urllib3/issues/2168

I tried to --upgrade the urllib3 but still not working, the result is:
pip3 install --upgrade urllib3
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: urllib3 in ./Library/Python/3.9/lib/python/site-packages (2.0.2)

Answer: I met this problem too. My old version is python 3.9.
"brew install openssl@1.1" doesn't work for me.
You can try:
pipenv install --python 3.11
That fixed my problem.
Title: SOLVED Why is np.empty == np.zeros True, and np.empty == np.zeros False
Tags: python,numpy

Question: New to python and maybe misunderstanding some fundamentals here, but having trouble making sense of the following.
print(numpy.empty(3) == numpy.zeros(3))

#Result
[True True True]

print(numpy.empty(3) == numpy.empty(3))

#Result
[False False False]

My original assumption was that .empty array, when the comparison is called, is initialized as .zeros()? But if that's the case, the latter wouldn't make sense.

Answer: The numpy.empty function creates a new array of the specified shape, but does not initialize it with values. Instead, it leaves the array elements uninitialized, which means they can contain random values that were in memory when the array was created.
And numpy.zeros also creates a new array of the specified shape, but initializes the elements in it as 0.0

So when you compare these two arrays, this is what happens:
[1.23456789e-312, 2.96439388e-323, 2.47032823e-323] == [0.0, 0.0, 0.0]
Title: SOLVED Why is np.empty == np.zeros True, and np.empty == np.zeros False
Tags: python,numpy

Question: New to python and maybe misunderstanding some fundamentals here, but having trouble making sense of the following.
print(numpy.empty(3) == numpy.zeros(3))

#Result
[True True True]

print(numpy.empty(3) == numpy.empty(3))

#Result
[False False False]

My original assumption was that .empty array, when the comparison is called, is initialized as .zeros()? But if that's the case, the latter wouldn't make sense.

Answer: So, it turns out it was just an environment thing in my Jupyter notebook. I asked the question because I was consistently getting True as an answer on the first check, but when I ran the same on local or online compiler, it gave a different answer. All answers above contributed.
Title: AWS lambda throwing import error because of URLLIB
Tags: python,amazon-web-services,urllib

Question: Im running a python script on aws lambda and its throwing the following error.
 {
   "errorMessage": "Unable to import module 'app': urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2k-fips  26 Jan 2017. See: https://github.com/urllib3/urllib3/issues/2168",
   "errorType": "Runtime.ImportModuleError",
   "stackTrace": [] }

It was running perfectly an hour ago , and even after I have made no deployments , it seems to be failing.
my python version is 3.7.
and Im only using urllib to parse and unquote urls .
namely
from urllib.parse import urlparse


and
from urllib.parse import unquote


like its mentioned in the GitHub url I can upgrade my python version, but doing so would break other things.
Are there any alternative librries I can use to get the same result?
from the GitHub link , it shows urllib no longer supports OpenSSL<1.1.1 but somehow some of our higher environments the same scripts is running without issues.

Answer: I hacked at this quite a bit and ended up doing a few things.
Im using python3.8.16

I installed the openssl11


sudo yum install openssl11


found the existing runtime.


whereis openssl
openssl: /usr/bin/openssl /usr/lib64/openssl /usr/include/openssl /usr/share/man/man1/openssl.1ssl.gz
3) renamed   the old openssl


sudo mv openssl openssl102


symlinked the new version


ln -s openssl11 openssl


in the appropriate build directory, i then rebuilt the target python modules under the supported version.


python3 -m pip install --target . urllib3==1.26.15
Title: AWS lambda throwing import error because of URLLIB
Tags: python,amazon-web-services,urllib

Question: Im running a python script on aws lambda and its throwing the following error.
 {
   "errorMessage": "Unable to import module 'app': urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2k-fips  26 Jan 2017. See: https://github.com/urllib3/urllib3/issues/2168",
   "errorType": "Runtime.ImportModuleError",
   "stackTrace": [] }

It was running perfectly an hour ago , and even after I have made no deployments , it seems to be failing.
my python version is 3.7.
and Im only using urllib to parse and unquote urls .
namely
from urllib.parse import urlparse


and
from urllib.parse import unquote


like its mentioned in the GitHub url I can upgrade my python version, but doing so would break other things.
Are there any alternative librries I can use to get the same result?
from the GitHub link , it shows urllib no longer supports OpenSSL<1.1.1 but somehow some of our higher environments the same scripts is running without issues.

Answer: I'm unsure what caused this error, but I could fix it by importing boto3 into my function's requirements.
Title: Is there a way to make SQLAlchemy accept a list as a named parameter with the IN operator?
Tags: python,sqlalchemy,pyodbc

Question: import pandas as pd
import sqlalchemy as sa

query = "SELECT * FROM students WHERE name IN :name"
t = as.text(query)
pd.read_sql(t, con=conn, params={'name': ['Ravi', 'Rami']}

This is what I tried but it results in a syntax error.
Is there a workaround to accept a list as a named parameter with the IN operator?

Answer: There is a typo
t = as.text(query)
Should be
t = sa.text(query)
Title: Pandas groupby time gaps
Tags: python,pandas,date,datetime

Question: I have a dataframe called df_merged which outputs like below:




index
count_date
time_gap
device
count
average_speed




0
2018-12-05
0 days 00:00:00
CAT17
0
0


1
2018-12-05
0 days 00:00:00
CAT17
0
0


2
2018-12-05
0 days 00:00:00
CAT17
0
0


3
2018-12-05
0 days 00:00:00
CAT17
0
0


4
2018-12-05
0 days 01:00:00
CAT17
0
0


...
...
...
...
...
...


154747
2023-05-04
0 days 22:00:00
CAT17
0
0


154748
2023-05-04
0 days 23:00:00
CAT17
4
16


154749
2023-05-04
0 days 23:00:00
CAT17
0
0


154750
2023-05-04
0 days 23:00:00
CAT17
1
13


154751
2023-05-04
0 days 23:00:00
CAT17
3
17




Here is the info on this df:
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 154752 entries, 0 to 154751
Data columns (total 5 columns):

     Column         Non-Null Count   Dtype                  
 0   count_date     154752 non-null  datetime64[ns]
 1   time_gap       0 non-null       category      
 2   device         154752 non-null  object        
 3   count          154752 non-null  int32         
 4   average_speed  154752 non-null  int32         
dtypes: category(1), datetime64[ns](1), int32(2), object(1)
memory usage: 3.7+ MB

time_gap is a category because I replaced it with a pd.cut() function. I don't know if it's best to change the dtype here to groupby.
I would like to groupby time_gap, knowing that the average_speed has to be weighted by count with this function:
def average_speed_mean(x):
    try: 
        return np.average(x["average_speed"], weights=x["count"])
    except ZeroDivisionError:
        return 0

I'm trying to have basically the same dataframe grouped like this:




index
count_date
time_gap
device
count
average_speed




0
2018-12-05
0 days 00:00:00
CAT17
0
0


1
2018-12-05
0 days 01:00:00
CAT17
0
0


...
...
...
...
...
...


38687
2023-05-04
0 days 22:00:00
CAT17
0
0


38688
2023-05-04
0 days 23:00:00
CAT17
8
16




I tried this:
df_merged = df_merged.groupby("time_gap").agg({("count", sum), ("average_speed", average_speed_mean)})

But it doesn't seem to work out and I have no idea how I could solve this.
Thank you in advance for your help.

Answer: Thank you, it's working very well and I find your solution pretty neat to understand it step by step.
Here is the output I get:




index
count_date
time_gap
device
count
average_speed




0
2018-12-05
0 days 00:00:00
CAT17
0
0.000000


1
2018-12-05
0 days 01:00:00
CAT17
0
0.000000


2
2018-12-05
0 days 02:00:00
CAT17
0
0.000000


3
2018-12-05
0 days 03:00:00
CAT17
0
0.000000


4
2018-12-05
0 days 04:00:00
CAT17
0
0.000000


...
...
...
...
...
...


38683
2023-05-04
0 days 19:00:00
CAT17
56
16.339286


38684
2023-05-04
0 days 20:00:00
CAT17
39
20.179487


38685
2023-05-04
0 days 21:00:00
CAT17
14
16.142857


38686
2023-05-04
0 days 22:00:00
CAT17
4
17.500000


38687
2023-05-04
0 days 23:00:00
CAT17
8
16.000000
Title: AttributeError: 'NoneType' object has no attribute 'lower' In a machine learning application in Python
Tags: python,machine-learning,scikit-learn,nonetype

Question: These are my codes that I get an error when I run:
def my_tokenizer(text):
    if text is None:
        return []
    else:
        return text.split()

vectorizer = CountVectorizer(tokenizer=my_tokenizer)
tag_dtm = vectorizer.fit_transform(tag_data['Tags'])

This is my error output:
---> 69     doc = doc.lower()
     70 if accent_function is not None:
     71     doc = accent_function(doc)

AttributeError: 'NoneType' object has no attribute 'lower'

I looked at the sample solutions and got my code from this:
vectorizer = CountVectorizer(tokenizer = lambda x: x.split())
tag_dtm = vectorizer.fit_transform(tag_data['Tags'])

I converted it to the above. But I still get the same error and I don't know what to fix:

Answer: By default CountVectorizer will try to convert all inputs to lowercase. Since you have None in your input, lower() cannot be applied.
To fix this particular problem you can provide lowercase = False argument when initializing the CountVectorizer. However, a safer approach would be to remove all occurrences of None from your input before passing to the vectorizer.
Title: Difference between two zlib data ( same value )
Tags: python,python-3.x,zlib,deflate

Question: I am working with zlib data from 3 days and I cannot get out of this problem:
The original zlib compressed data hex is as follows: 789c34c9410e82301005d0bbfc756b5a832ee62a94900146255620ed80314defae1b772f79050a5af6180df21feb06f2ce6002153c84930ec2dacf8b4a3a38821a7fbefcbed7c4a3805ab401775679f3c76e69b27bb6c259bd1d6cf3bc5d034c0978cd635a7300b993ab1dba5abf000000ffff
And the hex that I generate using the zlib python library is the following: 789c35c9410e82301005d0bbfc756b5a832ee62a94900146255620ed80314defae1b772f79050a5af6180df21feb06f2ce6002153c84930ec2dacf8b4a3a38821a7fbefcbed7c4a3805ab401775679f3c76e69b27bb6c259bd1d6cf3bc5d034c0978cd635a7300b993ab1dba5abfb1bd28150000ffff
Can anyone explain to me the difference between the two values?
import zlib, json
ZLIB_SUFFIX = b'\x00\x00\xff\xff'
data = json.dumps({
        "t": None,
        "s": None,
        "op": 10,
        "d": {
            "heartbeat_interval": 41250,
            "_trace": [
                '["gateway-prd-us-east1-b-4kf6",{"micros":0.0}]'
            ]
        }
    }, separators=(',', ':')).encode('utf-8')
deflate = zlib.compressobj(6, zlib.DEFLATED, zlib.MAX_WBITS)
result = deflate.compress(data) + deflate.flush() + ZLIB_SUFFIX
print(result)

Answer: The original stream is not terminated, and hence invalid, and ends with an empty stored block. The one you generated is terminated and valid, but is followed by an extraneous 00 00 ff ff. Both decompress to the same data, though the original is not validated with a check value. The one you generated is.
Your ZLIB_SUFFIX is not any such thing. What it is is a zero length and the complement of that length that would follow stored block header bits in a deflate stream. However it has no such meaning if it does not follow stored block header bits in a deflate stream, which is the case in the one you generated.
Title: Probema to delete from sqlite with tkinter
Tags: python,sqlite,tkinter

Question: import tkinter as tk
import sqlite3 as sql

class main_window:
    def __init__(self, window):
        self.wind = window
        self.wind.geometry('500x500')
        self.wind.resizable(0, 0)
        self.draw()
        self.refresh()

    def draw(self):
        tk.Label(text= 'Usuario').pack()
        self.user = tk.Entry()
        self.user.pack()


        tk.Label(text= 'Password').pack()
        self.password = tk.Entry()
        self.password.pack()

        self.frame = tk.Frame(self.wind)
        self.frame.pack()

        tk.Button(self.frame, text= 'Save', width= 5, height= 1, command= self.save).pack(side= 'left')
        tk.Button(self.frame, text= 'Delete', width= 5, height= 1, command= self.delete).pack(side= 'top')

        self.user_list = tk.Listbox(width= 40, height= 100)
        self.user_list.pack(padx= 4, pady= 8, side= 'left')

        self.password_list = tk.Listbox(width= 40, height= 100)
        self.password_list.pack(padx= 4, pady= 8, side= 'left')

    def refresh(self):
        self.user.delete(0, tk.END)
        self.password.delete(0, tk.END)

        self.user_list.delete(0, tk.END)
        self.password_list.delete(0, tk.END)
        
        db = sql.connect('accounts.db')
        curs = db.cursor()

        db_accounts_users = curs.execute("SELECT user FROM tb_accounts")
        for user in db_accounts_users:
            for uss in user:
                self.user_list.insert(tk.END, uss)

        db_accounts_pass = curs.execute("SELECT pass FROM tb_accounts")
        for password in db_accounts_pass:
            for fpass in password:
                self.password_list.insert(tk.END, fpass)

        db.commit()
        db.close()

    def save(self):
        db = sql.connect('accounts.db')
        curs = db.cursor()
        curs.execute("INSERT INTO tb_accounts(user, pass) VALUES (?, ?)", (self.user.get(), self.password.get()))
        db.commit()
        db.close()
        self.refresh()

    def delete(self):
        for user in self.user_list.curselection():
            select_user = self.user_list.get(user)
        db = sql.connect('accounts.db')
        curs = db.cursor()
        curs.execute("DELETE FROM tb_accounts WHERE user= ?", (select_user))
        db.commit()
        db.close()
        self.refresh()


obj_main_window = main_window(tk.Tk())
obj_main_window.wind.mainloop()

The idea is to delete a data with a button. But when I press the button I get the next error.
sqlite3.ProgrammingError: Incorrect number of bindings supplied. The current statement uses 1, and there are 4 supplied.
the number of supplied in the error its depend on how many characters have the user.
Thanks for any help with this.
This is just a personal practice.

Answer: I used list instead of tuple for query parameters:
curs.execute("DELETE FROM tb_accounts WHERE user= ?", [select_user])
Title: Cannot import model from trransformers/simpletransformers:
Tags: python,nlp,artificial-intelligence

Question: The instruction that causes the error is:
'from simpletransformers.conv_ai import ConvAIModel'
The error thrown mentions each time 'cached_path' from 'transformers' and points to transformers' init.py.
This is from within a virtual environment, with all dependencies installed and up to date. Python3.10, Linux.
All 6 other models except from the one above and NERModel are imported without error.
Any help appreciated.
Thanks.

Answer: As a workaround, I edited conv _ai_uitls.py to comment out the import of 'cached_path' and hard-coded the source. I don't like to do that sort of things but I didn't know what else... There is also the issue of collections, which does not import Iterable either. Edited again replacing collections with collections.abc. Sloppy but at least it works. Posted my own answer to help the other 3 guys who might be interested.
Title: Python import function/class using full path or base path
Tags: python,import

Question: What is recommended when importing something from a subpackage?
Option A:
from sklearn.model_selection import train_test_split
from sklearn import preprocessing

train_test_split()
preprocessing()

Option B:
import sklearn

sklearn.model_selection.train_test_split()
sklearn.preprocessing()

In my opinion, using option A you may not know where the function comes from when you see it many lines after it is imported. In option B, you always know where it comes from because it is more verbose. However, you always need to write the full path function. Is that a disadvantage?
What are your recommendations?

Answer: We generally used option A. It is also good practice.
Title: ValueError: `run` not supported when there is not exactly one input key, got ['question', 'documents']
Tags: python,langchain

Question: Getting the error while trying to run a langchain code.
ValueError: `run` not supported when there is not exactly one input key, got ['question', 'documents'].
Traceback:
File "c:\users\aviparna.biswas\appdata\local\programs\python\python37\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 565, in _run_script
    exec(code, module.__dict__)
File "D:\Python Projects\POC\Radium\Ana\app.py", line 49, in <module>
    answer = question_chain.run(formatted_prompt)
File "c:\users\aviparna.biswas\appdata\local\programs\python\python37\lib\site-packages\langchain\chains\base.py", line 106, in run
    f"`run` not supported when there is not exactly one input key, got ['question', 'documents']."

My code is as follows.
import os
from apikey import apikey

import streamlit as st
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain, SequentialChain
#from langchain.memory import ConversationBufferMemory
from docx import Document

os.environ['OPENAI_API_KEY'] = apikey

# App framework
st.title('🦜🔗 Colab Ana Answering Bot..')
prompt = st.text_input('Plug in your question here')


# Upload multiple documents
uploaded_files = st.file_uploader("Choose your documents (docx files)", accept_multiple_files=True, type=['docx'])
document_text = ""

# Read and combine Word documents
def read_docx(file):
    doc = Document(file)
    full_text = []
    for paragraph in doc.paragraphs:
        full_text.append(paragraph.text)
    return '\n'.join(full_text)

for file in uploaded_files:
    document_text += read_docx(file) + "\n\n"

with st.expander('Contextual Prompt'):
    st.write(document_text)

# Prompt template
question_template = PromptTemplate(
    input_variables=['question', 'documents'],
    template='Given the following documents: {documents}. Answer the question: {question}'
)

# Llms
llm = OpenAI(temperature=0.9)
question_chain = LLMChain(llm=llm, prompt=question_template, verbose=True, output_key='answer')

# Show answer if there's a prompt and documents are uploaded
if prompt and document_text:
    formatted_prompt = question_template.format(question=prompt, documents=document_text)
    answer = question_chain.run(formatted_prompt)
    st.write(answer['answer'])

I have gone through the documentations and even then I am getting the same error. I have already seen demos where multiple prompts are being taken by langchain.

Answer: I got the same error while on python 3.7.1 but when I upgraded my python to 3.10 and langchain to latest version I could get rid of that error. I noticed this since on colab it was running fine but locally it wasn't.
Title: How to check if a release exists on a GitLab repo
Tags: python,git,gitlab,gitlab-api,http.client

Question: I have a repo in GitLab and now I need to check if a release based on a tag exists using Python.
I have tried below Python code but I always get the response "301 Moved Permanently" independently if release exsits or not.
I am using Http Client instead of requests as I do not want to depend on any third-party and also because Http Client is faster.
conn = http.client.HTTPConnection("my.git.space")

headers = { 'PRIVATE-TOKEN': "XXX" }

conn.request("GET", "/api/v4/projects/497/releases/1.0.0.0", headers=headers)
res = conn.getresponse()

print(f"{res.status} {res.reason}")

conn.close()

If I use postman it works, if exists it returns 200, otherwise, 404.
Any ideas on how to do this?

Answer: Finally I got it by replacing:

http.client.HTTPConnection

to

http.client.HTTPSConnection
Title: ModuleNotFoundError: No module named 'numexpr.interpreter'
Tags: python,python-3.x,spyder,numexpr

Question: there's something wrong with my installation of numexpr. When I run the following code
import numexpr as ne

I get the following error message:

from numexpr.interpreter import MAX_THREADS, use_vml, BLOCK_SIZE1
ModuleNotFoundError: No module named 'numexpr.interpreter'

Anybody has any idea what could be the cause?
I've installed and uninstalled numexpr multiple times already.
Some more info:
Python v 3.10.11
numexpr 2.8.4
Full error log:

Traceback (most recent call last):
File
"C:\Users\bemajco\Miniconda3\envs\spyder-env\lib\site-packages\spyder_kernels\py3compat.py",
line 356, in compat_exec
exec(code, globals, locals)
File
"c:\users\bemajco\myprojects\hyphen\learning\hyphen_learning.py", line
11, in 
import numexpr as ne
File
"C:\Users\bemajco\Miniconda3\Lib\site-packages\numexpr_init_.py",
line 24, in 
from numexpr.interpreter import MAX_THREADS, use_vml, BLOCK_SIZE1
ModuleNotFoundError: No module named 'numexpr.interpreter'

Answer: Issue was on the version of numexpr 2.8.4. With 2.8.3 the error was gone...
Title: Thrift Tserializer and TJSONProtocol
Tags: python,java,c++,erlang,thrift

Question: I have a question about the Tserializer and the TJSONProtocol offered by Thrift.
I use Thrift to implement a RPC between server and client written in different programming languages. I need to add new functionality in my system implementing an integrity check on the data exchanged between client and server.
The idea is to convert in string the data exchanged between sender and receiver (defined in the IDL thrift) and use this string as an input to the algorithm for the integrity calculation.
For structured data types, I want to leverage on the Tserializer based on TJSONProtocol to obtain a JSON string (representing the data to protect) to provide as input to the algorithm for integrity calculation.
Is it correct to assume that the JSON string resulting from conversion is always the same (assuming to have the same input data) across different programming languages?
I mean, can I assume that the behaviour of TSerializer (based on TJSONProtocol) is the same across the different implementations of Thrift libraries available for the different programming languages?

Answer: TL,DR: No.
There are several factors that are not guaranteed. Just as one example, the order of list<> is preserved and guarateed, but this is not the case for set<> or map<> containers. Hence, even though the data before serialization and after deserialization are the same, the serialized data may differ just because of implementation differences.
Title: Pyrebase4 error cannot import name 'gaecontrib'
Tags: python,pyrebase,python-requests-toolbelt

Question: I have been trying to install pyrebase4 using pip install pyrebase4 but when ran it throws below error
"C:\Users\ChodDungaTujheMadhadchod\anaconda3\envs\sam_upgraded\lib\site-packages\requests_toolbelt\adapters\appengine.py", line 42, in <module> from .._compat import gaecontrib ImportError: cannot import name 'gaecontrib' from 'requests_toolbelt._compat'
As I see the error direct directly to requests_toolbelt , but I cannot figure out the possible way to fix it, I tried upgrading to latest version as which is requests-toolbelt==1.0.0 . So is there any way to fix it.

Answer: Okay so what I found that the latest requests-toolbelt 1.0.1 is currently throwing this issue. So downgrading it to next previous version requests-toolbelt==0.10.1 fixes the issue.
Title: Unable to query Twitter API V2
Tags: python,twitter,tweepy,tweets

Question: I'm trying to query the Twitter API v2 with free research access via tweepy in Python and it throws me this error: "When authenticating requests to the Twitter API v2 endpoints, you must use keys and tokens from a Twitter developer App that is attached to a Project. You can create a project via the developer portal."
bearer_token = 'YOUR BEARER TOKEN'

# Authenticate to Twitter API
client = tweepy.Client(bearer_token=bearer_token)

# Read the tweet ids from the file
df = pd.read_csv('tweet_training_set.tsv', sep='\t')
tweet_ids = df['tweet_id'].tolist()

# Rehydrate the tweets one batch at a time
tweets = []
batch_size = 50
for i in range(0, len(tweet_ids), batch_size):
    batch = tweet_ids[i:i+batch_size]
    tweets.extend(client.get_tweets(ids=batch))

I am rehydrating the tweets 1 batch at a time otherwise it throws me the error "HTTP 431 - header too large"
My app is attached to a project

Answer: What tier are you on? Free, Basic, or Enterprise?
Are you also approved for Academic Research?
Title: Geopandas gets creative when it can't find the street
Tags: python,geopandas,geopy

Question: I have noticed an interesting behavior when when working with Geopandas.  When Geopandas is fed a street that it presumably cannot find, it will somehow substitute an existing street that seems to be near the city of the address.  See below input/output:
> getGeoLoc(["2502 asdfasdf St, Albany NY"])
  0    POINT (-73.78173 42.66155)

The above jibberish returned coordinate is 500 Hamilton St Apartment 1, Albany, NY
What's even more bizarre is that varying the street number results in additional locations around the returned street.  This doesn't apparently work if you botch the number, city or state, which returns a null value.
This makes things a little tricky when I'm bulk converting addresses because I can't really tell if it is actually finding the street or if I've fed it a bad piece of data.
Can anybody explain this or tell me how to get an error for a bad street name?

Answer: The behavior in geopandas is due to the geocoding service. Those geocoding services (e.g., Google Maps API, OpenStreetMap Nominatim) convert addresses into geographic coordinates.
When you an address that cannot be found, the geocoding service attempts to perform approximate matching or infer a nearby location based on the available information. This can sometimes result in unexpected substitutions, where the geocoding service assigns coordinates to a similar-sounding or nearby street. As you point out, it gets creative.
You can however check the result of the geocoding operation and examine the quality or confidence of the returned coordinates. Most geocoding services provide a geocoding quality code or similar indicator that reflects the accuracy or reliability of the geocoded result.
My advice to you is to clean your data before you pass it to the geocoding service via geopandas. I recently had my tam spend two complete weeks doing this to be able to return accurate positions of thousands of stores....painful but necessary.
Title: Python packages imported in editable mode can't be resolved by pylance in VSCode
Tags: python,visual-studio-code

Question: I have several Python packages developed locally, which I often use in VSCode projects. For these projects, I create a new virtualenv and install these packages with pip install -e ../path/to/package. This succeeds, and I can use these packages in the project. However, VSCode underlines the package's import line in yellow, with this error:

Import "mypackage" could not be resolved Pylance(reportMissingImports)

Again, mypackage works fine in the project, but VSCode reports that error, and I lose all autocomplete and type hint features when calling mypackage in the project.
I ensured that the right Python interpreter is selected (the one from the project's virtualenv), but the error persists. The error and Pylance docs do not offer any other possible solutions.
VSCode version: 1.78.0

Answer: Usually, when you choose the correct interpreter, Pylance should take effect immediately. You can try adding "python.analysis.extraPaths": ["path/to/your/package"] to your settings.json.
You can also try clicking on the install prompt in vscode to see which environment it will install the package in. I still believe that the problem was caused by incorrect selection of the interpreter.