eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I'm have troubles with SQLalchemy in Flask app first minute after the app start (or restart). It looks like logger exceptions of a sort:  sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (2006, 'MySQL server has gone away') <---- most often one   sqlalchemy.exc.ResourceClosedError: This result object does not return rows. It has been closed automatically.   sqlalchemy.exc.NoSuchColumnError: "Could not locate column in row for column 'users.id'"   sqlalchemy.exc.NoSuchColumnError: "Could not locate column in row for column 'payments.id'"   sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (2013, 'Lost connection to MySQL server during query')  Then everything gets back to normal. It's not a critical issue but annoying. I tried to run warming up queries upon application start:  with app.app_context(): for _ in range(5): try: db.session.execute(statement) logger.info("DB connection is successfully established") return except Exception as e: logger.warning(e) time.sleep(1) raise Exception("Couldn't establish a DB connection")  It passes through just fine but then I see same issues. It doesn't happen in development environment, only in production where Flask app runs on uwsgi server. Is there a way to fix it? Update: connection URI looks like this: "mysql+mysqldb://user:password@localhost/mydb?unix_socket=/var/run/mysqld/mysqld.sock"

Answer: Seems How to correctly setup Flask + uWSGI + SQLAlchemy to avoid database connection issues worked for them. This is a copy paste of the answer just to avoid "link only answers". I would still recommend anyone arriving at this question to refer the original answer linked above.  The SQLAlchemy manual provides two examples how to approach this: Using Connection Pools with Multiprocessing. The first approach involving Engine.dispose() can be approached using uwsgidecorators.postfork as suggested by Hett - simple example that should work if there's only the default binding involved:   db = SQLAlchemy() def create_app(): app = Flask(__name__) app.config["SQLALCHEMY_DATABASE_URI"] = "postgres:///test" db.init_app(app) def _dispose_db_pool(): with app.app_context(): db.engine.dispose() try: from uwsgidecorators import postfork postfork(_dispose_db_pool) except ImportError: # Implement fallback when running outside of uwsgi... raise return app  Meta: I am posting this as an answer as per OP's comment
Question: I tried to import pytorch in Jupyter notebook: import torch When I run it I get this error message: NameError Traceback (most recent call last) <ipython-input-7-db4b44599dae> in <module> ----> 1 import torch 2 import numpy as np 3 from torch import nn 4 from tqdm.auto import tqdm 5 from torchvision import transforms /opt/anaconda3/lib/python3.8/site-packages/torch/__init__.py in <module> 463 raise # If __file__ is not None the cause is unknown, so just re-raise. 464 --> 465 for name in dir(_C): 466 if name[0] != '_' and not name.endswith('Base'): 467 __all__.append(name) NameError: name '_C' is not defined  I have tried potential solutions but nothing worked. I tried deleting and reinstalling Pytorch and Cython, but I kept getting the same error.

Answer: I had this same error, I found the issue was an outdated package. I had to manually specify the latest typing_extensions: pip3 install typing_extensions==4.9.0
Question: I have the following class: class YesOrNn(enum.Enum): YES = "Y" NO = "N"  I am getting my inputs such as YesOrNo("true") or YesOrNo("false"), To make this work, I think I need to change the class to: class YesOrNn(enum.Enum): YES = "true" NO = "false"  But, I also have a case where whenever a variable's value is saved as YesOrNo.YES.value, it should answer "Y". I can't seem to figure out how to pull that off

Answer: You want the _missing_ method: import enum class YesOrNo(enum.Enum): YES = "Y" NO = "N" @classmethod def _missing_(cls, value): if value.lower() in ('y', 'yes', 'true', 't'): return cls.YES elif value.lower() in ('n', 'no', 'false', 'f'): return cls.NO   Disclosure: I am the author of the Python stdlib Enum, the enum34 backport, and the Advanced Enumeration (aenum) library.
Question: I tried to check if a given path exists on Windows/Linux (using Path.exists()) A FileNotFoundError is raised on Linux only, obviously when looking at the POSIX path object it is not converted to POSIX. I would like to know why pathlib does not convert the Windows-style path to a POSIX path (currently running on Windows): from pathlib import PosixPath, WindowsPath, Path, PurePosixPath, PureWindowsPath raw_string = r'.\mydir\myfile' print(Path(raw_string)) print(PurePosixPath(raw_string))  Output: .\mydir\myfile .\mydir\myfile  Both Windows and Linux pathlib path imports of the raw_string show the same Windows path as the output, and such a Windows path is of course not usable on Linux (second output). Shouldn't pathlib take care of these conversions for Path objects to be platform-agnostic? So that the last output should look like: ./mydir/myfile 

Answer: Found a solution: Path(PureWindowsPath(raw_string)) works on either platform. It’s useful if you’re developing on Windows and want to simply copy paste long Windows formatted file paths. raw_string = r'.\mydir\myfile' print(Path(PureWindowsPath(raw_string)))  Output: mydir/myfile  And you need the PureWindowsPath. If on a Linux system, you just code print(Path(WindowsPath(raw_string))) instead, you will get the error: NotImplementedError: cannot instantiate 'WindowsPath' on your system
Question: can you help me with the correct syntax to connect to a DBus signal? This is one of my many tries which at least runs and matches the signature in the docs: from PySide6 import QtDBus from PySide6.QtCore import Q from PySide6.QtWidgets import QMainWindow class MainWindow(QMainWindow): __slots__ = ["__mainwidget"] __mainwidget:QWidget def __init__ (self, *args, **kwargs): super().__init__(*args, **kwargs) service = 'org.freedesktop.DBus' path = '/org/freedesktop/DBus' iface = 'org.freedesktop.DBus' conn = QtDBus.QDBusConnection.systemBus() conn.connect(service, path, iface, "NameOwnerChanged", self, "nochangeslot") #smp = QtDBus.QDBusInterface(service, path, iface, connection=QtDBus.QDBusConnection.systemBus() def nochangeslot(self, arg:object) -> None: print(arg) pass  But it doesn't work and looks weird with the slot as string... On the output I see: qt.dbus.integration: Could not connect "org.freedesktop.DBus" to ochangeslot  Please consider this is a PySide6 issue and not a PyQt5 issue, the signature of the call is slightly different and my code doesn't hang like in similar topics on stackoverflow. Thank you in advance for any help!

Answer: This es the full solution to my initial post, I got this running with the QT/PySide support and they also acknowledged the hangig bug and a Python crash: import sys from PySide6.QtWidgets import QMainWindow from PySide6 import QtDBus, QtCore from PySide6.QtCore import QLibraryInfo, qVersion, Slot from PySide6.QtWidgets import QApplication, QMainWindow class MainWindow(QMainWindow): def __init__ (self, *args, **kwargs): super().__init__(*args, **kwargs) service = "org.freedesktop.DBus" path = "/org/freedesktop/DBus" iface = "org.freedesktop.DBus" conn = QtDBus.QDBusConnection.systemBus() #without this, the conn.connect call hangs, seems to be a bug, is already reported and fixed. conn.registerObject('/', self) conn.connect(service, path, iface, "NameOwnerChanged", self, QtCore.SLOT("nameownerchanged(QString, QString, QString)")) pass @Slot(str, str, str) def nameownerchanged(self, arg1:str, arg2:str, arg3:str) -> None: print(arg1) print(arg2) print(arg3) pass if __name__ == '__main__': print('Python {}.{}.{} {}'.format(sys.version_info[0], sys.version_info[1], sys.version_info[2], sys.platform)) print(QLibraryInfo.build()) app = QApplication(sys.argv) window = MainWindow() window.setWindowTitle(qVersion()) window.resize(800, 480) window.show() sys.exit(app.exec())
Question: I am playing an mp3 file with Python on a Raspberry Pi: output_file = "sound.mp3" pygame.mixer.init() pygame.mixer.music.load(output_file) pygame.mixer.music.play() while pygame.mixer.music.get_busy(): pass sleep(0.2)  Now while the mp3 file is playing, I would like to get the current amplitude of the sound played. Is there a way to do this? The only examples I was able to find work on a microphone input but not with a file being played.

Answer: There's a VU meter code on github working with microphone. You can easily change its input stream to wave file. If you want to work with .mp3 files, check this SO link for converting .mp3 to .wav on air without storing data on storage. import wave def main(): audio = pyaudio.PyAudio() try: # from wav file chunk = 1024 wf = wave.open("kimi_no_shiranai.wav", 'rb') stream = audio.open(format = audio.get_format_from_width(wf.getsampwidth()), channels = wf.getnchannels(), rate = wf.getframerate(), output = True) data = wf.readframes(chunk) maximal = Amplitude() while data: # writing to the stream is what *actually* plays the sound. stream.write(data) data = wf.readframes(chunk) amp = Amplitude.from_data(data) if amp > maximal: maximal = amp amp.display(scale=100, mark=maximal) # from microphone # stream = audio.open(format=pyaudio.paInt16, # channels=2, # rate=RATE, # input=True, # frames_per_buffer=INPUT_FRAMES_PER_BLOCK # ) # maximal = Amplitude() # while True: # data = stream.read(INPUT_FRAMES_PER_BLOCK) # amp = Amplitude.from_data(data) # if amp > maximal: # maximal = amp # amp.display(scale=100, mark=maximal) finally: stream.stop_stream() stream.close() audio.terminate()  Edit: for converting .mp3 to .wav use this code. It doesn't have syncing problem. It stores converted wave into BytesIO buffer. from pydub import AudioSegment sound = AudioSegment.from_mp3("kimi_no_shiranai.mp3") wav_form = sound.export(format="wav") wf = wave.open(wav_form, 'rb') stream = audio.open(format = audio.get_format_from_width(sound.sample_width), channels = sound.channels, rate = sound.frame_rate, output = True)  SO link 1 SO link 2
Question: I need to create a docker image that features two versions of Python, lets say, 3.9 & 3.10. The idea is to install both versions of Python and then install the pip/wheel/lib directories from Python 3.9 as it is the default version we want to use. I then created sys links to the pip/wheel/lib directories. Then using a .sh script, we will pass the Python version via a build argument, and using that ARG, we will select the alternative version of Python, this case being 10, which will make the 3.10 version the version used by the build. But i'm not sure what the best way to do this, the dockerfile. My Dockerfile looks like this: COPY --from=python1 --chown=developer:developer /usr/local/bin/py* /usr/local/bin/ COPY --from=python1 --chown=developer:developer /usr/local/bin/pip* /usr/local/bin/ COPY --from=python1 --chown=developer:developer /usr/local/bin/wheel /usr/local/bin/ COPY --from=python1 --chown=developer:developer /usr/local/lib/lib* /usr/local/lib/ RUN for link in /usr/local/bin/py* /usr/local/bin/pip* /usr/local/bin/wheel /usr/local/lib/lib*; do \ [ -e "$link" ] || ln -s "$link" /usr/local/bin/; \ done # Copy Base Python Version COPY --from=python1 --chown=developer:developer /usr/local/lib/python3.9/ /usr/local/lib/python3.9/ # Copy Alternative Python Version COPY --from=python2 --chown=developer:developer /usr/local/lib/python3.10/ /usr/local/lib/python3.10/ # Copy the version-switching script COPY sagacity-cd/swap.sh /usr/local/bin/ # Run the script to set the default version based on the build argument USER root RUN chmod +x /usr/local/bin/swap.sh && \ /usr/local/bin/swap.sh ${DEFAULT_PYTHON_VERSION} USER developer # Copy the version-switching script COPY sagacity-cd/swap.sh /usr/local/bin/ # Run the script to set the default version based on the build argument USER root RUN chmod +x /usr/local/bin/swap.sh && \ /usr/local/bin/swap.sh ${DEFAULT_PYTHON_VERSION} USER developer  My swap.sh looks like this: #!/bin/bash if [ "$1" == "3.9" ]; then rm /usr/local/bin/python ln -s /usr/local/bin/python3.9 /usr/local/bin/python elif [ "$1" == "3.10" ]; then rm /usr/local/bin/python ln -s /usr/local/bin/python3.10 /usr/local/bin/python else echo "Invalid version specified. Usage: $0 [3.9|3.10]" exit 1 fi 

Answer:  generic image that has both and we want to just swap the version on our CI/CD build when we build the Lambda, via Terraform passing in build args  If you're already going to build the Lambda via Terraform, why don't you just build the Lambda with the correct version of Python? Say the Dockerfile for the Lambda is like this: ARG PY_VERSION FROM python:${PY_VERSION}-bookworm # then all your lambda stuff COPY . /app ENTRYPOINT /app/run.sh  Now you can build your lambda for different versions: docker build --build-arg="PY_VERSION=3.10" . docker build --build-arg="PY_VERSION=3.9" . docker build --build-arg="PY_VERSION=3.8" .  This is mentioned here in the docs. If you really want a base image for the different versions, you can use the same thing, and pre-build the different versions into your own container repo, and then use the same mechanism to build the next layer (but that seems like overkill to me).
Question: I can reach my couchbase cluster with a simple cURL: curl -u $CB_USERNAME:$CB_PASSWORD http://$CB_HOST/pools/default/buckets  but I do not manage to connect using the Python SDK: from datetime import timedelta from couchbase.auth import PasswordAuthenticator from couchbase.cluster import Cluster from couchbase.options import ClusterOptions import os # Configuration CB_HOST = os.environ.get('CB_HOST') CB_BUCKET = os.environ.get('CB_BUCKET') CB_USERNAME = os.environ.get('CB_USERNAME') CB_PASSWORD = os.environ.get('CB_PASSWORD') # Initialize Couchbase connection auth = PasswordAuthenticator(CB_USERNAME, CB_PASSWORD) options = ClusterOptions(auth) cluster = Cluster(f'couchbase://{CB_HOST}', options)  This gives this error: Traceback (most recent call last): File "/Users/luc/code/couchbase/examples/main.py", line 27, in <module> cluster = Cluster(f'couchbase://{CB_HOST}', options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/luc/code/couchbase/examples/venv/lib/python3.11/site-packages/couchbase/cluster.py", line 99, in __init__ self._connect() File "/Users/luc/code/couchbase/examples/venv/lib/python3.11/site-packages/couchbase/logic/wrappers.py", line 98, in wrapped_fn raise e File "/Users/luc/code/couchbase/examples/venv/lib/python3.11/site-packages/couchbase/logic/wrappers.py", line 82, in wrapped_fn ret = fn(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/luc/code/couchbase/examples/venv/lib/python3.11/site-packages/couchbase/cluster.py", line 105, in _connect raise ErrorMapper.build_exception(ret) couchbase.exceptions.UnAmbiguousTimeoutException: UnAmbiguousTimeoutException(<ec=14, category=couchbase.common, message=unambiguous_timeout (14), C Source=/Users/couchbase/jenkins/workspace/python/sdk/python-packaging-pipeline/py-client/src/connection.cxx:199>)  Any hints what I'm doing wrong ?

Answer: Just to eliminate possible network/connectivity issues, I suggest running SDK Doctor. From the docs:  SDK doctor is a tool to diagnose application-server-side connectivity issues with your Couchbase Cluster. It makes the same connections to the Couchbase Server cluster that Couchbase SDKs make during bootstrapping, and then reports on the state of the connections made — giving diagnostic information that can help to solve puzzling network issues.  Hopefully that will help you identify any issues.
Question: with this simple code the output is as expected but only in a 30% of the browser window. import networkx as nx from pyvis.network import Network G = nx.Graph() G.add_node(1) G.add_node(2) G.add_node(3) G.add_edge(1, 2) G.add_edge(1, 3) net = Network(height="100%", width="100%", directed=True, notebook=True) net.from_nx(G) net.show("simple.html")  is there a better parameter to use realy 100% of the browser ? or is there a better library to actully achiev something like that with python ? 

Answer: The reason why you cannot change the canvas size when you set it on 100% is that it is not 100% of browser window, it's 100% of its parent's height. If you change width and background color, you see the canvas is inside another frame. net = Network(height="500px", width="500px", directed=True, notebook=True, bgcolor="#ff0000")   You just need to give its size in pixel, and it will change its parent's size too. net = Network(height="600px", width="100%", directed=True, notebook=True)
Question: I loaded pdf files from a directory and I need to split them to smaller chunks to make a summary. The problem is that I can't iterate on documents object in a for loop and I get an error like this: AttributeError: 'tuple' object has no attribute 'page_content' How can I iterate on my document items to call the summary function for each of them? Here is my code: # Load the documents from langchain.document_loaders import DirectoryLoader document_directory = "pdf_files" loader = DirectoryLoader(document_directory) documents = loader.load() text_splitter = RecursiveCharacterTextSplitter(chunk_size=4000, chunk_overlap=50) # Iterate on long pdf documents to make chunks (2 pdf files here) for doc in documents: # it fails on this line texts = text_splitter.split_documents(doc) chain = load_summarize_chain(llm, chain_type="map_reduce", map_prompt=prompt, combine_prompt=prompt)  

Answer: And if you want to use a for loop, you can try: for file in os.listdir(pdf_folder_path): if file.endswith('.pdf'): pdf_path = os.path.join(pdf_folder_path, file) loader = PyPDFLoader(pdf_path) documents.extend(loader.load()) text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200) chunked_documents = text_splitter.split_documents(documents)
Question: I am using Python 3.11.5 with the below code: import asyncio from collections.abc import AsyncIterable # Leave this iterable be, the question is about # how to use many instances of this in parallel async def iterable() -> AsyncIterable[int]: yield 1 yield 2 yield 3 # How can one get multiple async iterables to work with asyncio.gather? # In other words, since asyncio.gather works with asyncio.Task, # (1) How can one convert an async iterable to a Task? # (2) How can one use asyncio.gather to run many of these tasks in parallel, # keeping the results 1-1 with the source iterables? results_1, results_2, results_3 = asyncio.gather(iterable(), iterable(), iterable())  To restate the question, how can one get:  An AsyncIterable as an asyncio task, where the task iterates until exhaustion Run multiple of these tasks in parallel, storing the results on a per-task basis  (e.g. for use with asyncio.gather)? I am looking for a 1 - 3 line snippet showing how to connect these dots.

Answer: According your code snippet, you're trying to pass an async generator function (iterable) directly to asyncio.gather, however, it expects awaitables (coroutines, Tasks, or Futures). So, to fix the issue, one possible solution is, to create a new coroutine that consumes the async iterable()s and collects their items in the lists. The code will be like this: async def collect(async_iterable): return [item async for item in async_iterable] results = asyncio.run(asyncio.gather( asyncio.create_task(collect(iterable())), asyncio.create_task(collect(iterable())), asyncio.create_task(collect(iterable())) ))  In this way, you would have three [1,2,3] lists, but still, there is a problem — you haven't gone through a concurrency manner among the iterable()s. I mean by adding a print within the collect() you see they are running sequentially. Therefore, to have an asynchronous behaviour between your iterables you need at least an await statement within the generator to make it as an awaitable coroutine used for switching. Here we can use await asyncio.sleep(0) as a trick! Here's the whole code: import asyncio from collections.abc import AsyncIterable async def iterable() -> AsyncIterable[int]: yield 1 await asyncio.sleep(0) yield 2 await asyncio.sleep(0) yield 3 async def collect(async_iterable: AsyncIterable[int]) -> list: result = [] async for item in async_iterable: print(item) result.append(item) return result async def main(): tasks = [ asyncio.create_task(collect(iterable())), asyncio.create_task(collect(iterable())), asyncio.create_task(collect(iterable())) ] results = await asyncio.gather(*tasks) return results results_1, results_2, results_3 = asyncio.run(main()) print(results_1, results_2, results_3)  Out: 1 1 1 2 2 2 3 3 3 [1, 2, 3] [1, 2, 3] [1, 2, 3]
Question: I have reponse object that contain excel file, which i am decoding like this xl = pd.ExcelFile(io.BytesIO(response.content))  Now i want to export this into a .xlsx file. This object had pandas.io.excel._base.Excelfile type. Is there any way to export this. I have tried .to_excel but that didn't worked.

Answer: ExcelFile's purpose is to read and parse excel files into pandas objects. If your object is already an excel file, why pass it to ExcelFile at all? Directly save the response.content to a file: with open('outfile.xlsx', 'wb') as f: f.write(response.content)  If you want to save the individual sheets, you can indeed use pandas: xl = pd.ExcelFile(io.BytesIO(response.content)) for name in xl.sheet_names: xl.parse(name).to_csv(f'{name}.xlsx')
Question: I don't know why my One-Hot encoding code; "pd.get_dummies" is returning Boolean values instead of the binaries of 0 1 df = pd.get_dummies(df) after writing the following line of code; df = pd.get_dummies(df) and also tried; df = pd.get_dummies(df, columns=['column_a', 'column_b', 'column_c']) the returning values of both were booleans True and False instead of 0 and 1

Answer: By default pd.get_dummies return Boolean, try : df = pd.get_dummies(df, dtype=int)
Question: I have a super simple test code to compute FFT in MATLAB, which I am trying to convert to Python but the computed values do not match. MATLAB Code: rect=zeros(100,1); ffrect=zeros(100,1); for j=45:55 rect(j,1)=1; end frect=fft(rect);  Python Code import numpy as np import matplotlib.pyplot as plt from scipy.fft import fft, ifft, fftshift, ifftshift rect = np.zeros((100, 1)) for j in range(44, 55): rect[j] = 1 frect = fft(rect)  Problem: The computed values of frect computed using MATLAB and Python do not match (or even look similar). The values computed by Python has only zero values for complex component and 1s for real.

Answer: scipy.fft.fft computes the FFT over the last axis unless you specify otherwise with the axis parameter. In the Python version, your input array is of shape (100, 1), so you're computing 100 different 1-point FFTs. To compute a single 100-point FFT, either  reshape rect to have its 100 entries in the last dimension (for example, by making it a 1D vector with shape (100,), or a 2D single-row array with shape (1, 100)), or pass axis=0 when calling scipy.fft.fft  fft(rect.reshape(-1)) # or fft(rect, axis=0)