eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have, in a single repo, two Python projects that both depend on a shared utility package. My goal is to package each of the two projects in a software distribution package (i.e. a .tzr.gz file) I am currently getting this done using setuptools and setup.py files and having a hard time of it. I would much rather use Poetry to manage and package each of the two projects separately. Please consider this "minimal repro" of my problem: repo project1/ __init__.py main_module.py pyproject.toml project2/ __init__.py main_module.py pyproject.toml util/ __init__.py util_module.py  I tried to get Poetry to include the util package when building project1 by modifying its project.toml this way: [tool.poetry] name = "project1" version = "0.1.0" description = "" authors = [""] packages = [ { include = "../util/*.py" } ] [tool.poetry.dependencies] python = "^3.9" [tool.poetry.dev-dependencies] pytest = "^5.2" [build-system] requires = ["poetry-core>=1.0.0"] build-backend = "poetry.core.masonry.api"  When I run poetry build I get this error: Building project1 (0.1.0) - Building sdist ValueError 'C:\\repo\\util\\__init__.py' is not in the subpath of 'C:\\repo\\project1' OR one path is relative and the other is absolute. at ~\.pyenv\pyenv-win\versions\3.9.6\lib\pathlib.py:929 in relative_to 925│ n = len(to_abs_parts) 926│ cf = self._flavour.casefold_parts 927│ if (root or drv) if n == 0 else cf(abs_parts[:n]) != cf(to_abs_parts): 928│ formatted = self._format_parsed_parts(to_drv, to_root, to_parts) → 929│ raise ValueError("{!r} is not in the subpath of {!r}" 930│ " OR one path is relative and the other is absolute." 931│ .format(str(self), str(formatted))) 932│ return self._from_parsed_parts('', root if n == 1 else '', 933│ abs_parts[n:])  Doesn't poetry support my use-case? If not, what am I missing? Alternatively, please suggest another approach to package my two projects separately, but both packages must include the shared util package.

Answer: This seems to be a issue with poetry. Check this out https://github.com/python-poetry/poetry/issues/5621.
Question: I am building and Inventory Web App and I am facing this error. I am trying to get Username from token to show a custom hello message and show that the user is logged in as someone. Here is my Views.py that gets Token from localStorage as logged in user: class UserDetails(APIView): def post(self, request): serializer = UserAccountTokenSendSerializer(data=request.data) global token if serializer.is_valid(): token = serializer.validated_data['token'] return Response(serializer.data) def get(self): user_id = Token.objects.get(key=token).user_id details = User.objects.get(id=user_id) serializer = UserDetails(details, many=False) return Response(serializer.data)  class UserDetails(serializers.ModelSerializer): class Meta: model = User fields = ( 'username', )  Here is my Urls.py: urlpatterns = [ path('get-user-token/', views.UserDetails.as_view()), path('get-user-details/', views.UserDetails.as_view()), ]  And this is the error that I get:  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch response = handler(request, *args, **kwargs) TypeError: get() takes 1 positional argument but 2 were given [15/Jun/2022 19:34:59] "GET /api/v1/get-user-details/ HTTP/1.1" 500 82239 

Answer: The .get(…) method [Django-doc] takes a request parameter as well, even if you do not use it, so: class UserDetails(APIView): # … def get(self, request): # … pass I would strongly advise, not to make token a global variable. The same webserver can be used for another user that thus has not authenticated (properly). Take a look at the Authentication section of the Django REST framework documentation for more information on how to authenticate and check credentials.   Note: In Django, class-based views (CBV) often have a …View suffix, to avoid a clash with the model names. Therefore you might consider renaming the view class to UserDetailsView, instead of UserDetails.
Question: I would like to add a column in a dataframe in the streamlit and this column be a checkbox, so that later I could list the lines that the checkboxes were marked, is this possible?

Answer: You can use the package streamlit-aggrid. Code """ pip install streamlit-aggrid """ import streamlit as st import pandas as pd from st_aggrid import AgGrid, GridUpdateMode from st_aggrid.grid_options_builder import GridOptionsBuilder data = { 'country': ['Japan', 'China', 'Thailand', 'France', 'Belgium', 'South Korea'], 'capital': ['Tokyo', 'Beijing', 'Bangkok', 'Paris', 'Brussels', 'Seoul'] } df = pd.DataFrame(data) gd = GridOptionsBuilder.from_dataframe(df) gd.configure_selection(selection_mode='multiple', use_checkbox=True) gridoptions = gd.build() grid_table = AgGrid(df, height=250, gridOptions=gridoptions, update_mode=GridUpdateMode.SELECTION_CHANGED) st.write('## Selected') selected_row = grid_table["selected_rows"] st.dataframe(selected_row)  Output
Question:  Python version: 3.7 (I have to use this version) OS: Linux Cloud Platform: Azure Resource: Azure function with python Goal: Load a model created with skit-learn version 1.0.2 with the following dependencies installed:   numpy: 1.17.3 joblib: 1.1.0 scipy: 1.7.3  I am using joblib to load a skit-learn model that I trained (By the way I created the model locally in my machine with python 3.9). However, I am getting the following error: Traceback (most recent call last): File \"/home/site/wwwroot/sortierung/__init__.py\", line 51, in main prediction_file_path) File \"/home/site/wwwroot/shared_code/custom_functions_prediction.py\", line 255, in predict result.update(classify_mail(m,s,X, stop_words, model_folder_path)) File \"/home/site/wwwroot/shared_code/custom_functions_prediction.py\", line 105, in classify_mail model = load(modelFilePath) File \"/home/site/wwwroot/.python_packages/lib/site-packages/joblib/numpy_pickle.py\", line 587, in load obj = _unpickle(fobj, filename, mmap_mode) File \"/home/site/wwwroot/.python_packages/lib/site-packages/joblib/numpy_pickle.py\", line 506, in _unpickle obj = unpickler.load() File \"/usr/local/lib/python3.7/pickle.py\", line 1088, in load dispatch[key[0]](self) File \"/usr/local/lib/python3.7/pickle.py\", line 1385, in load_stack_global self.append(self.find_class(module, name)) File \"/usr/local/lib/python3.7/pickle.py\", line 1426, in find_class __import__(module, level=0)\nModuleNotFoundError: No module named 'scipy.sparse._csr'  I checked in the scipy folder installed and I could not find this module. How could I solve this issue?. Tks in advance

Answer: To resolve this ModuleNotFoundError: No module named 'scipy.sparse._csr' error, try the following way: This error occurred because you have created a model in Python 3.9 but running it on Python 3.7. You can try creating a model in Python 3.7 or upgrade the Azure Python function app to a specific version of Python 3.9. To change the Python version to 3.9, according to documentation:  You can update the linuxFxVersion setting in the function app with the az functionapp config set command.  az functionapp config set --name <FUNCTION_APP> \ --resource-group <RESOURCE_GROUP> \ --linux-fx-version "python|3.9"  References: No module named 'scipy.sparse._csr' and How to change python version of azure function
Question: Goal: use Git Bash as Terminal in PyCharm. How can I have a normal Bash with Git integrated Terminal in PyCharm? File path for Git Bash: C:\Users\me\AppData\Local\Programs\Git\git-bash.exe --cd-to-home. I apply Git Bash in PyCharm Settings:  However, when I click New Session (new Terminal +), it launches as a Window: 

Answer: You are pointing at git-bash.exe, which is wrong. You should point at bash.exe which is located inside the bin folder. So your Shell path should be: "C:\Users\<username>\AppData\Local\Programs\Git\bin\bash.exe" --login  using "C:\Users\<username>\AppData\Local\Programs\Git\bin\sh.exe" --login  would also work. Also works without --login. Make sure the " are there.
Question: How do I dump the current configuration of the Python logging module? For example, if I use a module that configures logging for me, how can I see what it has done?

Answer: There does not appear to be a documented way to do so, but we can get hints by looking at how the logging module is implemented. All Loggers belong to a tree, with the root Logger instance at logging.root. The Logger instances do not track their own children but instead have a shared Manager that can be used to get a list of all loggers: >>> print(logging.root.manager.loggerDict) { 'rosgraph': <logging.PlaceHolder object at 0xffffa2851710>, 'rosgraph.network': <logging.Logger object at 0xffffa28517d0>, 'rosout': <rosgraph.roslogging.RospyLogger object at 0xffffa2526290>, 'rospy': <rosgraph.roslogging.RospyLogger object at 0xffffa2594250>, ... }  Each Logger instance has handlers and filters attributes which can help understand the behavior of the logger.
Question: I am a little confused about the method pyspark.sql.Window.rowsBetween that accepts Window.unboundedPreceding, Window.unboundedFollowing, and Window.currentRow objects as start and end arguments. Could you please explain how the function works and how to use Window objects correctly, with some examples? Thank you!

Answer: Rows between/Range between as the name suggests help with limiting the number of rows considered inside a window. Let us take a simple example. Starting with data: dfw = ( spark .createDataFrame( [ ("abc", 1, 100), ("abc", 2, 200), ("abc", 3, 300), ("abc", 4, 200), ("abc", 5, 100), ], "name string,id int,price int", ) ) # output +----+---+-----+ |name| id|price| +----+---+-----+ | abc| 1| 100| | abc| 2| 200| | abc| 3| 300| | abc| 4| 200| | abc| 5| 100| +----+---+-----+  Now over this data let's try to find of running max i.e max for each row: ( dfw .withColumn( "rm", F.max("price").over(Window.partitionBy("name").orderBy("id")) ) .show() ) #output +----+---+-----+---+ |name| id|price| rm| +----+---+-----+---+ | abc| 1| 100|100| | abc| 2| 200|200| | abc| 3| 300|300| | abc| 4| 200|300| | abc| 5| 100|300| +----+---+-----+---+  So as expected it looked at each price from top to bottom one by one and populated the max value it got this behaviour is known as start = Window.unboundedPreceding to end = Window.currentRow Now changing rows between values to start = Window.unboundedPreceding to end = Window.unboundedFollowing we will get as below: ( dfw .withColumn( "rm", F.max("price").over( Window .partitionBy("name") .orderBy("id") .rowsBetween(Window.unboundedPreceding, Window.unboundedFollowing) ) ) .show() ) #output +----+---+-----+---+ |name| id|price| rm| +----+---+-----+---+ | abc| 1| 100|300| | abc| 2| 200|300| | abc| 3| 300|300| | abc| 4| 200|300| | abc| 5| 100|300| +----+---+-----+---+  Now as you can see in the same window it's looking downwards in all values for a max instead of limiting it to the current row. Now third will be start = Window.currentRow and end = Window.unboundedFollowing ( dfw .withColumn( "rm", F.max("price").over( Window .partitionBy("name") .orderBy("id") .rowsBetween(Window.currentRow, Window.unboundedFollowing) ) ) .show() ) #output +----+---+-----+---+ |name| id|price| rm| +----+---+-----+---+ | abc| 1| 100|300| | abc| 2| 200|300| | abc| 3| 300|300| | abc| 4| 200|200| | abc| 5| 100|100| +----+---+-----+---+  Now it's looking down only for a max starting its row from the current one. Also, it's not limited to just these 3 to use as is you can even start = Window.currentRow-1 and end = Window.currentRow+1 so instead of looking for all values above or below it will only look at 1 row above and 1 row below. like this: ( dfw .withColumn( "rm", F.max("price").over( Window .partitionBy("name") .orderBy("id") .rowsBetween(Window.currentRow-1, Window.currentRow+1) ) ) .show() ) # output +----+---+-----+---+ |name| id|price| rm| +----+---+-----+---+ | abc| 1| 100|200| | abc| 2| 200|300| | abc| 3| 300|300| | abc| 4| 200|300| | abc| 5| 100|200| +----+---+-----+---+  So you can imagine it a window inside the window which works around the current row it's processing.
Question: Given a dataclass like below: class MessageHeader(BaseModel): message_id: uuid.UUID def dict(self, **kwargs): return json.loads(self.json())  I would like to get a dictionary of string literal when I call dict on MessageHeader The desired outcome of dictionary is like below: {'message_id': '383b0bfc-743e-4738-8361-27e6a0753b5a'}  I want to avoid using 3rd party library like pydantic & I do not want to use json.loads(self.json()) as there are extra round trips Is there any better way to convert a dataclass to a dictionary with string literal like above?

Answer: You can use dataclasses.asdict: from dataclasses import dataclass, asdict class MessageHeader(BaseModel): message_id: uuid.UUID def dict(self): return {k: str(v) for k, v in asdict(self).items()}  If you're sure that your class only has string values, you can skip the dictionary comprehension entirely: class MessageHeader(BaseModel): message_id: uuid.UUID dict = asdict
Question: What I am trying to do? I am trying to access my environment variable from the .env file and print its value in the terminal. What is the issue? When I run the script in the terminal I keep getting the error none Some more info: I am using Windows 10 The version of dotenv is 0.20.0 I downloaded it using  python -m pip install python-dotenv The code .env file - export PRIVATE_KEY = 0xc3c4e4fe27d8e6b06710e713878e4488c034ce346a578fdfa78bb3d335130eec The Python file - from dotenv import load_dotenv import os load_dotenv() print(os.getenv("PRIVATE_KEY")) 

Answer: I solved my issue but it wasn't after removing the "export" in the .env file (I tried with and without it both gave the same results in the terminal), I had to specify the full path of the .env file in the load_dotenv(), apparently I had to specify it but many code samples I saw in forums didn't need to do it I wonder why? Here is the new code... The Python code -  The .env file -  by the way it's not a real private key At some point before I found the solution I ran the Python script in the terminal and got a random private key (I don't have it pictured sadly), it might had been a key I set in the past but then I checked if I had other .env files in the folder but I had none I also didn't have any environment variables in System Properties > Advanced > Environment Variables on windows so where does dotenv gets the key values by dafault? also After that I reopened vscode and tried again but I got a none error...
Question: Is there any way to extract the "Most Replayed" (aka Video Activity Graph) Data from a YouTube video via API? What I'm referring to: 

Answer: One more time YouTube Data API v3 doesn't provide a basic feature. I recommend you to try out my open-source YouTube operational API. Indeed by fetching https://yt.lemnoslife.com/videos?part=mostReplayed&id=VIDEO_ID, you will get the most replayed graph values you are looking for in item["mostReplayed"]. With the video id XiCrniLQGYc you would get: { "kind": "youtube#videoListResponse", "etag": "NotImplemented", "items": [ { "kind": "youtube#video", "etag": "NotImplemented", "id": "XiCrniLQGYc", "mostReplayed": { "markers": [ { "startMillis": 0, "intensityScoreNormalized": 1 }, { "startMillis": 2580, "intensityScoreNormalized": 0.7083409245967562 }, { "startMillis": 5160, "intensityScoreNormalized": 0.6381007317793738 }, ... { "startMillis": 255420, "intensityScoreNormalized": 0.012864077773078256 } ], "timedMarkerDecorations": [ { "visibleTimeRangeStartMillis": 0, "visibleTimeRangeEndMillis": 10320 } ] } } ] }
Question: In .Net C#, there is a function Task.WhenAll that can take a list of tasks to await them. What should I use in python? I am trying to do the same with this: tasks = ... # list of coroutines for task in tasks: await task 

Answer: After adding tasks to a list, you should use asyncio.gather that gives coroutines as an argument list and executes them asynchronously. Also, you could use asyncio.create_task that takes a coroutine and calls concurrent tasks in the event loop. import asyncio async def coro(i): await asyncio.sleep(i//2) async def main(): tasks = [] for i in range(5): tasks.append(coro(i)) await asyncio.gather(*tasks) if __name__ == "__main__": loop = asyncio.get_event_loop() loop.run_until_complete(main()) loop.close()
Question: I wanna run my app without any problem, but I got this attached error. Could someone help or point me in the right direction regarding why this is happening? Traceback (most recent call last): File "/Users/enchant3dmango/Documents/GitHub/nexus/automation-api/app/main.py", line 4, in <module> from configurations import config # noqa # pylint: disable=unused-import File "/Users/enchant3dmango/Documents/GitHub/nexus/automation-api/app/configurations/config.py", line 7, in <module> from google.cloud import secretmanager File "/Users/enchant3dmango/opt/miniconda3/envs/nexus/lib/python3.9/site-packages/google/cloud/secretmanager.py", line 20, in <module> from google.cloud.secretmanager_v1 import SecretManagerServiceClient File "/Users/enchant3dmango/opt/miniconda3/envs/nexus/lib/python3.9/site-packages/google/cloud/secretmanager_v1/__init__.py", line 24, in <module> from google.cloud.secretmanager_v1.gapic import secret_manager_service_client File "/Users/enchant3dmango/opt/miniconda3/envs/nexus/lib/python3.9/site-packages/google/cloud/secretmanager_v1/gapic/secret_manager_service_client.py", line 25, in <module> import google.api_core.gapic_v1.client_info File "/Users/enchant3dmango/opt/miniconda3/envs/nexus/lib/python3.9/site-packages/google/api_core/gapic_v1/__init__.py", line 18, in <module> from google.api_core.gapic_v1 import config File "/Users/enchant3dmango/opt/miniconda3/envs/nexus/lib/python3.9/site-packages/google/api_core/gapic_v1/config.py", line 23, in <module> import grpc File "/Users/enchant3dmango/opt/miniconda3/envs/nexus/lib/python3.9/site-packages/grpc/__init__.py", line 22, in <module> from grpc import _compression File "/Users/enchant3dmango/opt/miniconda3/envs/nexus/lib/python3.9/site-packages/grpc/_compression.py", line 15, in <module> from grpc._cython import cygrpc ImportError: dlopen(/Users/enchant3dmango/opt/miniconda3/envs/nexus/lib/python3.9/site-packages/grpc/_cython/cygrpc.cpython-39-darwin.so, 2): Symbol not found: _CFRelease Referenced from: /Users/enchant3dmango/opt/miniconda3/envs/nexus/lib/python3.9/site-packages/grpc/_cython/cygrpc.cpython-39-darwin.so Expected in: flat namespace in /Users/enchant3dmango/opt/miniconda3/envs/nexus/lib/python3.9/site-packages/grpc/_cython/cygrpc.cpython-39-darwin.so  I'm running this in Apple M1. I already upgraded pip and setuptools before installing all the requirements in my virtual environment using conda. Here is my python, pip, and setuptools version: python 3.9.12 pip 21.2.4 setuptools 62.4.0 

Answer: I was able to get around the problem by rebuilding grpcio from source like this: pip uninstall grpcio export GRPC_PYTHON_LDFLAGS=" -framework CoreFoundation" pip install grpcio --no-binary :all:
Question: hi I'm getting an import error, does anyone know the solution? ImportError: Bindings generation error. Submodule name should always start with a parent module name. Parent name: cv2.cv2. Submodule name: cv2

Answer: If you are using opencv-python version 4.6.0.66, try to downgrade to 4.5.5.64 version, on Pycharm you can do that by go to File->Setting->Python Interpreter-> Double-click on opencv-python version->check the specify version box, then choose older version. Downgrade opencv also makes the auto-completion works again.
Question: I am working on a project which has dependency on another project. In live environment, the dependency is published and installed simply using pip install. On my local environment, I'd like to be able to install the local dependency instead, using the pip install -e command. The structure is as follow: - Home --- Project1 ----- docker-compose ----- Dockerfile --- relaton-py  In this structure, Project1 has dependency on relaton-py, thus I'd like to "install" this dependency using the local relaton-py. My docker-compose file looks like: volumes: - .:/code - /Users/myuser/Dev/Projects/relaton-py:/relaton-py  while the Dockerfile looks like: COPY requirements.txt /code/requirements.txt RUN ["pip", "install", "-e", "relaton-py"] WORKDIR /code RUN ["pip", "install", "-r", "requirements.txt"] # Copy the rest of the codebase COPY . /code  When trying to spin un the environment, I get the following error: => ERROR [local/web-precheck:latest 11/19] RUN ["pip", "install", "-e", "relaton-py"] 1.2s ------ > [local/web-precheck:latest 11/19] RUN ["pip", "install", "-e", "relaton-py"]: #27 0.685 ERROR: relaton-py is not a valid editable requirement. It should either be a path to a local project or a VCS URL (beginning with bzr+http, bzr+https, bzr+ssh, bzr+sftp, bzr+ftp, bzr+lp, bzr+file, git+http, git+https, git+ssh, git+git, git+file, hg+file, hg+http, hg+https, hg+ssh, hg+static-http, svn+ssh, svn+http, svn+https, svn+svn, svn+file). #27 0.889 WARNING: You are using pip version 22.0.4; however, version 22.1.2 is available. #27 0.889 You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command. ------ failed to solve: rpc error: code = Unknown desc = executor failed running [pip install -e relaton-py]: exit code: 1  However, if I try not to install this local dependency in the Dockerfile, the entire environment spins up and I am able to access the container and install the package manually with the same command pip install -e relaton-py. Has anyone had to deal with this sort of thing already? Any idea on how to make Dockerfile recognise the files in the mounted volumes?

Answer: For simpler packages, it may be enough to just mount it directly into the site-packages. In your example, you might want to try:  putting relaton-py (just the package in its source form, e.g. the Git repository) one directory above your Compose file, and adding this line to docker-compose.yml under volumes:  - ../relaton-py/relaton:/usr/local/lib/python3.10/site-packages/relaton:ro  Assuming you use Python 3.10, this should override whatever gets installed from PyPI during Docker image build with the directory on your host system, mounted in read-only mode to prevent accidental changes from within the container (but any source code changes you make on your main system, of course, will be propagated immediately). If your package includes native modules and requires compilation for particular OS/architecture, there may be extra steps involved in this.