eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am trying to change 1 row in my heatmap to a different color here is the dataset: m = np.array([[ 0.7, 1.4, 0.2, 1.5, 1.7, 1.2, 1.5, 2.5], [ 1.1, 2.5, 0.4, 1.7, 2. , 2.4, 2. , 3.2], [ 0.9, 4.4, 0.7, 2.3, 1.6, 2.3, 2.6, 3.3], [ 0.8, 2.1, 0.2, 1.8, 2.3, 1.9, 2. , 2.9], [ 0.9, 1.3, 0.8, 2.2, 1.8, 2.2, 1.7, 2.8], [ 0.7, 0.9, 0.4, 1.8, 1.4, 2.1, 1.7, 2.9], [ 1.2, 0.9, 0.4, 2.1, 1.3, 1.2, 1.9, 2.4], [ 6.3, 13.5, 3.1, 13.4, 12.1, 13.3, 13.4, 20. ]]) data = pd.DataFrame(data = m)  Right now I am using seaborn heatmap, I can only create something like this: cmap = sns.diverging_palette(240, 10, as_cmap = True) sns.heatmap(data, annot = True, cmap = "Reds") plt.show   I hope to change the color scheme of the last row, here is what I want to achieve (I did this in Excel):  Is it possible I achieve this in Python with seaborn heatmap? Thank you!

Answer: You can split in two, mask the unwanted parts, and plot separately: # Reds data1 = data.copy() data1.loc[7] = float('nan') ax = sns.heatmap(data1, annot=True, cmap="Reds") # Greens data2 = data.copy() data2.loc[:6] = float('nan') sns.heatmap(data2, annot=True, cmap="Greens")  output:  NB. you need to adapt the loc[…] parameter to your actual index names
Question: Some warning can cause problems when running tests with pytest. It might be desirable to ignore all warnings. I could not find a clear way to suppress all warnings.

Answer: This is the solution I found: pytest -W ignore test_script.py
Question: i have a geoJSON geo = {'type': 'Polygon', 'coordinates': [[[23.08437310100004, 53.15448536100007], [23.08459767900007, 53.15448536100007], [23.08594514600003, 53.153587050000056], (...) [23.08437310100004, 53.15448536100007]]]}  and i want to use these coordinates as an input to shapely.geometry.Polygon. The problem is that Polygon only accepts tuple values, meaning i have to convert this geojson to a polygon. When i try to input this type of data into a Polygon there's an error ValueError: A LinearRing must have at least 3 coordinate tuples I tried this: [tuple(l) for l in geo['coordinates']]  but this dosen't quite work since it only returns this [([23.08437310100004, 53.15448536100007], [23.08459767900007, 53.15448536100007], (...) [23.08437310100004, 53.15448536100007])]  and what i need is this (i think it's a tuple) ([(23.08437310100004, 53.15448536100007), (23.08459767900007, 53.15448536100007), (...) (23.08437310100004, 53.15448536100007)])  is there a function for this?

Answer: A generic solution is to use the shape function:  Returns a new, independent geometry with coordinates copied from the context.  This works for all geometries not just polygons. from shapely.geometry import shape from shapely.geometry.polygon import Polygon geo: dict = {'type': 'Polygon', 'coordinates': [[[23.08437310100004, 53.15448536100007], [23.08459767900007, 53.15448536100007], [23.08594514600003, 53.153587050000056], [23.08437310100004, 53.15448536100007]]]} polygon: Polygon = shape(geo)
Question: I want to import keras after I did pip install keras, but it shows message as shown below. I even can't call any function from keras library. Can anyone know about this? import keras  Error: AttributeError: module 'tensorflow.compat.v2.__internal__' has no attribute 'register_clear_session_function' 

Answer: you should use import tensorflow.keras instead of import keras. More info here.
Question: I cannot quite find clear documentation on how to ignore one or more specific Pyright checks:  Using a config file at the root of your project. At the top of a file, function, or method. Each ligne as a trailing comment.  Thanks in advance for sharing this information.

Answer: The usual mypy in-line comments like # type: ignore should work (see details), and for pyright specific config, you can put a pyrightconfig.json in your project root. You can find available config options here. It's just a JSON file, so it looks something like this: { "venvPath": "/home/username/.virtualenvs/", "venv": "myenv", "reportOptionalSubscript": false, "reportOptionalMemberAccess": false }  EDIT: In-source configuration can be as type-ignore statements as supported by mypy. # type: ignore is not a place holder for something else, it is literal. To narrow it down and ignore a specific error (it can be only one of the mypy error codes), like this: # type: ignore[error-code] To use the specific example of import mentioned in the comments, here are the two variants: from os import non_existent # type: ignore[attr-defined] from missing_module import name # type: ignore  This is all discussed in the link to the mypy docs I provided, and the list of error codes linked from there. pyright specific configuration can only be project wide (see EDIT3), either by specifying them in a [tool.pyright] section in your pyproject.toml file, or by creating a pyrightconfig.json like above in your top-level project directory. EDIT2: In the comments the OP raised the question how to find the mypy error-codes that correspond to a pyright config option. Unfortunately there's no easy way besides reading the docs thoroughly along with some understanding of the language; e.g. in the case of from os import name, Python is actually importing the attribute os.name of the module object os into the current namespace. The following interactive session should make this clear: In [1]: import os In [2]: type(os) Out[2]: module In [3]: locals()["curdir"] ------------------------------------------------------------------- KeyError Traceback (most recent call last) <ipython-input-3-a31c5782bef1> in <module> ----> 1 locals()["curdir"] KeyError: 'curdir' In [4]: from os import curdir In [5]: locals()["curdir"] Out[5]: '.' In [6]: os.curdir == curdir Out[6]: True  EDIT3: Pyright also seems to support file-level, and line-level directives, the documentation is hidden under "comments". In short, you can ask pyright to ignore a line or ignore specific pyright errors like this: import missing_module import name # pyright: ignore import missing_module import name # pyright: ignore[reportMissingImports]  You can find the list of errors in the configuration docs.
Question: Sending the output of Prettytable to Telegram This question is a followup to an earlier question. The code which i have is this: import telegram from prettytable import PrettyTable def send_msg(text): token = "*******:**************" chat_id = "***********" bot = telegram.Bot(token=token) bot.sendMessage(chat_id=chat_id, text=text) myTable = PrettyTable(["Student Name", "Class", "Section", "Percentage"]) myTable.add_row(["Leanord", "X", "B", "91.2 %"]) myTable.add_row(["Penny", "X", "C", "63.5 %"]) myTable.add_row(["Howard", "X", "A", "90.23 %"]) myTable.add_row(["Bernadette", "X", "D", "92.7 %"]) myTable.add_row(["Sheldon", "X", "A", "98.2 %"]) myTable.add_row(["Raj", "X", "B", "88.1 %"]) myTable.add_row(["Amy", "X", "B", "95.0 %"]) table_txt = myTable.get_string() with open('output.txt','w') as file: file.write(table_txt) new_list = [] with open("output.txt", 'r', encoding="utf-8") as file: send_msg(file.read())  The problem is that the message which is sent looks like this: +--------------+-------+---------+------------+ | Student Name | Class | Section | Percentage | +--------------+-------+---------+------------+ | Leanord | X | B | 91.2 % | | Penny | X | C | 63.5 % | | Howard | X | A | 90.23 % | | Bernadette | X | D | 92.7 % | | Sheldon | X | A | 98.2 % | | Raj | X | B | 88.1 % | | Amy | X | B | 95.0 % | +--------------+-------+---------+------------+  But when the message received in telegram looks like this:  How can i fix this? Telegram lets you send messages in code whereby i guess it would preserve the format. How can i send this message in   format?

Answer: You have already solved it yourself: you used three backticks in the title of your question. In Markdown (including here on SO), you can put three backticks around a block of code, and that makes it use the code block formatting.  ``` this is inside a code block ```  You can just add a line containing the three backticks in front and back of your text: backticked_text = "```\n" + text + "\n```"
Question: I have a dataset that contains the NBA Player's average statistics per game. Some player's statistics are repeated because of they've been in different teams in season. For example:  Player Pos Age Tm G GS MP FG 8 Jarrett Allen C 22 TOT 28 10 26.2 4.4 9 Jarrett Allen C 22 BRK 12 5 26.7 3.7 10 Jarrett Allen C 22 CLE 16 5 25.9 4.9  I want to average Jarrett Allen's stats and put them into a single row. How can I do this?

Answer: You can groupby and use agg to get the mean. For the non numeric columns, let's take the first value: df.groupby('Player').agg({k: 'mean' if v in ('int64', 'float64') else 'first' for k,v in df.dtypes[1:].items()})  output:  Pos Age Tm G GS MP FG Player Jarrett Allen C 22 TOT 18.666667 6.666667 26.266667 4.333333  NB. content of the dictionary comprehension: {'Pos': 'first', 'Age': 'mean', 'Tm': 'first', 'G': 'mean', 'GS': 'mean', 'MP': 'mean', 'FG': 'mean'}
Question: We're currently maintaining code written in several HTTP frameworks (Flask, aiohttp and FastAPI). Rewriting them so they all use the same framework is currently not feasible. There's some code that I'd like to share across those applications and that would be very well suited for middleware (logging-config, monitoring, authentication, ...). The initial implementation was done by subclassing Flask and worked really well across all Flask-based apps. But it's unusable in aiohttp or FastAPI. Creating a framework-agnostic implementation is doable (theoretically) and this morning I've taken one of the simpler cases and successfully converted it to WSGI middleware and am able to integrate it in the Flask applications. But ASGI is giving me some trouble because there's not much documentation out there for pure ASGI middleware. All the examples show how middleware is written for their framework. The official ASGI docs is also really really "skinny" on that topic. From what I can tell, it should look something like this (side-questions: "What's the second argument passed into the constructor?"): class MyMiddleware: def __init__(self, app, something) -> None: self.app = app self.something = something # <- what is this second argument? async def __call__(self, scope, receive, send): print("Hello from the Middleware") await self.app(scope, receive, send)  I took Starlette TimingMiddleware as inspiration, but I am unable to integrate it with aiohttp. Probably because their implementation is slightly different. Considering that there is a section of middleware in the ASGI specification, and that both aiohttp and Starlette implement that spec, shouldn't there be a way to write a middleware that works in both? If yes, what am I missing?

Answer:  Flask is a WSGI framework. Starlette is an ASGI framework. aiohttp neither supports WSGI nor ASGI. https://github.com/aio-libs/aiohttp/issues/2902  There is no way to have the same middleware support all three frameworks.
Question: Python recently has released match-case in version 3.10. The question is how can we do a default case in Python? I can do if/elif but don't know how to do else. Below is the code: x = "hello" match x: case "hi": print(x) case "hey": print(x) default: print("not matched")  I added this default myself. I want to know the method to do this in Python.

Answer: You can define a default case in Python. For this you use a wild card (_). The following code demonstrates it: x = "hello" match x: case "hi": print(x) case "hey": print(x) case _: print("not matched")
Question:  I looked at the standard documentation that I would expect to capture my need (Apache Arrow and Pandas), and I could not seem to figure it out. I know Python best, so I would like to use Python, but it is not a strict requirement. Problem I need to move Parquet files from one location (a URL) to another (an Azure storage account, in this case using the Azure machine learning platform, but this is irrelevant to my problem). These files are too large to simply perform pd.read_parquet("https://my-file-location.parquet"), since this reads the whole thing into an object. Expectation I thought that there must be a simple way to create a file object and stream that object line by line -- or maybe column chunk by column chunk. Something like import pyarrow.parquet as pq with pq.open("https://my-file-location.parquet") as read_file_handle: with pq.open("https://my-azure-storage-account/my-file.parquet", "write") as write_filehandle: for next_line in read_file_handle{ write_file_handle.append(next_line)  I understand it will be a little different because Parquet is primarily meant to be accessed in a columnar fashion. Maybe there is some sort of config object that I would pass which specifies which columns of interest, or maybe how many lines can be grabbed in a chunk or something similar. But the key expectation is that there is a means to access a parquet file without loading it all into memory. How can I do this? FWIW, I did try to just use Python's standard open function, but I was not sure how to use open with a URL location and a byte stream. If it is possible to do this via just open and skip anything Parquet-specific, that is also fine. Update Some of the comments have suggested using bash-like scripts, such as here. I can use this if there is nothing else, but it is not ideal because:  I would rather keep this all in a full language SDK, whether Python, Go, or whatever. If the solution moves into a bash script with pipes, it requires an external call since the final solution will not be written entirely bash, Powershell, or any scripting language. I really want to leverage some of the benefits of Parquet itself. As I mentioned in the comment below, Parquet is columnar storage. So if I have a "data frame" that is 1.1 billion rows and 100 columns, but I only care about 3 columns, I would love to be able to only download those 3 columns, saving a bunch of time and some money, too. 

Answer: Great post, based on @Micah's answer, I put my 2 cents in it, in case you don't want to read the docs. A small snippet is the following: import pandas as pd import numpy as np from pyarrow.parquet import ParquetFile # create a random df then save to parquet df = pd.DataFrame({ 'A': np.arange(10000), 'B': np.arange(10000), 'C': np.arange(10000), 'D': np.arange(10000), }) df.to_parquet('./test/test') # ****** below is @Micah Kornfield's answer ****** # 1. open parquet file batch = ParquetFile('./test/test') # 2. define generator of batches record = batch.iter_batches( batch_size=10, columns=['B', 'C'], ) # 3. yield pandas/numpy data print(next(record).to_pandas()) # pandas print(next(record).to_pydict()) # native python dict
Question: I'm making a pose estimation script for my game. However, it's working at 20-30 fps and not using the whole CPU even if there is no fps limit. It's not using whole GPU too. Can someone help me? Here is resource usage while playing a dance video: https://i.sstatic.net/6L8Rj.jpg Here is my code: import cv2 import mediapipe as mp import time inFile = '/dev/video0' capture = cv2.VideoCapture(inFile) FramesVideo = int(capture.get(cv2.CAP_PROP_FRAME_COUNT)) # Number of frames inside video FrameCount = 0 # Currently playing frame prevTime = 0 # some objects for mediapipe mpPose = mp.solutions.pose mpDraw = mp.solutions.drawing_utils pose = mpPose.Pose() while True: FrameCount += 1 #read image and convert to rgb success, img = capture.read() imgRGB = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) #process image results = pose.process(imgRGB) if results.pose_landmarks: mpDraw.draw_landmarks(img, results.pose_landmarks, mpPose.POSE_CONNECTIONS) #get landmark positions landmarks = [] for id, lm in enumerate(results.pose_landmarks.landmark): h, w, c = img.shape cx, cy = int(lm.x * w), int(lm.y * h) cv2.putText(img, str(id), (cx,cy), cv2.FONT_HERSHEY_PLAIN, 1, (255,0,0), 1) landmarks.append((cx,cy)) # calculate and print fps frameTime = time.time() fps = 1/(frameTime-prevTime) prevTime = frameTime cv2.putText(img, str(int(fps)), (30,50), cv2.FONT_HERSHEY_PLAIN, 3, (255,0,0), 3) #show image cv2.imshow('Video', img) cv2.waitKey(1) if FrameCount == FramesVideo-1: capture.release() cv2.destroyAllWindows() break 

Answer: Set the model_complexity of mp.Pose to 0. As the documentation states:  MODEL_COMPLEXITY Complexity of the pose landmark model: 0, 1 or 2. Landmark accuracy as well as inference latency generally go up with the model complexity. Default to 1.  This is the best solution I've found, also use this.
Question: When I want to ignore some fields using attr library, I can use repr=False option. But I cloud't find a similar option in pydantic Please see example code import typing import attr from pydantic import BaseModel @attr.s(auto_attribs=True) class AttrTemp: foo: typing.Any boo: typing.Any = attr.ib(repr=False) class Temp(BaseModel): foo: typing.Any boo: typing.Any # I don't want to print class Config: frozen = True a = Temp( foo="test", boo="test", ) b = AttrTemp(foo="test", boo="test") print(a) # foo='test' boo='test' print(b) # AttrTemp(foo='test')  However, it does not mean that there are no options at all, I can use the syntax print(a.dict(exclude={"boo"})) Doesn't pydantic have an option like repr=False?

Answer: It looks like this feature has been requested and also implemented not long ago. However, it seems like it hasn't made it into the latest release yet. I see two options how to enable the feature anyway: 1. Use the workaround provided in the feature request Define a helper class: import typing from pydantic import BaseModel, Field class ReducedRepresentation: def __repr_args__(self: BaseModel) -> "ReprArgs": return [ (key, value) for key, value in self.__dict__.items() if self.__fields__[key].field_info.extra.get("repr", True) ]  and use it in your Model definition: class Temp(ReducedRepresentation, BaseModel): foo: typing.Any boo: typing.Any = Field(..., repr=False) class Config: frozen = True a = Temp( foo="test", boo="test", ) print(a) # foo='test'  2. pip install the latest master I would suggest doing this in a virtual environment. This is what worked for me: Uninstall the existing version: $ pip uninstall pydantic ...  Install latest master: $ pip install git+https://github.com/samuelcolvin/pydantic.git@master ...  After that the repr argument should work out of the box: import typing from pydantic import BaseModel, Field class Temp(BaseModel): foo: typing.Any boo: typing.Any = Field(..., repr=False) class Config: frozen = True a = Temp( foo="test", boo="test", ) print(a) # foo='test'
Question: def percent(marks): return (marks[0]+marks[1]+marks[2]+marks[3]/400)*100 marks1=[54,65,85,54] percent1=percent(marks1) marks2=[54,52,65,85] percent2 = percent(marks2) print(percent1,percent2) 

Answer: The lines after return will not be executed anytime. So you can delete them and nothing will change. The message told you about it, because it is very unusual to have such code. I think you wanted this: def percent(marks): return (marks[0]+marks[1]+marks[2]+marks[3]/400)*100 marks1 = [54, 65, 85, 54] percent1 = percent(marks1) marks2 = [54, 52, 65, 85] percent2 = percent(marks2) print(percent1, percent2)  Spaces in Python code matter. In your code all lines are the part of the function. In fixed code they are not.
Question: I tried to use fake_useragent module with this block from fake_useragent import UserAgent ua = UserAgent() print(ua.random)  But when the execution reached this line ua = UserAgent(), it throws this error Traceback (most recent call last): File "/home/hadi/Desktop/excel/gatewayform.py", line 191, in <module> gate = GateWay() File "/home/hadi/Desktop/excel/gatewayform.py", line 23, in __init__ ua = UserAgent() File "/usr/local/lib/python3.9/dist-packages/fake_useragent/fake.py", line 69, in __init__ self.load() File "/usr/local/lib/python3.9/dist-packages/fake_useragent/fake.py", line 75, in load self.data = load_cached( File "/usr/local/lib/python3.9/dist-packages/fake_useragent/utils.py", line 250, in load_cached update(path, use_cache_server=use_cache_server, verify_ssl=verify_ssl) File "/usr/local/lib/python3.9/dist-packages/fake_useragent/utils.py", line 245, in update write(path, load(use_cache_server=use_cache_server, verify_ssl=verify_ssl)) File "/usr/local/lib/python3.9/dist-packages/fake_useragent/utils.py", line 178, in load raise exc File "/usr/local/lib/python3.9/dist-packages/fake_useragent/utils.py", line 154, in load for item in get_browsers(verify_ssl=verify_ssl): File "/usr/local/lib/python3.9/dist-packages/fake_useragent/utils.py", line 99, in get_browsers html = html.split('<table class="w3-table-all notranslate">')[1] IndexError: list index out of range  I use linux and I have installed the module using this command pip3 install fake_useragent --upgrade. Is there any solution for this issue? if not, is there a better module to use?

Answer: There is a solution for this, from Github pull request #110. Basically, all you need to do is change one character in one line of the fake_useragent/utils.py source code. To do this on your system, open /usr/local/lib/python3.9/dist-packages/fake_useragent/utils.py† in your favorite text editor using admin privileges. Go to line 99, and change the w3  html = html.split('<table class="w3-table-all notranslate">')[1] # ^^ change this  to ws:  html = html.split('<table class="ws-table-all notranslate">')[1] # ^^ to this  Save the file (with admin permissions), restart your Python session, and your code should work just fine.  † To find the fake_useragent directory in which utils.py resides, run the following code: import fake_useragent print(fake_useragent.__file__)  For example, on my Windows laptop, this printed 'C:\\Users\\mattdmo\\AppData\\Roaming\\Python\\Python310\\site-packages\\fake_useragent\\__init__.py'  so the folder to open is C:\Users\mattdmo\AppData\Roaming\Python\Python310\site-packages\fake_useragent.
Question: I'm getting the error of missing import when doing import in my Django project, I think it's because it's installed in a Docker Container. But how can I make it so VSCode somehow knows that the packages are installed? If I select an interpreter of a venv in which I have installed django or other packages it doesn't give me that warning, but I don't think creating an venv and installing all the packages there is the right thing. Or maybe it is?

Answer: It's recommended to install the packages individually, but if you want to reuse them, you can add the paths of them into the PYTHONPATH. You can do this to modify the PYTHONPATH:  Add these in the settings.json file to Modify the PYTHONPATH in the terminal: "terminal.integrated.env.windows": { "PYTHONPATH": "xxx/site-packages" }  Create a .env file under your workspace, and add these settings in it to modify the PYTHONPATH for the extension and debugger: PYTHONPATH=xxx/site-packages   You can refer to here to understand the effects of these two configurations.