eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: There are two similar pydantic object like that. The only difference is some fields are optionally. How can I just define the fields in one object and extend into another one? class ProjectCreateObject(BaseModel): project_id: str project_name: str project_type: ProjectTypeEnum depot: str system: str ... class ProjectPatchObject(ProjectCreateObject): project_id: str project_name: Optional[str] project_type: Optional[ProjectTypeEnum] depot: Optional[str] system: Optional[str] ... 

Answer: I find a good and easy way by __init__subclass__. The docs also can be generated successfully. class ProjectCreateObject(BaseModel): project_id: str project_name: str project_type: ProjectTypeEnum depot: str system: str ... def __init_subclass__(cls, optional_fields=(), **kwargs): """ allow some fields of subclass turn into optional """ super().__init_subclass__(**kwargs) for field in optional_fields: cls.__fields__[field].outer_type_ = Optional cls.__fields__[field].required = False _patch_fields = ProjectCreateObject.__fields__.keys() - {'project_id'} class ProjectPatchObject(ProjectCreateObject, optional_fields=_patch_fields): pass
Question: I've just begun playing around with Python's Data Classes, and I would like confirm that I am declaring Class Variables in the proper way.  Using regular python classes class Employee: raise_amount = .05 def __init__(self, fname, lname, pay): self.fname = fname self.lname = lname self.pay = pay  Using python Data Class @dataclass class Employee: fname: str lname: str pay: int raise_amount = .05  The class variable I am referring to is raise_amount. Is this a properly declared class variable using Data Classes? Or is there a better way of doing so?  I have tested the data class implementation already and it provides the expected functionality, but I am mainly wondering if my implementation is following best practices. 

Answer: To create a class variable, annotate the field as a typing.ClassVar or not at all. from typing import ClassVar from dataclasses import dataclass @dataclass class Foo: ivar: float = 0.5 cvar: ClassVar[float] = 0.5 nvar = 0.5 foo = Foo() Foo.ivar, Foo.cvar, Foo.nvar = 1, 1, 1 print(Foo().ivar, Foo().cvar, Foo().nvar) # 0.5 1 1 print(foo.ivar, foo.cvar, foo.nvar) # 0.5 1 1 print(Foo(), Foo(12)) # Foo(ivar=0.5) Foo(ivar=12)  There is a subtle difference in that the unannotated field is completely ignored by @dataclass, whereas the ClassVar field is stored but not converted to an attribute.   dataclasses — Data Classes   The member variables [...] are defined using PEP 526 type annotations.   Class variables One of two places where dataclass() actually inspects the type of a field is to determine if a field is a class variable as defined in PEP 526. It does this by checking if the type of the field is typing.ClassVar. If a field is a ClassVar, it is excluded from consideration as a field and is ignored by the dataclass mechanisms. Such ClassVar pseudo-fields are not returned by the module-level fields() function.
Question: How can I crop an image using OpenCV from the center? I think it has something to do with this line, but if there is a better way please inform me. crop_img = img[y:y+h, x:x+w] 

Answer: Just an additional comment for the Lenik's answer (It is the first time I want to contribute in StackOverflow and don't have enough reputation to comment the answer), you need to be sure x and y are integers. Probably in this case x and y would always be integers as most of resolutions are even, but is a good practice to keep the values inside an int(). center = image.shape x = center[1]/2 - w/2 y = center[0]/2 - h/2 crop_img = img[int(y):int(y+h), int(x):int(x+w)]
Question: I have a two variables that i want to compare. When printed, this is what they look like: 2020-05-20 13:01:30 2020-05-20 14:49:03  However, one is a string type, and the other a datetime type. If I want to convert the string one into date type so I can compare them, is the only way to use strptime? Because this seems a little redundant to me, since the string already has the exact format I want it to have. Basically, is there a function that does the same as strptime, but without re-formating it? As you can imagine, googling this problem is impossible, as all I'm getting is people trying to format any kind of string into datetime, so all the answers are just pointing at strptime.

Answer: If you work with Python 3.7+, for ISO 8601 compatible strings, use datetime.fromisoformat() as this is considerably more efficient than strptime or dateutil's parser. Ex: from datetime import datetime dtobj = datetime.fromisoformat('2020-05-20 13:01:30') print(repr(dtobj)) # datetime.datetime(2020, 5, 20, 13, 1, 30)  You can find a benchmark vs. strptime etc. here or here.
Question: I'm interested in automatizing reverse image search. Yandex in particular is great for busting catfishes, even better than Google Images. So, consider this Python code: import requests import webbrowser try: filePath = "C:\\path\\whateverThisIs.png" searchUrl = 'https://yandex.ru/images/' multipart = {'encoded_image': (filePath, open(filePath, 'rb')), 'image_content': ''} response = requests.post(searchUrl, files=multipart, allow_redirects=False) #fetchUrl = response.headers['Location'] print(response) print(dir(response)) print(response.content) input() except Exception as e: print(e) print(e.with_traceback) input()```  The script fails with KeyError, 'location' is not found. I know the code works cause if you substitute searchUrl with http://www.google.hr/searchbyimage/upload then the script returns the correct url. So, in short the expected outcome would be a url with an image search. In actuality we get a KeyError where that url was supposed to be stored. Evidently, Yandex doesn't work in exactly the same way, maybe the url is off (although I tried a heap ton of variations) or the reason may be completely different. Regardless of that, help in solving this problem is much appreciated!

Answer: You can get url with an image search by using this code. Tested on ubuntu 18.04, with python 3.7 and requests 2.23.0 import json import requests file_path = "C:\\path\\whateverThisIs.png" search_url = 'https://yandex.ru/images/search' files = {'upfile': ('blob', open(file_path, 'rb'), 'image/jpeg')} params = {'rpt': 'imageview', 'format': 'json', 'request': '{"blocks":[{"block":"b-page_type_search-by-image__link"}]}'} response = requests.post(search_url, params=params, files=files) query_string = json.loads(response.content)['blocks'][0]['params']['url'] img_search_url = search_url + '?' + query_string print(img_search_url)
Question: I'm trying to delete messages from /getUpdates in telegram API but I didn't know how.. I tried to use /deleteMessage https://api.telegram.org/bot<TOKEN>/deleteMessage?chat_id=blahblah&message_id=BlahBlah  But it didn't delete message from API database..

Answer: TL;DR: Call getUpdates() with the offset parameter set to the last message's id, incremented by 1  We'll need to let Telegram know which message's we've processed. To do this, set the offset parameter to the update_id + 1 of the last message your script has processed.  Call getUpdates() to get the update_id of the latest message https://api.telegram.org/<MY-TOKEN>/getUpdates  { "ok": true, "result": [ { "update_id": 343126593, # <-- Remember / Save this id "message": { ...   Increment the update_id by 1  On the next getUpdates() call, set the offset parameter to the id: https://api.telegram.org/<MY-TOKEN>/getUpdates?offset=343126594
Question: I've been trying to run pip list -o and pip list --outdated to see if any packages need to be updated but it enters a loop of printing: WARNING: Keyring is skipped due to an exception: Failed to create the collection: Prompt dismissed.. I've upgraded keyring and the version was already up-to-date. I've seen this keyring warning whilst using pip install {package} --upgrade to upgrade other packages as well.

Answer: I searched the web about that topic and find that GitHub issue. If your pip version is any version before "21.1", you can try to upgrade pip to the latest version with pip install --upgrade pip command. Also, as a workaround, you can consider the following answer of jrd from the above link:  Exporting PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring prevent python from using any keyring. PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring pipenv update does not ask me anything anymore. So, as a temporary solution, one might want to put this in a .env file.
Question: I follow this link: https://sendgrid.com/docs/ui/sending-email/sender-verification In my sendgrid account the from_email is set as verifie Single sender authentication, but when i send email verifications in my localhost, i still receive the same message : The from address does not match a verified Sender Identity my config:  EMAIL_HOST = 'smtp.sendgrid.net' EMAIL_HOST_USER = 'apikey' EMAIL_HOST_PASSWORD = 'your api generate password' EMAIL_PORT = 587 EMAIL_USE_TLS = True 

Answer: You need to add another line to your config containing the verified sender's address: DEFAULT_FROM_EMAIL = 'you@domain.com'
Question: I would like to get the keys from an unbound TypedDict subclass. What is the correct way to do so? Below I have a hacky method, and I'm wondering if there's a more standard way.  Current Method I used inspect.getmembers on the TypedDict subclass, and saw the __annotations__ attribute houses a mapping of the keys + type annotations. From there, I use .keys() to get access to all of the keys. from typing_extensions import TypedDict class SomeTypedDict(TypedDict): key1: str key2: int print(SomeTypedDict.__annotations__.keys())  Prints: dict_keys(['key1', 'key2']) This does work, but I am wondering, is there a better/more standard way?  Versions python==3.6.5 typing-extensions==3.7.4.2 

Answer: The code documentation explicitly states (referring to a sample derived class Point2D):  The type info can be accessed via the Point2D.__annotations__ dict, and the Point2D.__required_keys__ and Point2D.__optional_keys__ frozensets.  So if the modules code says this, there is no reason to look for another method. Note that your method only printed the names of the dictionary keys. You can get the names and the type simply by accessing the full dictionary: print(SomeTypedDict.__annotations__)  Which will get you back all the info: {'key1': <class 'str'>, 'key2': <class 'int'>}
Question: I am creating a 3d pong game using pygame. I wanted to add a thick black border layer to the rectangle to make it more stylish. Here's what I tried: pygame.draw.rect(screen, (0,0,255), (x,y,150,150), 0) pygame.draw.rect(screen, (0,0,0), (x-1,y-1,155,155), 1) pygame.draw.rect(screen, (0,0,0), (x-2,y-2,155,155), 1) pygame.draw.rect(screen, (0,0,0), (x-3,y-3,155,155), 1) pygame.draw.rect(screen, (0,0,0), (x-4,y-4,155,155), 1)  It worked but as the game I am trying to create is a 3d game this method was time consuming. Please tell me if there is any inbuilt method in pygame to draw borders around a rectangle. Sorry for my English.

Answer: You could also put it in a function, and make it more concise with for loops. First, you'll note that the four rectangles you drew were in a nice, easy pattern, so you could compact the drawing of the four rectangles like this: pygame.draw.rect(surface, (0,0,255), (x,y,150,150), 0) for i in range(4): pygame.draw.rect(surface, (0,0,0), (x-i,y-i,155,155), 1)  Then, because pygame draw functions do not need to be run in the global scope, you can put all this into a function: def drawStyleRect(surface): pygame.draw.rect(surface, (0,0,255), (x,y,150,150), 0) for i in range(4): pygame.draw.rect(surface, (0,0,0), (x-i,y-i,155,155), 1)  Then, in your mainloop, all you have to do is run: while not done: ... drawStyleRect(screen) # Or whatever you named the returned surface of 'pygame.display.set_mode()' ...  You could even put the drawing function in a separate module, if you really wanted to.
Question: I encountered it while executing from object_detection.utils import label_map_util in jupyter notebook. It is actually the tensorflow object detection tutorial notebook(it comes with the tensorflow object detection api) The complete error log: AttributeError Traceback (most recent call last) <ipython-input-7-7035655b948a> in <module> 1 from object_detection.utils import ops as utils_ops ----> 2 from object_detection.utils import label_map_util 3 from object_detection.utils import visualization_utils as vis_util ~\AppData\Roaming\Python\Python37\site-packages\object_detection\utils\label_map_util.py in <module> 25 import tensorflow as tf 26 from google.protobuf import text_format ---> 27 from object_detection.protos import string_int_label_map_pb2 28 29 ~\AppData\Roaming\Python\Python37\site-packages\object_detection\protos\string_int_label_map_pb2.py in <module> 19 syntax='proto2', 20 serialized_options=None, ---> 21 create_key=_descriptor._internal_create_key, 22 serialized_pb=b'\n2object_detection/protos/string_int_label_map.proto\x12\x17object_detection.protos\"\xc0\x01\n\x15StringIntLabelMapItem\x12\x0c\n\x04name\x18\x01 \x01(\t\x12\n\n\x02id\x18\x02 \x01(\x05\x12\x14\n\x0c\x64isplay_name\x18\x03 \x01(\t\x12M\n\tkeypoints\x18\x04 \x03(\x0b\x32:.object_detection.protos.StringIntLabelMapItem.KeypointMap\x1a(\n\x0bKeypointMap\x12\n\n\x02id\x18\x01 \x01(\x05\x12\r\n\x05label\x18\x02 \x01(\t\"Q\n\x11StringIntLabelMap\x12<\n\x04item\x18\x01 \x03(\x0b\x32..object_detection.protos.StringIntLabelMapItem' 23 ) AttributeError: module 'google.protobuf.descriptor' has no attribute '_internal_create_key' 

Answer: The protoc version I got through pip show protobuf and protoc --version were different. The version in pip was a bit outdated. After I upgraded the pip version with pip install --upgrade protobuf  the problem was solved.
Question: I am trying to create a virtual environment using mkvirtualenv with python 3 in Windows but the environment is created with python 2.7.My pip version is also from python 2.7 which i have avoided using  py -m pip install virtualenvwrapper-win  When i do mkvirtualenv test  environment is created with python 2.7 Please help me with a solution Thanks in advance:)

Answer: If you would like to create a virtualenv with python 3.X having the version 2.X You just have to pass a parameter argument for your virtual env. $ virtualenv venv -p $(which python3)  This command will point to your current python3 install folder, and create a virtualenv copied from your current python3 binaries. If you would like to see what this command does, just fire the command: $ which python3 #should print your current python3 binary folder.
Question: I read this Stack Overflow post on a similar issue, but the suggestions there don't seem to be working. I installed Visual Studio Code on my Windows machine and added the Python extension. Then I changed the Python path for my project to C:\Users\username\.conda\envs\tom\python.exe. The .vscode/settings.json has this in it: { "python.pythonPath": "C:\\Users\\username\\.conda\\envs\\tom\\python.exe" }  The status bar in Visual Studio Code also shows:  But when I do conda env list even after doing conda activate tom in the terminal I get the output: # conda environments: # base * C:\ProgramData\Anaconda3 tom C:\Users\username\.conda\envs\tom  Instead of: # conda environments: # base C:\ProgramData\Anaconda3 tom * C:\Users\username\.conda\envs\tom  Also the packages not installed in base don't get imported when I try python app.py. What should I do? where python runs, but it doesn't give any output. Also, import os import sys os.path.dirname(sys.executable)  gives 'C:\\Python38' 

Answer: First, open the Anaconda prompt (How to access Anaconda command prompt in Windows 10 (64-bit)), and type: conda activate tom  To activate your virtual environment. Then to open Visual Studio Code in this active environment, type code  And it should work.
Question: i tried to extract the tables from a pdf using camelot but it is showing this error message!  import camelot tables = camelot.read_pdf("C:/Users/shres/Desktop/PY/Arun District Council_ASR-2019.pdf", pages='all') tables tables.export("test.csv", f='csv') tables[0] tables[0].parsing_report { 'accuracy' : 99.02, 'whitespace':12.24, 'order': 1, 'page' : 1 } tables[0].to_csv('test.csv') tables[0].df ******error: the code shows this error****** ImportError: cannot import name 'TableList' from 'camelot.core' (C:\Users\shres\AppData\Local\Programs\Python\Python38\lib\site-packages\camelot\core\__init__.py) 

Answer: you might want to reinstall it. Camelot and camelot-py are two different packages but they have the same import name. pip uninstall camelot pip uninstall camelot-py pip install camelot-py[cv]
Question: Something is puzzling for me when it comes to keeping consumers alive. Let's say I have a topic to which data is constantly being written. But, in an hour in a day, there are no new messages. If I had set a timeout for my consumers, when there are no new messages, the consumer will get closed. Now, new messages arrive. But, there are not consumers alive to consume them.  How should I handle such scenarios? My consumers may consume all messages and get closed. What is the best way to keep them alive? Is there any way to invoke them automatically upon the arrival of new messages? What are the best practices for such scenarios?

Answer: Why not just import time from confluent_kafka import Consumer consumer = Consumer({ 'bootstrap.servers': 'localhost:9092', 'group.id': 'my-consumer-1', 'auto.offset.reset': 'earliest' }) consumer.subscribe(['topicName']) while True: try: message = consumer.poll(10.0) if not message: time.sleep(120) # Sleep for 2 minutes if message.error(): print(f"Consumer error: {message.error()}") continue print(f"Received message: {msg.value().decode('utf-8')}") except: # Handle any exception here ... finally: consumer.close() print("Goodbye")   I cannot comment on the requirement of "setting a timeout for consumers", but in most of the cases consumers are supposed to run "forever" and should also be added to consumer groups in a way that they are highly available.
Question: I'm using in my lambda function the magic library to determine the file`s type. I first deployed it to a local container to check that everything works. My DockerFile :  FROM lambci/lambda:build-python3.8 WORKDIR /app RUN mkdir -p .aws COPY requirements.txt ./ COPY credentials /app/.aws/ RUN mv /app/.aws/ ~/.aws/ RUN pip install --no-cache-dir -r requirements.txt RUN pip install --no-cache-dir -r requirements.txt -t "/app/dependencies/" WORKDIR /app/dependencies RUN zip -r lambda.zip *  requirements.txt :  python-magic libmagic  In my local container when I run tests on the lambda logic everything went ok and passed (including the part that uses the magic code..). I created a zip that contains the lambda.py code and with the python dependencies (last 3 lines in the docker file). When I upload the zip to aws and test the lambda I'm getting the following error :  { "errorMessage": "Unable to import module 'lambda': failed to find libmagic. Check your installation", "errorType": "Runtime.ImportModuleError" }  As you can see, on my local container I'm using baseline image lambci/lambda:build-python3.8 that should be the same aws uses when the lambda is launching. I tried also to add python-magic-bin==0.4.14 to the requirements.txt instead of the magic and libmagic but it didnt help either because it seems that this module is for windows. Into the lambda.zip I put also the lambda.py which is the file that includes my lambda function :  import boto3 import urllib.parse from io import BytesIO import magic def lambda_handler(event, context): s3 = boto3.client("s3") if event: print("Event : ", event) event_data = event["Records"][0] file_name = urllib.parse.unquote_plus(event_data['s3']['object']['key']) print("getting file: {}".format(file_name)) bucket_name = event_data['s3']['bucket']['name'] file_from_s3 = s3.get_object(Bucket=bucket_name, Key=file_name) file_obj = BytesIO(file_from_s3['Body'].read()) print(magic.from_buffer(file_obj.read(2048)))  What am I doing wrong ? 

Answer: While using filetype as suggested by other answers is much simpler, that library does not detect as many file types as magic does. You can make python-magic work on aws lambda with python3.8 by doing the following:  Add libmagic.so.1 to a lib folder at the root of the lambda package. This lib folder will be automatically added to LD_LIBRARY_PATH on aws lambda. This library can be found in /usr/lib64/libmagic.so.1 on an amazon linux ec2 instance for example. Create a magic file or take the one available on an amazon linux ec2 instance in /usr/share/misc/magic and add it to your lambda package. The Magic constructor from python-magic takes a magic_file argument. Make this point to your magic file. You can then create the magic object with magic.Magic(magic_file='path_to_your_magic_file') and then call any function from python-magic you like on that object.  These steps are not necessary on the python3.7 runtime as those libraries are already present in aws lambda.
Question: I'm working with the librosa library, and I would like to know what information is returned by the librosa.load function when I read a audio (.wav) file. Is it the instantaneous sound pressure in pa, or the just the instantaneous amplitude of the sound signal with no units?

Answer: To confirm the previous answer, librosa.load returns a time series that in librosa glossary is defined as: "time series: Typically an audio signal, denoted by y, and represented as a one-dimensional numpy.ndarray of floating-point values. y[t] corresponds to the amplitude of the waveform at sample t." The amplitude is usually measured as a function of the change in pressure around the microphone or receiver device that originally picked up the audio. (See more here).
Question: I am unable to understand the difference between the two. Though, I come to know that word_tokenize uses Penn-Treebank for tokenization purposes. But nothing on TweetTokenizer is available. For which sort of data should I be using TweetTokenizer over word_tokenize?

Answer: Well, both tokenizers almost work the same way, to split a given sentence into words. But you can think of TweetTokenizer as a subset of word_tokenize. TweetTokenizer keeps hashtags intact while word_tokenize doesn't. I hope the below example will clear all your doubts... from nltk.tokenize import TweetTokenizer from nltk.tokenize import word_tokenize tt = TweetTokenizer() tweet = "This is a cooool #dummysmiley: :-) :-P <3 and some arrows < > -> <-- @remy: This is waaaaayyyy too much for you!!!!!!" print(tt.tokenize(tweet)) print(word_tokenize(tweet)) # output # ['This', 'is', 'a', 'cooool', '#dummysmiley', ':', ':-)', ':-P', '<3', 'and', 'some', 'arrows', '<', '>', '->', '<--', '@remy', ':', 'This', 'is', 'waaaaayyyy', 'too', 'much', 'for', 'you', '!', '!', '!'] # ['This', 'is', 'a', 'cooool', '#', 'dummysmiley', ':', ':', '-', ')', ':', '-P', '<', '3', 'and', 'some', 'arrows', '<', '>', '-', '>', '<', '--', '@', 'remy', ':', 'This', 'is', 'waaaaayyyy', 'too', 'much', 'for', 'you', '!', '!', '!', '!', '!', '!']  You can see that word_tokenize has split #dummysmiley as '#' and 'dummysmiley', while TweetTokenizer didn't, as '#dummysmiley'. TweetTokenizer is built mainly for analyzing tweets. You can learn more about tokenizer from this link