eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I get this flycheck error pointing to the runtime=_lambda.. variable: Argument of type "(cls: Runtime) -> Runtime" cannot be assigned to parameter "runtime" of type "Runtime" in function "__init__" Type "(cls: Runtime) -> Runtime" cannot be assigned to type "Runtime" # create lambda function # executed as root function = _lambda.Function(self, "lambda_function", runtime=_lambda.Runtime.PYTHON_3_7, handler="lambda_handler.main", code=_lambda.Code.from_asset("./lambda"), environment={ 'EC2_INSTANCE_ID': instance.instance_id, 'S3_OUTPUT': output_s3.s3_url_for_object(), 'S3_INPUT': input_s3.s3_url_for_object() })  It's a rather cosmetic IDE issue, the code itself works

Answer: This was a bug in jsii, the library CDK uses to transpile TypeScript (the language in which CDK is written) to Python. Here is the PR that fixed it. The fix was released in 1.64.0 If you are using a version before 1.64.0, you can use casting to suppress the error: import typing ... function = lambda_.Function( self, "function", ... runtime=typing.cast(lambda_.Runtime, lambda_.Runtime.PYTHON_3_7) )  Or just append # type: ignore to the end of the line to disable type checking on that particular line.
Question: Recently when setting up a breakpoint using ipdb.set_trace(context=20) I can see the command I'm inputing the first time, after hitting return, next time I write an instruction or command in my ipdb prompt is not showing. When I hit enter it executes it and shows it in the previous lines. This wasn't happening until very recently. I'm using mac, with iterm, latest ipdb and pytest. EDIT 2022-3-29  I've been trying to play with the shell settings, disconnect ozsh, antigen plugins, to see it was related, but doesn't seem to affect.  I've also tried with terminal, instead of iterm.  Here is a recording of what I'm describing:    EDIT 2022-3-31  I've realized this only happens with one of my projects The prompt disappears after an exception occurs no matter which type, otherwise it always works fine. After the exception prompt starts failing, but sometimes it's not in the first command after I've written a simple python program to run with the same setup and it doesn't happen, so there's something else messing with this  EDIT 2022-3-31 (2.0)  After spending some time playing with this, I discovered this was only happening in some tests, the ones decorated with freezegun  I'm using freezegun 1.2.1 and pytest 6.2.5. When I run this code if I execute print a couple times, cursor disappears. This is the most basic reproduction test I've been able to come up with.  import ipdb from freezegun import freeze_time @freeze_time("2022-3-12") def test_prompt_ipdb(): ipdb.set_trace() test_prompt_ipdb()  I now believe this a bug in one of these two, most likely freezegun doing something fancy.

Answer: This doesn't seem like a bug in ipdb (nor in IPython for that matter, with which this reproduces as well). The problem is between freezegun and prompt-toolkit, which IPython (and consequently ipdb) rely on. I'm hoping they will accept this PR, but until then this behavior can be resolved by adding prompt_toolkit to the ignore-list using the extend_ignore_list argument, like so: import ipdb import freezegun freezegun.configure(extend_ignore_list=['prompt_toolkit']) @freezegun.freeze_time("2022-3-12") def test_prompt_ipdb(): ipdb.set_trace() test_prompt_ipdb()
Question: Any ideas on why I get this error? My project was working fine. I copied it to an external drive and onto my laptop to work on the road; it worked fine. I copied it back to my desktop and had a load of issues with invalid interpreters etc, so I made a new project and copied just the scripts in, made a new requirements.txt and installed all the packages, but when I run it, I get this error: Traceback (most recent call last): File "E:\Dev\spot_new\flask_blog\run.py", line 1, in <module> from flaskblog import app File "E:\Dev\spot_new\flask_blog\flaskblog\__init__.py", line 3, in <module> from flask_bcrypt import Bcrypt File "E:\Dev\spot_new\venv\lib\site-packages\flask_bcrypt.py", line 21, in <module> from werkzeug.security import safe_str_cmp ImportError: cannot import name 'safe_str_cmp' from 'werkzeug.security' (E:\Dev\spot_new\venv\lib\site-packages\werkzeug\security.py)  I've tried uninstalling Python, Anaconda, PyCharm, deleting every reg key and environment variable I can find that looks pythonic, reinstalling all from scratch but still no dice.

Answer: Werkzeug released v2.1.0 today, removing werkzeug.security.safe_str_cmp. You can probably resolve this issue by pinning Werkzeug~=2.0.0 in your requirements.txt file (or similar). pip install Werkzeug~=2.0.0  After that it is likely that you will also have an AttributeError related to the jinja package, so if you have it, also run: pip install jinja2~=3.0.3
Question: I'm trying to run a simple dash app in a conda environment in Pycharm, however I'm running into the error in the title. Weirdly enough, I couldn't find a place on the internet which has a mention of this bug, except for here. The code is simple, as all I'm trying to run is a simple dashapp; code obtained the code from here. I have tried switching between python versions in conda (back and forth between python 3.9, 3.8 and 3.7) but the error seems to be persistent. I know I have also correctly installed all its dependencies as I'm not getting any import error. Would appreciate if anyone could help with this. Edit: Versions of Dash installed, as requested by @coralvanda :  Basically, I just did a pip install of everything so all the versions of packages are the latest. Screenshot of a full traceback of the error: 

Answer: I've been in the same problem. Uninstall the wrong version with: pip uninstall werkzeug  Install the right one with: pip install -v https://github.com/pallets/werkzeug/archive/refs/tags/2.0.3.tar.gz
Question: Below is my code- Elasticsearch is not using https protocol, it's using http protocol. pip uninstall elasticsearch pip install elasticsearch==7.13.4 import elasticsearch.helpers from elasticsearch import Elasticsearch # from elasticsearch import Elasticsearch, RequestsHttpConnection es_host = '<>' es_port = '<>' es_username = '<>' es_password = '><' es_index = '<>' es = Elasticsearch([{'host':str(es_host),'port':str(es_port)}], http_auth=(str(es_username), str(es_password))) es.indices.refresh(index=es_index)  Error- 10 es = Elasticsearch([{'host': str(es_host), 'port': str(es_port)}],http_auth=(str(es_username), str(es_password))) 11 12 es.indices.refresh(index=es_index) 3 frames /usr/local/lib/python3.7/dist-packages/elasticsearch/_sync/client/__init__.py in __init__(self, hosts, cloud_id, api_key, basic_auth, bearer_auth, opaque_id, headers, connections_per_node, http_compress, verify_certs, ca_certs, client_cert, client_key, ssl_assert_hostname, ssl_assert_fingerprint, ssl_version, ssl_context, ssl_show_warn, transport_class, request_timeout, node_class, node_pool_class, randomize_nodes_in_pool, node_selector_class, dead_node_backoff_factor, max_dead_node_backoff, serializer, serializers, default_mimetype, max_retries, retry_on_status, retry_on_timeout, sniff_on_start, sniff_before_requests, sniff_on_node_failure, sniff_timeout, min_delay_between_sniffing, sniffed_node_callback, meta_header, timeout, randomize_hosts, host_info_callback, sniffer_timeout, sniff_on_connection_fail, http_auth, maxsize, _transport) /usr/local/lib/python3.7/dist-packages/elasticsearch/_sync/client/utils.py in client_node_configs(hosts, cloud_id, **kwargs) /usr/local/lib/python3.7/dist-packages/elasticsearch/_sync/client/utils.py in hosts_to_node_configs(hosts) /usr/local/lib/python3.7/dist-packages/elasticsearch/_sync/client/utils.py in host_mapping_to_node_config(host) TypeError: __init__() missing 1 required positional argument: 'scheme'  When I add "scheme" Code- es = Elasticsearch([{'host':str(es_host),'port':str(es_port)}], http_auth=(str(es_username), str(es_password)), scheme="http",verify_certs=False)  Error- __init__() got an unexpected keyword argument 'scheme'  I checked and tried connection to ES but its not connecting.

Answer: I ran into a similar error. I am using elasticsearch==8.3.1. When you construct your url with the list of dictionaries, you need to define the schema. Add "scheme": "https" to your dictionary and that will solve the missing argument. es = Elasticsearch( [ {'host': 'localhost', 'port': '9200', "scheme": "https"} ], basic_auth=('elastic', '<password>') )  In your case, you should convert your instantiation to as follows: es = Elasticsearch( [ { 'host':str(es_host), 'port':str(es_port), 'scheme': "https" } ], http_auth=(str(es_username), str(es_password)) )  I am not sure if the scheme is http or https, that is something you'll need to dig into.
Question: I get two deprecation warnings whenever I try running any python code. They are: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. MIN_CHEMFILES_VERSION = LooseVersion("0.9") DeprecationWarning: Please use netcdf_file from the scipy.io namespace, the scipy.io.netcdf namespace is deprecated. I am not sure how to use packaging.version instead of distuils and netcdf file. I am running python 3.8. I tried updating my virtualenv as suggested here: DeprecationWarning in Python 3.6 and 3.7 (with Pillow, distutils, imp) This doesn't work for me. Any help will be appreciated. I could not find results for the second deprecation warning.

Answer: The DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.  message is caused by the distutils module being overridden since setuptools 60.0.0. You would notice because the distutils.__file__ variable evals to .../site-packages/setuptools/_distutils/__init__.py, regardless of your python interpreter. Using any python 3.8 or 3.9 will not fix the warning unless your installation features setuptools<60. And even then, the warning can still be triggered by setting SETUPTOOLS_USE_DISTUTILS=local in the environment. See the breaking changes section in https://github.com/pypa/setuptools/blob/main/CHANGES.rst#v6000 Workarounds to suppress this warnings can be either to downgrade to setuptools<60 or to start your python interpreter with the SETUPTOOLS_USE_DISTUTILS=stdlib environment variable. But I discourage any of these approaches, be ready for worse side effects if you go this way. Simply embrace the warning and try to contribute actual patches fixing this for any upstream library you were using before python 3.12 is released.
Question: I am trying to implement paddleocr. I have installed it using: #Github repo installation for paddle ! python3 -m pip install paddlepaddle -i https://mirror.baidu.com/pypi/simple #install paddle ocr !pip install paddleocr !git clone https://github.com/PaddlePaddle/PaddleOCR.git  But while importing from paddleocr import PaddleOCR,draw_ocr  I'm getting this error: ImportError: cannot import name 'inference' from 'paddle'

Answer: I had the same error. My solution was to: pip install paddlepaddle  Then I got another error (luckily you will not get this one but just in case) telling me to downgrade protoc to a version between 3.19 and 3.20, which I fixed by executing the following command: pip install protobuf==3.19.0  After this I was able to execute the script that imported from paddleocr
Question: I'm loading a json file on my computer. I can load it in without specifying the encoding on Kaggle, no, errors. On my PC I get the error in the title. with open('D:\soccer\statsbomb360\matches.json') as f: data = json.load(f, encoding = 'utf8')  Adding errors = 'ignore' or changing encoding to 'latin' doesn't work either. I'm a bit lost on what to try next, can you give me an idea? The json is from statsbombs freely available data. Interestingly from the same dataset I have some files that give me this error on Kaggle/Colab but not on my pc, but there specifying encoding = 'latin' did the trick. thank you!

Answer: Try with open('D:\soccer\statsbomb360\matches.json', encoding="utf8") as f: data = json.load(f)  per @mark-tolonen Also see this post: UnicodeDecodeError: 'charmap' codec can't decode byte X in position Y: character maps to <undefined>
Question: I am using faiss indexflatIP to store vectors related to some words. I also use another list to store words (the vector of the nth element in the list is nth vector in faiss index). I have two questions:  Is there a better way to relate words to their vectors? Can I update the nth element in the faiss? 

Answer: You can do both.   Is there a better way to relate words to their vectors?   Call index.add_with_ids(vectors, ids) Some index types support the method add_with_ids, but flat indexes don't. If you call the method on a flat index, you will receive the error add_with_ids not implemented for this type of index If you want to use IDs with a flat index, you must use index2 = faiss.IndexIDMap(index)   Can I update the nth element in the faiss?   If you want to update some encodings, first remove them, then add them again with add_with_ids If you don't remove the original IDs first, you will have duplicates and search results will be messed up. To remove an array of IDs, call index.remove_ids(ids_to_replace) Nota bene: IDs must be of np.int64 type.
Question: With the pdfplumber library, you can extract the text of a PDF page, or you can extract the tables from a pdf page. The issue is that I can't seem to find a way to extract text and tables. Essentially, if the pdf is formatted in this way: text1 tablename ___________ | Header 1 | ------------ | row 1 | ------------ text 2  I would like the output to be: ["text 1", "table name", [["header 1"], ["row 1"]], "text 2"]  In this example you could run extract_text from pdfplumber: with pdfplumber.open("example.pdf") as pdf: for page in pdf.pages: page.extract_text()  but that extracts text and tables as text. You could run extract_tables, but that only gives you the tables. I need a way to extract both text and tables at the same time. Is this built into the library some way that I don't understand? If not, is this possible? Edit: Answered This comes directly from the accepted answer with a slight tweak to fix it. Thanks so much! from operations import itemgetter def check_bboxes(word, table_bbox): """ Check whether word is inside a table bbox. """ l = word['x0'], word['top'], word['x1'], word['bottom'] r = table_bbox return l[0] > r[0] and l[1] > r[1] and l[2] < r[2] and l[3] < r[3] tables = page.find_tables() table_bboxes = [i.bbox for i in tables] tables = [{'table': i.extract(), 'top': i.bbox[1]} for i in tables] non_table_words = [word for word in page.extract_words() if not any( [check_bboxes(word, table_bbox) for table_bbox in table_bboxes])] lines = [] for cluster in pdfplumber.utils.cluster_objects( non_table_words + tables, itemgetter('top'), tolerance=5): if 'text' in cluster[0]: lines.append(' '.join([i['text'] for i in cluster])) elif 'table' in cluster[0]: lines.append(cluster[0]['table'])  Edit July 19th 2022: Updated a param to include itemgetter, which is now required for pdfplumber's cluster_objects function (rather than a string)

Answer: You can get tables' bounding boxes and then filter out all of the words inside them, something like this: def check_bboxes(word, table_bbox): """ Check whether word is inside a table bbox. """ l = word['x0'], word['top'], word['x1'], word['bottom'] r = table_bbox return l[0] > r[0] and l[1] > r[1] and l[2] < r[2] and l[3] < r[3] tables = page.find_tables() table_bboxes = [i.bbox for i in tables] tables = [{'table': i.extract(), 'doctop': i.bbox[1]} for i in tables] non_table_words = [word for word in page.extract_words() if not any( [check_bboxes(word, table_bbox) for table_bbox in table_bboxes])] lines = [] for cluster in pdfplumber.utils.cluster_objects(non_table_words+tables, 'doctop', tolerance=5): if 'text' in cluster[0]: lines.append(' '.join([i['text'] for i in cluster])) elif 'table' in cluster[0]: lines.append(cluster[0]['table'])
Question: I am working on a CNN Sentiment analysis machine learning model which uses the IMDb dataset provided by the Torchtext library. On one of my lines of code vocab = Vocab(counter, min_freq = 1, specials=('\<unk\>', '\<BOS\>', '\<EOS\>', '\<PAD\>')) I am getting a TypeError for the min_freq argument even though I am certain that it is one of the accepted arguments for the function. I am also getting UserWarning Lambda function is not supported for pickle, please use regular python function or functools partial instead. Full code from torchtext.datasets import IMDB from collections import Counter from torchtext.data.utils import get_tokenizer from torchtext.vocab import Vocab tokenizer = get_tokenizer('basic_english') train_iter = IMDB(split='train') test_iter = IMDB(split='test') counter = Counter() for (label, line) in train_iter: counter.update(tokenizer(line)) vocab = Vocab(counter, min_freq = 1, specials=('\<unk\>', '\<BOS\>', '\<EOS\>', '\<PAD\>'))  Source Links towardsdatascience github Legacy to new I have tried removing the min_freq argument and use the functions default as follows vocab = Vocab(counter, specials=('\<unk\>', '\<BOS\>', '\<EOS\>', '\<PAD\>')) however I end up getting the same type error but for the specials argument rather than min_freq. Any help will be much appreciated Thank you.

Answer: As https://github.com/pytorch/text/issues/1445 mentioned, you should change "Vocab" to "vocab". I think they miss-type the legacy-to-new notebook. correct code: from torchtext.datasets import IMDB from collections import Counter from torchtext.data.utils import get_tokenizer from torchtext.vocab import vocab tokenizer = get_tokenizer('basic_english') train_iter = IMDB(split='train') test_iter = IMDB(split='test') counter = Counter() for (label, line) in train_iter: counter.update(tokenizer(line)) vocab = vocab(counter, min_freq = 1, specials=('\<unk\>', '\<BOS\>', '\<EOS\>', '\<PAD\>'))  my environment:  python 3.9.12 torchtext 0.12.0 pytorch 1.11.0
Question: I am compiling some existing examples from the PyTorch tutorial website. I am working especially on the CPU device no GPU. When running a program the type of error below is shown. Does it become I'm working on the CPU device or setup issue? raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str)) from e RuntimeError: DataLoader worker (pid(s) 15876, 2756) exited unexpectedly`. How can I solve it? import torch import torch.functional as F import torch.nn as nn import torch.optim as optim import torchvision import torchvision.transforms as transforms import matplotlib.pyplot as plt import numpy as np from torch.utils.tensorboard import SummaryWriter from torch.utils.data import DataLoader from torchvision import datasets device = 'cpu' if torch.cuda.is_available() else 'cuda' print(device) transform = transforms.Compose( [transforms.ToTensor(), transforms.Normalize((0.5,), (0.5,))] ) #Store separate training and validations splits in data training_set = datasets.FashionMNIST( root='data', train=True, download=True, transform=transform ) validation_set = datasets.FashionMNIST( root='data', train=False, download=True, transform=transform ) training_loader = DataLoader(training_set, batch_size=4, shuffle=True, num_workers=2) validation_loader = DataLoader(validation_set, batch_size=4, shuffle=False, num_workers=2) classes = ('T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle Boot') def matplotlib_imshow(img, one_channel=False): if one_channel: img = img.mean(dim=0) img = img/2+0.5 #unnormalize npimg = img.numpy() if one_channel: plt.imshow(npimg, cmap="Greys") else: plt.imshow(np.transpose(npimg, (1, 2, 0))) dataiter = iter(training_loader) images, labels = dataiter.next() img_grid = torchvision.utils.make_grid(images) matplotlib_imshow(img_grid, one_channel=True) 

Answer: You need to first figure out why the dataLoader worker crashed. A common reason is out of memory. You can check this by running dmesg -T after your script crashes and see if the system killed any python process.
Question: Has anyone ever had a problem that Jupyter is busy (stuck) when executing input() inside the while statement? The problem is randomly happening to me. Sometimes the command box is prompted next to the cell, and sometimes the input() box never prompts. Here is the simpler version of my code: from IPython.display import clear_output class game_quit(Exception): def _render_traceback_(self): pass def the_game(player_status = "Offering"): while True: clear_output() print("The Game", flush = True) print(f"Status: {player_status}", flush = True) if (player_status == "Offering") or (player_status == "Wrong input!"): play_offer = input("Would you like to play the game (Y or N)? ") if play_offer.upper() == "Y": player_status = "Playing" play_accepted = True clear_output() break elif play_offer.upper() == "N": play_accepted = False clear_output() break else: player_status = "Wrong input!" clear_output() continue else: player_status = "Playing" play_accepted = True clear_output() break while play_accepted: the_play(player_status) else: raise game_quit() def the_play(player_status): while True: clear_output() print("The Game", flush = True) print(f"Status: {player_status}", flush = True) pet_offer = input("Do you want to go with a (D)og, a (C)at, or a (P)arakeet? ") if pet_offer.upper() == "D": player_pet = "Dog" clear_output() break elif pet_offer.upper() == "C": player_pet = "Cat" clear_output() break elif pet_offer.upper() == "P": player_pet = "Parakeet" clear_output() break else: player_status = "Wrong input!" clear_output() continue while pet_offer: clear_output() print(f"Your companion is a {player_pet}", flush = True) play_again = input("Would you like to continue playing the game (Y or N)? ") if play_again.upper() == "Y": play_continue = True clear_output() break elif play_again.upper() == "N": play_continue = False clear_output() break else: player_status = "Wrong input!" clear_output() continue if play_continue: player_status = "Playing" the_game(player_status) else: raise game_quit()  Step to reproduce the problem:  Execute the code.  the_game()   The user wants to play the game.  play_offer = input("Would you like to play the game (Y or N)? ")   The user chooses his pet.  pet_offer = input("Do you want to go with a (D)og, a (C)at, or a (P)arakeet? ")   The user wants to replay the game.  play_again = input("Would you like to continue playing the game (Y or N)? ")   The user should expect the text box to choose his pet.  pet_offer = input("Do you want to go with a (D)og, a (C)at, or a (P)arakeet? ")   A  The problem: The text box is not showing. The running code is stuck there.   B  But sometimes: The text box is showing, and the user can choose his pet.  So far, my only solution is to restart the kernel. Has anyone had a solution for this kind of problem? Regards, Ade

Answer: Someone gave me an insight. The problem is because clear_output() asynchronous problem. Then I created this function: from time import sleep from IPython.display import clear_output def refresh_screen(): clear_output() sleep(0.02)  and replaced all clear_output() with refresh_screen() in my code. The problem is gone. A little delay solved the problem.