eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: My repo contains a main.py that generates a html map and save results in a csv. I want the action to:  execute the python script (-> this seems to be ok) that the file generated would then be in the repo, hence having the file generated to be added, commited and pushed to the main branch to be available in the page associated with the repo.   name: refresh map on: schedule: - cron: "30 11 * * *" #runs at 11:30 UTC everyday jobs: getdataandrefreshmap: runs-on: ubuntu-latest steps: - name: checkout repo content uses: actions/checkout@v3 # checkout the repository content to github runner. - name: setup python uses: actions/setup-python@v4 with: python-version: 3.8 #install the python needed - name: Install dependencies run: | if [ -f requirements.txt ]; then pip install -r requirements.txt; fi - name: execute py script uses: actions/checkout@v3 run: | python main.py git config user.name github-actions git config user.email github-actions@github.com git add . git commit -m "crongenerated" git push  The github-action does not pass when I include the 2nd uses: actions/checkout@v3 and the git commands. Thanks in advance for your help

Answer: If you want to run a script, then you don't need an additional checkout step for that. There is a difference between steps that use workflows and those that execute shell scripts directly. You can read more about it here. In your configuration file, you kind of mix the two in the last step. You don't need an additional checkout step because the repo from the first step is still checked out. So you can just use the following workflow: name: refresh map on: schedule: - cron: "30 11 * * *" #runs at 11:30 UTC everyday jobs: getdataandrefreshmap: runs-on: ubuntu-latest steps: - name: checkout repo content uses: actions/checkout@v3 # checkout the repository content to github runner. - name: setup python uses: actions/setup-python@v4 with: python-version: 3.8 #install the python needed - name: Install dependencies run: | if [ -f requirements.txt ]; then pip install -r requirements.txt; fi - name: execute py script run: | python main.py git config user.name github-actions git config user.email github-actions@github.com git add . git commit -m "crongenerated" git push  I tested it with a dummy repo and everything worked.
Question: Given an arbitrary numpy array (its size and shape don't seem to play a role) import numpy as np a = np.array([1.]) print(a.dtype) # float64  it changes its dtype if you multiply it with a number equal or larger than 10**20 print((a*10**19).dtype) # float64 print((a*10**20).dtype) # object a *= 10**20 # Throws TypeError: ufunc 'multiply' output (typecode 'O') # could not be coerced to provided output parameter (typecode 'd') # according to the casting rule ''same_kind'' a *= 10.**20 # Throws numpy.core._exceptions._UFuncOutputCastingError: # Cannot cast ufunc 'multiply' output from dtype('float64') to # dtype('int32') with casting rule 'same_kind'  However, this doesn't happen if you multiply element-wise a[0] *= 10**20 print(a, a.dtype) # [1.e+20] float64  or specifically convert the number to a float (or int) a *= float(10**20) print(a, a.dtype) # [1.e+20] float64   Just for the record, if you do the multiplication outside of numpy, there are no issues b = 1. print(type(b), type(10**20), type(10.**20)) # float int float b *= 10**20 print(type(b)) # float 

Answer: I expect it is the size a "natural" integer can take on the system. print(sys.maxsize, sys.getsizeof(sys.maxsize)) => 9223372036854775807 36 print(10**19, sys.getsizeof(10**19)) => 10000000000000000000 36  And this is where on my system the conversion to object starts, when I do for i in range(1, 24): print(f'type of a*10**{i}:', (a * 10**i).dtype)  I do expect it is linked to the implementation of the integer:  PEP 0237: Essentially, long renamed to int. That is, there is only one built-in integral type, named int; but it behaves mostly like the old long type.  See https://docs.python.org/3.1/whatsnew/3.0.html#integers To notice this, one could use numpy.multiply with a forced output type. This will throw an error and not silently convert (similar to your *= example).
Question: When I try to run the quick start notebook of this repo, I get the error ModuleNotFoundError: No module named 'huggingface_hub.snapshot_download'. How can I fix it? I already installed huggingface_hub using pip. I get the error after compiling the following cell: !CUDA_VISIBLE_DEVICES=0 python -u ../scripts/main.py --summarizer gpt3_summarizer --controller longformer_classifier longformer_classifier --loader alignment coherence --controller-load-dir emnlp22_re3_data/ckpt/relevance_reranker emnlp22_re3_data/ckpt/coherence_reranker --controller-model-string allenai/longformer-base-4096 allenai/longformer-base-4096 --save-outline-file output/outline0.pkl --save-complete-file output/complete_story0.pkl --log-file output/story0.log  Here's the entire output: Traceback (most recent call last): File "../scripts/main.py", line 20, in <module> from story_generation.edit_module.entity import * File "/home/jovyan/emnlp22-re3-story-generation/story_generation/edit_module/entity.py", line 20, in <module> from story_generation.common.util import * File "/home/jovyan/emnlp22-re3-story-generation/story_generation/common/util.py", line 13, in <module> from sentence_transformers import SentenceTransformer File "/opt/conda/lib/python3.8/site-packages/sentence_transformers/__init__.py", line 3, in <module> from .datasets import SentencesDataset, ParallelSentencesDataset File "/opt/conda/lib/python3.8/site-packages/sentence_transformers/datasets/__init__.py", line 3, in <module> from .ParallelSentencesDataset import ParallelSentencesDataset File "/opt/conda/lib/python3.8/site-packages/sentence_transformers/datasets/ParallelSentencesDataset.py", line 4, in <module> from .. import SentenceTransformer File "/opt/conda/lib/python3.8/site-packages/sentence_transformers/SentenceTransformer.py", line 25, in <module> from .evaluation import SentenceEvaluator File "/opt/conda/lib/python3.8/site-packages/sentence_transformers/evaluation/__init__.py", line 5, in <module> from .InformationRetrievalEvaluator import InformationRetrievalEvaluator File "/opt/conda/lib/python3.8/site-packages/sentence_transformers/evaluation/InformationRetrievalEvaluator.py", line 6, in <module> from ..util import cos_sim, dot_score File "/opt/conda/lib/python3.8/site-packages/sentence_transformers/util.py", line 407, in <module> from huggingface_hub.snapshot_download import REPO_ID_SEPARATOR ModuleNotFoundError: No module named 'huggingface_hub.snapshot_download' 

Answer: Updating to the latest version of sentence-transformers fixes it (no need to install huggingface-hub explicitly): pip install -U sentence-transformers  I've proposed a pull request for this in the original repo.
Question: from pydantic import BaseModel class A(BaseModel): date = '' class B(A): person: float def __init__(self): self.person = 0 B()  tried to initiate class B but raised error AttributeError: 'B' object has no attribute '__fields_set__', why is it?

Answer: It's because you override the __init__ and do not call super there so Pydantic cannot do it's magic with setting proper fields. With pydantic it's rare you need to implement your __init__ most cases can be solved different way: from pydantic import BaseModel class A(BaseModel): date = "" class B(A): person: float = 0 B()
Question: My goal is to learn a notebook. It has recall 97% while I am struggling with F1 Score 'Attrited Customer' 77.9%. The problem is the notebook uses LightGBM. I am unable to install LightGBM. What I've tried:  pip install lightgbm -> it throws error python setup.py egg_info did not run successfully. Then, I did pip install wheel -> now it throws error python setup.py bdist_wheel did not run successfully. Then, I did pip install Cmake, pip install --upgrade pip setuptools, brew install libomp -> the error persisted.  The full error  × python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [80 lines of output] INFO:root:running bdist_wheel /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools. warnings.warn( INFO:root:running build INFO:root:running build_py INFO:root:creating build INFO:root:creating build/lib INFO:root:creating build/lib/lightgbm 

Answer: As of this writing, no official release of lightgbm (the Python package for LightGBM) supports the M1 Macs (which us ARM chips). osx-arm64 builds of lightgbm are supported by the lightgbm conda-forge feedstock, so you can install lightgbm on an M1 Mac using conda. conda install \ --yes \ -c conda-forge \ 'lightgbm>=3.3.3'  Progress towards officially supporting M1 Mac builds of LightGBM can be tracked in microsoft/LightGBM#5269 and microsoft/LightGBM#5328.
Question: According to pep-0585 for the latest versions of Python, it appears we can use List and list interchangeably for type declarations. So which should I use? Assume:  no requirement for backward compatibility using the latest version of python  from typing import List def hello_world_1(animals: list[str]) -> list[str]: return animals def hello_world_2(animals: List[str]) -> List[str]: return animals  How can I set up a python linter to enforce consistency and only allow either upper or lowercase List for example?

Answer: According to the current Python docs, typing.List and similar are deprecated. The docs further state  The deprecated types will be removed from the typing module in the first Python version released 5 years after the release of Python 3.9.0. See details in PEP 585—Type Hinting Generics In Standard Collections.  Concerning enforcing consistency, it says  It is expected that type checkers will flag the deprecated types when the checked program targets Python 3.9 or newer.  However, I use Pylint personally (not a type checker, admittedly) and I don't believe I've noticed it flagging the deprecated types yet.
Question: I have the following folder structure in my project my_project notebook |-- some_notebook.ipynb src |-- preprocess |-- __init__.py |-- some_processing.py __init__.py  Now, inside some_notebook.ipynb I simply want to get the methods from some_processing.py. Now we I run from src.preprocess import some_processing  from some_notebook.ipynb it always throws ModuleNotFoundError: No module named 'src'  I found multiple questions regarding this and played around with sys.path.append(<path-to-src>). But I couldn't solve it. Which path do I provide? Something like ../src didnt work? I checked for example the AlphaFold project from DeepMind and they are using it also with this structure. I tried to replicate exactly like they did. How can I solve this? Which path do I provide in sys.path.append()? I appreciate any help!

Answer: I found the answer. Running sys.path.insert(1, os.path.join(sys.path[0], '../src'))  made it possible to import anything from parent module src.
Question: I want to restrict the possible input arguments by using typing.Literal. The following code works just fine, however, mypy is complaining. from typing import Literal def literal_func(string_input: Literal["best", "worst"]) -> int: if string_input == "best": return 1 elif string_input == "worst": return 0 literal_func(string_input="best") # works just fine with mypy # The following call leads to an error with mypy: # error: Argument "string_input" to "literal_func" has incompatible type "str"; # expected "Literal['best', 'worst']" [arg-type] input_string = "best" literal_func(string_input=input_string) 

Answer: Unfortunately, mypy does not narrow the type of input_string to Literal["best"]. You can help it with a proper type annotation: input_string: Literal["best"] = "best" literal_func(string_input=input_string)  Perhaps worth mentioning that pyright works just fine with your example.  Alternatively, the same can be achieved by annotating the input_string as Final: from typing import Final, Literal ... input_string: Final = "best" literal_func(string_input=input_string)
Question: I know that I can to the following. But is it possible to combine them to give me a percent with fixed decimal? >>> print(f'{0.123:%}') 12.300000% >>> print(f'{0.123:.2f}') 0.12  But what I want is this output: 12.30% 

Answer: You can specify the number of decimal places before %: >>> f'{0.123:.2%}' '12.30%'
Question: I am using nbconvert for converting my .ipynb into an .pdf file. When doing so the resulting .pdf file contains a header with the filename and the current date below. How can I disable that? I was looking in the docs but cannot find how to do it. CLI command jupyter nbconvert --to pdf filename.ipynb  Actual  Wanted 

Answer: I found some helpful pointer in the docs. Just follow these steps:  Run jupyter --paths in your command-line. Copy the path who looks like /Users/username/.venv/venvName/share/jupyter (I run nbconvert from a venv. Could be different for you). Go to the path and duplicate the folder latex Name the folder hide_header or whatever you want Open base.tex.j2 and delete the line ((* block maketitle *))\maketitle((* endblock maketitle *)) Save run jupyter nbconvert --to pdf filename.ipynb --template=hide_header
Question: I'm trying to set up a simple server/client connection using the socket module on a Raspberry Pi Pico W running the latest nightly build image rp2-pico-w-20221123-unstable-v1.19.1-713-g7fe7c55bb.uf2 which I've downloaded from https://micropython.org/download/rp2-pico-w/ The following code runs fine for the first connection (network connectivity can be assumed at this point). import socket def await_connection(): print(' >> Awaiting connection ...') try: host_addr = socket.getaddrinfo('0.0.0.0', 46321)[0][-1] sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.bind(host_addr) sock.listen(1) con, addr = sock.accept() while True: # keep receiving commands on the open connection until the client stops sending cmd = con.recv(1024) if not cmd: print(f' >> {addr} disconnected') break elif cmd.decode() == 'foo': response = 'bar'.encode() else: response = cmd print(f"Received {cmd.decode()}") print(f"Returning {response.decode()}") con.sendall(response) except OSError as e: print(f' >> ERROR: {e}') finally: # appearantly, context managers are currently not supported in MicroPython, therefore the connection is closed manually con.close() print(' >> Connection closed.') while True: # main loop, causing the program to await a new connection as soon as the previous one is closed await_connection()  If the client closes the connection and tries to re-connect, the infamous [Errno 98] EADDRINUSE is thrown:  Please note that I've already implemented the sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) statement, as recommended here, but to no avail. However, if I run the exact same code on a Raspberry Pi 3 B with python 3.7.3 within the same network, everything works as expected - the client can disconnect and reconnect multiple times without issues:  How do I get the Pico to reuse the address after the initial connection, just like it is working in python 3.7.3?

Answer: While I was able to mitigate the reconnection crash by adding a sock.close() statement after the con.close(), the main issue with my code was the structure itself, as Steffen Ullrich pointed out. The actual fix was to move the operations on the sock object out of the loop. import socket def await_connection(): print(' >> Awaiting connection ...') try: con, addr = sock.accept() while True: # keep receiving commands on the open connection until the client stops sending cmd = con.recv(1024) if not cmd: print(f' >> {addr} disconnected') break else: response = cmd print(f"Received {cmd.decode()}") print(f"Returning {response.decode()}") con.sendall(response) except OSError as e: print(f' >> ERROR: {e}') finally: # appearantly, context managers are currently not supported in MicroPython, therefore the connection is closed manually con.close() print(' >> Connection closed.') host_addr = socket.getaddrinfo('0.0.0.0', 46321)[0][-1] sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.bind(host_addr) sock.listen(1) while True: # main loop, causing the program to await a new connection as soon as the previous one is closed await_connection()
Question: Python version: 3.11 Installing dependencies for an application by pip install -r requirements.txt gives the following error: socket.c -o build/temp.linux-armv8l-cpython-311/aiohttp/_websocket.o aiohttp/_websocket.c:198:12: fatal error: 'longintrepr.h' file not found #include "longintrepr.h" ^~~~~~~ 1 error generated. error: command '/data/data/com.termux/files/usr/bin/arm-linux-androideabi-clang' failed with exit code 1 [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for aiohttp Failed to build aiohttp ERROR: Could not build wheels for aiohttp, which is required to install pyproject.toml-based projects  This error is specific to Python 3.11 version. On Python with 3.10.6 version installation goes fine. Related question: yarl/_quoting.c:196:12: fatal error: 'longintrepr.h' file not found - 1 error generated

Answer: Solution for this error: need to update requirements.txt. Not working versions of modules with Python 3.11: aiohttp==3.8.1 yarl==1.4.2 frozenlist==1.3.0  Working versions: aiohttp==3.8.2 yarl==1.8.1 frozenlist==1.3.1  Links to the corresponding issues with fixes:  https://github.com/aio-libs/aiohttp/issues/6600 https://github.com/aio-libs/yarl/issues/706 https://github.com/aio-libs/frozenlist/issues/305
Question: I ran into an interesting case today wherein a = 10 b = 10 print (a is b)  logged out True. I did some searching and came across the concept of interning. Now that explains why True is correct for the range [-5, 256]. However, I get the same results even while using floats. Please help me understand why. Here is the part I don't get- a = 1000.00 b = 999.99 + 0.01 print (a is b) # Output was 'True'  I expect the output to be False since a's value is assigned before running the program, whereas b's value is determined at run-time as a result of expression evaluation and hence should have a different memory address. I'd also appreciate it if you could point out a case where a==b is True, but a is b evaluates to False where both a and b are of type float

Answer: What you are looking at in your case is called "constant folding". That's an implementation detail, not a language specification - meaning there is no guarantee that this behaviour will remain the same and you should not rely on it in your code. But, in general it comes down to the fact that things that can be calculated "statically" are optimised away to their simplest form before the code is actually being run. Take a look at the output of the following snippet import dis def f(): return 0.1 + 0.9 def g(): a = 0.1 b = 0.9 return a+b dis.dis(f) dis.dis(g)  You will notice that there is no addition performed in f, even though looking at the code addition is clearly there. As for two same-valued variables having different memory addresses, its not very difficult - any calculations that cannot be optimised away pre-runtime will yield you variables at different memory addresses if they are not within interning range. a = 1 b = 1 for _ in range(1000): a += 1 b += 1 print(f"{id(a) = }") print(f"{id(b) = }") print(f"{a == b = }") print(f"{a is b = }")
Question: We have the following two data frames temp = pd.DataFrame(np.array([['I am feeling very well',1],['It is hard to believe this happened',0], ['What is love?',1], ['No new friends',0], ['I love this show',1],['Amazing day today',1]]), columns = ['message','sentiment']) temp_truncated = pd.DataFrame(np.array([['I am feeling very',1],['It is hard to believe',1], ['What is',1], ['Amazing day',1]]), columns = ['message','cutoff'])  My idea is to create a third DataFrame that would represent the inner join between temp and temp_truncated by finding matches in temp that start with / contain the strings in temp_truncated Desired Output:  message sentiment cutoff 0 I am feeling very well 1 1 1 It is hard to believe this happened 0 1 2 What is love 1 1 3 Amazing day today 1 1 

Answer: You can use: import re pattern = '|'.join(map(re.escape, temp_truncated['message'])) key = temp['message'].str.extract(f'({pattern})', expand=False) out = (temp .merge(temp_truncated.rename(columns={'message': 'sub'}), left_on=key, right_on='sub') .drop(columns='sub') )  Output:  message sentiment cutoff 0 I am feeling very well 1 1 1 It is hard to believe this happened 0 1 2 What is love? 1 1 3 Amazing day today 1 1
Question: I am faced with a challenge. I have an Python method implemented and the SonarLint plugin of my PyCharm warns me with the message: "Refactor this function to reduce its Cognitive Complexity from 19 to the 15 allowed." but I can't see how to reduce the complexity. My Python method is: def position(key): if key == 'a': return 0 elif key == 'b': return 1 elif key == 'c': return 2 elif key == 'd': return 3 elif key == 'e': return 4 elif key == 'f': return 5 elif key == 'g': return 6 elif key == 'h': return 7 elif key == 'i': return 8 elif key == 'j': return 9 elif key == 'k': return 10 elif key == 'l': return 11 elif key == 'm': return 12 elif key == 'n': return 13 elif key == 'ñ': return 14 elif key == 'o': return 15 elif key == 'p': return 16 elif key == 'q': return 17 else: logger.info('error')  And the warning of SonarLint is:  And if I click on show issue locations it gives me the explanation of how the Cognitive Complexity is calculated:  I can't see how to reduce the complex of this function. I know that I can implement another method with the same behaviour using things like the ascii code, but it's not the point of this question. The summary of the question is how can I follow the suggestion of SonarLint, I mean, how can I reduce the Cognitive Complexity from 19 to the 15 of this particular method. Something I've noticed is that if I remove elif statements until I have only 14 characters cases, the warning magically disappears.

Answer: I've found what's happening. The Plugin SonarLint has a maximum number of Cognitive Complexity, as you can see in this capture:  So SonarLint doesn't say you that you can simplify your method, it tells you that this method is more complex than the prefixed limit that SonarLint has setted. This is the reason because if I delete elif until I reach the magic number of 15 for the Cognitive Complexity the warning magically disappears. ¡¡WARNING!! It's not recommendable to increase the SonarLint limit of Cognitive Complexity. The most advisable option is to refactor your method and find another way to do the same. In my case I've implemented the next method instead (there are another shorter solutions, but because I have to use spanish alphabet I decided to use what for mi is the most readable solution):  def position(key: str) -> (int, None): char_map = { 'a': 0, 'b': 1, 'c': 2, 'd': 3, 'e': 4, 'f': 5, 'g': 6, 'h': 7, 'i': 8, 'j': 9, 'k': 10, 'l': 11, 'm': 12, 'n': 13, 'ñ': 14, 'o': 15, 'p': 16, 'q': 17, 'r': 18, 's': 19, 't': 20, 'u': 21, 'v': 22, 'w': 23, 'x': 24, 'y': 25, 'z': 26 } try: output = char_map[key] except KeyError: logger.error('KeyError in position() with the key: ' + str(key)) return None return output
Question: I wonder why we need to use a try-finally when using a the @contextmanager decorator. The provided example suggests: from contextlib import contextmanager @contextmanager def managed_resource(*args, **kwds): resource = acquire_resource(*args, **kwds) try: yield resource finally: release_resource(resource)  It seems to me, however, that this will do the exact same thing: @contextmanager def managed_resource(*args, **kwds): resource = acquire_resource(*args, **kwds) yield resource release_resource(resource)  I'm sure I must be missing something. What am I missing?

Answer: Because a finally statement is guaranteed to run no matter what (except a power outage), before the code can terminate. So writing it like this guarantees that the resource is always released