eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Is it possible to import the SageMath functions inside a python session? What I wish to do, from a user perspective is something like this: >>> import sage >>> sage.kronecker_symbol(3,5) # ...or any other sage root functions  instead of accessing kronecker_symbol(3,5) from a sagemath session. If possible, it would be very handy, as would allow embedding all the functionalities of SageMath within the python world.

Answer: Importing SageMath functions in a Python session There are several ways to achieve that. SageMath from the operating system's package manager Some operating systems have Sage packaged natively, for example Arch Linux, Debian, Fedora, Gentoo, NixOS, and their derivatives (Linux Mint, Manjaro, Ubuntu...). See the dedicated "Distribution" page on the Sage wiki:  SageMath distribution and packaging  If using one of those, use the package manager to install sage or sagemath and then the Sage library will be installed on the system's Python, and in that Python it will become possible to do things like >>> from sage.arith.misc import kronecker >>> kronecker(3, 5) -1  Another option is to use a cross-platform package manager such as Conda, Guix and Nix. These should work on most Linux distributions and macOS. Yet another option would be to run a Docker container. I will detail the Conda case below. SageMath with Conda Install Sage with Conda and you will get that. Instructions are here:  SageMath installation: install from conda-forge  and start by installing a Conda distribution, either Miniconda, Minimamba or Anaconda, and then creating a sage conda environment. Once a sage conda environment is installed, activate it: conda activate sage  With that sage conda environment active, run python  and importing the sage module or importing functions such as kronecker from that module should work.
Question: This is really strange to me, because by default I thought unpacking gives tuples. In my case I want to use the prefix keys for caching, so a tuple is preferred. # The r.h.s is a tuple, equivalent to (True, True, 100) *prefix, seed = ml_logger.get_parameters("Args.attn", "Args.memory_gate", "Args.seed") assert type(prefix) is list  But I thought unpacking would return a tuple instead. Here is the relevant PEP: https://www.python.org/dev/peps/pep-3132/ -- Update -- Given the comment and answers bellow, specifically I was expecting the unpacking to give a tuple because in function arguments a spread arg is always a tuple instead of a list.  As Jason pointed out, during unpacking one would not be able to know the length of the result ahead of time, so implementation-wise the catch-all has to start as a list for dynamic appends. Converting it to a list is a waste of effort the majority of the time. Semantically, I would prefer to have a tuple for consistency.

Answer: This issue was mentioned in that PEP (PEP 3132):  After a short discussion on the python-3000 list [1], the PEP was accepted by Guido in its current form. Possible changes discussed were: [...]  Try to give the starred target the same type as the source iterable, for example, b in a, *b = 'hello' would be assigned the string 'ello'. This may seem nice, but is impossible to get right consistently with all iterables.  Make the starred target a tuple instead of a list. This would be consistent with a function's *args, but make further processing of the result harder.    But as you can see, these features currently are not implemented: In [1]: a, *b, c = 'Hello!' In [2]: print(a, b, c) H ['e', 'l', 'l', 'o'] !  Maybe, mutable lists are more appropriate for this type of unpacking.
Question: i installed pyenv and switched to python 3.6.9 (using pyenv global 3.6.9). How do i go back to my system python? Running pyenv global system didnt work

Answer: Your system Python might be /usr/bin/python or /usr/bin/python3. You have a couple options:  Execute that Python interpreter directly:  /usr/bin/python --version  If you want to run it from a script and you're on a *nix machine, put #!/usr/bin/python  at the top of the file, then give it execute permissions (chmod +x my-script.py) and run it directly: ./my-script.py.  Turn off pyenv's path hacks. This could mean removing the eval "$(pyenv init -)" from your ~/.bashrc or ~/.bash_profile and loading a new shell. Use the pyenv register plugin - https://github.com/doloopwhile/pyenv-register (or use/build something similar). Here's a portion of the README  Installation: git clone https://github.com/doloopwhile/pyenv-register.git $(pyenv root)/plugins/pyenv-register # clone plugin exec "$SHELL" # reload shell  Usage: pyenv register /usr/bin/python pyenv versions
Question: How can I update the fields of a dataclass using a dict? Example: @dataclass class Sample: field1: str field2: str field3: str field4: str sample = Sample('field1_value1', 'field2_value1', 'field3_value1', 'field4_value1') updated_values = {'field1': 'field1_value2', 'field3': 'field3_value2'}  I want to do something like  sample.update(updated_values) 

Answer: One way is to make a small class and inherit from it: class Updateable(object): def update(self, new): for key, value in new.items(): if hasattr(self, key): setattr(self, key, value) @dataclass class Sample(Updateable): field1: str field2: str field3: str field4: str  You can read this if you want to learn more about getattr and setattr
Question: I need to extract tables from pdf, these tables can be of any type, multiple headers, vertical headers, horizontal header etc. I have implemented the basic use cases for both and found tabula doing a bit better than camelot still not able to detect all tables perfectly, and I am not sure whether it will work for all kinds or not. So seeking suggestions from experts who have implemented similar use case. Example PDFs: PDF1 PDF2 PDF3 Tabula Implementation: import tabula tab = tabula.read_pdf('pdfs/PDF1.pdf', pages='all') for t in tab: print(t, "\n=========================\n")  Camelot Implementation: import camelot tables = camelot.read_pdf('pdfs/PDF1.pdf', pages='all', split_text=True) tables for tabs in tables: print(tabs.df, "\n=================================\n") 

Answer: Please read this: https://camelot-py.readthedocs.io/en/master/#why-camelot The main advantage of Camelot is that this library is rich in parameters, through which you can improve the extraction. Obviously, the application of these parameters requires some study and various attempts. Here you can find comparision of Camelot with other PDF Table Extraction libraries.
Question: When using an optional import, i.e. the package is only imported inside a function as I want it to be an optional dependency of my package, is there a way to type hint the return type of the function as one of the classes belonging to this optional dependency? To give a simple example with pandas as an optional dependency: def my_func() -> pd.DataFrame: import pandas as pd return pd.DataFrame() df = my_func()  In this case, since the import statement is within my_func, this code will, not surprisingly, raise:  NameError: name 'pd' is not defined  If the string literal type hint were used instead, i.e.: def my_func() -> 'pd.DataFrame': import pandas as pd return pd.DataFrame() df = my_func()  the module can now be executed without issue, but mypy will complain:  error: Name 'pd' is not defined  How can I make the module execute successfully and retain the static type checking capability, while also having this import be optional?

Answer: Try sticking your import inside of an if typing.TYPE_CHECKING statement at the top of your file. This variable is always false at runtime but is treated as always true for the purposes of type hinting.  For example: # Lets us avoid needing to use forward references everywhere # for Python 3.7+ from __future__ import annotations from typing import TYPE_CHECKING if TYPE_CHECKING: import pandas as pd def my_func() -> pd.DataFrame: import pandas as pd return pd.DataFrame()  You can also do if False:, but I think that makes it a little harder for somebody to tell what's going on. One caveat is that this does mean that while pandas will be an optional dependency at runtime, it'll still be a mandatory one for the purposes of type checking.  Another option you can explore using is mypy's --always-true and --always-false flags. This would give you finer-grained control over which parts of your code are typechecked. For example, you could do something like this: try: import pandas as pd PANDAS_EXISTS = True except ImportError: PANDAS_EXISTS = False if PANDAS_EXISTS: def my_func() -> pd.DataFrame: return pd.DataFrame()  ...then do mypy --always-true=PANDAS_EXISTS your_code.py to type check it assuming pandas is imported and mypy --always-false=PANDAS_EXISTS your_code.py to type check assuming it's missing. This could help you catch cases where you accidentally use a function that requires pandas from a function that isn't supposed to need it -- though the caveats are that (a) this is a mypy-only solution and (b) having functions that only sometimes exist in your library might be confusing for the end-user.
Question: The problem I was new to web scraping and I was trying to create a scraper which looks at a playlist link and gets the list of the music and the author. But the site kept rejecting my connection because it thought that I was a bot, so I used UserAgent to create a fake useragent string to try and bypass the filter. It sort of worked? But the problem was that when you visited the website by a browser, you could see the contents of the playlist, but when you tried to extract the html code with requests, the contents of the playlist was just a big blank space. Mabye I have to wait for the page to load? Or there is a stronger bot filter? My code import requests from bs4 import BeautifulSoup from fake_useragent import UserAgent ua = UserAgent() melon_site="http://kko.to/IU8zwNmjM" headers = {'User-Agent' : ua.random} result = requests.get(melon_site, headers = headers) print(result.status_code) src = result.content soup = BeautifulSoup(src,'html.parser') print(soup)  Link of website playlist link html I get when using requests html with blank space where the playlist was supposed to be

Answer: You wanna check out this link to get the content you wish to grab. The following attempt should fetch you the artist names and their song names. import requests from bs4 import BeautifulSoup url = 'https://www.melon.com/mymusic/playlist/mymusicplaylistview_listSong.htm?plylstSeq=473505374' r = requests.get(url,headers={"User-Agent":"Mozilla/5.0"}) soup = BeautifulSoup(r.text,"html.parser") for item in soup.select("tr:has(#artistName)"): artist_name = item.select_one("#artistName > a[href*='goArtistDetail']")['title'] song = item.select_one("a[href*='playSong']")['title'] print(artist_name,song)  Output are like: Martin Garrix - 페이지 이동 Used To Love (feat. Dean Lewis) 재생 - 새 창 Post Malone - 페이지 이동 Circles 재생 - 새 창 Marshmello - 페이지 이동 Here With Me 재생 - 새 창 Coldplay - 페이지 이동 Cry Cry Cry 재생 - 새 창  Note: your BeautifulSoup version should be 4.7.0 or later in order for the script to support pseudo selector.
Question: I am using this:  blur = cv2.GaussianBlur(dst,(5,5),0)  And I wanted to show the kernel matrix by this: print(cv2.getGaussianKernel(ksize=(5,5),sigma=0))  But I am getting a type error: TypeError: an integer is required (got type tuple)  If I only put 5, I get a 5x1 matrix. Isn't the blur kernel 5x5? Or am I missing on something fundamental?

Answer: The Gaussian kernel is separable. Therefore, the kernel generated is 1D. The GaussianBlur function applies this 1D kernel along each image dimension in turn. The separability property means that this process yields exactly the same result as applying a 2D convolution (or 3D in case of a 3D image). But the amount of work is strongly reduced. For your 5x5 kernel, the 2D convolution does 25 multiplications and additions, the separable implementation does only 5+5=10. For larger kernels, the gains are increasingly significant. To see the full 2D kernel, apply the GaussianBlur function to an image that is all zeros and has a single pixel in the middle set to 1. This is the discrete equivalent to the Dirac delta function, which we can use to analyze linear time-invariant functions (==convolution filters).
Question: I have a c++ vector with std::pair<unsigned long, unsigned long> objects. I am trying to generate permutations of the objects of the vector using std::next_permutation(). However, I want the permutations to be of a given size, you know, similar to the permutations function in python where the size of the expected returned permutation is specified. Basically, the c++ equivalent of  import itertools list = [1,2,3,4,5,6,7] for permutation in itertools.permutations(list, 3): print(permutation)  Python Demo (1, 2, 3) (1, 2, 4) (1, 2, 5) (1, 2, 6) (1, 2, 7) (1, 3, 2) (1, 3, 4) .. (7, 5, 4) (7, 5, 6) (7, 6, 1) (7, 6, 2) (7, 6, 3) (7, 6, 4) (7, 6, 5) 

Answer: You might use 2 loops:  Take each n-tuple iterate over permutations of that n-tuple  template <typename F, typename T> void permutation(F f, std::vector<T> v, std::size_t n) { std::vector<bool> bs(v.size() - n, false); bs.resize(v.size(), true); std::sort(v.begin(), v.end()); do { std::vector<T> sub; for (std::size_t i = 0; i != bs.size(); ++i) { if (bs[i]) { sub.push_back(v[i]); } } do { f(sub); } while (std::next_permutation(sub.begin(), sub.end())); } while (std::next_permutation(bs.begin(), bs.end())); }  Demo
Question: I'm try to test the api I wrote with Fastapi. I have the following method in my router : @app.get('/webrecord/check_if_object_exist') async def check_if_object_exist(payload: WebRecord) -> bool: key = get_key_of_obj(payload.data) if payload.key is None else payload.key return await check_if_key_exist(key)  and the following test in my test file : client = TestClient(app) class ServiceTest(unittest.TestCase): ..... def test_check_if_object_is_exist(self): webrecord_json = {'a':1} response = client.get("/webrecord/check_if_object_exist", json=webrecord_json) assert response.status_code == 200 assert response.json(), "webrecord should already be in db, expected : True, got : {}".format(response.json())  When I run the code in debug I realized that the break points inside the get method aren't reached. When I changed the type of the request to post everything worked fine. What am I doing wrong?

Answer: In order to send data to the server via a GET request, you'll have to encode it in the url, as GET does not have any body. This is not advisable if you need a particular format (e.g. JSON), since you'll have to parse the url, decode the parameters and convert them into JSON. Alternatively, you may POST a search request to your server. A POST request allows a body which may be of different formats (including JSON). If you still want GET request  @app.get('/webrecord/check_if_object_exist/{key}') async def check_if_object_exist(key: str, data: str) -> bool: key = get_key_of_obj(payload.data) if payload.key is None else payload.key return await check_if_key_exist(key) client = TestClient(app) class ServiceTest(unittest.TestCase): ..... def test_check_if_object_is_exist(self): response = client.get("/webrecord/check_if_object_exist/key", params={"data": "my_data") assert response.status_code == 200 assert response.json(), "webrecord should already be in db, expected : True, got : {}".format(response.json())  This will allow to GET requests from url mydomain.com/webrecord/check_if_object_exist/{the key of the object}. One final note: I made all the parameters mandatory. You may change them by declaring to be None by default. See fastapi Docs
Question: I tried to compare these two snippets and see how many iterations could be done in one second. Turns out that Julia achieves 2.5 million iterations whereas Python 4 million. Isn't Julia supposed to be quicker. Or maybe these two snippets are not equivalent? Python: t1 = time.time() i = 0 while True: i += 1 if time.time() - t1 >= 1: break  Julia: function f() i = 0 t1 = now() while true i += 1 if now() - t1 >= Base.Dates.Millisecond(1000) break end end return i end 

Answer: This is kind of an odd performance comparison since typically one measures the time it takes to compute something of substance, rather than seeing how many trivial iterations one can do in a certain amount of time. I had trouble getting your Python and Julia codes to work, so I modified the Julia code to work and just didn't run the Python code. As noted by @chepner in a comment, using now() and doing time comparisons with DateTime objects is fairly expensive. The Python time.time() function just returns a floating-point value. As it turns out, there's a Julia function called time() that does the exact same thing: julia> time() 1.587648091474481e9  Here's the timing of your original f() function (modified to work) on my system: julia> using Dates julia> function f() i = 0 t1 = now() while true i += 1 if now() - t1 >= Millisecond(1000) break end end return i end f (generic function with 1 method) julia> f() 4943739  It did almost 5 million iterations before time was up. As I said, I wasn't able to get your Python code to run on my system without significant fiddling (which I didn't bother doing). But here's a version of f() that uses time() instead, which I will imaginatively call g(): julia> function g() i = 0 t1 = time() while true i += 1 if time() - t1 >= 1 break end end return i end g (generic function with 1 method) julia> g() 36087637  This version did 36 million iterations. So I guess Julia is faster at looping? Yay! Well, actually the main work in this loop is the calls to time() so... Julia is faster at generating lots of time() calls! Why is it odd to time this? As I said, most of actual work here is calling time(). The rest of the loop doesn't really do anything. In an optimizing compiled language, if the compiler sees a loop that doesn't do anything, it will eliminate it entirely. For example: julia> function h() t = 0 for i = 1:100_000_000 t += i end return t end h (generic function with 1 method) julia> h() 5000000050000000 julia> @time h() 0.000000 seconds 5000000050000000  Woah, zero seconds! How is that possible? Well, let's look at the LLVM code (kind of like machine code but for an imaginary machine that is used as an intermediate representation) this lowers to: julia> @code_llvm h() ; @ REPL[16]:1 within `h' define i64 @julia_h_293() { top: ; @ REPL[16]:6 within `h' ret i64 5000000050000000 }  The compiler sees the loop, figures out that the result is the same every time, and just returns that constant value instead of actually executing the loop. Which, of course, takes zero time.
Question: I have a personal implementation of a RL algorithm that generates performance metrics every x time steps. That metric is simply a scalar, so I have an array of scalars that I want to display as a simple graph such as:  I want to display it in real time in tensorboard like my above example. Thanks in advance

Answer: If you really want to use tensorboard you can start looking at tensorflow site and this datacamp tutorial on tensorboard. With tensorflow you can use summary.scalar to plot your custom data (as the example), no need for particular format, as the summary is taking care of that, the only condition is that data has to be a real numeric scalar value, convertible to a float32 Tensor. import tensorflow as tf import numpy as np import os import time now = time.localtime() subdir = time.strftime("%d-%b-%Y_%H.%M.%S", now) summary_dir1 = os.path.join("stackoverflow", subdir, "t1") summary_writer1 = tf.summary.create_file_writer(summary_dir1) for cont in range(200): with summary_writer1.as_default(): tf.summary.scalar(name="unify/sin_x", data=np.math.sin(cont) ,step=cont) tf.summary.scalar(name="unify/sin_x_2", data=np.math.sin(cont/2), step=cont) summary_writer1.flush()   That said, if you are not planning on using tensorflow with your implementation, I would suggest you just use matplotlib as this library also enables you to plot data in real time https://youtu.be/Ercd-Ip5PfQ?t=444.
Question: The lark parser predefines some common terminals, including a string. It is defined as follows: _STRING_INNER: /.*?/ _STRING_ESC_INNER: _STRING_INNER /(?<!\\)(\\\\)*?/ ESCAPED_STRING : "\"" _STRING_ESC_INNER "\""  I do understand _STRING_INNER. I also understand how ESCAPED_STRING is composed. But what I don't really understand is _STRING_ESC_INNER. If I read the regex correctly, all it says is that whenever I find two consecutive literal backslashes, they must not be preceeded by another literal backslash? How can I combine those two into a single regex? And wouldn't it be required for the grammar to only allow escaped double quotes in the string data?

Answer: Preliminaries:  .*? Non-greedy match, meaning the shortest possible number of repetitions of . (any symbol). This only makes sense when followed by something else. So .*?X on input AAXAAX would match only the AAX part, instead of expanding all the way to the last X. (?<!...) is a "negative look-behind assertion" (link): "Matches if the current position in the string is not preceded by a match for ....". So .*(?<!X)Y would match AY but not XY.  Applying this to your example:  ESCAPED_STRING: The rule says: "Match ", then _STRING_ESC_INNER, and then " again". _STRING_INNER: Matches the shortest possible number of repetitions of any symbol. As said before, this only makes sense when considering the regular expression that comes after it. _STRING_ESC_INNER: We want this to match the shortest possible string that does not contain a closing quote. That is, for an input "abc"xyz", we want to match "abc", instead of also consuming the xyz" part. However, we have to make sure that the " is really a closing quote, in that it should not be itself escaped. So for input "abc\"xyz", we do not want to match only "abc\", because the \" is escaped. We observe that the closing " has to be directly preceded by an even number of \ (with zero being an even number). So " is ok, \\" is ok, \\\\" is ok etc. But as soon as " is preceded by an odd number of \, that means the " is not really a closing quote. (\\\\) matches \\. The (?<!\\) says "the position before should not have \". So combined (?<!\\)(\\\\) means "match \\, but only if it is not preceded by \". The following *? then does the smallest possible repetitions of this, which again only makes sense when considering the regular expression that comes after this, which is the " from the ESCAPED_STRING rule (possible point of confusion: the \" in the ESCAPED_STRING refers to a literal " in the actual input we want to match, in the same way that \\\\ refers to \\ in the input). So (?<!\\)(\\\\)*?\" means "match the shortest amount of \\ that is followed by " and not preceded by \. So in other words, (?<!\\)(\\\\)*?\" matches only " that are preceded by an even number of \ (including blocks of size 0). Now combining it with the preceding _STRING_INNER, the _STRING_ESC_INNER rule then says: Match the first " preceded by an even number of \, so in other words, the first " where the \ is not itself escaped.