eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am trying to use different versions of Python on my Windows pc and I'm getting this error when using pip:  Fatal error in launcher: Unable to create process using '"c:\users\mypc\appdata\local\programs\python\python38\python.exe" "C:\Python38\Scripts\pip.exe" ': The system cannot find the file specified.  I understand that this might mean there's two PATH to each of those locations so it's confused, but c:\users\mypc\appdata\local\programs\python\python38\python.exe doesn't even exist on my computer nor in my PATH. Output of where python is: C:\Python38\python.exe`  Here is the PATH in readable format C:\Program Files\Intel\WiFi\bin\ C:\Program Files\Common Files\Intel\WirelessCommon\ C:\WINDOWS\system32 C:\WINDOWS C:\WINDOWS\System32\Wbem C:\WINDOWS\System32\WindowsPowerShell\v1.0\ C:\WINDOWS\System32\OpenSSH\ C:\Program Files\nodejs\ C:\Program Files\Git\cmd C:\Program Files\wkhtmltopdf C:\Program Files\Docker\Docker\resources\bin C:\ProgramData\DockerDesktop\version-bin C:\Python38\Scripts C:\Python38\ "C:\Users\mypc\AppData\Local\Microsoft\WindowsApps C:\bin" C:\Program Files\Intel\WiFi\bin\ C:\Program Files\Common Files\Intel\WirelessCommon\ C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2019.2.1\bin C:\Program Files (x86)\Nmap C:\Users\mypc\AppData\Local\Microsoft\WindowsApps C:\Users\mypc\AppData\Local\Programs\Microsoft VS Code\bin C:\Users\mypc\AppData\Roaming\npm C:\Users\mypc\AppData\Local\Programs\MiKTeX 2.9\miktex\bin\x64\ C:\Program Files\wkhtmltopdf C:\ProgramData\mypc\atom\bin C:\Program Files\JetBrains\PyCharm Community Edition 2019.3.3\bin C:\Program Files\MongoDB\Server\4.2\bin 

Answer: it seems like python team may have implemented some security measures. The new method now is just to prefix python -m before your commands. Let's say you are trying to install pygame (any package) with pip. For that, you'll use python -m pip install pygame //Or any package name  Also, upgrading pip and all other commands will also use the same command structure: python -m pip install --upgrade pip
Question: Python code to request the URL: agent = {"User-Agent":'Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36'} #using agent to solve the blocking issue response = requests.get('https://www.naukri.com/jobs-in-andhra-pradesh', headers=agent) #making the request to the link  Output when printing the html : <!DOCTYPE html> <html> <head> <title>Naukri reCAPTCHA</title> #the title in the actual title of the URL that I am requested for <meta name="robots" content="noindex, nofollow"> <link rel="stylesheet" href="https://static.naukimg.com/s/4/101/c/common_v62.min.css" /> <script src="https://www.google.com/recaptcha/api.js" async defer></script> </head> </html> 

Answer: Using Google Cache along with a referer (in the header) will help you bypass the captcha. Things to note:  Don't send more than 2 requests/sec. You may get blocked. The result you receive is a cache. This will not be effective if you are trying to scrape a real-time data. Example:  header = { "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36" , 'referer':'https://www.google.com/' } r = requests.get("http://webcache.googleusercontent.com/search?q=cache:www.naukri.com/jobs-in-andhra-pradesh",headers=header)  This gives: >>> r.content [Squeezed 2554 lines]
Question: I was recently surprised to find that the "splat" (unary *) operator always captures slices as a list during item unpacking, even when the sequence being unpacked has another type: >>> x, *y, z = tuple(range(5)) >>> y [1, 2, 3] # list, was expecting tuple  Compare to how this assignment would be written without unpacking: >>> my_tuple = tuple(range(5)) >>> x = my_tuple[0] >>> y = my_tuple[1:-1] >>> z = my_tuple[-1] >>> y (1, 2, 3)  It is also inconsistent with how the splat operator behaves in function arguments: >>> def f(*args): ... return args, type(args) ... >>> f() ((), <class 'tuple'>)  In order to recover y as a tuple after unpacking, I now have to write: >>> x, *y, z = tuple(range(5)) >>> y = tuple(y)  Which is still much better that the slice-based syntax, but nonetheless suffers from what I consider to be a very unnecessary and unexpected loss of elegance. Is there any way to recover y as a tuple instead of a list without post-assignment processing? I tried to force python to interpret y as a tuple by writing x, *(*y,), z = ..., but it still ended up as a list. And of course silly things like x, *tuple(y), z don't work in python. I am currently using Python 3.8.3 but solutions/suggestions/explanations involving higher versions (as they become available) are also welcome.

Answer: This is by design. Quoting the official docs about Assignment:  ...The first items of the iterable are assigned, from left to right, to the targets before the starred target. The final items of the iterable are assigned to the targets after the starred target. A list of the remaining items in the iterable is then assigned to the starred target (the list can be empty).  It is highly probable that the Python user wants to mutate your y afterwards, so the list type was chosen over the tuple. Quoting the Acceptance section of PEP 3132 that I found through a link in this related question:  After a short discussion on the python-3000 list [1], the PEP was accepted by Guido in its current form. Possible changes discussed were:  Only allow a starred expression as the last item in the exprlist. This would simplify the unpacking code a bit and allow for the starred expression to be assigned an iterator. This behavior was rejected because it would be too surprising.  Try to give the starred target the same type as the source iterable, for example, b in a, *b = "hello" would be assigned the string "ello". This may seem nice, but is impossible to get right consistently with all iterables.  Make the starred target a tuple instead of a list. This would be consistent with a function's *args, but make further processing of the result harder.    So converting with y = tuple(y) afterwards is your only option.
Question: I have face this weird behavior I can not find explications about. MWE: l = [1] l += {'a': 2} l [1, 'a'] l + {'B': 3} Traceback (most recent call last): File "<input>", line 1, in <module> TypeError: can only concatenate list (not "dict") to list  Basically, when I += python does not raise an error and append the key to the list while when I only compute the + I get the expected TypeError. Note: this is Python 3.6.10

Answer: l += ... is actually calling object.__iadd__(self, other) and modifies the object in-place when l is mutable  The reason (as @DeepSpace explains in his comment) is that when you do l += {'a': 2} the operation updates l in place only and only if l is mutable. On the other hand, the operation l + {'a': 2} is not done in place resulting into list + dictionary -> TypeError.  (see here)  l = [1] l = l.__iadd__({'a': 2}) l #[1, 'a']  is not the same as + that calls object.__add__(self, other) l + {'B': 3}  TypeError: can only concatenate list (not "dict") to list
Question: I have tabe i have to map with two values in NY,CAits Domestic, WT its OUTSIDE, and other than that its has to OVERSEAS di = {"NY": "Domestic","CA": "Domestic","WT":"OUTSIDE"} df.replace({'Territory': di})  How to give OVERSEAS in the above code. So by default it has(nothing in the dictionary) to OVERSEAS

Answer: Use Series.map which return missing values for no match values, so added Series.fillna for replace them to default value: df = pd.DataFrame({'Territory':['NY','CA','WT','SK','DE']}) di = {"NY": "Domestic","CA": "Domestic","WT":"OUTSIDE"} print (df) Territory 0 NY 1 CA 2 WT 3 SK 4 DE df['Territory'] = df['Territory'].map(di).fillna('OVERSEAS') print (df) Territory 0 Domestic 1 Domestic 2 OUTSIDE 3 OVERSEAS 4 OVERSEAS
Question: I am building an application which receives data from Kafka. When using standard avro library provided by Apache ( https://pypi.org/project/avro-python3/ ) the results are correct, however, the deserialization process is terribly slow.  class KafkaReceiver: data = {} def __init__(self, bootstrap='192.168.1.111:9092'): self.client = KafkaConsumer( 'topic', bootstrap_servers=bootstrap, client_id='app', api_version=(0, 10, 1) ) self.schema = avro.schema.parse(open("Schema.avsc", "rb").read()) self.reader = avro.io.DatumReader(self.schema) def do(self): for msg in self.client: bytes_reader = io.BytesIO(msg.value) decoder = BinaryDecoder(bytes_reader) self.data = self.reader.read(decoder)  While reading why this is so slow I've found fastavro which should be much faster. I am using this this way:   def do(self): schema = fastavro.schema.load_schema('Schema.avsc') for msg in self.client: bytes_reader = io.BytesIO(msg.value) bytes_reader.seek(0) for record in reader(bytes_reader, schema): self.data = record  And, since everything is working when using Apache's librabry, I would expect that everything will be working the same way with fastavro. However, when running this, I am getting   File "fastavro/_read.pyx", line 389, in fastavro._read.read_map File "fastavro/_read.pyx", line 290, in fastavro._read.read_utf8 File "fastavro/_six.pyx", line 22, in fastavro._six.py3_btou UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 3: invalid start byte  I don't usually program in Python, so I don't exactly know how to approach this. Any ideas?

Answer: The fastavro.reader expects the avro file format that includes the header. It looks like what you have is a serialized record without the header. I think you might be able to read this using the fastavro.schemaless_reader. So instead of: for record in reader(bytes_reader, schema): self.data = record  You would do: self.data = schemaless_reader(bytes_reader, schema)
Question: I'm trying to extract the data from an image using pytesseract. This module has image_to_data and image_to_osd methods. These two methods provide lots of info (TextLineOrder, WritingDirection, ScriptDetection, Orientation, etc...) as output. The image below is the output of the image_to_data method. What do the values of these columns (level, block_num, par_num, line_num, word_num) mean?  The output of image_to_osd looks as presented below. What is the meaning each term in it? Page number: 0 Orientation in degrees: 0 Rotate: 0 Orientation confidence: 16.47 Script: Latin Script confidence: 4.00  I referred to docs but I did not find any info regarding these parameters.

Answer: Column Level:  Item with no block_num, paragraph_num, line_num, word_num Item with block_num and with no paragraph_num, line_num, word_num Item with block_num, paragraph_num and with no line_num, word_num Item with block_num, paragraph_num, line_num, and with no word_num Item with all those numbers  Column block_num: Block number of the detected text or item  Column par_num: Paragraph number of the detected text or item  Column line_num: Line number of the detected text or item  Column word_num: word number of the detected text or item  But above all 4 columns are interconnected.If the item comes from new line then word number will start counting again from 0, it doesn't continue from previous line last word number. Same goes with line_num, par_num, block_num.  Check out the below image for reference. 1st column: block_num  2nd column: par_num  3rd column: line_num  4rth column: word_num
Question: I would like to create optional asynchronious semaphore. In case of asyncio.Semaphore does not support None values, i decided to create asyncio.Semaphore, if connections limit is specified, else - some kind of dummy object There is a contextlib.nullcontext, but it supports only synchorious with I`ve created my own dummy: @contextlib.asynccontextmanager async def asyncnullcontext(): yield None  It there any default asynchronious null context manager?

Answer:  It there any default asynchronious null context manager?  You can use contextlib.AsyncExitStack(). ExitStack() was similarly the way to create a quick-and-dirty null context manager before the introduction of nullcontext.
Question: I have checked my pip version and got the following output: Requirement already up-to-date: pip in ./anaconda3/envs/runlee_python3/lib/python3.8/site-packages (20.1)  I have a specific situation in which I have to use version 1.15 of Tensorflow, but when I try to install it, it seems like it can‘t find this specific version. pip install tensorflow==1.15 ERROR: Could not find a version that satisfies the requirement tensorflow==1.15 (from versions: 2.2.0rc1, 2.2.0rc2, 2.2.0rc3) ERROR: No matching distribution found for tensorflow==1.15  I can also not find version 1.15 when listing all available options. What am I missing?

Answer: You are using python 3.8, which was not officially supported when tensorflow was at version 1.15. You can also check on pypi, there are no files available for cp38, even for 2.10 Onle the versions listed by your command have a cp38 whl file available, see here Since you have conda, simply create a virtual env with the required version conda create -n tf python=3.7  then install tensorflow in this env
Question: I noticed that both scipy and sklearn have a cosine similarity/cosine distance functions. I wanted to test the speed for each on pairs of vectors: setup1 = "import numpy as np; arrs1 = [np.random.rand(400) for _ in range(60)];arrs2 = [np.random.rand(400) for _ in range(60)]" setup2 = "import numpy as np; arrs1 = [np.random.rand(400) for _ in range(60)];arrs2 = [np.random.rand(400) for _ in range(60)]" import1 = "from sklearn.metrics.pairwise import cosine_similarity" stmt1 = "[float(cosine_similarity(arr1.reshape(1,-1), arr2.reshape(1,-1))) for arr1, arr2 in zip(arrs1, arrs2)]" import2 = "from scipy.spatial.distance import cosine" stmt2 = "[float(1 - cosine(arr1, arr2)) for arr1, arr2 in zip(arrs1, arrs2)]" import timeit print("sklearn: ", timeit.timeit(stmt1, setup=import1 + ";" + setup1, number=1000)) print("scipy: ", timeit.timeit(stmt2, setup=import2 + ";" + setup2, number=1000))  sklearn: 11.072769448000145 scipy: 1.9755544730005568  sklearn runs almost 10 times slower than scipy (even if you remove the array reshape for the sklearn example and generate data that's already in the right shape). Why is one significantly slower than the other?

Answer: As mentioned in the comments section, I don't think the comparison is fair mainly because the sklearn.metrics.pairwise.cosine_similarity is designed to compare pairwise distance/similarity of the samples in the given input 2-D arrays. On the other hand, scipy.spatial.distance.cosine is designed to compute cosine distance of two 1-D arrays. Maybe a more fair comparison is to use scipy.spatial.distance.cdist vs. sklearn.metrics.pairwise.cosine_similarity, where both computes pairwise distance of samples in the given arrays. However, to my surprise, that shows the sklearn implementation is much faster than the scipy implementation (which I don't have an explanation for that currently!). Here is the experiment: import numpy as np from sklearn.metrics.pairwise import cosine_similarity from scipy.spatial.distance import cdist x = np.random.rand(1000,1000) y = np.random.rand(1000,1000) def sklearn_cosine(): return cosine_similarity(x, y) def scipy_cosine(): return 1. - cdist(x, y, 'cosine') # Make sure their result is the same. assert np.allclose(sklearn_cosine(), scipy_cosine())  And here is the timing result: %timeit sklearn_cosine() 10 loops, best of 3: 74 ms per loop %timeit scipy_cosine() 1 loop, best of 3: 752 ms per loop
Question: Example: a = {a:'1', b:'2'} len(a)  What is the time complexity of len(a) ?

Answer: Inspecting the c-source of dictobject.c shows that the structure contains a member responsible for maintaining an explicit count (dk_size) layout: +---------------+ | dk_refcnt | | dk_size | | dk_lookup | | dk_usable | | dk_nentries | +---------------+ ...  Thus it will have order O(1)
Question: I am working on invoice management system in which user can add invoice data and it will save in database and whenever user logged in the data will appear on home page but whenever user logout and try to access home page but it is giving following error. TypeError at / 'AnonymousUser' object is not iterable  i tried AnonymousUser.is_authenticated method but still not working. i want if user is logged in then home.html should open otherwise intro.html here is my code views.py  from django.shortcuts import render from django.contrib.auth.mixins import LoginRequiredMixin, UserPassesTestMixin from django.views.generic import ( ListView, DetailView, CreateView, UpdateView, DeleteView ) from .models import Invoicelist def home(request): if request.user.is_authenticated(): context = { 'invoices': Invoicelist.objects.all() } return render(request, 'invoicedata/home.html', context) else: return render(request, 'invoicedata/intro.html', context)  home.html {% extends "invoicedata/base.html" %} {% block content %} {% for invoice in invoices %} <article class="media content-section"> <div class="media-body"> <div class="article-metadata"> <small class="text-muted">{{ invoice.date_posted|date:"F d, Y" }}</small> <h2><a class="article-title" href="{% url 'invoice-detail' invoice.id %}">{{ invoice.issuer }}</a></h2> </div> <p class="article-content">{{ invoice.invoice_number }}</p> <p class="article-content">{{ invoice.date }}</p> <p class="article-content">{{ invoice.amount }}</p> <p class="article-content">{{ invoice.currency }}</p> <p class="article-content">{{ invoice.other }}</p> <div class="article-metadata"> <small class="text-muted">{{ invoice.author }}</small> </div> </div> </article> {% endfor %} {% endblock content %}  intro.html {% extends "invoicedata/base.html" %} {% block content %} <h2>login to your portal for great auditing services</h2> {% endblock content %} 

Answer: Finally i got the solution that work for me here it is Django provides LoginRequiredMixin i used this in my invoicelistview function from django.contrib.auth.mixins import LoginRequiredMixin, UserPassesTestMixin class InvoiceListView(LoginRequiredMixin,ListView): model = Invoicelist template_name = 'invoicedata/home.html' context_object_name = 'invoices' def get_queryset(self): return self.model.objects.all().filter(author=self.request.user).order_by('-date_posted')[:2]  and that's it. Now whenever user logout then it will redirect to login page
Question: In a post I was reviewing recently, I read that it's advised not to use 'sudo pip' to install certain items. Can someone clarify why this is and what the downsides/upsides are? Thanks!

Answer: Your OS has a Python interpreter to run Python software controlled by your package manager, be it apt, yum, or App Store. Any Python package installed to the system Python installation are dependencies of such software, or that software itself.  By installing or updating packages in your system Python, you can break that software. Also, your modifications would be overwritten with the next update of something that required a dependency you've overwritten ("upgraded"), which often occurs when you install something with many dependencies. This can bite you at the most inopportune moment. If you value your time and sanity, always use virtualenv or your favorite wrapper over it. Preferably have one virtualenv per project, and separate virtualenvs for stuff like AWS CLI. Never sudo pip install anything for your development.
Question: I have a list in Python which includes up to 50 elements. In order for me to easily add/subtract elements, I'd prefer to either code it vertically (each list element on one Python code line) or alternatively, import a separate CSV file? list_of_elements = ['AA','BB','CC','DD','EE','FF', 'GG'] for i in list_of_elements: more code...  I'd prefer code like this: list_of_elements = ['AA', 'BB', 'CC', 'DD', 'EE', 'FF', 'GG'] for i in list_of_elements: more code...  Just to clarify, it's not about printing, but about coding. I need to have a better visual overview of all the list elements inside the Python code.

Answer: The first line should contain the first element, like this:  list_of_elements = ['AA', 'BB', 'CC', 'DD', 'EE', 'FF', 'GG']  or as Naufan Rusyda Faikar commented: Put backslash next to = Or put the left bracket next to =. list_of_elements = \ ['AA', 'BB', 'CC', 'DD', 'EE', 'FF', 'GG'] list_of_elements = [ 'AA', 'BB', 'CC', 'DD', 'EE', 'FF', 'GG']  All three will work.
Question: While learning from an online course on visualising data, I came across this line of code. import pandas as pd pd.plotting.register_matplotlib_converters() import matplotlib.pyplot as plt %matplotlib inline import seaborn as sns  Can someone please tell me what is the use of  pd.plotting.register_matplotlib_converters()  I referred to the official documentation, but a clear explanation is not given. Documentation

Answer: I found this in the documentation: This function modifies the global matplotlib.units.registry dictionary. Pandas adds custom converters for  pd.Timestamp pd.Period np.datetime64 ...  So I guess it makes sure that pandas datatypes like pd.Timestamp can be used in matplotlib plots without having to cast them to another type.
Question: I have a complex Django-Pytest test suite with lots of tests that are running in parallel processes. I'd like to see the exact timepoint at which each test started and ended. How can I get that information out of Pytest?

Answer: The start/stop timestamps for each call phase are stored in the CallInfo objects. However, accessing those for reporting is not very convenient, so it's best to store both timestamps in the report objects. Put the following code in a conftest.py file in your project/test root dir: import pytest @pytest.hookimpl(hookwrapper=True) def pytest_runtest_makereport(item, call): outcome = yield report = outcome.get_result() report.start = call.start report.stop = call.stop  Now that you have each report enhanced with start/stop times, process them the way you need to, for example by printing them in a custom section after the test execution. Enhance your conftest.py file with: def pytest_terminal_summary(terminalreporter): terminalreporter.ensure_newline() terminalreporter.section('start/stop times', sep='-', bold=True) for stat in terminalreporter.stats.values(): for report in stat: if report.when == 'call': start = datetime.fromtimestamp(report.start) stop = datetime.fromtimestamp(report.stop) terminalreporter.write_line(f'{report.nodeid:20}: {start:%Y-%m-%d %H:%M:%S} - {stop:%Y-%m-%d %H:%M:%S}')  A test execution of sample tests def test_spam(): time.sleep(1) def test_eggs(): time.sleep(2)  now yields: test_spam.py .. [100%] ------------------------------ start/stop times ------------------------------- test_spam.py::test_spam: 2020-04-26 13:29:05 - 2020-04-26 13:29:06 test_spam.py::test_eggs: 2020-04-26 13:29:06 - 2020-04-26 13:29:08 ============================== 2 passed in 3.03s ==============================  Notice that in the above pytest_terminal_summary hookimpl example, I only print the time of the call phase (execution times of the test function). If you want to see or include the timestamps of the test setup/teardown phases, filter terminalreporter.stats by report objects with report.when == 'setup'/report.when == 'teardown', respectively.
Question: Python installation screenshot 1.i Uninstall everything of python with advance uninstaller . ( register file and ...) 2.i download the last version of Python from python.org 3. i add the Include PATH file when start the installation of Python. BUT I don't KNOW WHY ITS NOT Installed !

Answer: Error Code 0x80070643 I found it. if you are not administrator of the system , and change the location of installation , for example (c:\python) this error will be appear . so you must be install python on (c:\users\'your username'\App Data\Local\Programs\Python) and after installation python go to system environment and add the path into this.