eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I'm making a little graphic interface with Python 3 which should download a youtube video with its URL. I used the youtube_dl module for that. This is my code : import youtube_dl # Youtube_dl is used for download the video ydl_opt = {"outtmpl" : "/videos/%(title)s.%(ext)s", "format": "bestaudio/best"} # Here we give some advanced settings. outtmpl is used to define the path of the video that we are going to download def operation(link): """ Start the download operation """ try: with youtube_dl.YoutubeDL(ydl_opt) as yd: # The method YoutubeDL() take one argument which is a dictionary for changing default settings video = yd.download([link]) # Start the download result.set("Your video has been downloaded !") except Exception: result.set("Sorry, we got an error.") operation("https://youtube.com/watch?v=...")  When I execute my code, I get this error: ERROR: YouTube said: Unable to extract video data  I saw here that it was because it doesn't find any video info, how can I resolve this problem?

Answer: Updating youtube-dl helped me. Depending on the way you installed it, here are the commands:  youtube-dl --update (self-update) pip install -U youtube-dl (via python) brew upgrade youtube-dl (macOS + homebrew) choco upgrade youtube-dl (Windows + Chocolatey)
Question: I need to use python3.7 as default python version to use in google colab(via this notebook ) for testing the faceswap GitHub project, by this codes: %cd "/content/faceit" !rm -rf faceswap !git clone https://github.com/deepfakes/faceswap.git %cd faceswap !python setup.py  The reason is that,when i try to install faceswap in google colab i get this error: /content/faceit Cloning into 'faceswap'... remote: Enumerating objects: 7725, done. remote: Total 7725 (delta 0), reused 0 (delta 0), pack-reused 7725 Receiving objects: 100% (7725/7725), 194.20 MiB | 31.66 MiB/s, done. Resolving deltas: 100% (5338/5338), done. /content/faceit/faceswap INFO Running as Root/Admin INFO The tool provides tips for installation and installs required python packages INFO Setup in Linux 4.19.112+ INFO Installed Python: 3.6.9 64bit ERROR Please run this script with Python version 3.7 or 3.8 64bit and try again.  So based of the different python module which needs to be installed by different files, it needs to install python 3.7 and set it as python default command. I would appropriate, any help to solve it. Thanks.

Answer: According to this post, there are different ways to run a specific version of Python on Colab:  Installing Anaconda Adding (fake) google.colab library Starting Jupyterlab Accessing it with ngrok  The code sample is below # install Anaconda3 !wget -qO ac.sh https://repo.anaconda.com/archive/Anaconda3-2020.07-Linux-x86_64.sh !bash ./ac.sh -b # a fake google.colab library !ln -s /usr/local/lib/python3.6/dist-packages/google \ /root/anaconda3/lib/python3.8/site-packages/google # start jupyterlab, which now has Python3 = 3.8 !nohup /root/anaconda3/bin/jupyter-lab --ip=0.0.0.0& # access through ngrok, click the link !pip install pyngrok -q from pyngrok import ngrok print(ngrok.connect(8888))  Additionally, I recommend you to use it by specifying the Python version to run a script on colab. # Install the python version !apt-get install python3.7 # Select the version !python3.7 setup.py  You can see this example I have tried.  If you will use multiple library versions, you can also use virtualenv on colab by specify the python version with --python option. For example: virtualenv env --python=python3.7
Question: I am trying to import the rpy2 library into a Jupyter Notebook but I cannot get past this error. The PATH 'C:\Program Files\R\R-4.0.2\bin\x64' has been added. This is the only version of R installed on my computer. I have completely uninstalled and reinstalled R/Rstudio/Anaconda with no luck. Here is the full error: --------------------------------------------------------------------------- OSError Traceback (most recent call last) <ipython-input-7-098f0d39b3a3> in <module> ----> 1 from rpy2.robjects import pandas2ri C:\Anaconda\lib\site-packages\rpy2\robjects\__init__.py in <module> 14 from functools import partial 15 import types ---> 16 import rpy2.rinterface as rinterface 17 import rpy2.rlike.container as rlc 18 C:\Anaconda\lib\site-packages\rpy2\rinterface.py in <module> 4 import typing 5 from typing import Union ----> 6 from rpy2.rinterface_lib import openrlib 7 import rpy2.rinterface_lib._rinterface_capi as _rinterface 8 import rpy2.rinterface_lib.embedded as embedded C:\Anaconda\lib\site-packages\rpy2\rinterface_lib\openrlib.py in <module> 42 rlib = _rinterface_cffi.lib 43 else: ---> 44 rlib = _dlopen_rlib(R_HOME) 45 46 C:\Anaconda\lib\site-packages\rpy2\rinterface_lib\openrlib.py in _dlopen_rlib(r_home) 35 raise ValueError('The library path cannot be None.') 36 else: ---> 37 rlib = ffi.dlopen(lib_path) 38 return rlib 39 OSError: cannot load library 'C:\Program Files\R\R-4.0.2\bin\x64\R.dll': error 0x7e  edit: Here is the code I run to import rpy2 library: from rpy2.robjects import r, pandas2ri 

Answer: 1 - Windows + IDE For those not using Anaconda, add the following in Windows' environment variables PATH: C:\Program Files\R\R-4.0.3\bin\x64 Your R version may differ from "R-4.0.3" 2 - Anaconda Otherwise, check Grayson Felt's reply:  I found a solution here. Adding the PATH  C:\Users\username\Anaconda2;C:\Users\username\Anaconda2\Scripts;C:\Users\username\Anaconda2\Library\bin;C:\Users\username\Anaconda2\Library\mingw-w64\lib;C:\Users\username\Anaconda2\Library\mingw-w64\bin  and subsequently restarting Anaconda fixed the issue.  3 - Code header Windows basic Alternatively, following Bruno's suggestion (and being more sohpisticated): try: import rpy2.robjects as robjects except OSError as e: try: import os import platform if ('Windows', 'Microsoft') in platform.system(): os.environ["R_HOME"] = 'C:/Program Files/R/R-4.0.3/bin/x64' # Your R version here 'R-4.0.3' os.environ["PATH"] = "C:/Program Files/R/R-4.0.3/bin/x64" + ";" + os.environ["PATH"] import rpy2.robjects as robjects except OSError: raise(e)  This code won't be effective for non-Windows platform. Also adjustments may be necessary for different R versions. If it gets more complicated than this, you should probably just go for solutions 1 or 2. NOTE: You may also face this issue if your Python and R versions are in different architechtures (x86 vs x64)
Question: I installed pip3 using sudo apt-get install python3-pip after that when I run the following command to install django sudo pip3 install django I get this error:  Traceback (most recent call last): File "/usr/bin/pip3", line 9, in from pip import main File "/usr/lib/python3/dist-packages/pip/init.py", line 14, in from pip.utils import get_installed_distributions, get_prog File "/usr/lib/python3/dist-packages/pip/utils/init.py", line 23, in from pip.locations import ( File "/usr/lib/python3/dist-packages/pip/locations.py", line 9, in from distutils import sysconfig ImportError: cannot import name 'sysconfig' from 'distutils' (/usr/lib/python3.8/distutils/init.py)  How do I fix this?

Answer: I have tried recently manually installing python3.9 version in my Ubuntu from 3.6 version using apt install python3.9. Then pip3 was broken. The issue is because distutils were not build for the 3.9 version. So in my case I ran apt install python3.9-distutils to resolve my issue. In your case make sure to modify 3.x version in distutils command.
Question: In my python project, after upgrading mypy from 0.770 to 0.782 an error is received in files where there were previously no type errors: my_pkg_name\__init__.py: error: Source file found twice under different module names: 'top_pkg.my_pkg_name' and 'my_pkg_name' Found 1 error in 1 file (checked 1 source file)  I'm pretty sure this is related to Issue #8944 on mypy and the way which vscode-python executes mypy on the open files. I've tried adding various mypy flags (e.g. --namespace-packages, --no-namespace-packages) but this did not resolve the issue. my_pkg_name does contain an __init__.py and so does top_pkg. With mypy==0.770 this was not a problem. Looking at the extension's output this is how mypy is invoked: > ~\.virtualenvs\OfflineSystem.38\Scripts\python.exe ` c:\Users\***\.vscode\extensions\ms-python.python-2020.8.108011\pythonFiles\pyvsc-run-isolated.py mypy ` --ignore-missing-imports --follow-imports=silent --show-column-numbers ` d:\***\top_pkg\my_pkg_name\sub_pkg\my_file.py  Should change something in the mypy-related vscode settings for this to work?

Answer: I had a similar issue, but not via VSCode. The fix in my case was to remove an __init__.py file from a directory that was being included by adding it to the MYPYPATH, and so wasn't actually being treated as a module (so it shouldn't really have had the __init__.py file). You said you tried adding the --namespace-packages flag, but I think you would need --no-namespace-packages to disable the new checker which might be causing your problem.
Question: I am trying to run a "local" web app on Google Colab using FastAPI / Uvicorn like some of the Flask app sample code I've seen but cannot get it to work. Has anyone been able to do this? Appreciate it. Installed FastAPI & Uvicorn successfully !pip install FastAPI -q !pip install uvicorn -q  Sample app from fastapi import FastAPI app = FastAPI() @app.get("/") async def root(): return {"message": "Hello World"}  Run attempts #attempt 1 if __name__ == "__main__": uvicorn.run("/content/fastapi_002:app", host="127.0.0.1", port=5000, log_level="info")   #attempt 2 #uvicorn main:app --reload !uvicorn "/content/fastapi_001.ipynb:app" --reload 

Answer: You can use ngrok to export a port as an external url. Basically, ngrok takes something available/hosted on your localhost and exposes it to the internet with a temporary public URL. First install the dependencies !pip install fastapi nest-asyncio pyngrok uvicorn  Create your app from fastapi import FastAPI from fastapi.middleware.cors import CORSMiddleware app = FastAPI() app.add_middleware( CORSMiddleware, allow_origins=['*'], allow_credentials=True, allow_methods=['*'], allow_headers=['*'], ) @app.get('/') async def root(): return {'hello': 'world'}  Then run it down. import nest_asyncio from pyngrok import ngrok import uvicorn ngrok_tunnel = ngrok.connect(8000) print('Public URL:', ngrok_tunnel.public_url) nest_asyncio.apply() uvicorn.run(app, port=8000)
Question: Have everyone already had this problem, where VS Code keeps loading all the time and won't open a ipynb file? I've tried to use python 3.7 but same problem. Also tried to reinstall both VS Code and Anaconda, no success.  Here is my environment data:  VS Code version: 1.49.0 Python extension version:v2020.8.108011 OS and version: Ubuntu 20.04 Python version (& distribution if applicable, e.g. Anaconda): Anaconda python 3.8.3 Type of virtual environment used: using conda base environment Value of the python.languageServer setting: "Pylance" ipython version: 7.16.1 jedi version: 0.17.1 ipykernel version: 5.3.2 

Answer: In their official GitHub page, they are tracking this issue already. There is also a solution (kind of) right now. You have to maximize the terminal panel below and then restore the panel size (basically max and min with the arrow button). Then the Notebook loads and everything works fine. :D The workaround was in this comment: https://github.com/microsoft/vscode-python/issues/13901#issuecomment-691625412 Not perfect but at least all the features are there and I can work with my notebooks again :)
Question: I want to add another button next to the edit and delete icons on flask admin list view. In addition, I want to send that row data to a route as a post request. I know that I have to edit the admin/model/list.html template, but I am not getting how to add this functionalities. Can you provide any guidance? 

Answer: You need to define custom action buttons for your view. This process is not described in the Flask-Admin tutorial but it is mentioned in the API description. POST method If you need to create a button for a POST method you should implement a jinja2 macro like this delete_row action. It may look like this (I named the file "custom_row_actions.html"): {% macro copy_row(action, row_id, row) %} <form class="icon" method="POST" action="{{ get_url('.copy_view') }}"> <input type="hidden" name="row_id" value="{{ get_pk_value(row) }}"/> <button type="submit" title="{{ _gettext('Copy record') }}"> <span class="glyphicon glyphicon-copy"></span> </button> </form> {% endmacro %}  Then you create a template for your list of records and import the macro library in it (I named it "my_list.html"): {% extends 'admin/model/list.html' %} {% import 'custom_row_actions.html' as custom_row_actions with context %}  After that you have to make a couple of changes in your view: from flask_admin import expose from flask_admin.contrib.sqla.view import ModelView from flask_admin.model.template import TemplateLinkRowAction class MyView(ModelView): list_template = "my_list.html" # Override the default template column_extra_row_actions = [ # Add a new action button TemplateLinkRowAction("custom_row_actions.copy_row", "Copy Record"), ] @expose("/copy", methods=("POST",)) def copy_view(self): """The method you need to call"""  GET method Creating a button for a GET method is much simpler. You don't need to override templates, just add an action to your view: from flask_admin import expose from flask_admin.contrib.sqla.view import ModelView from flask_admin.model.template import EndpointLinkRowAction class MyView(ModelView): column_extra_row_actions = [ # Add a new action button EndpointLinkRowAction("glyphicon glyphicon-copy", ".copy_view"), ] @expose("/copy", methods=("GET",)) def copy_view(self): """The method you need to call"""  Glyphicons Glyphicons is the icon library which is bundled with the Bootstrap v3 library which is used by the Flask-Admin. You can use it if you chose this Bootstrap version on Flask-Admin initialization: from flask_admin import Admin admin = Admin(template_mode="bootstrap3")  You can look at the available icons in the Bootsrap v3 documentation.
Question: I have a numpy array: A = np.array([8, 2, 33, 4, 3, 6])  What I want is to create another array B where each element is the pairwise max of 2 consecutive pairs in A, so I get: B = np.array([8, 33, 33, 4, 6])  Any ideas on how to implement? Any ideas on how to implement this for more then 2 elements? (same thing but for consecutive n elements) Edit: The answers gave me a way to solve this question, but for the n-size window case, is there a more efficient way that does not require loops? Edit2: Turns out that the question is equivalent for asking how to perform 1d max-pooling of a list with a window of size n. Does anyone know how to implement this efficiently?

Answer: A loop-free solution is to use max on the windows created by skimage.util.view_as_windows: list(map(max, view_as_windows(A, (2,))))  [8, 33, 33, 4, 6]  Copy/pastable example: import numpy as np from skimage.util import view_as_windows A = np.array([8, 2, 33, 4, 3, 6]) list(map(max, view_as_windows(A, (2,))))
Question: I'm following the Prefect tutorial available at: https://docs.prefect.io/core/tutorial/01-etl-before-prefect.html. The code can be downloaded from the git: https://github.com/PrefectHQ/prefect/tree/master/examples/tutorial The tutorials have a dependency to aircraftlib which is a directory under tutorials. I can execute the Flows through the terminal with: python 02_etl_...  and it executes perfectly! I've created a project, and added the Flow to that project. Through the Prefect Server UI I can run the Flow, but it fails with the error message:  State Message: Failed to load and execute Flow's environment: ModuleNotFoundError("No module named 'aircraftlib'")  How should I handle the dependency when executing the Flows through the Prefect Server UI?

Answer: This depends partially on the type of Flow Storage and Agent you are using. Since you are running with Prefect Server, I assume you are using Local Storage + a Local Agent; in this case, you need to make sure the aircraftlib directory is on your local importable Python PATH. There are a few ways of doing this:  run your Prefect Agent in the tutorial directory; your Local Agent's path will then be inherited by the flows it submits manually add the tutorial/ directory to your global python path (I don't recommend this) add the tutorial/ directory to your Agent's path with the -p CLI flag; for example: prefect agent start -p ~/Developer/prefect/examples/tutorial (this is the approach I recommend)
Question: I've been looking into type hinting my code but noticed that Python programmers typically do not type hint self in their programs Even when I look at the docs, they do not seem to type hint self, see here. This is from version 3.10 post forward declarations  def __init__(self, value: T, name: str, logger: Logger) -> None:  I can understand why this is an issue before type annotations were introduced in 3.7 with Forward declarations More info here and here  The reason this seems useful to me is mypy seems able to catch bugs with this problem example: from __future__ import annotations class Simple(object): def __init__(self: Simple): print(self.x)  would return this from mypy mypy test.py test.py:5: error: "Simple" has no attribute "x" Found 1 error in 1 file (checked 1 source file)  Which if you remove the type from self becomes Success: no issues found in 1 source file    Is there a reason that self is not annotated or is this only convention? Are there trade offs I'm missing or is my annotation of self wrong for some reason? 

Answer: mypy usually handles the type of self without needing an explicit annotation. You're running into a different problem - a method with no argument or return type annotations is not type-checked at all. For a method with no non-self arguments, you can avoid this by annotating self, but you can also avoid it by annotating the return type.
Question: Celery - bottom line: I want to get the task name by using the task id (I don't have a task object) Suppose I have this code: res = chain(add.s(4,5), add.s(10)).delay() cache.save_task_id(res.task_id)  And then in some other place: task_id = cache.get_task_ids()[0] task_name = get_task_name_by_id(task_id) #how? print(f'Some information about the task status of: {task_name}')  I know I can get the task name if I have a task object, like here: celery: get function name by task id?. But I don't have a task object (perhaps it can be created by the task_id or by some other way? I didn't see anything related to that in the docs). In addition, I don't want to save in the cache the task name. (Suppose I have a very long chain/other celery primitives, I don't want to save all their names/task_ids. Just the last task_id should be enough to get all the information regarding all the tasks, using .parents, etc) I looked at all the relevant methods of AsyncResult and AsyncResult.Backend objects. The only thing that seemed relevant is backend.get_task_meta(task_id), but that doesn't contain the task name. Thanks in advance PS: AsyncResult.name always returns None: result = AsyncResult(task_id, app=celery_app) result.name #Returns None result.args #Also returns None 

Answer: Finally found an answer. For anyone wondering: You can solve this by enabling result_extended = True in your celery config. Then: result = AsyncResult(task_id, app=celery_app) result.task_name #tasks.add
Question: I am using BeautifulSoup to extract data from HTML files. I want to get all of the information between two tags. This means that if I have an HTML section like this: <h1></h1> Text <i>here</i> has no tag <div>This is in a div</div> <h1></h1>  Then if I wanted all of the information between the first h1 and the second h1, the output would look like this: Text <i>here</i> has no tag <div>This is in a div</div>  I've tried nextsibling loops, but there always seems to be a catch. Is there a command in beautifulsoup that simply pulls everything (Text, newlines, divs, special characters) that is between element "A" and element "B"?

Answer: One solution is to .extract() all content in front of first <h1> and after second <h1> tag: from bs4 import BeautifulSoup html_doc = ''' This I <b>don't</b> want <h1></h1> Text <i>here</i> has no tag <div>This is in a div</div> <h1></h1> This I <b>don't</b> want too ''' soup = BeautifulSoup(html_doc, 'html.parser') for c in list(soup.contents): if c is soup.h1 or c.find_previous('h1') is soup.h1: continue c.extract() for h1 in soup.select('h1'): h1.extract() print(soup)  Prints: Text <i>here</i> has no tag <div>This is in a div</div>
Question: I have a very minimalist Dockerfile for my production Django application: FROM python:3.8 ENV PYTHONDONTWRITEBYTECODE 1 ENV PYTHONUNBUFFERED 1 RUN apt-get update && apt-get -y upgrade WORKDIR /app COPY requirements.txt ./ RUN pip install --upgrade pip && \ pip install -r requirements.txt COPY . . EXPOSE 8000 CMD [ "gunicorn", "api.wsgi:application", "--bind=0.0.0.0" ]  Do I need to run apt-get update && apt-get -y upgrade? From my understanding, the two commands (1) download the latest listing of available packages and (2) upgrade already installed packages. Why does the official python docker image not do this already? If I don't need to run them in this minimalist Dockerfile, when do I need to run them? I've noticed they're commonly run when installing other packages.

Answer: The base Docker Hub Linux distribution images like ubuntu:18.04 actually update themselves fairly regularly: if you docker pull ubuntu:18.04, wait a week, and repeat it, you will get a newer image. You're somewhat dependent on intermediate images, like python:3.8, doing the same thing. It is unusual, but not unheard-of, to run apt-get update and similar "upgrade everything" commands in a Dockerfile; it is more common to either assume the base image is up-to-date already, or to have specific provenance requirements and build everything from scratch on top of a base distribution image. If you're using a major-version or minor-version image tag (python:3, python:3.8, python:3.8-buster) you're probably okay, so long as you make sure to update your base image to something listed on the Docker Hub image page periodically. If you're forcing a specific patch level (python:3.8.4) you're at some risk, since these images stop getting updates once a newer upstream version is released. You do need to run apt-get update if you need to install any OS-level packages, and for Docker layer-caching reasons you should do it in the same RUN command as the corresponding apt-get install # Doesn't usually have an "upgrade" RUN apt-get update \ && DEBIAN_FRONTEND=noninteractive \ apt-get install --no-install-recommends --assume-yes \ a-package \ another-package \ more-packages