eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: What's a pythonic way to validate the init arguments before instantiation w/o overriding dataclasses built-in init?  I thought perhaps leveraging the __new__ dunder-method would be appropriate?  from dataclasses import dataclass @dataclass class MyClass: is_good: bool = False is_bad: bool = False def __new__(cls, *args, **kwargs): instance: cls = super(MyClass, cls).__new__(cls, *args, **kwargs) if instance.is_good: assert not instance.is_bad return instance 

Answer: Define a __post_init__ method on the class; the generated __init__ will call it if defined: from dataclasses import dataclass @dataclass class MyClass: is_good: bool = False is_bad: bool = False def __post_init__(self): if self.is_good: assert not self.is_bad  This will even work when the replace function is used to make a new instance.
Question: I use VS code since a while with some Extensions. All is perfect expect when I use Flask. Prettier put all flask code glued together, and intellisence is not working with flask code: {% extends "layout.html" %} {% block style %} body {color: red; } {% endblock %} {% block body %} you must provide a name {% endblock %}  What can I do to make it work with flask (trie flask-snippets)? I run it in virtuel env (run before lauch vscode). Thanks in advance,

Answer: Batteries included solution Here is a solution that gives you code highlighting, tag-autocompletion and customizable auto-formatting:  Install Better Jinja or Django (better syntax highlighting within double quotes) plugin Install djLint plugin Press CTRL + Shift + P and type open settings json + Enter  This is my config and it works great for my jinja templates. djLint has more filetype-specific options to offer (see extension-description in VS Code). // settings.json // My jinja-templates use the extension `.jinja2`. To make it work with the "Django" plugin I add this to my settings: "files.associations": { "*.jinja2": "django-html" }, // Add emmet tag autocomplete for jinja and django templates "emmet.includeLanguages": { "jinja2": "html", "jinja-html": "html", "django-html": "html", }, // Set djLint as formatter for html, jinja, jinja-html, django-html "[html][jinja][jinja-html][django-html]": { "editor.formatOnSave": true, "editor.defaultFormatter": "monosans.djlint", "editor.detectIndentation": false, "editor.linkedEditing": true, "editor.tabSize": 2 }, // Add specific formatting rules as needed, e.g.: "djlint.enableLinting": true, "djlint.closeVoidTags": true, "djlint.formatAttributeTemplateTags": true, "djlint.formatCss": true, "djlint.formatJs": true, "djlint.lineBreakAfterMultilineTag": false, "djlint.maxBlankLines": 2, "djlint.maxLineLength": 100, "djlint.maxAttributeLength": 100
Question: Can you define the type hint for a variable defined with the with syntax? with example() as x: print(x)  I would like to type hint the above to say that x is a str (as an example). The only work around that I've found is to use an intermediate variable, but this feels hacky. with example() as x: y: str = x print(y)  I can't find an example in the typing documentation.

Answer: PEP 526, which has been implemented in Python 3.6, allows you to annotate variables. The variable used in a with statement can be annotated like this: x: str with example() as x: [...]
Question: I want to compute the MAD (median absolute deviation) which is defined by MAD = median(|x_i - mean(x)|)  for a list of numbers x x = list(range(0, 10)) + [1000]  However, the results differ significantly using numpy, pandas, and an hand-made implementation:  from scipy import stats import pandas as pd import numpy as np print(stats.median_absolute_deviation(x, scale=1)) # prints 3.0 print(pd.Series(x).mad()) # prints 164.54 print(np.median(np.absolute(x - np.mean(x)))) # prints 91.0  What is wrong?

Answer: The median absolute deviation is defined as: median(|x_i - median(x)|  The method mad in Pandas returns the mean absolute deviation instead. You can calculate MAD using following methods: x = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 1000] stats.median_absolute_deviation(x, scale=1) # 3.0 np.median(np.absolute(x - np.median(x))) # 3.0 x = pd.Series(x) (x - x.median()).abs().median() # 3.0
Question: Used the install guide on pytorch.org on how to install it and the command I'm using is pip install torch===1.4.0 torchvision===0.5.0 -f https://download.pytorch.org/whl/torch_stable.html  But it's coming up with this error;  ERROR: Could not find a version that satisfies the requirement torch===1.4.0 (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2) ERROR: No matching distribution found for torch===1.4.0  Is this even a me-related issue? Can other people use this command? Pip is installed and works for other modules, Python 3.8, CUDA version 10.1, Windows 10 Home 2004

Answer: Looks like this issue is related to virtual environment. Did you try recommended installation line in another/new one virtual environment? If it doesn't help the possible solution might be installing package using direct link to PyTorch and TorchVision builds for your system: Windows: pip install https://download.pytorch.org/whl/cu101/torch-1.4.0-cp38-cp38-win_amd64.whl pip install https://download.pytorch.org/whl/cu101/torchvision-0.5.0-cp38-cp38-win_amd64.whl  Ubuntu (Linux): pip install https://download.pytorch.org/whl/cu101/torch-1.4.0-cp38-cp38-linux_x86_64.whl pip install https://download.pytorch.org/whl/cu101/torchvision-0.5.0-cp38-cp38-linux_x86_64.whl
Question: I am trying to submit data from HTML forms and validate it with a Pydantic model. Using this code from fastapi import FastAPI, Form from pydantic import BaseModel from starlette.responses import HTMLResponse app = FastAPI() @app.get("/form", response_class=HTMLResponse) def form_get(): return '''<form method="post"> <input type="text" name="no" value="1"/> <input type="text" name="nm" value="abcd"/> <input type="submit"/> </form>''' class SimpleModel(BaseModel): no: int nm: str = "" @app.post("/form", response_model=SimpleModel) def form_post(form_data: SimpleModel = Form(...)): return form_data  However, I get the HTTP error: "422 Unprocessable Entity" { "detail": [ { "loc": [ "body", "form_data" ], "msg": "field required", "type": "value_error.missing" } ] }  The equivalent curl command (generated by Firefox) is curl 'http://localhost:8001/form' -H 'Content-Type: application/x-www-form-urlencoded' --data 'no=1&nm=abcd'  Here the request body contains no=1&nm=abcd. What am I doing wrong?

Answer: I found a solution that can help us to use Pydantic with FastAPI forms :) My code: class AnyForm(BaseModel): any_param: str any_other_param: int = 1 @classmethod def as_form( cls, any_param: str = Form(...), any_other_param: int = Form(1) ) -> AnyForm: return cls(any_param=any_param, any_other_param=any_other_param) @router.post('') async def any_view(form_data: AnyForm = Depends(AnyForm.as_form)): ...  It's shown in the Swagger as a usual form. It can be more generic as a decorator: import inspect from typing import Type from fastapi import Form from pydantic import BaseModel from pydantic.fields import ModelField def as_form(cls: Type[BaseModel]): new_parameters = [] for field_name, model_field in cls.__fields__.items(): model_field: ModelField # type: ignore new_parameters.append( inspect.Parameter( model_field.alias, inspect.Parameter.POSITIONAL_ONLY, default=Form(...) if model_field.required else Form(model_field.default), annotation=model_field.outer_type_, ) ) async def as_form_func(**data): return cls(**data) sig = inspect.signature(as_form_func) sig = sig.replace(parameters=new_parameters) as_form_func.__signature__ = sig # type: ignore setattr(cls, 'as_form', as_form_func) return cls  And the usage looks like @as_form class Test(BaseModel): param: str a: int = 1 b: str = '2342' c: bool = False d: Optional[float] = None @router.post('/me', response_model=Test) async def me(request: Request, form: Test = Depends(Test.as_form)): return form
Question: I ran this code in Python: from __future__ import unicode_literals import youtube_dl ydl_opts = { 'format': 'bestaudio/best', 'postprocessors': [{ 'key': 'FFmpegExtractAudio', 'preferredcodec': 'mp3', 'preferredquality': '192', }], } with youtube_dl.YoutubeDL(ydl_opts) as ydl: ydl.download(['YOUTUBE URL'])  I was hoping it would convert the Youtube video to a URL file. I got a really long error which basically repeated this:  [0;31mERROR:[0m Unable to download webpage: (caused by URLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1108)')))  I have searched online but a unsure on how to solve this problem?

Answer: Add the no-check-certificate parameter to the command: youtube-dl --no-check-certificate  This option was renamed to --no-check-certificates starting with version 2021.10.09 (inclusive).
Question: With pandas 1.0.0 the use of .to_markdown() to show the content of a dataframe in this forum in markdown is going to proliferate. Is there a convenient way to load the data back into a dataframe? Maybe an option to .from_clipboard(markdown=True)? 

Answer: None of the answers so far read the data from the clipboard. They all require the data to be in a string. This led me to look into the source code of pandas.read_clipboard(), and lo and behold, the method internally uses pandas.read_csv(), and passes all arguments to it. This automatically leads to the following solution: t = pd.DataFrame({"a": [0, 1], "b":[2, 3]}).to_markdown() print(t)  | | a | b | |---:|----:|----:| | 0 | 0 | 2 | | 1 | 1 | 3 |  Mark the table and copy it to the clipboard (Ctrl-C on Windows). Building on the above answers: pd.read_clipboard(sep="|", header=0, index_col=1, skipinitialspace=True).dropna( axis=1, how="all" ).iloc[1:]  The next step would be to integrate this into https://pyjanitor-devs.github.io/pyjanitor/ so it can be easily called with a markdown=True parameter.
Question: I have not found some R package ( there is no one, trust me) for my tasks, but there is one in python. So i wrote python script and used reticulaye::py_run_file('my_script.py') in some functions. But after building and installation, package can't find that script. Where should i put this script in order to use it after installation directly from package. And another thing, i need to install miniconda reticulate::install_miniconda(). Does anyone know the way to install it automatically after install.package command ? 

Answer: Typically non-R code goes in ./inst/python/your_script.py (likewise for JS, etc). Anything in the inst folder will be installed into your package's root directory unchanged. To call these files in your package functions, use something like: reticulate::py_run_file(system.file("python", "your_script.py", package = "yourpkg"))  See: http://r-pkgs.had.co.nz/inst.html  For your second question, you should prompt the user before installing anything, but you would usually call any external installers in a special function called .onLoad with arguments libname and pkgname. This is a function that is automatically executed when you call library(yourpkg). .onLoad <- function(libname, pkgname) { user_permission <- utils::askYesNo("Install miniconda? downloads 50MB and takes time") if (isTRUE(user_permission)) { reticulate::install_miniconda() } else { message("You should run `reticulate::install_miniconda()` before using this package") } }  You can put this function in any of your package R files.
Question: I'm trying to use vscode with jupyter via the python extension. My pipfile looks like this: [[source]] name = "pypi" url = "https://pypi.org/simple" verify_ssl = true [packages] opencv-python = "*" [requires] python_version = "3.6" [dev-packages] ipykernel = "*" ipython = "*" jupyter = "*"  To start the ipython interpreter i follow these steps:  $ pipenv install $ pipenv shell $ code . using the Python: Select interpreter, i select the pipenv environment run code  when i got to the code block by pressing shift + enter, i see the errors:  Code block: #%% import cv2  I have also tried using all dependencies in the [packages] section, reinstalling my pipenv from scratch, and repeating the above process. Always the same error, what am i missing? $ code -v 1.41.1 26076a4de974ead31f97692a0d32f90d735645c0 x64 

Answer: I had the same issue do the following 1- remove python and fresh install new python the latest version has pip in it 2- open terminal as administrator and run the following command:  pip install ipykernel --trusted-host=pypi.python.org --trusted-host=pypi.org --trusted-host=files.pythonhosted.org
Question: I am trying to create an anaconda python package. My meta.yaml looks like this: package: name: liveprint-lib version: "0.1.0" build: number: 0 requirements: build: - pip - python=3.7 - setuptools run: - python=3.7 - numpy - opencv about: home: https://github.com/monomonedula/liveprint license: Apache License 2.0 license_file: LICENSE.txt summary: Python utility library for dynamic animations projections  build.sh: $PYTHON setup.py install  The folder structure: . ├── bld.bat ├── build.sh ├── LICENSE.txt ├── liveprint ├── meta.yaml ├── README.md ├── resources ├── setup.py └── test  The error I get when running conda build . is the following: /home/vhhl/programs/anaconda3/conda-bld/liveprint-lib_1581422598848/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/bin/python: can't open file 'setup.py': [Errno 2] No such file or directory  What am I doing wrong?

Answer: Your meta.yaml file is missing a source section. Also, it's usually best to keep your recipe files in a directory of their own, rather than in the top repo. I recommend the following: mkdir conda-recipe mv meta.yaml build.sh bld.bat conda-recipe  Then, edit meta.yaml to add a source section, which points to the top-level directory of your repo. package: name: liveprint-lib version: "0.1.0" source: # Relative path to the parent directory. path: .. build: number: 0 requirements: build: - pip - python=3.7 - setuptools run: - python=3.7 - numpy - opencv  Then try: conda build conda-recipe
Question: So I was following a tutorial to connect to my jupyter notebook which is running on my remote server so that I can access it on my local windows machine. These were the steps that I followed. On my remote server : jupyter notebook --no-browser --port=8889  Then on my local machine ssh -N -f -L localhost:8888:localhost:8889 *******@**********.de.gyan.com  But I am getting an error CreateProcessW failed error:2 ssh_askpass: posix_spawn: No such file or directory Host key verification failed.  How do I resolve this? Or is there is any other way to achieve the same?

Answer: If you need the DISPLAY variable set because you want to use VcXsrc or another X-Server in Windows 10 the workaround is to add the host you want to connect to your known_hosts file. This can be done by calling ssh-keyscan -t rsa host.example.com | Out-File ~/.ssh/known_hosts -Append -Encoding ASCII;
Question: I'm trying to implement Pydantic Schema Models for the following JSON. { "description": "Best Authors And Their Books", "authorInfo": { "KISHAN": { "numberOfBooks": 10, "bestBookIds": [0, 2, 3, 7] }, "BALARAM": { "numberOfBooks": 15, "bestBookIds": [10, 12, 14] }, "RAM": { "numberOfBooks": 6, "bestBookIds": [3,5] } } }  Here are the schema objects in Pydantic from typing import List, Type, Dict from pydantic import BaseModel class AuthorBookDetails(BaseModel): numberOfBooks: int bestBookIds: List[int] class AuthorInfoCreate(BaseModel): __root__: Dict[str, Type[AuthorBookDetails]] #pass class ScreenCreate(BaseModel): description: str authorInfo: Type[AuthorInfoCreate]  I'm parsing the AuthorInfoCreate as follows: y = AuthorBookDetails( numberOfBooks = 10, bestBookIds = [3,5]) print(y) print(type(y)) x = AuthorInfoCreate.parse_obj({"RAM" : y}) print(x)  I see the following error. numberOfBooks=10 bestBookIds=[3, 5] <class '__main__.AuthorBookDetails'> Traceback (most recent call last): File "test.py", line 44, in <module> x = AuthorInfoCreate.parse_obj({"RAM": y}) File "C:\sources\rep-funds\env\lib\site-packages\pydantic\main.py", line 402, in parse_obj return cls(**obj) File "C:\sources\rep-funds\env\lib\site-packages\pydantic\main.py", line 283, in __init__ raise validation_error pydantic.error_wrappers.ValidationError: 1 validation error for AuthorInfoCreate __root__ -> RAM subclass of AuthorBookDetails expected (type=type_error.subclass; expected_class=AuthorBookDetails)  I want to understand how can I change AuthorInfoCreate so that I have the json schema mentioned.

Answer: Actually you should remove Type from type annotations. You need an instance of a class, not an actual class. Try the solution below: from typing import List,Dict from pydantic import BaseModel class AuthorBookDetails(BaseModel): numberOfBooks: int bestBookIds: List[int] class AuthorInfoCreate(BaseModel): __root__: Dict[str, AuthorBookDetails] class ScreenCreate(BaseModel): description: str authorInfo: AuthorInfoCreate
Question: I've created a setup.py for my application. Some of the dependencies i set in install_requires require pip version 19.3.1 or greater. Is there a way to check for pip version as part of setup.py? and to upgrade pip prior to build?

Answer: This is not your responsibility to build workarounds in your project for the issues in the packaging of other projects. This is kind of a bad practice. There is also not much point in doing this as part of a setup.py anyway since in many cases this file is not executed during install time. The best thing you can do is try and fix the faulty packaging of these dependency projects directly: contact the maintainers, file an issue, propose a fix, etc. The second best thing is to inform the users of your project. Clearly state this problem in the documentation of your own project and how to prevent it (i.e. "install pip version 19.3.1 or greater").  Update: If you decide to enforce a check in setup.py anyway, here are some techniques that might help... But I would still recommend against those, since your setup.py is not actually at fault here, but the issue seems to lie in the packaging of the dependencies. 1. __requires__ = ['pip >= 19.3.1'] # make sure it's before 'import setuptools' import setuptools setuptools.setup( # ... )  This would trigger an exception: pkg_resources.DistributionNotFound: The 'pip>=19.3.1' distribution was not found and is required by the application  The drawback of this technique is that it doesn't trigger when called from pip (for example: pip install .), since in that case the __main__ module is not setup.py but a module of pip. Reference:  https://setuptools.readthedocs.io/en/stable/pkg_resources.html?highlight=requires#workingset-objects  2. import pkg_resources import setuptools pkg_resources.require(['pip >= 19.3.1']) setuptools.setup( # ... )  This would trigger a pkg_resources.VersionConflict exception. This should work even if called from pip, but... This doesn't seem to work with build isolation (PEP 517, pyproject.toml), because in such a case there is usually no pip at all in the build environment. Reference:  https://setuptools.readthedocs.io/en/stable/pkg_resources.html?highlight=require#basic-workingset-methods
Question: I stumbled upon a problem, when I was working on my ETL pipeline. I am using dataclasses dataclass to parse JSON objects. One of the keywords of the JSON object is a reserved keyword. Is there a way around this: from dataclasses import dataclass import jsons out = {"yield": 0.21} @dataclass class PriceObj: asOfDate: str price: float yield: float jsons.load(out, PriceObj)  This will obviously fail because yield is reserved. Looking at the dataclasses field definition, there doesn't seem to be anything in there that can help. Go, allows one to define the name of the JSON field, wonder if there is such a feature in the dataclass?

Answer: You can decode / encode using a different name with the dataclasses_json lib, from their docs: from dataclasses import dataclass, field from dataclasses_json import config, dataclass_json @dataclass_json @dataclass class Person: given_name: str = field(metadata=config(field_name="overriddenGivenName")) Person(given_name="Alice") # Person('Alice') Person.from_json('{"overriddenGivenName": "Alice"}') # Person('Alice') Person('Alice').to_json() # {"overriddenGivenName": "Alice"}
Question: I am getting above error when i try airflow -version and airflow initdb  File "/home/ravi/sandbox/bin/airflow", line 26, in <module> from airflow.bin.cli import CLIFactory File "/home/ravi/sandbox/lib/python3.6/site-packages/airflow/bin/cli.py", line 70, in <module> from airflow.www.app import (cached_app, create_app) File "/home/ravi/sandbox/lib/python3.6/site-packages/airflow/www/app.py", line 37, in <module> from airflow.www.blueprints import routes File "/home/ravi/sandbox/lib/python3.6/site-packages/airflow/www/blueprints.py", line 25, in <module> from airflow.www import utils as wwwutils File "/home/ravi/sandbox/lib/python3.6/site-packages/airflow/www/utils.py", line 39, in <module> from flask_admin.model import filters File "/home/ravi/sandbox/lib/python3.6/site-packages/flask_admin/model/__init__.py", line 2, in <module> from .base import BaseModelView File "/home/ravi/sandbox/lib/python3.6/site-packages/flask_admin/model/base.py", line 8, in <module> from werkzeug import secure_filename ImportError: cannot import name 'secure_filename' ``` 

Answer: With this, it worked, thanks: pip install werkzeug==0.16.0 DB: sqlite:////home/centos/airflow/airflow.db [2020-02-07 12:02:02,523] {db.py:368} INFO - Creating tables