eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I would like to install opencv to my conda environment from source. Since I'm using Jetson, there is no pip or conda packages that are available for opencv. I use this command for installing from source,  -D BUILD_EXAMPLES=OFF -D BUILD_opencv_python2=ON -D BUILD_opencv_python3=ON -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=${PREFIX} -D CUDA_ARCH_BIN=5.3,6.2,7.2 -D CUDA_ARCH_PTX= -D CUDA_FAST_MATH=ON -D CUDNN_VERSION='8.0' -D EIGEN_INCLUDE_PATH=/usr/include/eigen3 -D ENABLE_NEON=ON -D OPENCV_DNN_CUDA=ON -D OPENCV_ENABLE_NONFREE=ON -D OPENCV_EXTRA_MODULES_PATH=/tmp/build_opencv/opencv_contrib/modules -D OPENCV_GENERATE_PKGCONFIG=ON -D WITH_CUBLAS=ON -D WITH_CUDA=ON -D WITH_CUDNN=ON -D WITH_GSTREAMER=ON -D WITH_LIBV4L=ON -D WITH_OPENGL=ON"  How do I install the python dependencies to my conda environment instead of installing it to usr/local/python?

Answer: By default it will install to your system Python path which you can see by entering: which python  in the terminal. In your cmake commands (the above list you posted) you need to tell it which python executable path you want to build to. At the moment your build is pointing to the above default Python location, and now you want to point it to your Conda Python path. So for example, my base path for my Python environment in Anaconda is: /home/robert/anaconda3/  You can get a list of your Anaconda environments and their location by entering this in the terminal: conda env list  To do this, you'll need to update the cmake commands to tell it where the Python path which you want to build to is located. I've used this post before to help me correctly specify the Python executable build path, and it has worked for me when specifying the Python path for a venv. For example, if I wanted to install to one of my Anaconda environments I would do something like this in my cmake: -D PYTHON_DEFAULT_EXECUTABLE=$(/home/robert/anaconda3/envs/venv_openvcv/python3)  When you build cmake, scroll through the output and pay particular attention to the line which says something like: Python (for build): /home/robert/anaconda3/envs/venv_openvcv/python3  This is your way of confirming if it is about to build opencv to the correct Python executable (the Anaconda one you have specified). Edit: Additionally here is a tutorial which outlines in detail the steps to compile OpenCV for an Anaconda environment - Installing OpenCV for Conda Virtual Environments
Question: I just newly install python 3.8 via anaconda installer and install pytorch using command conda install pytorch torchvision cpuonly -c pytorch  when i try to import torch, I got this error message. OSError: [WinError 126] The specified module could not be found. Error loading "C:\Users\chunc\anaconda3\lib\site-packages\torch\lib\asmjit.dll" or one of its dependencies.  I can see dll files are still in the directory. I ran Dependency Walker and it gave me this result. I am with this problem for a day. What should i do if i want to use PyTorch module?

Answer: I had the same problem, you should check if you installed Microsoft Visual C++ Redistributable, because if you didn't this may lead to the DLL load failure. Here is a link to download it: https://aka.ms/vs/16/release/vc_redist.x64.exe
Question: I'm trying to create a fastapi API endpoint that relies on HTTP GET parameters, has them documented and uses fastapi's validation capabilities. Consider the following minimal example: import fastapi app = fastapi.FastAPI( ) @app.get("/endpoint") def example_endpoint( par1: int = fastapi.Query( None, description="example documentation1", ), par2: int = fastapi.Query( None, description="example documentation2", ), ): return {"test": par1 + par2}  This has the documentation support and works over HTTP GET parameters, but doesn't validate them - http://localhost:8000/endpoint?par1=2&par2=3 works fine, but http://localhost:8000/endpoint crashes with an internal server error, instead of notifying the user that a parameter was expected. Is there a way to make par1 and par2 required and keep the documentation feature?

Answer: You can use Ellipsis, If you hadn't seen that ... before: it is a special single value that makes query required from fastapi import Query Query(...,description="example documentation1")  So in your case answer below can do the job @app.get("/endpoint") def example_endpoint( par1: int = fastapi.Query(..., description="example documentation1",), par2: int = fastapi.Query(..., description="example documentation2",), ): if par1 and par2: return {"test": par1 + par2} raise ValueError("Missing query parameters")  Also you can use example=1 Query(..., description="example documentation2", example=1)
Question: from datetime import datetime from werkzeug.security import generate_password_hash from werkzeug.security import check_password_hash from flask_login import UserMixin from app import db class User(UserMixin, db.Model): id = db.Column(db.Integer, primary_key=True) username = db.Column(db.String(64), index=True, unique=True) email = db.Column(db.String(64), index=True, unique=True) password_hash = db.Column(db.String(64)) posts = db.relationship('Post', backref='author', lazy='dynamic') def set_password(self, password): self.password_hash = generate_password_hash(password) def check_password(self, password): return check_password_hash(self.password_hash, password) def __repr__(self): return '<User{}>'.format(self.username)  After reading the official documentation I couldn't yet understand the purpose of UserMixin. Can anyone please describe it in brief?

Answer: Flask-login requires a User model with the following properties:  has an is_authenticated() method that returns True if the user has provided valid credentials has an is_active() method that returns True if the user’s account is active has an is_anonymous() method that returns True if the current user is an anonymous user has a get_id() method which, given a User instance, returns the unique ID for that object  UserMixin class provides the implementation of this properties. Its the reason you can call for example is_authenticated to check if login credentials provide is correct or not instead of having to write a method to do that yourself.
Question: I have two pandas DataFrames / Series containing one row each. df1 = pd.DataFrame([1, 2, 3, 4]) df2 = pd.DataFrame(['one', 'two', 'three', 'four'])  I now want to get all possible combinations into an n*n matrix / DataFrame with values for all cross-products being the output from a custom function. def my_function(x, y): return f"{x}:{y}"  This should therefore result in: df = pd.DataFrame([['1:one', '2:one', '3:one', '4:one'], ['1:two', '2:two', '3:two', '4:two'], ['1:three', '2:three', '3:three', '4:three'], ['1:four', '2:four', '3:four', '4:four']]) 0 1 2 3 0 1:one 2:one 3:one 4:one 1 1:two 2:two 3:two 4:two 2 1:three 2:three 3:three 4:three 3 1:four 2:four 3:four 4:four  While I can build my own matrix through itertools.product, this seems like a very inefficient way for larger datasets and I was wondering if there is a more pythonic way. Thank you in advance.

Answer: Let us try np.add.outer df = pd.DataFrame(np.add.outer(df1[0].astype(str).values,':'+df2[0].values).T) Out[258]: 0 1 2 3 0 1:one 2:one 3:one 4:one 1 1:two 2:two 3:two 4:two 2 1:three 2:three 3:three 4:three 3 1:four 2:four 3:four 4:four
Question: My initial dataframe is:  Name Info1 Info2 0 Name1 Name1-Info1 Name1-Info2 1 Name1 Name1-Info1 Name1-Info2 2 Name1 Name1-Info1 Name1-Info2 3 Name2 Name2-Info1 Name2-Info2 4 Name2 Name2-Info1 Name2-Info2  and i would like to return the number of repetitions of each row as such:  Name Info1 Info2 Count 0 Name1 Name1-Info1 Name1-Info2 3 1 Name2 Name2-Info1 Name2-Info2 2  How can I count a pandas dataframe over duplications?

Answer: df.groupby(['Name', 'Info1', 'Info2']).size().reset_index().rename(columns={0:"count"})
Question: Imagine I have a dash application where I want the global data to refresh on page reload. I'm using a function to serve the layout as described here. However, I'm note sure how/where I should define df such that I can use it in callbacks (like in a case where I'd like to subset the df based on some input and pass it to a layout table). My code below reloads the data on page refresh, but the callback cannot access the df. I'm very new to dash so apologies in advance for potentially dumb question. def serve_layout(): df = # Fetch data from DB return # Layout app.layout = serve_layout @app.callback() def my_func: # Here I want to reference df 

Answer: The most common approach for sharing data between callbacks is to save the data in a dash_core_components.Store object, def serve_layout(): df = # Fetch data from DB store = Store(id="mystore", data=df.to_json()) # The store must be added to the layout return # Layout  You can then add the store as a State argument for the callbacks that need access to the data, @app.callback(..., [State("mystore", "data")]) def my_func(..., data): df = pd.read_json(data)  The main drawback of this approach is that the data is exchanged between the client and the server each time a callback is invoked. If the data frame is small, it doesn't really matter, but if it is large, the data exchange (and the serialization to/from JSON) might cause severe performance issues. It can be avoided by caching the data frame server side, either manually as demonstrated in the documentation or using the enriched components from dash-extensions. Here is a small example of the latter, import dash_core_components as dcc import dash_html_components as html import numpy as np import pandas as pd from dash_extensions.enrich import Dash, ServersideOutput, Output, Input, Trigger app = Dash() app.layout = html.Div([dcc.Store(id="store"), # this is the store that holds the data html.Div(id="onload"), # this div is used to trigger the query_df function on page load html.Div(id="log")]) @app.callback(ServersideOutput("store", "data"), Trigger("onload", "children")) def query_df(): return pd.DataFrame(data=np.random.rand(int(10)), columns=["rnd"]) # some random example data @app.callback(Output("log", "children"), Input("store", "data")) def print_df(df): return df.to_json() # do something with the data if __name__ == '__main__': app.run_server()  tested with dash-extensions==0.0.27rc1. Disclaimer: I am the author of dash-extensions.
Question: I have installed wget on my Python, and I'm downloading files from different URLs with it. So far my code looks like this: import wget urls = ['https://www.iedb.org/downloader.php?file_name=doc/epitope_full_v3.zip', 'https://www.iedb.org/downloader.php?file_name=doc/tcell_full_v3.zip', 'https://www.iedb.org/downloader.php?file_name=doc/bcell_full_v3.zip', 'https://www.iedb.org/downloader.php?file_name=doc/mhc_ligand_full_single_file.zip'] path = '/home/david/data/files/zip_files' for url in urls: wget.download(url, path)  I want my code to overwrite the downloaded files if they exist, so that way every time I run the code I get the latest version of that files, instead of keeping the old ones and downloading the new ones with a different name (e.g, if epitope_full_v3.zip already exists, when I execute the code it will download it again, but will keep the old one and rename the new one to epitope_full_v3_1.zip). I know that wget can take an -O argument in the shell that allows you to do that, but I have not seen that for the python version on the documentation. I appreciate your help.

Answer: Although wget doesn't mentioned that, you could change it by yourself.Use os.path.basename() to get the filename, and check whether it exists.Like this: import wget import os urls = ['https://www.iedb.org/downloader.php?file_name=doc/epitope_full_v3.zip', 'https://www.iedb.org/downloader.php?file_name=doc/tcell_full_v3.zip', 'https://www.iedb.org/downloader.php?file_name=doc/bcell_full_v3.zip', 'https://www.iedb.org/downloader.php?file_name=doc/mhc_ligand_full_single_file.zip'] path = '/home/david/data/files/zip_files' for url in urls: filename = path + '/' + os.path.basename(url) # get the full path of the file if os.path.exists(filename): os.remove(filename) # if exist, remove it directly wget.download(url, out=filename) # download it to the specific path.
Question: I have a pandas dataframe and want to create a BigQuery table from it. I understand that there are many posts asking about this question, but all the answers I can find so far require explicitly specifying the schema of every column. For example: from google.cloud import bigquery as bq client = bq.Client() dataset_ref = client.dataset('my_dataset', project = 'my_project') table_ref = dataset_ref.table('my_table') job_config = bq.LoadJobConfig( schema=[ bq.SchemaField("a", bq.enums.SqlTypeNames.STRING), bq.SchemaField("b", bq.enums.SqlTypeNames.INT64), bq.SchemaField("c", bq.enums.SqlTypeNames.FLOAT64), ] ) client.load_table_from_dataframe(my_df, table_ref, job_config=job_config).result()  However, sometimes I have a dataframe of many columns (for example, 100 columns), it's really non-trival to specify all the columns. Is there a way to do it efficiently? Btw, I found this post with similar question: Efficiently write a Pandas dataframe to Google BigQuery But seems like bq.Schema.from_dataframe does not exist: AttributeError: module 'google.cloud.bigquery' has no attribute 'Schema' 

Answer: Here's a code snippet to load a DataFrame to BQ: import pandas as pd from google.cloud import bigquery # Example data df = pd.DataFrame({'a': [1,2,4], 'b': ['123', '456', '000']}) # Load client client = bigquery.Client(project='your-project-id') # Define table name, in format dataset.table_name table = 'your-dataset.your-table' # Load data to BQ job = client.load_table_from_dataframe(df, table)  If you want to specify only a subset of the schema and still import all the columns, you can switch the last row with # Define a job config object, with a subset of the schema job_config = bigquery.LoadJobConfig(schema=[bigquery.SchemaField('b', 'STRING')]) # Load data to BQ job = client.load_table_from_dataframe(df, table, job_config=job_config)
Question: I want to make a script for detecting and reading QR codes from photos. I would like to use PyZbar for that, but I have a problem with some errors. I'm working in google colaboratory !sudo apt install tesseract-ocr !pip install pytesseract !pip install pyzbar[scripts] import shutil import os import random import re import cv2 import numpy as np import pytesseract from pytesseract import Output %matplotlib inline import matplotlib as mpl import matplotlib.pyplot as plt import matplotlib.pylab as pylab import glob import pyzbar.pyzbar from PIL import Image  this is an error I'm struggling with: ImportError Traceback (most recent call last) <ipython-input-25-d8758fa4db37> in <module>() 24 import glob 25 # ZBAR - Bar Code Reader is an open source software suite for reading bar codes from various sources, such as video streams, image files and raw intensity sensors ---> 26 import pyzbar.pyzbar 27 # PIL - Python Imaging Library 28 from PIL import Image 4 frames /usr/local/lib/python3.6/dist-packages/pyzbar/zbar_library.py in load() 63 path = find_library('zbar') 64 if not path: ---> 65 raise ImportError('Unable to find zbar shared library') 66 libzbar = cdll.LoadLibrary(path) 67 dependencies = [] ImportError: Unable to find zbar shared library  Thank You ind advance for your answers

Answer: Before you can !pip install pyzbar, you need to install libzbar with this command. !apt install libzbar0  Then, pyzbar should work.
Question: I am getting the following error message when attempting to execute Python code in PyCharm:  Cannot run program "/Users/x/.virtualenvs/untitled/bin/python" (in directory "/Users/x/PycharmProjects/untitled"): error=2, No such file or directory  I made sure everything was updated and restarted my computer, but I still get the same error. I have no idea what the problem is.  Edit I just opened my terminal and was faced with this error message: virtualenvwrapper_run_hook:12: no such file or directory: /usr/local/bin/python3.7 virtualenvwrapper.sh: There was a problem running the initialization hooks. If Python could not import the module virtualenvwrapper.hook_loader, check that virtualenvwrapper has been installed for VIRTUALENVWRAPPER_PYTHON=/usr/local/bin/python3.7 and that PATH is set properly. I have no idea what happened here. I certainly didn't touch any of this.  Edit 2 If I execute Python3 --version, I get Python 3.8.5.  Edit 3 I followed this, but this error remains:   Edit 4 This is the current state:  I think this is related.

Answer: If it helps at all this is what my venv settings looks like. I don't have the answer as to why it happens, but I find its usually when renaming the project.  In the past i've recreated the project and copied the project files directly from the old folder to the new one in a file explorer (not pycharm) and its fixed it.
Question: In Python, we can define a class and print it as such: class a: num = 1 b = a() print(b)  And we would get the output: <class '__main__.a'>  I'm trying to identify unique classes, and I have some classes with longer paths. I would like to extract the "class path", or __main__.a in the case above. For example, if I print some longer class I would get: <class 'django_seed.tests.Game'> <class 'django_seed.tests.Player'> <class 'django_seed.tests.Action'>  And I would like to extract: 'django_seed.tests.Game' 'django_seed.tests.Player' 'django_seed.tests.Action'  Since I can cast <class 'django_seed.tests.Game'> to a string, I can substring it quite easily with '<class 'django_seed.tests.Game'>'[8:-2], but I'm sure there must be a cleaner way. Thanks!

Answer: The __module__ attribute can be used to access the "path" of a class (where it was defined) and the __name__ attribute returns the name of the class as a string print(f'{YourClass.__module__}.{YourClass.__name__}')
Question: Is there a way to convert a \x escaped string like "\\xe8\\xaa\\x9e\\xe8\\xa8\\x80" into readable form: "語言"? >>> a = "\\xe8\\xaa\\x9e\\xe8\\xa8\\x80" >>> print(a) \xe8\xaa\x9e\xe8\xa8\x80  I am aware that there is a similar question here, but it seems the solution is only for latin characters. How can I convert this form of string into readable CJK characters?

Answer: Decode it first using 'unicode-escape', then as 'utf8': a = "\\xe8\\xaa\\x9e\\xe8\\xa8\\x80" decoded = a.encode('latin1').decode('unicode_escape').encode('latin1').decode('utf8') print(decoded) # 語言  Note that since we can only decode bytes objects, we need to transparently encode it in between, using 'latin1'.
Question: I have an array, given number of items in a group and number of groups, I need to print the array cyclically in a loop. Array-[1,2,3,4,5,6] Group- 4 Iterations- 7 Output should be: ['1', '2', '3', '4'] ['5', '6', '1', '2'] ['3', '4', '5', '6'] ['1', '2', '3', '4'] ['5', '6', '1', '2'] ['3', '4', '5', '6'] ['1', '2', '3', '4'] 

Answer: One solution is to combine itertools.cycle with itertools.slice. from itertools import cycle, islice def format_print(iterable, group_size, iterations): iterable = cycle(iterable) for _ in range(iterations): print(list(islice(iterable, 0, group_size))) format_print(range(1, 7), 4, 7)  Output: [1, 2, 3, 4] [5, 6, 1, 2] [3, 4, 5, 6] [1, 2, 3, 4] [5, 6, 1, 2] [3, 4, 5, 6] [1, 2, 3, 4]  If it is required to print string lists, cycle(iterable) can be replaced with cycle(map(str, iterable)).
Question: I have a list of elements like [1,3,5,6,8,7]. I want a list of sums of two consecutive elements of the list in a way that the last element is also added with the first element of the list. I mean in the above case, I want this list: [4,8,11,14,15,8] But when it comes to the addition of the last and the first element during for loop, index out of range occurs. Consider the following code: List1 = [1,3,5,6,8,7] List2 = [List1[i] + List1[i+1] for i in range (len(List1))] print(List2) 

Answer: List2 = [List1[i] + List1[(i+1)%len(List1)] for i in range (len(List1))]
Question: In pandas lot's of methods have the keyword argument inplace. This means if inplace=True, the called function will be performed on the object itself, and returns None, on the other hand if inplace=False the original object will be untouched, and the method is performed on the returned new instance. I've managed to implement this functionality as follows: from copy import copy class Dummy: def __init__(self, x: int): self.x = x def increment_by(self, increment: int, inplace=True): if inplace: self.x += increment else: obj = copy(self) obj.increment_by(increment=increment, inplace=True) return obj def __copy__(self): cls = self.__class__ klass = cls.__new__(cls) klass.__dict__.update(self.__dict__) return klass if __name__ == "__main__": a = Dummy(1) a.increment_by(1) assert a.x == 2 b = a.increment_by(2, inplace=False) assert a.x == 2 assert b.x == 4  It works as expected. However I have many methods where I repeat that same template: def function(self, inplace=True, **kwds) if inplace: # do something else: obj = copy(self) obj.function(inplace=True, *args, **kwds) return obj  To avoid repetition, I would like to create a decorator and mark functions which can be executed inplace and also non-inplace. I would like to use it this way from copy import copy class Dummy: def __init__(self, x: int): self.x = x @inplacify def increment_by(self, increment: int): self.x += increment # just the regular inplace way def __copy__(self): cls = self.__class__ klass = cls.__new__(cls) klass.__dict__.update(self.__dict__) return klass  and I expect it to behave as the same as the example ahove. I've tried writing different decorators (something starting like this def inplacify(method): def inner(self, *method_args, **method_kwds): inplace = method_kwds.pop("inplace", True) def new_method(inplace, *method_args, **method_kwds):  ) but I got stuck every time. I need the reference for self in order to return a copy of the class, which I don't have there. Also it feels a little vague to change the function signature with a decorator. I have several questions: Is this behaviour can be implemented? Do I need a class decorator? Is it considered to be a bad practice, and if so, what would be the best option to deal with such issue?

Answer: If your method has return self, the following works: import copy def inplacify(method): def wrap(self,*a,**k): inplace = k.pop("inplace",True) if inplace: method(self,*a,**k) else: return method(copy.copy(self),*a,**k) return wrap class classy: def __init__(self,n): self.n=n @inplacify def func(self,val): self.n+=val return self  I tested it: inst = classy(5) print(inst.n) inst.func(4) print(inst.n) obj = inst.func(3,inplace=False) print(inst.n,obj.n)  And got the expected result: 5 9 9 12  Hope this works for your needs.
Question: According to the official documents, using train() or eval() will have effects on certain modules. However, now I wish to achieve a similar thing with my custom module, i.e. it does something when train() is turned on, and something different when eval() is turned on. How can I do this?

Answer: Yes, you can. As you can see in the source code, eval() and train() are basically changing a flag called self.training (note that it is called recursively): def train(self: T, mode: bool = True) -> T: self.training = mode for module in self.children(): module.train(mode) return self def eval(self: T) -> T: return self.train(False)  This flag is available in every nn.Module. If your custom module inherits this base class, then it is quite simple to achieve what you want: import torch.nn as nn class MyCustomModule(nn.Module): def __init__(self): super().__init__() # [...] def forward(self, x): if self.training: # train() -> training logic else: # eval() -> inference logic