eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am trying to configure venv on Windows Subsystem for Linux with Ubuntu. What I have tried: 1) Installing venv through pip (pip3, to be exact) pip3 install venv  I get the following error ERROR: Could not find a version that satisfies the requirement venv (from versions: none) ERROR: No matching distribution found for venv  2) Installing venv through apt and apt-get sudo apt install python3-venv  In this case the installation seems to complete, but when I try to create a virtual environment with python3 -m venv ./venv, I get an error, telling me to do apt-get install python3-venv (which I just did!) The virtual environment was not created successfully because ensurepip is not available. On Debian/Ubuntu systems, you need to install the python3-venv package using the following command. apt-get install python3-venv You may need to use sudo with that command. After installing the python3-venv package, recreate your virtual environment. Failing command: ['/mnt/c/Users/Vicubso/.../code/venv/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']  I have also read the following posts post 1, post 2, and several others. None of these seem to solve my problem. Any help would be much appreciated.

Answer: Give this approach a shot: Install the pip: sudo apt-get install python-pip  Install the virtual environment: sudo pip install virtualenv  Store your virtual environments somewhere: mkdir ~/.storevirtualenvs  Now you should be able to create a new virtualenv virtualenv -p python3 yourVenv  To activate: source yourVenv/bin/activate  To exit your new virtualenv, just deactivate
Question: I want to repeatedly call a script via cron in a docker container, but when I switch from one time execution to execution via cron the official python image suddenly can't seem to find python. Dockerfile: FROM python:3.7-slim COPY main.py /home/main.py #A: works CMD [ "python", "/home/main.py" ] #B: doesn't work #RUN apt-get update && apt-get -y install -qq --force-yes cron #COPY hello-cron /etc/cron.d/hello-cron #CMD ["cron", "-f"]  main.py import time for i in range(90000): print(i) time.sleep(5000)  hello-cron: * * * * * root python /home/main.py > /proc/1/fd/1 2> /proc/1/fd/2 #  When I switch A for B in the Dockerfile the error message is: /bin/sh: 1: python: not found Thank you all for he quick responses! Adding PATH=/usr/local/bin in the cron file solved my problem.

Answer: Cron doesn't set up the PATH environment variable the same as a normal login shell so python can't be found. It should work if you specify a complete path to the Python executable, e.g. replace python with /usr/bin/python (or whatever the path to your Python executable happens to be). Alternatively you can explicitly set the PATH environment variable in the Cron configuration file to include the directory where Python can be found.
Question: How can I use Plotly to produce a line plot with a shaded standard deviation? I am trying to achieve something similar to seaborn.tsplot. Any help is appreciated. 

Answer: The following approach is fully flexible with regards to the number of columns in a pandas dataframe and uses the default color cycle of plotly. If the number of lines exceed the number of colors, the colors will be re-used from the start. As of now px.colors.qualitative.Plotly can be replaced with any hex color sequence that you can find using px.colors.qualitative: Alphabet = ['#AA0DFE', '#3283FE', '#85660D', '#782AB6', '#565656', '#1... Alphabet_r = ['#FA0087', '#FBE426', '#B00068', '#FC1CBF', '#C075A6', '... [...]   Complete code: # imports import plotly.graph_objs as go import plotly.express as px import pandas as pd import numpy as np # sample data in a pandas dataframe np.random.seed(1) df=pd.DataFrame(dict(A=np.random.uniform(low=-1, high=2, size=25).tolist(), B=np.random.uniform(low=-4, high=3, size=25).tolist(), C=np.random.uniform(low=-1, high=3, size=25).tolist(), )) df = df.cumsum() # define colors as a list colors = px.colors.qualitative.Plotly # convert plotly hex colors to rgba to enable transparency adjustments def hex_rgba(hex, transparency): col_hex = hex.lstrip('#') col_rgb = list(int(col_hex[i:i+2], 16) for i in (0, 2, 4)) col_rgb.extend([transparency]) areacol = tuple(col_rgb) return areacol rgba = [hex_rgba(c, transparency=0.2) for c in colors] colCycle = ['rgba'+str(elem) for elem in rgba] # Make sure the colors run in cycles if there are more lines than colors def next_col(cols): while True: for col in cols: yield col line_color=next_col(cols=colCycle) # plotly figure fig = go.Figure() # add line and shaded area for each series and standards deviation for i, col in enumerate(df): new_col = next(line_color) x = list(df.index.values+1) y1 = df[col] y1_upper = [(y + np.std(df[col])) for y in df[col]] y1_lower = [(y - np.std(df[col])) for y in df[col]] y1_lower = y1_lower[::-1] # standard deviation area fig.add_traces(go.Scatter(x=x+x[::-1], y=y1_upper+y1_lower, fill='tozerox', fillcolor=new_col, line=dict(color='rgba(255,255,255,0)'), showlegend=False, name=col)) # line trace fig.add_traces(go.Scatter(x=x, y=y1, line=dict(color=new_col, width=2.5), mode='lines', name=col) ) # set x-axis fig.update_layout(xaxis=dict(range=[1,len(df)])) fig.show()
Question: I am currently using SHAP Package to determine the feature contributions. I have used the approach for XGBoost and RandomForest and it worked really well. Since the data I am working on is a sequential data I tried using LSTM and CNN to train the model and then get the feature importance using the SHAP's DeepExplainer; but it is continuously throwing error. The error I am getting is: AssertionError: <class 'keras.callbacks.History'> is not currently a supported model type!.  I am attaching the sample code as well (LSTM). It would be helpful if someone could help me with it. shap.initjs() model = Sequential() model.add(LSTM(n_neurons, input_shape=(X.shape[1],X.shape[2]), return_sequences=True)) model.add(LSTM(n_neurons, return_sequences=False)) model.add(Dense(1)) model.compile(loss='mean_squared_error', optimizer='adam') h=model.fit(X, y, epochs=nb_epochs, batch_size=n_batch, verbose=1, shuffle=True) background = X[np.random.choice(X.shape[0],100, replace=False)] explainer = shap.DeepExplainer(h,background) 

Answer: The returned value of model.fit is not the model instance; rather, it's the history of training (i.e. stats like loss and metric values) as an instance of keras.callbacks.History class. That's why you get the mentioned error when you pass the returned History object to shap.DeepExplainer. Instead, you should pass the model instance itself: explainer = shap.DeepExplainer(model, background)
Question:  How can I combined multiple audio files (wav) to one file in Python? I found this: import wave infiles = ["sound_1.wav", "sound_2.wav"] outfile = "sounds.wav" data= [] for infile in infiles: w = wave.open(infile, 'rb') data.append( [w.getparams(), w.readframes(w.getnframes())] ) w.close() output = wave.open(outfile, 'wb') output.setparams(data[0][0]) output.writeframes(data[0][1]) output.writeframes(data[1][1]) output.close()  but this appends one audio file to the other. What I would like to have is code, that "stacks" the audio files (with volume controll please). Is this even possible in Python?

Answer: You can use the pydub module. It's one of the easiest ways to cut, edit, merge audio files using Python. Here's an example of how to use it to combine audio files with volume control: from pydub import AudioSegment sound1 = AudioSegment.from_file("/path/to/sound.wav", format="wav") sound2 = AudioSegment.from_file("/path/to/another_sound.wav", format="wav") # sound1 6 dB louder louder = sound1 + 6 # sound1, with sound2 appended (use louder instead of sound1 to append the louder version) combined = sound1 + sound2 # simple export file_handle = combined.export("/path/to/output.mp3", format="mp3")  To overlay sounds, try this: from pydub import AudioSegment sound1 = AudioSegment.from_file("1.wav", format="wav") sound2 = AudioSegment.from_file("2.wav", format="wav") # sound1 6 dB louder louder = sound1 + 6 # Overlay sound2 over sound1 at position 0 (use louder instead of sound1 to use the louder version) overlay = sound1.overlay(sound2, position=0) # simple export file_handle = overlay.export("output.mp3", format="mp3")  Full documentation here pydub API Documentation
Question: How can I add grid lines (vertically and horizontally) to a seaborn catplot? I found a possibility to do that on a boxplot, but I have multiple facets and therefore need a catplot instead. And in contrast to this other answer, catplot does not allow an ax argument.  This code is borrowed from here. import seaborn as sns sns.set(style="ticks") exercise = sns.load_dataset("exercise") g = sns.catplot(x="time", y="pulse", hue="kind", data=exercise) plt.show()  Any ideas? Thank you! EDIT: The provided answer is working, but for faceted plots, only the last plot inherits the grid.  import seaborn as sns sns.set(style="ticks") exercise = sns.load_dataset("exercise") g = sns.catplot(x="time", y="pulse", hue="kind", col="diet", data=exercise) plt.grid() plt.show()  Can someone explain to me why and how to fix it?

Answer: You can set the grid over seaborn plots in two ways: 1. plt.grid() method: You need to use the grid method inside matplotlib.pyplot. You can do that like so: import seaborn as sns import matplotlib.pyplot as plt sns.set(style="ticks") exercise = sns.load_dataset("exercise") g = sns.catplot(x="time", y="pulse", hue="kind", data=exercise) plt.grid() #just add this plt.show()  Which results in this graph:  2. sns.set_style() method You can also use sns.set_style which will enable grid over all subplots in any given FacetGrid. You can do that like so: import seaborn as sns import matplotlib.pyplot as plt sns.set(style="ticks") exercise = sns.load_dataset("exercise") sns.set_style("darkgrid") g = sns.catplot(x="time", y="pulse", hue="kind", col="diet", data=exercise) plt.show()  Which returns this graph:
Question: Is it possible to opencv (using python) as default read an image as order of RGB ? in the opencv documentation imread method return image as order of BGR but in code imread methods return the image as RGB order ? I am not doing any converting process. Just used imread methods and show on the screen. It shows as on windows image viewer. is it possible ?  EDIT 1: my code is below. left side cv.imshow() methods and the other one plt.imshow() methods. cv2.imshow() methods shows image as RGB and plt show it as opencv read (BGR) the image. image_file = 'image/512-2-1001-18-RGB.jpg' # img = imp.get_image(image_file) img = cv2.imread(image_file) plt.imshow(img) plt.show() cv2.imshow('asd', img) cv2.waitKey(0) cv2.destroyAllWindows()   EDIT 2 : Some how opencv imshow methods is showing the image as RGB below I have attached the first pixel's value of image and next image is photoshop pixel values   EDIT 3 : below just reading image and with imshow and second image is original RGB image. after imshow method image looks same as original image and this confused me  Original image in order of RGB. 

Answer: OpenCV is entirely consistent within itself. It reads images into Numpy arrays with the channels in BGR order, keeps the images in BGR order and its cv2.imshow() and cv2.imwrite() also expect images in BGR order. All your JPEG/PNG/BMP/TIFF files remain in their normal RGB order on disk. Other libraries, such as PIL/Pillow, scikit-image, matplotlib, pyvips store images in conventional RGB order in memory. So, you will only get colour issues if you mix OpenCV with any other library. If you go from/to OpenCV from any of the others, you will need to reverse the channel order. It is the same process either way, you are swapping the first and third channel: RGBimage = BGRimage[...,::-1]  or BGRimage = RGBimage[...,::-1]  Or, you can use OpenCV cvtColor() to do the transform: RGBimage = cv2.cvtColor(BGRimage, cv2.COLOR_BGR2RGB)   You don't need to necessarily make a whole copy in a new variable every time. Say you read an image with OpenCV, in OpenCV BGR order obviously, and you briefly want to display it with matplotlib, you can just reverse the channels as you pass it: # Load image with OpenCV and process in BGR order im = cv2.imread(something) # Briefly display with matplotlib, which will want RGB order plt.imshow(img[...,::-1]) plt.show() # Carry on processing in BGR order with OpenCV ...
Question: I am trying to escape a string such as this: string = re.split(")(", other_string)  Because not escaping those parentheses gives me an error. But if I do this: string = re.split("\)\(", other_string)  I get a warning from PEP8 that it's an invalid escape sequence. Is there a way to do this properly? Putting 'r' in front of the string does not fix it.

Answer: You probably are looking for this which would mean your string would be written as string = r")(" which would be escaped. Though that is from 2008 and in python 2 which is being phased out. What the r does is make it a "raw string" See: How to fix "<string> DeprecationWarning: invalid escape sequence" in Python? as well. What you're dealing with is string literals which can be seen in the PEP 8 Style Guide UPDATE For Edit in Question: If you're trying to split on ")(" then here's a working example that doesn't throw PEP8 warnings: import re string = r"Hello my darling)(!" print(string) string = re.split(r'\)\(', string) print(string)  Which outputs: Hello )(world! ['Hello ', 'world!']  Or you can escape the backslash explicitly by using two backslashes: import re string = r"Hello )(world!" print(string) string = re.split('\\)\\(', string) print(string)  Which outputs the same thing: Hello )(world! ['Hello ', 'world!']
Question: I installed a program with pip (e.g. simple-plotter) in windows using the following command: py -m pip install simple-plotter  How do I run the program I installed? In Linux, I can just type the command in a terminal, but if I type in windows, I get a "not recognized as an internal or external command" or if I run in Python I get a "No module named" error.

Answer: I think the error occurs because the Scripts folder is not on PATH in the environment variables, while in Linux it probably is (I don't know a lot of Linux so I don't know how environment variables work there, anyway): the best way to solve this is by adding the Scripts python folder to PATH: In my case it is C:\Users\MyUser\AppData\Local\Programs\Python\Python37-32\Scripts So, press WIN(windows key) + PAUSE_BREAK(near print screen key)  Then click on Advanced System Configuration (Sorry if it's not accurate, my system is in portuguese)  After that click in Environment Variables  Then, select Path and click edit  Click in new, add the path and hit the OKs
Question: Code snippet 1: a = True, True, True b = (True, True, True) print(a == b)  returns True. Code snippet 2: (True, True, True) == True, True, True  returns (False, True, True).

Answer: Operator precedence. You're actually checking equality between (True, True, True) and True in your second code snippet, and then building a tuple with that result as the first item. Recall that in Python by specifying a comma-separated "list" of items without any brackets, it returns a tuple: >>> a = True, True, True >>> print(type(a)) <class 'tuple'> >>> print(a) (True, True, True)  Code snippet 2 is no exception here. You're attempting to build a tuple using the same syntax, it just so happens that the first element is (True, True, True) == True, the second element is True, and the third element is True. So code snippet 2 is equivalent to: (((True, True, True) == True), True, True) And since (True, True, True) == True is False (you're comparing a tuple of three objects to a boolean here), the first element becomes False.
Question: I am an advocate of using perceptually uniform colormaps when plotting scientific data as grayscale images and applying false colorings. I don't know who invented these, but these colormaps are fantastic and I would not use anything else. Anyways to be honest, I've gotten a bit bored of the 5 colormaps (viridis, plasma, inferno, magma, cividis) which have been implemented in many popular graphing softwares (R-ggplot, python-matplotlib, matlab, JMP, etc.). I'm sure some of you also feel the same monotony... So in addition to those 5 colormaps, what are some other colormaps which are perceptually uniform? BONUS: Is there some algorithm to derive colormaps with perceptually uniform qualities (maybe not since color perception has a psychological aspect)? but if so, what is it? Some examples & refs:   https://www.youtube.com/watch?v=xAoljeRJ3lU

Answer: If you follow this page: http://bids.github.io/colormap/, you will find all the details required to produce Viridis, Magma, Inferno and Plasma. All the details are too long to enumerate as an answer but using the aforementioned page and viscm, you can regenerate them and some more interactively. Alternatively, and using Colour: import colour import numpy as np CAM16UCS = colour.convert(['#ff0000', '#00ff00'], 'Hexadecimal', 'CAM16UCS') gradient = colour.algebra.lerp( np.linspace(0, 1, 20)[..., np.newaxis], CAM16UCS[0][np.newaxis], CAM16UCS[1][np.newaxis], ) RGB = colour.convert(gradient, 'CAM16UCS', 'Output-Referred RGB') colour.plotting.plot_multi_colour_swatches( [colour.plotting.ColourSwatch(RGB=np.clip(x, 0, 1)) for x in RGB]) print(colour.convert(RGB, 'Output-Referred RGB', 'Hexadecimal')) ['#fe0000' '#fb3209' '#f74811' '#f35918' '#ef671e' '#ea7423' '#e67f28' '#e18a2c' '#dc9430' '#d79e34' '#d1a738' '#cbb03b' '#c4b93d' '#bcc23e' '#b2cc3d' '#a6d53a' '#97df36' '#82e92e' '#62f321' '#00ff00']   Note that the two boundary colours are given as hexadecimal values but you could obviously choose any relevant colourspace. Likewise, CAM16 could be swapped for JzAzBz or alike. You can try that online with this Google Colab notebook.
Question: Historically I have always used the following for reading files in python: with open("file", "r") as f: for line in f: # do thing to line  Is this still the recommend approach? Are there any drawbacks to using the following: from pathlib import Path path = Path("file") for line in path.open(): # do thing to line  Most of the references I found are using the with keyword for opening files for the convenience of not having to explicitly close the file. Is this applicable for the iterator approach here? with open() docs

Answer: If all you wanted to do was read or write a small blob of text (or bytes), then you no longer need to use a with-statement when using pathlib: >>> import pathlib >>> path = pathlib.Path("/tmp/example.txt") >>> path.write_text("hello world") 11 >>> path.read_text() 'hello world' >>> path.read_bytes() b'hello world'  These methods still use a context-manager internally (src). Opening a large file to iterate lines should still use a with-statement, as the docs show: >>> with path.open() as f: ... for line in f: ... print(line) ... hello world
Question: When bumping my python version from 3.7 to 3.8 in poetry, reinstalling all the dependencies fail with a version of the following: ERROR: No matching distribution found for...  The distribution for that version is available at pypa, and is often the most recent version. Simply removing the offending package doesn't fix the issue, as poetry will likely fail with other packages. After some investigation, it appears that somehow poetry isn't using pip3 to install underneath, but is instead using pip2.7. Indeed this is supported by a deprecation alert, and the error is always reproducible if I attempt to install the same version with pip (globally or otherwise) and not pip3. This issue is frustrating and deleting the venv alone doesn't seem to help. How can I resolve this dependency issue that shouldn't exist in the first place?

Answer: There are two issues here which feed into each other.  poetry seems to consistently botch the upgrade of a venv when you modify the python versions. According to finswimmer, the upgrade should create a new virtual env for the new python version, however this process can fail when poetry uses the wrong pip version or loses track of which virtual env it's using.  poetry uses whatever pip is no questions asked - with no way to override and force usage of pip3.   Here are the distilled steps I used to solve this issue  delete the virtual env ( sometimes poetry loses track of the venv/thinks it's already activated. Best to clear the slate )  rm -rf `poetry env list --full-path`   create a new virtual env ( the command should fail, but the venv will be created )  poetry install   manually activate the virtual env  source "$(poetry env list --full-path | tail -1 | sed 's/.\{12\}$//')/bin/activate"   poetry install within the virtual env ( this ensures poetry is using the correct version of pip )  poetry install
Question: Let's say I have some BaseModel, and I want to check that it's options list is not empty. I can perfectly do it with a validator: class Trait(BaseModel): name: str options: List[str] @validator("options") def options_non_empty(cls, v): assert len(v) > 0 return v  Are there any other, more elegant, way to do this?

Answer: If you want to use a @validator: return v if v else doSomething  Python assumes boolean-ess of an empty list as False If you don't want to use a @validator: In Pydantic, use conlist: from pydantic import BaseModel, conlist from typing import List class Trait(BaseModel): name: str options: conlist(str, min_length=1)