eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am trying to get the timestamp of monday at 00:00 of the current week in python. I know that for a specific date, the timestamp can be found using  baseTime = int(datetime.datetime.timestamp(datetime.datetime(2020,1,1)))  However, I want my program to automatically find out, based on the date, which date monday of the current week was, and then get the timestamp. That is to say, it would return different dates this week and next week, meaning different timestamps.  I know that the current date can be found using  import datetime today = datetime.date.today()  Thanks in advance

Answer:  I am trying to get the timestamp of monday at 00:00 of the current week in python  You could use timedelta method from datetime package. from datetime import datetime, timedelta now = datetime.now() monday = now - timedelta(days = now.weekday()) print(monday)  Output 2020-01-27 08:47:01
Question: So I am using a docker for this python chat app project.  I originally had python manage.py runserver 0.0.0.0:8000 as my command in docker-compose.  I found that I should switch to gunicorn if I want to deploy my app on web (like heroku). The tutorial I found say simply change the command in docker-compose to gunicorn myproject.wsgi -b 0.0.0.0:8000. I did that, and all the websocket connections broke. There's a fail to send due to websocket is still in CONNECTING state, then after a while handshake fails with a status code of 404. All the setup were the same as before, except that one line. Just wondering what else I need to change to make websocket work with gunicorn? Thanks EDIT: after some digging on the internet it seems that gunicorn wasn't supposed to be run with websocket (wsgi asgi difference I suppose?) If anyone could tell me something I could use instead of gunicorn for web server it would be extremely appreciated, or if there's any way I can run gunicorn with my django channels still working? Thanks!!

Answer: When using ASGI, for asynchronous servers (websockets), you should use an asynchronous server, like Daphne or Uvicorn, the Django documentation has examples on how to deploy for both of them. If you want to use uvicorn directly you could do something like: uvicorn myproject.asgi:application --host 0.0.0.0 --port 8000  You can also run uvicorn through gunicorn using the worker class: gunicorn myproject.asgi:application -k uvicorn.workers.UvicornWorker -b 0.0.0.0:8000
Question: I am trying to show a plot to the notebook from a python script, but all I get is a text output showing me the type() output of the figure.I have something like this:  This is my script (a very simplified version of my actual script, but same concept). import matplotlib.pyplot as plt x=[1,2,3,4,5,6,5,3,2,4,2,3,4,2] plt.plot(x) plt.show()  I have also tried setting the backend to notebook, but I get the same result. If I plot some data from the console (writing python directly into the code textbox, the plot works fine). I have searched for the answer in a lot of forums, and even the official Google notebook to plotting in Colab says that the plot should work fine from within a script.

Answer: Instead of calling !python plot.py  Use this instead %run plot.py  It will show the plot normally.
Question: We all now the question: Change data type of columns in Pandas where it is really nice explained how to change the data type of a column, but what if I have a dataframe df with the following df.dtypes: A object B int64 C int32 D object E int64 F float32  How could I change this without explicity mention the column names that all int64 types are converted to int32 types? So the desired outcome is: A object B int32 C int32 D object E int32 F float32 

Answer: You can create dictionary by all columns with int64 dtype by DataFrame.select_dtypes and convert it to int32 by DataFrame.astype, but not sure if not fail if big integers numbers: df = pd.DataFrame({ 'A':list('abcdef'), 'B':[4,5,4,5,5,4], 'C':[7,8,9,4,2,3], 'D':[1,3,5,7,1,0], 'E':[5,3,6,9,2,4], 'F':list('aaabbb') }) d = dict.fromkeys(df.select_dtypes(np.int64).columns, np.int32) df = df.astype(d) print (df.dtypes) A object B int32 C int32 D int32 E int32 F object dtype: object
Question: I need help. VSCode will NEVER find poetry virtualenv interpreter no matter what I try.  Installed poetry Python package manager using a standard $ curl method as explained in the official documentation.  Started a project by $ poetry new finance-essentials_37-64, installed poetry environment with $ poetry install.  So now I can see that I indeed have a virtual environment by: Jaepil@Jaepil-PC MINGW64 /e/VSCodeProjects/finance_essentials_37-64 $ poetry env list >> finance-essentials-37-64-SCQrHB_N-py3.7 (Activated)  and this virtualenv is installed at: C:\Users\Jaepil\AppData\Local\pypoetry\Cache\virtualenvs, which has finance-essentials-37-64-SCQrHB_N-py3.7 directory.  However, VSCode is unable to find this virtualenv in its 'select interpreter' command. I only see a bunch of Anaconda and Pipenv environments but not the poetry environment's interpreter that I've just made.  I also added "python.venvPath": "~/.cache/pypoetry/virtualenvs", to my settings.json as suggested in here, but to no avail. Still doesn't work.  I also tried an absolute path, by adding "python.venvPath": "C:\\Users\\Jaepil\\AppData\\Local\\pypoetry\\Cache\\virtualenvs", to the same settings, but it also doesn't work.  VSCode settings reference states that it has python.poetryPath as a default but it doesn't seem to work either. Should I change the default value "poetry" in this case?   python.poetryPath  "poetry" Specifies the location of the Poetry dependency manager executable, if installed. The default value "poetry" assumes the executable is in the current path. The Python extension uses this setting to install packages when Poetry is available and there's a poetry.lock file in the workspace folder.  I'm on Windows 10 pro 64bit & Has Python 3.7.6 installed on the system.  PS C:\Users\Jaepil> python Python 3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)] on win32 

Answer: You just need to type in your shell: poetry config virtualenvs.in-project true  The virtualenv will be created inside the project path and vscode will recognize. Consider adding this to your .bashrc or .zshrc.  If you already have created your project, you need to re-create the virtualenv to make it appear in the correct place: poetry env list # shows the name of the current environment poetry env remove <current environment> poetry install # will create a new environment using your updated configuration
Question: I have a function which can take a fixed list of values: e.g. def func(mode="a"): if mode not in ["a", "b"]: raise AttributeError("not ok")  is there a way to type hint it can only be one of these two values?

Answer: I think you want a literal type: def func(mode: Literal["a", "b"] = "a"): if mode not in ["a", "b"]: raise AttributeError("not ok")  This was introduced in Python 3.8, via PEP 586.
Question: I'm trying to use pytest to test if my function is logging the expected text, such as addressed this question (the pyunit equivalent would be assertLogs). Following the pytest logging documentation, I am passing the caplog fixture to the tester. The documentation states:  Lastly all the logs sent to the logger during the test run are made available on the fixture in the form of both the logging.LogRecord instances and the final log text.  The module I'm testing is: import logging logger = logging.getLogger(__name__) def foo(): logger.info("Quinoa")  The tester is: def test_foo(caplog): from mwe16 import foo foo() assert "Quinoa" in caplog.text  I would expect this test to pass. However, running the test with pytest test_mwe16.py shows a test failure due to caplog.text being empty: ============================= test session starts ============================== platform linux -- Python 3.7.3, pytest-5.3.0, py-1.8.0, pluggy-0.13.0 rootdir: /tmp plugins: mock-1.12.1, cov-2.8.1 collected 1 item test_mwe16.py F [100%] =================================== FAILURES =================================== ___________________________________ test_foo ___________________________________ caplog = <_pytest.logging.LogCaptureFixture object at 0x7fa86853e8d0> def test_foo(caplog): from mwe16 import foo foo() > assert "Quinoa" in caplog.text E AssertionError: assert 'Quinoa' in '' E + where '' = <_pytest.logging.LogCaptureFixture object at 0x7fa86853e8d0>.text test_mwe16.py:4: AssertionError ============================== 1 failed in 0.06s ===============================  Why is caplog.text empty despite foo() sending text to a logger? How do I use pytest such that caplog.text does capture the logged text, or otherwise verify that the text is being logged?

Answer: The documentation is unclear here. From trial and error, and notwithstanding the "all the logs sent to the logger during the test run are made available" text, it still only captures logs with certain log levels. To actually capture all logs, one needs to set the log level for captured log messages using caplog.set_level or the caplog.at_level context manager, so that the test module becomes: import logging def test_foo(caplog): from mwe16 import foo with caplog.at_level(logging.DEBUG): foo() assert "Quinoa" in caplog.text  Now, the test passes: ============================= test session starts ============================== platform linux -- Python 3.7.3, pytest-5.3.0, py-1.8.0, pluggy-0.13.0 rootdir: /tmp plugins: mock-1.12.1, cov-2.8.1 collected 1 item test_mwe16.py . [100%] ============================== 1 passed in 0.04s ===============================
Question: I made the following scatter plot with Plotly:  import plotly import plotly.plotly as py from plotly.graph_objs import Scatter import plotly.graph_objs as go trace1 = go.Scatter( x=x1_tsne, # x-coordinates of trace y=y1_tsne, # y-coordinates of trace mode="markers ", # scatter mode (more in UG section 1) text=label3, opacity=1, textposition="top center", marker=dict(size=25, color=color_4, symbol=marker_list_2, line=dict(width=0.5)), textfont=dict( color="black", size=18, # can change the size of font here family="Times New Roman", ), ) layout = { "xaxis": { "showticklabels": False, "showgrid": False, "zeroline": False, "linecolor": "black", "linewidth": 2, "mirror": True, "autorange": False, "range": [-40, 40], }, "yaxis": { "showticklabels": False, "showgrid": False, "zeroline": False, "linecolor": "black", "linewidth": 2, "mirror": True, "autorange": False, "range": [-40, 40], }, } data = [trace1] fig = go.Figure(data=data, layout=layout) py.iplot(fig)   I want the height, width, and the markers to be like the plot shown below (made in Matplotlib):  I tried tuning the range, and using Autorange, but they did not help.  EDIT: The following code worked for me: trace1 = go.Scatter( x=x1_tsne, # x-coordinates of trace y=y1_tsne, # y-coordinates of trace mode="markers +text ", # scatter mode (more in UG section 1) text=label3, opacity=1, textposition="top center", marker=dict(size=25, color=color_4, symbol=marker_list_2, line=dict(width=0.5)), textfont=dict( color="black", size=18, # can change the size of font here family="Times New Roman", ), ) data = [trace1] layout = go.Layout( autosize=False, width=1000, height=1000, xaxis=go.layout.XAxis(linecolor="black", linewidth=1, mirror=True), yaxis=go.layout.YAxis(linecolor="black", linewidth=1, mirror=True), margin=go.layout.Margin(l=50, r=50, b=100, t=100, pad=4), ) fig = go.Figure(data=data, layout=layout) py.iplot(fig, filename="size-margins") 

Answer: Consider using: fig.update_layout( autosize=False, width=800, height=800, )  ...and then eventually reduce the size of your marker. Full code: import plotly.graph_objs as go trace1 = go.Scatter( x=x1_tsne, # x-coordinates of trace y=y1_tsne, # y-coordinates of trace mode="markers +text ", # scatter mode (more in UG section 1) text=label3, opacity=1, textposition="top center", marker=dict(size=12, color=color_4, symbol=marker_list_2, line=dict(width=0.5)), textfont=dict( color="black", size=18, # can change the size of font here family="Times New Roman", ), ) data = [trace1] layout = go.Layout( autosize=False, width=1000, height=1000, xaxis=go.layout.XAxis(linecolor="black", linewidth=1, mirror=True), yaxis=go.layout.YAxis(linecolor="black", linewidth=1, mirror=True), margin=go.layout.Margin(l=50, r=50, b=100, t=100, pad=4), ) fig = go.Figure(data=data, layout=layout)
Question: I have data inside an xarray.DataArray that I want to manipulate, however, it do not manage to change individual entries in the DataArray.  Example:  import numpy as np import xarray as xr data = np.random.rand(2,2) times = [1998,1999] locations = ['It','Be'] A = xr.DataArray(data, coords = [times, locations], dims = [time, space])  this gives me a DataArray. Now I want to set the entry for (1998,'It') manually to 5, but the following does not work:  A.sel(time = 1998, space = 'It').values = 5  neither this works:  A.sel(time = 1998, space = 'It').values = array(5)  the data remains as it is. However, strangely the following works out well:  A.sel(time = 1998).values[0] = 5  could you please explain me the logic behind this? 

Answer: Xarray's assignment does not allow you to assign values to arrays using sel or isel. This is described in the documentation here. For your application, you probably want to use the .loc property: A.loc[dict(time=1998, space='It')] = 5  It is also possible to use DataArray.where to replace values.
Question: Right now my requirement.txt contains following package list:  asgiref==3.2.3 beautifulsoup4==4.8.2 certifi==2019.11.28 chardet==3.0.4 Click==7.0 Django==3.0.2 idna==2.8 pytz==2019.3 requests==2.22.0 six==1.14.0 soupsieve==1.9.5 sqlparse==0.3.0 urllib3==1.25.7  I just want the package name only, so that pip3 always installs the latest version.

Answer: Use sed to remove version info from your requirements.txt file. e.g. sed 's/==.*$//' requirements.txt  will give output asgiref beautifulsoup4 certifi chardet Click Django idna pytz requests six soupsieve sqlparse urllib3  and this can be piped into pip to do the install sed 's/==.*$//' requirements.txt | xargs pip install
Question: Is it possible to use multiple format specifiers in a Python f-string? For example, let's say we want to round up numbers to two decimal points and also specify a width for print. Individually it looks like this: In [1]: values = [12.1093, 13.95123] In [2]: for v in values: print(f'{v:.2}') 1.2e+01 1.4e+01 In [3]: for v in values: print(f'{v:<10} value') 12.1093 value 13.95123 value  But, is it possible to combine both? I tried: for v in values: print(f'{v:.2,<10} value')  But I got Invalid format specifier error.

Answer: Dependent on the result you want, you can combine them normally such as; for v in values: print(f"{v:<10.2} value") #1.2e+01 value #1.4e+01 value  However, your result does not seem like the result you're looking for. To force the fixed notation of the 2 you need to add f: for v in values: print(f"{v:<10.2f} value") #12.11 value #13.95 value  You can read more on format specifications here.
Question: I am trying to display a progress bar when I perform "vector" progress_apply operations on pandas dataframes, in MS Visual Studio Code. In VS Code with the Python extension enabled, I tried in a cell import pandas as pd from tqdm import tqdm_notebook, tqdm_pandas tqdm_notebook().pandas() df = pd.DataFrame({'a' : ['foo', 'bar'], 'b' : ['spam', 'eggs']}) df.progress_apply(lambda row: row['a'] + row['b'], axis = 1)  And the result is not OK (edit: this may actually render fine on more recent versions of VS Code).  How can I visualize the progress bar when I run pandas progress_apply in vscode?

Answer: Revisiting this in 2022 (VS Code 1.63.2), the code below will work fine in VS code, and may be more appealing visually than the other solution I previously had for this: import pandas as pd from tqdm.notebook import tqdm tqdm.pandas() df = pd.DataFrame({'a' : ['foo', 'bar'], 'b' : ['spam', 'eggs']}) df.progress_apply(lambda row: row['a'] + row['b'], axis = 1)
Question: The Prettier Visual Studio Code extension does not support Django template tags {% %}. How can I fix this? Do I have to disable Prettier for HTML files, or is there another solution? See this GitHub issue too: No Django template tags support

Answer: February 2022 Based on @Al Mahdi's comment: Prettier does not support prettier.disableLanguages option anymore. Therefore, to ignore certain files you have to create a .prettierignore file akin a .gitignore file (for people who use Git). The file lives in the root folder of your project. Source of my examples below. To ignore one file you could type in a particular filename: # Ignoring just one file my_cool_html_file.html  Or you could use a blanket statement: # Ignoring all html files *.html  There is also the pragma option (<!--prettier-ignore-->) which lets you ignore bits and pieces of code from particular files. Suppose in your my_cool_html_file.html you want to not have Prettier format some lines in it, you could: <!-- prettier-ignore --> <div class="x" >hello world</div > <!-- prettier-ignore-attribute --> <div (mousedown)=" onStart ( ) " (mouseup)=" onEnd ( ) " ></div> <!-- prettier-ignore-attribute (mouseup) --> <div (mousedown)="onStart()" (mouseup)=" onEnd ( ) " ></div>  July 2020 (old answer) You can do two things:  Disable Prettier on HTML files by adding this command in the 'settings.json' file: "prettier.disableLanguages": ["html"]  This will ensure, if you have enabled it, VS Code's inherent HTML formatting.   OR  You can install a Django extension like this one. However, the problem with this extension is that it disables VS Codes inherent HTML intellisense (which I personally like).  Hope this helps.
Question: I'm trying to learn pybind11 and the first Google result is this page, where you should be guided towards compiling and running some test cases. From this page, I have installed bybind11 by: pip3 install pybind11  and I have installed: sudo apt install python3-dev cmake  as instructed in the original page. But I don't know how to go to the next step which is to  mkdir build ...  and the rest of the steps to compile the test cases. I suppose this should be inside the pybind11 installation folder installed via pip3.  my environment is:  Ubuntu 18.04.3 LTS bionic Python3 3.6.9 pip 20.0.2  and my questions are:  where is the path to the presumed test cases where I can follow the rest of the tutorial from is this the correct/best way to install pybind11? if not what is the recommended method of installation?  P.S.1. using pip3 show pybind11 I realized that I have version 2.4.3 installed and the installation folder is /usr/<userName>/.local/lib/python3.6/sitepackages. However, inside the pybind11 folder there are no test cases as far as I can see.  P.S.2. From here I installed via sudo apt install python-pybind11 and from here using dpkg --listfiles python-pybind11 I found the installation folder at /usr/lib/python2.7/dist-packages/. Not only there are no test cases in this folder either, but this is also python2.7 which I don't want to use!

Answer: You need to install pybind11 as instructed here by cloning the GitHub repository: python3 -m pip install pytest numpy scipy sudo apt install -y cmake python3-dev libeigen3-dev libboost-dev git git clone https://github.com/pybind/pybind11.git cd pybind11 cmake -DDOWNLOAD_CATCH=1 mkdir build cd build cmake .. sudo make install cd ..  Then you can run the tests by going to the folder cd tests. Next, follow steps in the tutorial, starting with mkdir build. P.S. You may also need to make sure your Python packages are up to date, following the instructions here.
Question: I want to generate a list of dates between two dates and store them in a list in string format. This list is useful to compare with other dates I have.  My code is given below:  from datetime import date, timedelta sdate = date(2019,3,22) # start date edate = date(2019,4,9) # end date def dates_bwn_twodates(start_date, end_date): for n in range(int ((end_date - start_date).days)): yield start_date + timedelta(n) print(dates_bwn_twodates(sdate,edate))  My present output:  <generator object dates_bwn_twodates at 0x000002A8E7929410>  My expected output:  ['2019-03-22',.....,'2019-04-08']  Something wrong in my code. 

Answer: You can use pandas.date_range() for this: import pandas pandas.date_range(sdate,edate-timedelta(days=1),freq='d')   DatetimeIndex(['2019-03-22', '2019-03-23', '2019-03-24', '2019-03-25', '2019-03-26', '2019-03-27', '2019-03-28', '2019-03-29', '2019-03-30', '2019-03-31', '2019-04-01', '2019-04-02', '2019-04-03', '2019-04-04', '2019-04-05', '2019-04-06', '2019-04-07', '2019-04-08'], dtype='datetime64[ns]', freq='D')
Question: I would like to perform operation similar to pandas.io.json.json_normalize is pyspark dataframe. Is there an equivalent function in spark? https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.io.json.json_normalize.html

Answer: Spark has a similar function explode() but it is not entirely identical. Here is how explode works at a very high level. >>> from pyspark.sql.functions import explode, col >>> data = {'A': [1, 2]} >>> df = spark.createDataFrame(data) >>> df.show() +------+ | A| +------+ |[1, 2]| +------+ >>> df.select(explode(col('A')).alias('normalized')).show() +----------+ |normalized| +----------+ | 1| | 2| +----------+  On the other hand you could convert the Spark DataFrame to a Pandas DataFrame using:  spark_df.toPandas() --> leverage json_normalize() and then revert back to a Spark DataFrame.  To revert back to a Spark DataFrame you would use spark.createDataFrame(pandas_df).   Please note that this back and forth solution is not ideal as calling toPandas(), results in all records of the DataFrame to be collected (.collect()) to the driver and could result in memory errors when working with larger datasets. The link below provides more insight on using toPandas(): DF.topandas() throwing error in pyspark Hope this helps and good luck!
Question: I'm trying to run the following Colab project, but when I want to split the training data into validation and train parts I get this error: KeyError: "Invalid split train[:70%]. Available splits are: ['train']"  I use the following code:  (training_set, validation_set), dataset_info = tfds.load( 'tf_flowers', split=['train[:70%]', 'train[70%:]'], with_info=True, as_supervised=True, )  How I can fix this error? 

Answer: According to the Tensorflow Dataset docs the approach you presented is now supported. Splitting is possible by passing split parameter to tfds.load like so split="test[:70%]". (training_set, validation_set), dataset_info = tfds.load( 'tf_flowers', split=['train[:70%]', 'train[70%:]'], with_info=True, as_supervised=True, )  With the above code the training_set has 2569 entries, while validation_set has 1101. Thank you Saman for the comment on API deprecation: In previous Tensorflow version it was possible to use tfds.Split API which is now deprecated: (training_set, validation_set), dataset_info = tfds.load( 'tf_flowers', split=[ tfds.Split.TRAIN.subsplit(tfds.percent[:70]), tfds.Split.TRAIN.subsplit(tfds.percent[70:]) ], with_info=True, as_supervised=True, )