eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have an array of images where each image is stored as the following dimension  (3, 32, 32)  if I wanted to show an image using plt.imshow(img)  then I am getting the following error:  TypeError: Invalid shape (3, 32, 32) for image data  I understand why I am getting this error, because according to imshow documentation, it takes an array of shape  (M, N), or (M, N, 3), or (M, N, 4)  How can I convert the image such that it has the required dimensions without losing any of its data? Thanks!

Answer: Try transposing: img.T This will reverse the order of the dimensions, making it (M,N,3).
Question: I create a virtual environment (test_venv) and I activate it. So far, successful. HOWEVER, the path of the Python Interpreter doesn't change. I have illustrated the situation below. For clarification, the python path SHOULD BE ~/Desktop/test_venv/bin/python. >>> python3 -m venv Desktop/test_venv >>> source Desktop/test_venv/bin/activate (test_venv) >>> which python /usr/bin/python 

Answer: It is not an answer specifically to your question, but it corresponds the title of the question. I faced similar problem and couldn't find solution on Internet. Maybe someone use my experience. I created virtual environment for my python project. Some time later my python interpreter also stopped changing after virtual environment activation. Similar to how you described. My problem was that I moved the project folder to a different directory some time ago. And if I return the folder to its original directory, then everything starts working again. There is following problem resolution. You save all package requirements (for example, using 'pip freeze' or 'poetry') and remove 'venv'-folder (or in your case 'test_venv'-folder). After that we create virtual environment again, activate it and install all requirements. This approach resolved my problem.
Question: I was trying to download a GUI, but the terminal kept giving me this error:  Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.  I'm trying to install it using this command: python -m pip install --upgrade pip setuptools virtualenv 

Answer: You need to download Python from https://python.org. When in the installation, be sure to check the option that adds Python to PATH.
Question: I'm trying to get tensorflow working on my MacBook pro M1. However, I keep getting the following error when trying to import: zsh: illegal hardware instruction python I have downloaded and installed tensorflow via this link. These were my installation steps:  install a venv: python3 -m venv venv. drag the install_venv.sh (which is located within the downloaded folder) file to the terminal, add -p at the end. select the directory of the venv as the location where tensorflow should be installed. activate the venv. type "python". try to import tensorflow: import tensorflow as tf.  I'm using Python 3.8.2.

Answer: This worked for me after trying a bunch of solutions to no avail. Step 1 Using pyenv install python version 3.8.5 and set it as your default python version. This tutorial(https://realpython.com/intro-to-pyenv/) is helpful for getting pyenv configured properly. Step 1.1 Use this post(https://github.com/pyenv/pyenv/issues/1446) if you have troubles running pyenv in zsh. Step 1.2 Once you have python version 3.8.5 running which you can check by running python -V which should output: Python 3.8.5  Step 2 Install virtualenv via pip install virtualenv Step 2.1 Create a virtual environment by running virtualenv ENV Step 2.2 Activate that virtual environment by running source ENV/bin/activate Step 3 Install the tensorflow wheel called tensorflow-2.4.1-py3-none-any.whl located at this public google drive link https://drive.google.com/drive/folders/1oSipZLnoeQB0Awz8U68KYeCPsULy_dQ7 Step 3.1 Assuming you simply installed the wheel to downloads run pip install ~/Downloads/tensorflow-2.4.1-py3-none-any.whl in your activated virtual environment Step 4 Type python which will bring up >>>in your terminal and type >>> import tensorflow >>>  If there is no 'zsh illegal hardware instruction" error you should be good to go. Note: If you are using anaconda, the above will also work. You can skip the virtual env steps (assuming you have a virtual env activated through Conda) and just go straight to the pip install as mentioned above (steps 3 and later).
Question: I'm using pydantic and want to create classes which contain pandas dataframes. I was looking for this online for quite a time and did not find anything. My code for the custom types looks as following. I named the type for dataframes pd.DataFrame but obviously its not correct. Does anyone know how to declare a pandas dataframe type? import pandas as pd from pydantic import BaseModel class SubModelInput(BaseModel): a: pd.DataFrame b: pd.DataFrame class ModelInput(BaseModel): SubModelInput: SubModelInput a: pd.DataFrame b: pd.DataFrame c: pd.DataFrame  Thanks for any help!

Answer: You can activate Arbitrary Types Allowed: import pandas as pd from pydantic import BaseModel class SubModelInput(BaseModel): a: pd.DataFrame b: pd.DataFrame class Config: arbitrary_types_allowed = True class ModelInput(BaseModel): SubModelInput: SubModelInput a: pd.DataFrame b: pd.DataFrame c: pd.DataFrame class Config: arbitrary_types_allowed = True
Question: I see that asyncio.to_thread() method is been added @python 3.9+, its description says it runs blocking codes on a separate thread to run at once. see example below: def blocking_io(): print(f"start blocking_io at {time.strftime('%X')}") # Note that time.sleep() can be replaced with any blocking # IO-bound operation, such as file operations. time.sleep(1) print(f"blocking_io complete at {time.strftime('%X')}") async def main(): print(f"started main at {time.strftime('%X')}") await asyncio.gather( asyncio.to_thread(blocking_io), asyncio.sleep(1)) print(f"finished main at {time.strftime('%X')}") asyncio.run(main()) # Expected output: # # started main at 19:50:53 # start blocking_io at 19:50:53 # blocking_io complete at 19:50:54 # finished main at 19:50:54  By explanation, it seems like using thread mechanism and not context switching nor coroutine. Does this mean it is not actually an async after all? is it same as a traditional multi-threading as in concurrent.futures.ThreadPoolExecutor? what is the benefit of using thread this way then?

Answer: Source code of to_thread is quite simple. It boils down to awaiting run_in_executor with a default executor (executor argument is None) which is ThreadPoolExecutor. In fact, yes, this is traditional multithreading, сode intended to run on a separate thread is not asynchronous, but to_thread allows you to await for its result asynchronously. Also note that the function runs in the context of the current task, so its context variable values will be available inside the func. async def to_thread(func, /, *args, **kwargs): """Asynchronously run function *func* in a separate thread. Any *args and **kwargs supplied for this function are directly passed to *func*. Also, the current :class:`contextvars.Context` is propogated, allowing context variables from the main thread to be accessed in the separate thread. Return a coroutine that can be awaited to get the eventual result of *func*. """ loop = events.get_running_loop() ctx = contextvars.copy_context() func_call = functools.partial(ctx.run, func, *args, **kwargs) return await loop.run_in_executor(None, func_call)
Question: I'm tring to save an np array as an image. The problem is that, if I write the path in the imwrite function it works, but if i store it in a variable and then use this variable as path it doesn't work and returns an error. This works: cv2.imwrite('my/path/to/image.png', myarray[...,::-1])  This doesn't work new_image_path = path/'{}+.png'.format(np.random.randint(1000)) cv2.imwrite(new_image_path, myarray[...,::-1])  And it returns this error: SystemError: <built-in function imwrite> returned NULL without setting an error 

Answer: The method cv2.imwrite does not work with the Path object. Simply convert it to string when you call the method: cv2.imwrite(str(new_image_path), myarray[...,::-1])
Question: The %load line-magic command loads the content of a given file into the current cell, for instance, executing: [cell 1] %load hello_world.py  ... transform the cell into: [cell 1] # %load hello_world.py print("hello, world")  I would like to create a %load_next line-magic command which would instead load this file into the next cell. For example, executing cell 1 in the following notebook: [cell 1] %load_next hello_world.py [cell 2] print("hello, cruel world") # original content  ... would keep cell 1 unchanged and update cell 2 with the new content: [cell 1] %load_next hello_world.py [cell 2] print("hello, world")  I have tried this: from IPython.core.magic import Magics, magics_class, line_magic from pathlib import Path @magics_class class MyMagics(Magics): @line_magic def load_next(self, line): new_content = Path(line).read_text() self.shell.set_next_input(new_content, replace=False) ip = get_ipython() ip.register_magics(MyMagics)  But it inserts the content between the current and the next cell: [cell 1] %load_next hello_world.py [cell 2] print("hello, world") [cell 3] print("hello, cruel world") # original content  Is it possible to make it either replace the next cell, or delete the next cell before inserting it?

Answer: You can run below script. There is no way to get all cells, so I decided to run javascript code to remove the next cell. Js part finds all cells and remove the next cell from the current cell. I have tested on Jupyter Notebook and Jupyter Lab. from IPython.display import display, HTML, Javascript from IPython.core.magic import Magics, magics_class, line_magic from pathlib import Path @magics_class class MyMagics(Magics): @line_magic def load_next(self, line): js_script = r"""<script> if (document.getElementById('notebook-container')) { //console.log('Jupyter Notebook'); allCells = document.getElementById('notebook-container').children; selectionClass = /\bselected\b/; jupyter = 'notebook'; } else if (document.getElementsByClassName('jp-Notebook-cell').length){ //console.log('Jupyter Lab'); allCells = document.getElementsByClassName('jp-Notebook-cell'); selectionClass = /\bjp-mod-selected\b/; jupyter = 'lab'; } else { console.log('Unknown Environment'); } if (typeof allCells !== 'undefined') { for (i = 0; i < allCells.length - 1; i++) { if(selectionClass.test(allCells[i].getAttribute('class'))){ allCells[i + 1].remove(); // remove output indicators of current cell window.setTimeout(function(){ if(jupyter === 'lab') { allCells[i].setAttribute('class', allCells[i].getAttribute('class') + ' jp-mod-noOutputs'); allCells[i].getElementsByClassName('jp-OutputArea jp-Cell-outputArea')[0].innerHTML = ''; } else if(jupyter === 'notebook'){ allCells[i].getElementsByClassName('output')[0].innerHTML = ''; } }, 20); break; } } } </script>""" # remove next cell display(HTML(js_script)) new_content = Path(line).read_text() self.shell.set_next_input(new_content, replace=False) ip = get_ipython() ip.register_magics(MyMagics)
Question: I'm Making a telegram bot using Python-Telegram-bot. I wanna make it send a message to one specific user (myself in this case) to select an option. after that, it should take that option as a command and work as usual. but after 30 min... it should send me the same message making me choose an option just like before. How can I make i work? def start(update: Update, context: CallbackContext) -> None: user = update.message.from_user.username print(user) context.bot.send_message(chat_id=update.effective_chat.id, text= "Choose an option. ('/option1' , '/option 2', '/...')") def main(): updater = Updater("<MY-BOT-TOKEN>", use_context=True) updater.dispatcher.add_handler(CommandHandler('start', start)) updater.start_polling() updater.idle() if __name__ == '__main__': main()  I want this to run without getting the /start command (the update). but after this... there will be normal functions that'll get updates and after 30 min I wanna run this "start" function again without getting an update.

Answer: You can get the bot object either from the updater or the dispatcher: updater = Updater('<bot-token>') updater.bot.sendMessage(chat_id='<user-id>', text='Hello there!') # alternative: updater.dispatcher.bot.sendMessage(chat_id='<user-id>', text='Hello there!')
Question: I can impute the mean and most frequent value using dask-ml like so, this works fine: mean_imputer = impute.SimpleImputer(strategy='mean') most_frequent_imputer = impute.SimpleImputer(strategy='most_frequent') data = [[100, 2, 5], [np.nan, np.nan, np.nan], [70, 7, 5]] df = pd.DataFrame(data, columns = ['Weight', 'Age', 'Height']) df.iloc[:, [0,1]] = mean_imputer.fit_transform(df.iloc[:,[0,1]]) df.iloc[:, [2]] = most_frequent_imputer.fit_transform(df.iloc[:,[2]]) print(df) Weight Age Height 0 100.0 2.0 5.0 1 85.0 4.5 5.0 2 70.0 7.0 5.0  But what if I have 100 million rows of data it seems that dask would do two loops when it could have done only one, is it possible to run both imputers simultaneously and/or in parallel instead of sequentially? What would be a sample code to achieve that?

Answer: You can used dask.delayed as suggested in docs and Dask Toutorial to parallelise the computation if entities are independent of one another. Your code would look like: from dask.distributed import Client client = Client(n_workers=4) from dask import delayed import numpy as np import pandas as pd from dask_ml import impute mean_imputer = impute.SimpleImputer(strategy='mean') most_frequent_imputer = impute.SimpleImputer(strategy='most_frequent') def fit_transform_mi(d): return mean_imputer.fit_transform(d) def fit_transform_mfi(d): return most_frequent_imputer.fit_transform(d) def setdf(a,b,df): df.iloc[:, [0,1]]=a df.iloc[:, [2]]=b return df data = [[100, 2, 5], [np.nan, np.nan, np.nan], [70, 7, 5]] df = pd.DataFrame(data, columns = ['Weight', 'Age', 'Height']) a = delayed(fit_transform_mi)(df.iloc[:,[0,1]]) b = delayed(fit_transform_mfi)(df.iloc[:,[2]]) c = delayed(setdf)(a,b,df) df= c.compute() print(df) client.close()  The c object is a lazy Delayed object. This object holds everything we need to compute the final result, including references to all of the functions that are required and their inputs and relationship to one-another.
Question: My question is about the right way of making response in aiohttp Official aiohttp documentation gives us the example of making an async query: session = aiohttp.ClientSession() async with session.get('http://httpbin.org/get') as resp: print(resp.status) print(await resp.text()) await session.close()  I cannot understand, why is the context manager here. All i have found is that __aexit__() method awaits resp.release() method. But the documentation also tells that awaiting resp.release() is not necessary at general. That all really confuses me. Why should i do that way if i find the code below more readable and not so nested? session = aiohttp.ClientSession() resp = await session.get('http://httpbin.org/get') print(resp.status) print(await resp.text()) # I finally have not get the essence of this method. # I've tried both using and not using this method in my code, # I've not found any difference in behaviour. # await resp.release() await session.close()  I have dug into aiohttp.ClientSession and its context manager sources, but i have not found anything that could clarify the situation. In the end, my question: what's the difference?

Answer: Explicitly managing a response via async with, is not necessary but advisable. The purpose of async with for response objects is to safely and promptly release resources used by the response (via a call to resp.release()). That is, even if an error occurs the resources are freed and available for further requests/responses. Otherwise, aiohttp will also release the response resources but without guarantee of promptness. The worst case is that this is delayed for an arbitrary time, i.e. up to the end of the application and timeout of external resources (such as sockets).  The difference is not noticeable if no errors occur (in which case aiohttp cleans up unused resources) and/or if the application is short (in which case there are enough resources to not need re-use). However, since errors may occur unexpectedly and aiohttp is designed for many requests/responses, it is advisable to always default to prompt cleanup via async with.
Question: Every time a new jupyter notebook instance is opened, it requires %config Completer.use_jedi = False command to be run, before autocomplete functionality starts working. This is tiring every time, to config use_jedi to False before coding. kindly suggest if there is a permanent fix to have autocomplete in juypter notebook.

Answer: I launch my jupyterlab from docker and catch this problem. I solved like this: COPY ipython_kernel_config.py /root/.ipython/profile_default/ipython_kernel_config.py  Content ipython_kernel_config.py: c.Completer.use_jedi = False  idea: https://github.com/ipython/ipython/issues/11530
Question: Hi have this very basic test: def test_long_diff(): long_str1 = "ABCDEFGHIJ " * 10 long_str2 = "ABCDEFGHIJ " * 5 + "* " + "ABCDEFGHIJ " * 5 assert long_str1 == long_str2  Using: Python 3.8.5, pytest-6.2.1, PyCharm 2020.2, MacOs Running with pytest from a shell, the output is "useable" and the error message will point out the faulty character in the long string: (venv) ~/dev/testdiff/> pytest longdiff.py ========== test session starts =========== platform darwin -- Python 3.8.5, pytest-6.2.1, py-1.10.0, pluggy-0.13.1 [...] > assert long_str1 == long_str2 E AssertionError: assert 'ABCDEFGHIJ A...J ABCDEFGHIJ ' == 'ABCDEFGHIJ A...J ABCDEFGHIJ ' E Skipping 45 identical leading characters in diff, use -v to show E - BCDEFGHIJ * ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ E ? -- E + BCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ  Using pytest-clarity and the -vv option, I get coloured diff (not rendered bellow) and different details: E left: "ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ " E right: "ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ * ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ " E E left and right have different lengths: E len(left) == 110, len(right) == 112  But if I let Pycharm run the test (same Python version, same .venv, I just right-click the test and pick "Run 'pytest for ...'"), the output in the Run Console is almost unusable because "something along the way" turns the long strings into tuples of shorter strings before applying the diff: FAILED [100%] longdiff.py:0 (test_long_diff) ('ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ' 'ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ') != ('ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ * ABCDEFGHIJ ' 'ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ABCDEFGHIJ ') <Click to see difference>  Clicking <Click to see difference> basically shows the same garbled output in a larger window What's causing this Pycharm output? And is there a way to prevent this behaviour ? Ideally I'd like to see the output from pytest-clarity in the Run console.

Answer: So it turns out this is an hard-coded behaviour of the pytest plugin used by PyCharm. The plugin always applies pprint.pformat() to the left and right values. The behaviour described in the question then occurs when the strings are longer than 80 characters and contain white spaces. One possible workaround is to override the plugin's pytest_assertrepr_compare hook. Here's a version that worked for me. Simply paste it in your conftest.py. import pprint import pytest @pytest.hookimpl(tryfirst=True) def pytest_assertrepr_compare(config, op, left, right): if op in ('==', '!='): return ['{0} {1} {2}'.format(pprint.pformat(left, width=999), op, pprint.pformat(right, width=999))]  Another possible hack it to monkeypatch pprint.pformat: import pytest import pprint from functools import partial @pytest.fixture(scope='function', autouse=True) def fix_long_string_diffs(monkeypatch): monkeypatch.setattr(pprint, 'pformat', partial(pprint.pformat, width=999))
Question: In vscode I want to use Prettier as my default formatter, but not for Python, where I will just use autopep8. I have the following settings now: { "workbench.iconTheme": "vscode-icons", "workbench.editorAssociations": [ { "viewType": "jupyter.notebook.ipynb", "filenamePattern": "*.ipynb" } ], "git.confirmSync": false, "editor.formatOnPaste": true, "editor.formatOnSave": true, "editor.defaultFormatter": "esbenp.prettier-vscode", "python.formatting.provider": "autopep8", "explorer.confirmDelete": false, "python.showStartPage": false, "explorer.confirmDragAndDrop": false }  When I save a python file, it gives me the message: "Extension 'Pretier - code formatter cannot format etc...'. So, apparently it still uses the wrong formatter for python files. How do I change this?!

Answer: If I disabled Prettier as the default formatter, it would not format on save anymore, but my Python would be formatted by autopep8 on save. With this in mind, the following solution worked for me to have both Prettier working for other languages and autopep8 for Python: { "workbench.iconTheme": "vscode-icons", "workbench.editorAssociations": [ { "viewType": "jupyter.notebook.ipynb", "filenamePattern": "*.ipynb" } ], "git.confirmSync": false, "editor.formatOnPaste": true, "editor.formatOnSave": true, "python.formatting.provider": "autopep8", "explorer.confirmDelete": false, "python.showStartPage": false, "explorer.confirmDragAndDrop": false, "python.linting.pylintArgs": ["--load-plugins=pylint_django"], "javascript.updateImportsOnFileMove.enabled": "always", "editor.defaultFormatter": "esbenp.prettier-vscode", "[python]": { "editor.defaultFormatter": "ms-python.python" } }  Let me know if somebody finds a better solution!
Question: Here's my settings.py: INSTALLED_APPS = [ 'rest_framework', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'api.apps.ApiConfig' ] 

Answer: If you are using VSCode, Ctrl + Shift + P -> Type and select 'Python: Select Interpreter' and enter into your projects virtual environment. This is what worked for me.
Question: I am working in Python on a Jupyter Notebook, and I got this warning:  WARNING:root:'PYARROW_IGNORE_TIMEZONE' environment variable was not set. I tried to remove it, but I couldn't. I tried to set PYARROW_IGNORE_TIMEZONE to 1, as I saw on some forums but it didn't work. Here is my code : PYARROW_IGNORE_TIMEZONE=1 import databricks.koalas as ks import pyspark from pyspark.sql import SparkSession, functions from pyspark.sql.types import * import datetime  What's wrong with it ? I am using spark and koalas.

Answer: If you want to set environment variable, you should use os. Otherwise you're just setting the variable in Python, but it doesn't get exported to the environment. import os os.environ["PYARROW_IGNORE_TIMEZONE"] = "1"