Question: I've been a developer for about three years, primarily working in TypeScript and Node.js. I'm trying to broaden my skillset by learning Python (and eventually expanding my learning of computer vision, machine learning (ML), etc.), but I feel incredibly frustrated by trying to get Python to work consistently on my machine. Surely I'm doing something wrong, but I just can't really understand what it is. I've run into these problems mostly when using ML packages (TensorFlow, Whisper, OpenCV (although I was eventually able to resolve this), so I don't know if it's related to M1 support of one of the common dependencies, etc. My current understanding of Python is that:  M1 support of Python is version-dependent at best. venv is the only environment manager I should need to use I should use pyenv to install Python versions so as to not conflict with OS-installed python (macOS dependencies)  I'll use the latest project I'm working on as an example. My machine and environment  Mac Pro M1, macOS v12.6 (Monterey)  pyenv 2.3.9  Python 3.7.13  Fish shell version 3.5.1  My general workflow to get a project started:  Create a virtual environment using venv python3 -m venv <some_environment_name> Open created directory using Visual Studio Code, and activate the virtual environment   Here is where I encounter my first issue, which seems to be persistent.  source /Users/paal/src/whisper_transcription  Output: /bin/activate.fish functions: Function '_old_fish_prompt' does not exist ~/src/whisper_transcription/bin/activate.fish (line 18): functions -c _old_fish_prompt fish_prompt ^ in function 'deactivate' with arguments 'nondestructive' called on line 30 of file ~/src/whisper_transcription/bin/activate.fish from sourcing file ~/src/whisper_transcription/bin/activate.fish (Type 'help functions' for related documentation) functions: Function 'fish_prompt' does not exist ~/src/whisper_transcription/bin/activate.fish (line 47): functions -c fish_prompt _old_fish_prompt ^ from sourcing file ~/src/whisper_transcription/bin/activate.fish (Type 'help functions' for related documentation) fish: Unknown command: _old_fish_prompt ~/src/whisper_transcription/bin/activate.fish (line 71): _old_fish_prompt ^ in function 'fish_prompt' in command substitution (whisper_transcription)  So, to resolve this, I add the following if statement to the fish.config file. if type -q $program _old_fish_prompt end  Looking at GitHub issues, this seems to be a persistent issue for the Fish shell and this seems to at least temporarily resolve it. Or, I just switch to Z shell (executable zsh). OK, so with that resolved I move on. The environment is activated, I'm using Z shell now, and I can successfully run a Python script that prints "hello world" to the console. Then comes the nightmare of installing any packages. It seems like any project I start has some weird edge case of compatibility issues. Between M1 processors, Python versions, builds not working correctly, etc. For example, import whisper ... # The rest of the file  This with any other code or even by itself throws the following error: Traceback (most recent call last): File "main.py", line 1, in <module> import whisper File "/Users/paal/src/whisper_transcription/lib/python3.7/site-packages/whisper/__init__.py", line 12, in <module> from .decoding import DecodingOptions, DecodingResult, decode, detect_language File "/Users/paal/src/whisper_transcription/lib/python3.7/site-packages/whisper/decoding.py", line 514 if prefix := self.options.prefix: ^ SyntaxError: invalid syntax  This appears to be some problem with the Python version. From what I understand, the := operator isn't valid syntax until Python 3.8. However, dependencies of Whisper (PyTorch) only seems to be supported up to version 3.7.9. So, you can see, it seems like I just end up in these bizarre circular problems where some dependency of some package I want to use isn't supported by either the platform or the current Python version, and they seem basically unsolvable (at least with my current knowledge of Python). Why is this seemingly so complicated? I'm clearly doing something wrong here, and obviously I'm out of my comfort and knowledge zone, but these issues feel very daunting and opaque, and difficult to actually troubleshoot this in any consistent or clear way. Is there a resource that makes this stuff more clear? Is Python development on M1 chips just this broken at the moment? How can I get past these seemingly basic issues to start actually learning? I'm not necessarily looking for a solution to this specific problem here, but if there’s general advice about environment management and how to make things work somewhat reliably, I'm fine troubleshooting. I just feel like every time I start trying to learn, I end up in these rabbit holes that take hours and hours to fix and sometimes don't even really resolve things.

Answer: Instead of creating virtual environments using venv, you can use a more sophisticated tool like Poetry which enables you to manage you environments in a much better manner. I have been working on Python projects and one thing that sucks is compatibility issue with other host machines. This is where Poetry comes into the picture. There are a handful of videos available on Poetry that you can refer to. One great feature I have to mention is it tells you which package version is compatible with your Python version. This saves a lot of time of searching on the Internet for the particular package version that works with our Python version. Every time I work on a new project, I generally create a new virtual environment inside that project only using Poetry. So when I want to give this project to someone else or run on other host machine, I give the project with the virtual environment present in the project directory itself. This way they can just run the project from the virtual environment we gave them along with the project instead of having to install all the dependencies on their system and all that. For even advanced users, there is the option of Docker where you containerize your application and stuff, and I myself am learning that right now. But looking at your scenario, I would suggest using the Poetry package for managing virtual environments and their dependencies. Here is a great tutorial on the Poetry package