Update Dockerfiles to install packages inside a virtual environment
Author: Sai-Suraj-27
Pull request: yes

# What does this PR do?
Since we are explicitly setting this `UV_PYTHON` to use the global python interpreter in the containers. All packages will be installed globally.
```bash
ENV UV_PYTHON=/usr/local/bin/python
```
No use of creating a virtual environment as we are not using it anywhere. Right now it's just creating an extra virtual environment in the container with no packages being installed. So, `uv venv` can be removed.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@ydshieh @Rocketknight1 

Comments:
Hi @Sai-Suraj-27 . I have some doubts regarding this. See below.

My global python is `/usr/bin/python3`. Using `uv venv` with `UV_PYTHON=/usr/bin/python3` will install the package `zipp` to the newly created environment `.venv`, and yet at this point, the global python environment won't have it.

They only share the python interpreter, but all the packages are isolated.

```bash
root@fc6ee5f0b28d:~# python3 -m pip freeze
uv==0.7.16
root@fc6ee5f0b28d:~# python3 -c 'import sys; print(sys.executable)'
/usr/bin/python3
root@fc6ee5f0b28d:~# UV_PYTHON=/usr/bin/python3
root@fc6ee5f0b28d:~# echo $UV_PYTHON
/usr/bin/python3


root@fc6ee5f0b28d:~# uv venv && uv pip install zipp && uv pip freeze
Using CPython 3.10.12 interpreter at: /usr/bin/python3
Creating virtual environment at: .venv
Activate with: source .venv/bin/activate
Resolved 1 package in 31ms
Installed 1 package in 3ms
 + zipp==3.23.0
zipp==3.23.0


root@fc6ee5f0b28d:~# python3 -m pip freeze
uv==0.7.16


root@fc6ee5f0b28d:~# uv pip freeze
zipp==3.23.0


root@fc6ee5f0b28d:~#
root@fc6ee5f0b28d:~# uv run python3 -c 'import sys; print(sys.executable)'
/root/.venv/bin/python3
root@fc6ee5f0b28d:~# ls -la /root/.venv/bin/python3
lrwxrwxrwx 1 root root 6 Jun 28 11:15 /root/.venv/bin/python3 -> python

```

> ```shell
> root@fc6ee5f0b28d:~# python3 -m pip freeze
> uv==0.7.16
> root@fc6ee5f0b28d:~# python3 -c 'import sys; print(sys.executable)'
> /usr/bin/python3
> root@fc6ee5f0b28d:~# UV_PYTHON=/usr/bin/python3
> root@fc6ee5f0b28d:~# echo $UV_PYTHON
> /usr/bin/python3
> 
> 
> root@fc6ee5f0b28d:~# uv venv && uv pip install zipp && uv pip freeze
> Using CPython 3.10.12 interpreter at: /usr/bin/python3
> Creating virtual environment at: .venv
> Activate with: source .venv/bin/activate
> Resolved 1 package in 31ms
> Installed 1 package in 3ms
>  + zipp==3.23.0
> zipp==3.23.0
> 
> 
> root@fc6ee5f0b28d:~# python3 -m pip freeze
> uv==0.7.16
> 
> 
> root@fc6ee5f0b28d:~# uv pip freeze
> zipp==3.23.0
> 
> 
> root@fc6ee5f0b28d:~#
> root@fc6ee5f0b28d:~# uv run python3 -c 'import sys; print(sys.executable)'
> /root/.venv/bin/python3
> root@fc6ee5f0b28d:~# ls -la /root/.venv/bin/python3
> lrwxrwxrwx 1 root root 6 Jun 28 11:15 /root/.venv/bin/python3 -> python
> ```


Hi, @ydshieh. Thanks for immediate reply. I understand that in your code it is by default installing packages in the `.venv` created without activating it. But if we build the actual Dockerfiles (`tf-light.dockerfile` for example) & inspect the container, this is not happening.

Inside the `.venv/` the site-packages directory does not contain any packages. But inside the `usr/local/lib/python3.9/site-packages/` all the installed packages are present.
I am showing you the output for both commands inside the `tf-light.dockerfile` image container. For the command `ls -la /usr/local/lib/python3.9/site-packages/` i just trimmed the output, as the actual output is very long listing all the packages installed.

```bash
root@a7ed4cb588e7:/# ls -la .venv/lib/python3.9/site-packages/
total 20
drwxr-xr-x 2 root root 4096 Jun 28 11:35 .
drwxr-xr-x 3 root root 4096 Jun 28 11:35 ..
-rw-r--r-- 1 root root   18 Jun 28 11:35 _virtualenv.pth
-rw-r--r-- 1 root root 4342 Jun 28 11:35 _virtualenv.py
root@a7ed4cb588e7:/# ls -la /usr/local/lib/python3.9/site-packages/
total 11308
drwxr-xr-x  1 root root    4096 Jun 28 11:38 .
drwxr-xr-x  1 root root    4096 Jun 10 23:51 ..
drwxr-xr-x  2 root root    4096 Jun 28 11:38 GitPython-3.1.18.dist-info
drwxr-xr-x  2 root root    4096 Jun 28 11:38 MarkupSafe-3.0.2.dist-info
drwxr-xr-x  2 root root    4096 Jun 28 11:38 PIL
drwxr-xr-x  2 root root    4096 Jun 28 11:38 PyYAML-6.0.2.dist-info
drwxr-xr-x  2 root root    4096 Jun 28 11:38 PyYAML.libs
-rw-r--r--  1 root root     119 Jun 10 23:51 README.txt
-rwxr-xr-x  1 root root  981728 Jun 28 11:37 _cffi_backend.cpython-39-x86_64-linux-gnu.so
drwxr-xr-x  2 root root    4096 Jun 28 11:35 _distutils_hack
-rw-r--r--  1 root root    6573 Jun 28 11:37 _hypothesis_ftz_detector.py
-rw-r--r--  1 root root    1214 Jun 28 11:37 _hypothesis_globals.py
-rw-r--r--  1 root root   20112 Jun 28 11:37 _hypothesis_pytestplugin.py
drwxr-xr-x  2 root root    4096 Jun 28 11:38 _multiprocess
drwxr-xr-x  8 root root    4096 Jun 28 11:38 _pytest
-rw-r--r--  1 root root    5783 Jun 28 11:37 _soundfile.py
drwxr-xr-x  2 root root    4096 Jun 28 11:38 _soundfile_data
drwxr-xr-x  2 root root    4096 Jun 28 11:38 _yaml
drwxr-xr-x  5 root root    4096 Jun 28 11:38 absl
drwxr-xr-x  3 root root    4096 Jun 28 11:38 absl_py-2.3.0.dist-info
..
..
..
```
Let me know, if I am missing something.

Thanks for sharing. Do you know why this is happening within our docker build? Maybe we are doing something wrong there.

I could take a look next week, but in case you want to dive into it.

I think it is better to figure out the way so the packages are installed inside the venv, and they are used during CI runtime 



Thanks for sharing. Do you know why this is happening within our docker buold? Maybe we are doing something wrong there.

I could take a look next week, but in case you want to dive into it.

I think it is better to figure out the way so the packages are installed inside the venv, and they are used during CI runtime 

I am also a bit surprised that we don't get problems at ci runtime even with the issue you mentioned 

> Thanks for sharing. Do you know why this is happening within our docker build? Maybe we are doing something wrong there.
> 
> I could take a look next week, but in case you want to dive into it.
> 
> I think it is better to figure out the way so the packages are installed inside the venv, and they are used during CI runtime

I am also not 100% sure but I think it is because we are doing this: `ENV UV_PYTHON=/usr/local/bin/python
`. I will try to update the Dockerfiles such that packages are installed in a `venv` & not globally. Will get back to you. Thanks!

Hi, @ydshieh. As expected this line was causing the problem: `ENV UV_PYTHON=/usr/local/bin/python`. By default if we create a virtual environment using `uv venv` and install packages with `uv pip install ...` uv will install all the packages inside the created `.venv` instead of installing globally. But since we are setting this `UV_PYTHON=/usr/local/bin/python`. It is causing uv to install all packages globally.

Reference:
https://docs.astral.sh/uv/reference/environment/#uv_python
https://docs.astral.sh/uv/pip/environments/#using-arbitrary-python-environments

I built a Docker image by removing this line: `ENV UV_PYTHON=/usr/local/bin/python` & as expected all packages are present inside the `.venv/` & not globally.
```bash
root@9433e06e57ea:/# ls -la /usr/local/lib/python3.9/site-packages/
total 60
drwxr-xr-x 1 root root 4096 Jun 29 13:07 .
drwxr-xr-x 1 root root 4096 Jun 10 23:51 ..
-rw-r--r-- 1 root root  119 Jun 10 23:51 README.txt
drwxr-xr-x 3 root root 4096 Jun 10 23:51 _distutils_hack
-rw-r--r-- 1 root root  152 Jun 10 23:51 distutils-precedence.pth
drwxr-xr-x 5 root root 4096 Jun 10 23:51 pip
drwxr-xr-x 2 root root 4096 Jun 10 23:51 pip-23.0.1.dist-info
drwxr-xr-x 5 root root 4096 Jun 10 23:51 pkg_resources
drwxr-xr-x 7 root root 4096 Jun 10 23:51 setuptools
drwxr-xr-x 2 root root 4096 Jun 10 23:51 setuptools-58.1.0.dist-info
drwxr-xr-x 3 root root 4096 Jun 29 13:07 uv
drwxr-xr-x 3 root root 4096 Jun 29 13:07 uv-0.7.16.dist-info
drwxr-xr-x 4 root root 4096 Jun 10 23:51 wheel
drwxr-xr-x 2 root root 4096 Jun 10 23:51 wheel-0.45.1.dist-info
root@9433e06e57ea:/# ls -la .venv/lib/python3.9/site-packages/
total 11328
drwxr-xr-x  1 root root   20480 Jun 29 13:10 .
drwxr-xr-x  1 root root    4096 Jun 29 13:07 ..
drwxr-xr-x  2 root root    4096 Jun 29 13:10 GitPython-3.1.18.dist-info
drwxr-xr-x  2 root root    4096 Jun 29 13:10 MarkupSafe-3.0.2.dist-info
drwxr-xr-x  2 root root    4096 Jun 29 13:10 PIL
drwxr-xr-x  2 root root    4096 Jun 29 13:10 PyYAML-6.0.2.dist-info
drwxr-xr-x  2 root root    4096 Jun 29 13:10 PyYAML.libs
-rwxr-xr-x  1 root root  981728 Jun 29 13:08 _cffi_backend.cpython-39-x86_64-linux-gnu.so
drwxr-xr-x  2 root root    4096 Jun 29 13:07 _distutils_hack
-rw-r--r--  1 root root    6573 Jun 29 13:08 _hypothesis_ftz_detector.py
-rw-r--r--  1 root root    1214 Jun 29 13:08 _hypothesis_globals.py
-rw-r--r--  1 root root   20112 Jun 29 13:08 _hypothesis_pytestplugin.py
drwxr-xr-x  2 root root    4096 Jun 29 13:10 _multiprocess
drwxr-xr-x  8 root root    4096 Jun 29 13:10 _pytest
-rw-r--r--  1 root root    5783 Jun 29 13:08 _soundfile.py
drwxr-xr-x  2 root root    4096 Jun 29 13:10 _soundfile_data
-rw-r--r--  1 root root      18 Jun 29 13:07 _virtualenv.pth
-rw-r--r--  1 root root    4342 Jun 29 13:07 _virtualenv.py
drwxr-xr-x  2 root root    4096 Jun 29 13:10 _yaml
drwxr-xr-x  5 root root    4096 Jun 29 13:10 absl
drwxr-xr-x  3 root root    4096 Jun 29 13:10 absl_py-2.3.0.dist-info
..
..
```

Let me know if anything needs to be changed in the CI config side.

Thanks for checking.

However, I think something strange here within the docker. Because I tried to some the same thing (set UV_PYTHON=/usr/local/bin/python) in a runner (not the one used by CircleCI), and yet get

https://github.com/huggingface/transformers/pull/39098#issuecomment-3015192823

which corresponding to what mentioned in uv's documentation, i.e. only the python interpreter will be linked (i.e. share the same python interpreter, but the packages will be installed in the 2 different places.

I will have to check it in more details why there is an issue within docker (build).

> Thanks for checking.
> 
> However, I think something strange here within the docker. Because I tried to some the same thing (set UV_PYTHON=/usr/local/bin/python) in a runner (not the one used by CircleCI), and yet get
> 
> [#39098 (comment)](https://github.com/huggingface/transformers/pull/39098#issuecomment-3015192823)
> 
> which corresponding to what mentioned in uv's documentation, i.e. only the python interpreter will be linked (i.e. share the same python interpreter, but the packages will be installed in the 2 different places.
> 
> I will have to check it in more details why there is an issue within docker (build).

Hi, @ydshieh I think the reason `uv` is installing in the `.venv` here in  [#39098 (comment)](https://github.com/huggingface/transformers/pull/39098#issuecomment-3015192823) is because you are setting `UV_PYTHON=/usr/bin/python3` which just creates a shell variable & It is not passed down to any child processes that you start from that shell. Meaning when you are next running `uv pip install zipp`, it will not see the `UV_PYTHON` variable. In contrast, if you try `export UV_PYTHON=/usr/bin/python3`  or `UV_PYTHON=/usr/bin/python3 && uv pip install zipp` then uv will not install in the `.venv`.

So, in Dockerfiles whe we are setting `ENV UV_PYTHON=/usr/local/bin/python` this is equivalent to `export UV_PYTHON=/usr/local/bin/python` So any command that comes after this ENV line in the Dockerfile will have access to UV_PYTHON as an exported environment variable & uv is installing packages globally due to this.

Very nice, I will try it to make sure, and determine what to do 🙏 

@Sai-Suraj-27 You are completely right about `export`.

> I built a Docker image by removing this line: ENV UV_PYTHON=/usr/local/bin/python & as expected all packages are present inside the .venv/ & not globally.

I guess I miss this comment (or misread it). That is great. I will built the (testing) images and see if they work well when using with our CircleCI workflow.

Thank you a lot for finding this, pointing out about `export` and find a direction to go.

@ArthurZucker 

Do you remember why we use `ENV UV_PYTHON=/usr/local/bin/python` in the docker file for the CircleCI docker images?

Using it will install everything (packages) to the global python's site-packages, and `.venv` is empty.

As suggested by @Sai-Suraj-27 , we can either remove `uv venv` command (still everything is installed to global python), or remove `ENV UV_PYTHON=/usr/local/bin/python`  (packages are install to the virtual env).

Either works for me, but maybe you can confirm we don't miss any detail why you use `ENV UV_PYTHON=/usr/local/bin/python` back then.

When I used `ENV UV_PYTHON=/usr/local/bin/python` it was because UV was just not working without this at the time!
Things have changed, `uv` is a lot more stable now! 

I will try

> remove ENV UV_PYTHON=/usr/local/bin/python (packages are install to the virtual env).

when I am back 

I have made it to work with the `.venv` created by `uv`. 

It requires a bit more change to correctly use it however, and I don't think it brings a lot of value over the current approach of using the default python.

@Sai-Suraj-27 / @ArthurZucker Could you take a look.

build-doc

ok, let's simply using 

> ENV UV_PYTHON=/usr/local/bin/python

but 

change

> ["uv venv && uv pip install ."]

to 

> [ uv pip install ."]

will merge once updated

@Sai-Suraj-27 

I just update the PR and verified with the new built images and updated circleci files.

(I think it is just what you suggested initially 😅 , sorry for taking so long)

Would you like to take a final look and give a ✅  before I merge?)

Thanks for the patience and the initiative!

> @Sai-Suraj-27
> 
> I just update the PR and verified with the new built images and updated circleci files.
> 
> (I think it is just what you suggested initially 😅 , sorry for taking so long)
> 
> Would you like to take a final look and give a ✅ before I merge?)
> 
> Thanks for the patience and the initiative!

Yes, Thanks for reviewing the PR and working on it. Changes look good to me 👍🏻.