ktrapeznikov commited on
Commit
8933f91
·
verified ·
1 Parent(s): 4fba3fd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +50 -2
README.md CHANGED
@@ -1,4 +1,52 @@
1
- # SAFE Example Submission
2
 
3
- The key requirements is to have a `script.py` file in the top level directory of the repo.
 
4
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SAFE Example Submission with Custom Environment
2
 
3
+ This is an example of a submission that includes a custom environment
4
+ So instead of using the default environment of the submission, we first create a custome environment, package it with conda-pack and then push to the model hub in addition to the rest of the files.
5
 
6
+ Here are the steps:
7
+
8
+ 1. Determine the environment that you want and save to the `requirements-custom.txt` file.
9
+ - The best way is to start from the base competition image, create a new environment, test it with your model and use `pip freeze` to export it.
10
+
11
+ to start the container:
12
+
13
+ ```bash
14
+ docker run -it --rm --gpus all -v /path/to/local:/app/custom-env huggingface/competitions:latest bash
15
+ ```
16
+
17
+ to create an environment:
18
+ ```bash
19
+ conda init
20
+ source ~/.bashrc
21
+ conda create -n custom python=3.10 -y
22
+ conda activate custom
23
+ ```
24
+
25
+
26
+ 2. Export the environment with `conda-pack` to a `.tar.gz` file.
27
+ Run `docker-compose up`. This will pull the base competition image, copy the files into the container and create a new environment save it.
28
+ Change the local place where the volume `/app/custom-env` mounts in the docker-compose file. It will run `make_custom_env.sh` inside the container`.
29
+
30
+ ```bash
31
+ conda init
32
+ source ~/.bashrc
33
+ conda create -n custom python=3.10 -y
34
+ conda activate custom
35
+ pip install -r requirements-custom.txt
36
+ export OUTPUT=/app/custom-env/custom.tar.gz
37
+ conda deactivate
38
+ pip install conda-pack
39
+ conda-pack -n custom -o $OUTPUT
40
+ ```
41
+
42
+ 3. Upload the enviroment. Since the environment file is large. It's better to upload it seperately using fast HF_TRANSFER.
43
+ Make sure to install `pip install huggingface_hub[hf_transder]`.
44
+
45
+ ```bash
46
+ bash upload_env_file
47
+ ```
48
+
49
+ Make sure to change the path to your env file in the script.
50
+
51
+
52
+ 5. During runtime, `script.py` unpack the environment and run your model in your custome environment