rortizmerino commited on
Commit
1f5f5a1
β€’
1 Parent(s): 7163618

Update README.md

Browse files

small formatting and typo change

Files changed (1) hide show
  1. README.md +169 -168
README.md CHANGED
@@ -1,168 +1,169 @@
1
- ---
2
- license: apache-2.0
3
- ---
4
-
5
- Set the Cookiecutter-MLOps in Hugging Face
6
- ==============================================
7
-
8
- 1 - Create Model repository in Hugging Face (e.g. myHFrepo)
9
- 2 - Clone your Hugging face repo to your local directory:
10
-
11
- cd /path/to/parent directory of project folder
12
- git clone git@hf.co:USERNAME/myHFrepo
13
-
14
- For ssh connection check [here](https://huggingface.co/docs/hub/security-git-ssh#add-a-ssh-key-to-your-account)
15
-
16
- 3 - Create your virtual environment (e.g. jointvenv)
17
-
18
- cd myHFrepo
19
- python -m venv jointvenv
20
- source jointvenv/bin/activate
21
-
22
- 4 - Transfer and set DagsHub's cookiecutter template employing MLOps best practices to your Huggingface repo
23
-
24
- git clone https://dagshub.com/DagsHub/Cookiecutter-MLOps.git
25
-
26
- 4.1 - Delete git files cloned from Cookiecutter-MLOps repo
27
-
28
- rm -r /path/to/myHFrepo/Cookiecutter-MLOps/.git
29
-
30
- 4.2 - dResolve conflicts with .gitattributes and README.md
31
-
32
- cat /path/to/myHFrepo/Cookiecutter-MLOps/.gitattributes >> /path/to/myHFrepo/.gitattributes
33
- rm /path/to/myHFrepo/Cookiecutter-MLOps/.gitattributes
34
- git add .gitattributes
35
- git commit -m "Paste .gitattributes info from DagsHub/Cookiecutter-MLOps"
36
-
37
- cat /path/to/myHFrepo/Cookiecutter-MLOps/README.md >> /path/to/myHFrepo/README.md
38
- rm /path/to/myHFrepo/Cookiecutter-MLOps/README.md
39
- git add README.md
40
- git commit -m "Paste README info from DagsHub/Cookiecutter-MLOps"
41
-
42
- 4.3 - Move remaining files from DagsHub/Cookiecutter-MLOps yo your Hugging Face repo .gitattributes and README.md
43
-
44
- cd /path/to/myHFrepo/Cookiecutter-MLOps
45
- mv * .[^.]* ..
46
- cd /path/to/myHFrepo
47
- rm -r /path/to/myHFrepo/Cookiecutter-MLOps
48
-
49
- 5 - Add venv folder to.gitignore
50
-
51
- echo '' >> .gitignore
52
- echo '#'Virtual Environment >> .gitignore
53
- echo jointvenv/ >> .gitignore
54
- git add .
55
- git commit -m "add remaining DagsHub/Cookiecutter-MLOps repo content"
56
-
57
- 6 - Run step 2 from DagsHub/Cookiecutter-MLOps
58
-
59
- make dirs
60
-
61
- 7 - Run step 4 from DagsHub/Cookiecutter-MLOps
62
-
63
- make requirements
64
-
65
- 8 - Keep record of your own requirements
66
-
67
- mv requirements.txt requirementsCookiecutter-MLOps.txt
68
- git add requirementsCookiecutter-MLOps.txt
69
- git commit -m "external requirements from Cookiecutter-MLOps"
70
-
71
- pip freeze > requirements.txt
72
- git add requirements.txt
73
- git commit -m "First report venv requirements"
74
-
75
- 9 - Push your changes to the remote Hugging face repository
76
-
77
- git push origin main
78
-
79
- 10 - Optional Create Model repository in your Hugging Face organization (e.g. myHFrepo)
80
-
81
- git remote add dcc git@hf.co:MYORG/mywslHFrepo
82
- git pull dcc main --allow-unrelated-histories
83
-
84
- Resolve conflicts in .gitattributes and README.md
85
-
86
- git add .
87
- git commit -m "Merge HuggingFace individual and organization repos"
88
- git push dcc main
89
-
90
- ==============================
91
- Cookiecutter-MLOps
92
- ==============================
93
-
94
- A cookiecutter template employing MLOps best practices, so you can focus on building machine learning products while
95
- having MLOps best practices applied.
96
-
97
- Instructions
98
- ------------
99
- 1. Clone the repo.
100
- 2. Run `make dirs` to create the missing parts of the directory structure described below.
101
- 3. *Optional:* Run `make virtualenv` to create a python virtual environment. Skip if using conda or some other env manager.
102
- 1. Run `source env/bin/activate` to activate the virtualenv.
103
- 4. Run `make requirements` to install required python packages.
104
- 5. Put the raw data in `data/raw`.
105
- 6. To save the raw data to the DVC cache, run `dvc add data/raw`
106
- 7. Edit the code files to your heart's desire.
107
- 8. Process your data, train and evaluate your model using `dvc repro` or `make reproduce`
108
- 9. To run the pre-commit hooks, run `make pre-commit-install`
109
- 10. For setting up data validation tests, run `make setup-setup-data-validation`
110
- 11. For **running** the data validation tests, run `make run-data-validation`
111
- 12. When you're happy with the result, commit files (including .dvc files) to git.
112
-
113
- Project Organization
114
- ------------
115
-
116
- β”œβ”€β”€ LICENSE
117
- β”œβ”€β”€ Makefile <- Makefile with commands like `make dirs` or `make clean`
118
- β”œβ”€β”€ README.md <- The top-level README for developers using this project.
119
- β”œβ”€β”€ data
120
- β”‚Β Β  β”œβ”€β”€ processed <- The final, canonical data sets for modeling.
121
- β”‚Β Β  └── raw <- The original, immutable data dump
122
- β”‚
123
- β”œβ”€β”€ models <- Trained and serialized models, model predictions, or model summaries
124
- β”‚
125
- β”œβ”€β”€ notebooks <- Jupyter notebooks. Naming convention is a number (for ordering),
126
- β”‚ the creator's initials, and a short `-` delimited description, e.g.
127
- β”‚ `1.0-jqp-initial-data-exploration`.
128
- β”œβ”€β”€ references <- Data dictionaries, manuals, and all other explanatory materials.
129
- β”œβ”€β”€ reports <- Generated analysis as HTML, PDF, LaTeX, etc.
130
- β”‚Β Β  └── figures <- Generated graphics and figures to be used in reporting
131
- β”‚Β Β  └── metrics.txt <- Relevant metrics after evaluating the model.
132
- β”‚Β Β  └── training_metrics.txt <- Relevant metrics from training the model.
133
- β”‚
134
- β”œβ”€β”€ requirements.txt <- The requirements file for reproducing the analysis environment, e.g.
135
- β”‚ generated with `pip freeze > requirements.txt`
136
- β”‚
137
- β”œβ”€β”€ setup.py <- makes project pip installable (pip install -e .) so src can be imported
138
- β”œβ”€β”€ src <- Source code for use in this project.
139
- β”‚Β Β  β”œβ”€β”€ __init__.py <- Makes src a Python module
140
- β”‚ β”‚
141
- β”‚Β Β  β”œβ”€β”€ data <- Scripts to download or generate data
142
- β”‚Β Β  β”‚Β Β  β”œβ”€β”€ great_expectations <- Folder containing data integrity check files
143
- β”‚Β Β  β”‚Β Β  β”œβ”€β”€ make_dataset.py
144
- β”‚Β Β  β”‚Β Β  └── data_validation.py <- Script to run data integrity checks
145
- β”‚ β”‚
146
- β”‚Β Β  β”œβ”€β”€ models <- Scripts to train models and then use trained models to make
147
- β”‚ β”‚ β”‚ predictions
148
- β”‚Β Β  β”‚Β Β  β”œβ”€β”€ predict_model.py
149
- β”‚Β Β  β”‚Β Β  └── train_model.py
150
- β”‚ β”‚
151
- β”‚Β Β  └── visualization <- Scripts to create exploratory and results oriented visualizations
152
- β”‚Β Β  └── visualize.py
153
- β”‚
154
- β”œβ”€β”€ .pre-commit-config.yaml <- pre-commit hooks file with selected hooks for the projects.
155
- β”œβ”€β”€ dvc.lock <- constructs the ML pipeline with defined stages.
156
- └── dvc.yaml <- Traing a model on the processed data.
157
-
158
-
159
- --------
160
-
161
- <p><small>Project based on the <a target="_blank" href="https://drivendata.github.io/cookiecutter-data-science/">cookiecutter data science project template</a>. #cookiecutterdatascience</small></p>
162
-
163
-
164
- ---
165
-
166
- To create a project like this, just go to https://dagshub.com/repo/create and select the **Cookiecutter DVC** project template.
167
-
168
- Made with 🐢 by [DAGsHub](https://dagshub.com/).
 
 
1
+ ---
2
+ license: apache-2.0
3
+ ---
4
+
5
+ Set the Cookiecutter-MLOps in Hugging Face
6
+ ==============================================
7
+
8
+ 1 - Create Model repository in Hugging Face (e.g. myHFrepo)
9
+
10
+ 2 - Clone your Hugging face repo to your local directory:
11
+
12
+ cd /path/to/parent directory of project folder
13
+ git clone git@hf.co:USERNAME/myHFrepo
14
+
15
+ For ssh connection check [here](https://huggingface.co/docs/hub/security-git-ssh#add-a-ssh-key-to-your-account)
16
+
17
+ 3 - Create your virtual environment (e.g. jointvenv)
18
+
19
+ cd myHFrepo
20
+ python -m venv jointvenv
21
+ source jointvenv/bin/activate
22
+
23
+ 4 - Transfer and set DagsHub's cookiecutter template employing MLOps best practices to your Huggingface repo
24
+
25
+ git clone https://dagshub.com/DagsHub/Cookiecutter-MLOps.git
26
+
27
+ 4.1 - Delete git files cloned from Cookiecutter-MLOps repo
28
+
29
+ rm -r /path/to/myHFrepo/Cookiecutter-MLOps/.git
30
+
31
+ 4.2 - dResolve conflicts with .gitattributes and README.md
32
+
33
+ cat /path/to/myHFrepo/Cookiecutter-MLOps/.gitattributes >> /path/to/myHFrepo/.gitattributes
34
+ rm /path/to/myHFrepo/Cookiecutter-MLOps/.gitattributes
35
+ git add .gitattributes
36
+ git commit -m "Paste .gitattributes info from DagsHub/Cookiecutter-MLOps"
37
+
38
+ cat /path/to/myHFrepo/Cookiecutter-MLOps/README.md >> /path/to/myHFrepo/README.md
39
+ rm /path/to/myHFrepo/Cookiecutter-MLOps/README.md
40
+ git add README.md
41
+ git commit -m "Paste README info from DagsHub/Cookiecutter-MLOps"
42
+
43
+ 4.3 - Move remaining files from DagsHub/Cookiecutter-MLOps to your Hugging Face repo .gitattributes and README.md
44
+
45
+ cd /path/to/myHFrepo/Cookiecutter-MLOps
46
+ mv * .[^.]* ..
47
+ cd /path/to/myHFrepo
48
+ rm -r /path/to/myHFrepo/Cookiecutter-MLOps
49
+
50
+ 5 - Add venv folder to.gitignore
51
+
52
+ echo '' >> .gitignore
53
+ echo '#'Virtual Environment >> .gitignore
54
+ echo jointvenv/ >> .gitignore
55
+ git add .
56
+ git commit -m "add remaining DagsHub/Cookiecutter-MLOps repo content"
57
+
58
+ 6 - Run step 2 from DagsHub/Cookiecutter-MLOps
59
+
60
+ make dirs
61
+
62
+ 7 - Run step 4 from DagsHub/Cookiecutter-MLOps
63
+
64
+ make requirements
65
+
66
+ 8 - Keep record of your own requirements
67
+
68
+ mv requirements.txt requirementsCookiecutter-MLOps.txt
69
+ git add requirementsCookiecutter-MLOps.txt
70
+ git commit -m "external requirements from Cookiecutter-MLOps"
71
+
72
+ pip freeze > requirements.txt
73
+ git add requirements.txt
74
+ git commit -m "First report venv requirements"
75
+
76
+ 9 - Push your changes to the remote Hugging face repository
77
+
78
+ git push origin main
79
+
80
+ 10 - Optional Create Model repository in your Hugging Face organization (e.g. myHFrepo)
81
+
82
+ git remote add dcc git@hf.co:MYORG/mywslHFrepo
83
+ git pull dcc main --allow-unrelated-histories
84
+
85
+ Resolve conflicts in .gitattributes and README.md
86
+
87
+ git add .
88
+ git commit -m "Merge HuggingFace individual and organization repos"
89
+ git push dcc main
90
+
91
+ ==============================
92
+ Cookiecutter-MLOps
93
+ ==============================
94
+
95
+ A cookiecutter template employing MLOps best practices, so you can focus on building machine learning products while
96
+ having MLOps best practices applied.
97
+
98
+ Instructions
99
+ ------------
100
+ 1. Clone the repo.
101
+ 2. Run `make dirs` to create the missing parts of the directory structure described below.
102
+ 3. *Optional:* Run `make virtualenv` to create a python virtual environment. Skip if using conda or some other env manager.
103
+ 1. Run `source env/bin/activate` to activate the virtualenv.
104
+ 4. Run `make requirements` to install required python packages.
105
+ 5. Put the raw data in `data/raw`.
106
+ 6. To save the raw data to the DVC cache, run `dvc add data/raw`
107
+ 7. Edit the code files to your heart's desire.
108
+ 8. Process your data, train and evaluate your model using `dvc repro` or `make reproduce`
109
+ 9. To run the pre-commit hooks, run `make pre-commit-install`
110
+ 10. For setting up data validation tests, run `make setup-setup-data-validation`
111
+ 11. For **running** the data validation tests, run `make run-data-validation`
112
+ 12. When you're happy with the result, commit files (including .dvc files) to git.
113
+
114
+ Project Organization
115
+ ------------
116
+
117
+ β”œβ”€β”€ LICENSE
118
+ β”œβ”€β”€ Makefile <- Makefile with commands like `make dirs` or `make clean`
119
+ β”œβ”€β”€ README.md <- The top-level README for developers using this project.
120
+ β”œβ”€β”€ data
121
+ β”‚Β Β  β”œβ”€β”€ processed <- The final, canonical data sets for modeling.
122
+ β”‚Β Β  └── raw <- The original, immutable data dump
123
+ β”‚
124
+ β”œβ”€β”€ models <- Trained and serialized models, model predictions, or model summaries
125
+ β”‚
126
+ β”œβ”€β”€ notebooks <- Jupyter notebooks. Naming convention is a number (for ordering),
127
+ β”‚ the creator's initials, and a short `-` delimited description, e.g.
128
+ β”‚ `1.0-jqp-initial-data-exploration`.
129
+ β”œβ”€β”€ references <- Data dictionaries, manuals, and all other explanatory materials.
130
+ β”œβ”€β”€ reports <- Generated analysis as HTML, PDF, LaTeX, etc.
131
+ β”‚Β Β  └── figures <- Generated graphics and figures to be used in reporting
132
+ β”‚Β Β  └── metrics.txt <- Relevant metrics after evaluating the model.
133
+ β”‚Β Β  └── training_metrics.txt <- Relevant metrics from training the model.
134
+ β”‚
135
+ β”œβ”€β”€ requirements.txt <- The requirements file for reproducing the analysis environment, e.g.
136
+ β”‚ generated with `pip freeze > requirements.txt`
137
+ β”‚
138
+ β”œβ”€β”€ setup.py <- makes project pip installable (pip install -e .) so src can be imported
139
+ β”œβ”€β”€ src <- Source code for use in this project.
140
+ β”‚Β Β  β”œβ”€β”€ __init__.py <- Makes src a Python module
141
+ β”‚ β”‚
142
+ β”‚Β Β  β”œβ”€β”€ data <- Scripts to download or generate data
143
+ β”‚Β Β  β”‚Β Β  β”œβ”€β”€ great_expectations <- Folder containing data integrity check files
144
+ β”‚Β Β  β”‚Β Β  β”œβ”€β”€ make_dataset.py
145
+ β”‚Β Β  β”‚Β Β  └── data_validation.py <- Script to run data integrity checks
146
+ β”‚ β”‚
147
+ β”‚Β Β  β”œβ”€β”€ models <- Scripts to train models and then use trained models to make
148
+ β”‚ β”‚ β”‚ predictions
149
+ β”‚Β Β  β”‚Β Β  β”œβ”€β”€ predict_model.py
150
+ β”‚Β Β  β”‚Β Β  └── train_model.py
151
+ β”‚ β”‚
152
+ β”‚Β Β  └── visualization <- Scripts to create exploratory and results oriented visualizations
153
+ β”‚Β Β  └── visualize.py
154
+ β”‚
155
+ β”œβ”€β”€ .pre-commit-config.yaml <- pre-commit hooks file with selected hooks for the projects.
156
+ β”œβ”€β”€ dvc.lock <- constructs the ML pipeline with defined stages.
157
+ └── dvc.yaml <- Traing a model on the processed data.
158
+
159
+
160
+ --------
161
+
162
+ <p><small>Project based on the <a target="_blank" href="https://drivendata.github.io/cookiecutter-data-science/">cookiecutter data science project template</a>. #cookiecutterdatascience</small></p>
163
+
164
+
165
+ ---
166
+
167
+ To create a project like this, just go to https://dagshub.com/repo/create and select the **Cookiecutter DVC** project template.
168
+
169
+ Made with 🐢 by [DAGsHub](https://dagshub.com/).