File size: 1,603 Bytes
b585c7f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
# Python Wheel
## Build
The wheel adds dependencies including optional dependencies, except flash-attention, wiki-processing, metric, and training. To build do:
```bash
python setup.py sdist bdist_wheel
```
To install the default dependencies do:
```bash
pip install dist/h2ogpt-0.1.0-py3-none-any.whl
```
replace `0.1.0` with actual version built if more than one.
To install additional dependencies, for instance for faiss on GPU, do:
```bash
pip install dist/h2ogpt-0.1.0-py3-none-any.whl
pip install dist/h2ogpt-0.1.0-py3-none-any.whl[FAISS]
```
once `whl` file is installed, two new scripts will be added to the current environment: `h2ogpt_finetune`, and `h2ogpt_generate`.
The wheel is not required to use h2oGPT locally from repo, but makes it portable with all required dependencies.
See [setup.py](../setup.py) for controlling other options via `extras_require`.
## Run
```python
from h2ogpt.generate import main
main()
```
See `src/gen.py` for all documented options one can pass to `main()`. E.g. to start LLaMa7B:
```python
from h2ogpt.generate import main
main(base_model='meta-llama/Llama-2-7b-chat-hf',
prompt_type='llama2',
save_dir='save_gpt7',
score_model=None,
max_max_new_tokens=2048,
max_new_tokens=1024,
num_async=10,
top_k_docs=-1)
```
## Checks
Once the wheel is built, if you do:
```bash
python -m pip check
```
you may see:
```text
h2ogpt 0.1.0 has requirement numpy==1.24.3, but you have numpy 1.23.5.
h2ogpt 0.1.0 has requirement pandas==2.0.2, but you have pandas 1.5.3.
```
but that is expected.
|