lhlib 0.2.0a0
Common Library for the FI Lakehouse Framework
License: MIT
Requires Python: >=3.10.0,<4.0.0

# Lakehouse lib

> PwC's library to build data lake houses with Databricks.

## Maintainers

- [Jan Schäfer](mailto:jan.schafer@pwc.com)
- [Benjamin Fadl](mailto:benjamin.fadl@pwc.com)

## Set up

To set up your dev environment, you will need [poetry](https://python-poetry.org/) installed and set up your python
environment:

```bash
brew install poetry
poetry install
```

## Testing

Run the tests within the poetry environment:

```bash
poetry run pytest
```

## Linting

All linters are installed, after installing with poetry. To lint the code, you can run the linters within the
poetry environment:

```bash
poetry run black --check --diff lhlib/ tests/;
poetry run ruff --check --diff lhlib/ tests/;
poetry run mypy lhlib/;
```

## Building

### Local

The following instructions describe how to get started with the local develoment of the LhLib.

`Important`:
Steps `2` and `6` highly differ depending on your choice of tools. Therefore, we cannot include the debugging process in
these instructions.

1. Install (`poetry`)[https://python-poetry.org/docs/#installation] on your local machine.
2. Ensure poetry is installed correctly by typing `poetry` into the prefered command-line interface (CLI). If not, you
   probably have to configure your `PATH` variable (e.g. `%APPDATA%\Python\Scripts`)
3. Navigate to the repo-folder containing the `pyproject.toml`
4. If so, enter `poetry shell` into your CLI.
5. Enter `poetry install --with=group`. Wait for the installation to finish.
6. Open a Jupyter Notebook. Ensure the correct interpreter for the Kernel ist selected. The first time you probably need
   to set it up first. Enter `where python` (or `which python`/`poetry run where python` depending on your CLI) in the
   poetry shell to find the location of `python.exe` within the (poetry) enviroment. This path probably has to be added
   manually (try 'venvFolders' under "Settings/Preferences" in VSC first).
7. To test your setup enter `import lhlib` into a cell within the JupyterNotebook.

### Using Azure DevOps

This part of the instructions assumes, that local development of the LhLib has been concluded and a respective feature
branch already exists. However, the conclusion of local development is not a prerequisit for the following steps to work
(`The feature branch on remote is!`).

1. Make sure, your (local) feature branch is pushed to `remote`.
2. Go to `Azure DevOps` > `DigitalOperationsAnalytics` > `FactoryIntelligence2.0`
3. In the NavBar lefthandside: `Pipelines` > `Pipelines` > search for "`lhlib-ci`" or use this [link](https://dev.azure.com/DigitalOperationsAnalytics/FactoryIntelligence2.0/_build?definitionId=669).
4. In the top-right corner click "`Run pipeline`" and select your under "Branch/tag".
   Optional: Under "`Stages to run`" you can select only those stages you want to test.
5. Always run the complete pipeline before merging to "`main`". You should be able to find the resulting .WHL-file
   in the `Blob Container "lhlib"`.

File will be available under:
- ht<span>tps</span>://st<`app`>gllh<`env`>.blob.core.windows.net/lhlib/lhlib-<`version`>-py3-none-any.whl
- example:
  ht<span>tps</span>://st`fi2`gllh`d`.blob.core.windows.net/lhlib/lhlib-`0.0.5.dev58%2Ba9c2df3`.none-py3-none-any.whl
   
   [link to "Conntecting to Azure Storage using Microsoft Azure Storage Explorer"]