lending-borrowing 0.2.0
BHD lending dashboard for risk management suite.
Requires Python: >=3.8,<4.0

# Brevan Howard Digital Lending & Borrowing Dashboard

[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg?style=flat-square)](https://opensource.org/licenses/MIT) ![GitHub issues](https://img.shields.io/github/issues/NethermindEth/bhd-lending-borrowing-dashboard?style=flat-square) ![GitHub contributors](https://img.shields.io/github/contributors/NethermindEth/bhd-lending-borrowing-dashboard?style=flat-square)

Table of Contents:

- [Brevan Howard Digital Lending \& Borrowing Dashboard](#brevan-howard-digital-lending--borrowing-dashboard)
  - [About](#about)
    - [Business Requirements](#business-requirements)
  - [Getting Started](#getting-started)
    - [Prerequisites](#prerequisites)
    - [Development](#development)
  - [Airflow and Postgres](#airflow-and-postgres)
  - [Release tagging process](#release-tagging-process)
  - [Documentation and more](#documentation-and-more)
    - [Roadmap](#roadmap)
    - [Contacts](#contacts)

---

## About

Brevan Howard Digital requested Nethermind to build a a lending dashboard which would be a part of the Risk Management technology suite.

> Requested: Q4 2022

### Business Requirements

1. Dashboard would be used by senior management & portfolio managers
2. The main feature would be to get info on lending & rates from external sources / counterparties and display them on GUI, allowing for basic trend analytics.
   1. It is expected to onboard feeds from 10 counterparties
   2. The dashboard would work Like a traditional cash management tool
   3. Data update cadence would be weekly
   4. Charts & Diagrams should be easily customizable by the Users
3. Currently, rates & data are sent via emails in different formats (this could be standardized as CSV format to ease the integration)
   1. In the future, we would expect APIs to source the data
4. BHD sees it as a separate app / stand-alone product, and as of now, they are unlikely to integrate with any existing system.
5. There’s an expectation to have a separate dashboard/screen for USDC/USDT borrows.
6. The data should be stored on BHD premises (Cloud Machines) so that further analytics can be put in place

**Full Business requirements:** [here](https://www.notion.so/nethermind/ext-Lending-Borrowing-Dashboards-947a412a72544676ae6c0562a4704010).

**Development task tracker:** [here](https://www.notion.so/nethermind/ef56cbe13d6d465da84217f37059be94?v=f22aa14608724cf7bec9c6c22e175a26).

---

## Getting Started

### Prerequisites

These application dependencies are required to run the app:

- Python 3.8.x ([GCP Composer/Airflow version restriction - Latest version on Feb/23](https://cloud.google.com/composer/docs/concepts/versioning/composer-versions))
- Poetry - [how to install](https://python-poetry.org/docs/#installation)
- Curl - [how to install](https://curl.se/download.html)

### Development

1. Clone the repo: `git clone git@github.com:NethermindEth/bhd-lending-borrowing-dashboard.git`
2. Install Poetry packages: `poetry install`
3. Setup **`.env`** file from the sample data and fill settings accordingly: `cp .env.example .env`
   1. Application database settings:
      1. `POSTGRES_HOST` : Postgres database hostname/ip
      2. `POSTGRES_PORT` : Postgres database port exposed on host
      3. `POSTGRES_PASSWORD` : Postgres database password
      4. `POSTGRES_USER` : Postgres database user
   2. Application email settings:
      1. `IMAP_SERVER` : IMAP server hostname/ip
      2. `EMAIL_USER` : Email address handling all incoming messages for parsing
      3. `EMAIL_PASSWORD` : Email password (or granted [app password by google/gmail](https://support.google.com/mail/answer/185833))
   3. Application Json-RPC settings:
      1. `MAINNET_RPC_URL` : Ethereum's RPC endpoint for on-chain data workers.
      2. `MATIC_RPC_URL` : Polygon's RPC endpoint for on-chain data workers.
4. Install pre-commit hooks: `poetry run pre-commit install --hook-type commit-msg --hook-type pre-push`
5. Run pre-commit checks: `poetry run pre-commit run --all-files`
6. Run tests:
   1. For simple test execution: `poetry run pytest tests -m unit`
   2. For test coverage: `poetry run pytest tests --cov=lending_borrowing --cov-report term-missing --cov-report html --cov-branch -m unit -v --cov-fail-under=80`
7. Start docker environment (airflow + postgres): `docker-compose up -d`

---

## Airflow and Postgres

In order to run the application locally, we use docker compose to spin up airflow and postgres containers.

Build airflow image with latest dependencies:

```sh
rm -f dist/*.whl && poetry build --format wheel
export DOCKER_DEFAULT_PLATFORM=linux/amd64
docker-compose build --no-cache
```

Start airflow and postgres services:

```sh
export DOCKER_DEFAULT_PLATFORM=linux/amd64
docker-compose up -d
```

> Airflow DAGs will be automatically loaded from `dags` folder.

To check when airflow ui is ready:

```sh
curl http://localhost:8080/health
# login: airflow
# password: airflow
```

Stop metabase and postgres services:

```sh
docker-compose down
```

To stop and remove volumes:

```sh
docker-compose down --volumes --remove-orphans
```

## Release tagging process

In order to tag a new release, read the [release tagging process document](RELEASE.md).:

## Documentation and more

For solution design, cronjob frequencies and counterparty coverage, please refer to the [handover documentation](https://www.notion.so/nethermind/Tech-Handover-BHD-Lending-Borrowing-a54dc5c69f0442f196b9ca332dd132f4).

### Roadmap

- [x] Add GET endpoints
- [x] Set up cronjobs
- [x] Metabase Design
- [ ] On Premise Deployment
- [x] Testing
  - [x] Locally
  - [ ] BHD premises

See the [open issues](https://github.com/NethermindEth/BHD_Lending_Borrowing_Dashboard/issues) for a full list of proposed features (and known issues).

### Contacts

- Eli Barbieri (Tech Lead) ---- eli@nethermind.io
- Saul Martin (Cloud Expert) ---- saul@nethermind.io
- Fabio Bressler (Databases & System Design) ---- fabio@nethermind.io