django-cryptofeed-werks 0.1.0
Django Cryptofeed Werks aggregates candlesticks using tick data from financial exchanges
License: MIT
Requires Python: >=3.8,<3.10
Keywords: django,cryptocurrency,candles,trading

# What?

Django Cryptofeed Werks aggregates candlesticks using tick data from financial exchanges. 

Raw ticks are aggregated if equal symbol, timestamp, nanoseconds and tick rule. As described in the accompanying project [cryptofeed-werks](https://github.com/globophobe/cryptofeed-werks), aggregating trades in this way can increase information, as they are either orders of size or stop loss cascades.

As well, the number of rows can be reduced by 30-50%

By filtering aggregated rows, for example only writing a row when an aggregated trade is greater than `min_volume >= 1000`, the number of rows can be reduced more.


# Why?

Candlesticks aggregated by `django-cryptofeed-werks` are informationally dense. Such data can be useful for analyzing financial markets, and for machine learning. As an example, refer to ["The Volume Clock: Insights into the High Frequency Paradigm"](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2034858).


# How?

Whenever possible, data is downloaded from the exchange's AWS S3 repositories. Otherwise, it is downloaded using their REST APIs. 

Raw tick data may be backfilled either locally, or from the cloud.

A database, preferably PostgreSQL, is required. Raw tick data is saved in gzipped CSV files, or optionally in Google BigQuery. Aggregated trades are saved in PostgreSQL, with candles generated at 1 minute intervals.

Storing raw tick data in BigQuery is recommended, as it is cost performant. For the price of a fancy cup of coffee each month, years of historical data can be stored. It is cheaper than most S3 and GCS storage classes. As part of the [free tier](https://cloud.google.com/free), a terabyte of queries is free each month. A further [2 terabytes of streaming inserts](https://cloud.google.com/bigquery/pricing#data_ingestion_pricing) is free each month, using the BigQuery Storage Write API.  

[BigQuery Notes](https://github.com/globophobe/django-cryptofeed-werks/blob/main/NOTES.md).


Supported exchanges
-------------------

:white_medium_square: Alpaca Markets

:white_check_mark: Binance REST API (requires API key)

:white_check_mark: Bitfinex REST API

:white_medium_square: Bitflyer REST API

:white_check_mark: BitMEX REST API, and [S3](https://public.bitmex.com/) repository

:white_check_mark: Bybit REST API, and [S3](https://public.bybit.com/) repository

:white_check_mark: Coinbase Pro REST API

:white_medium_square: Deribit REST API

:white_check_mark: FTX REST API

Note: Exchanges without paginated REST APIs will never be supported.

For deployment, there are Dockerfiles. As well there are invoke tasks for rapid deployment to both GCP Compute and Google Cloud Run.


Installation
------------

For convenience, `django-cryptofeed-werks` can be installed from PyPI:

```
pip install django-cryptofeed-werks
```

With BigQuery:
```
pip install django-cryptofeed-werks[bigquery]
```

Environment
-----------

To use the scripts or deploy to GCP, rename `.env.sample` to `.env`, and add the required settings.


Future Plans
------------

Allow use as an aggr historical data source as decribed by [aggr](https://github.com/Tucsky/aggr#implement-historical-data).