cvxcovariance 0.1.9
Estimating covariance matrices of financial returns
Requires Python: >=3.10

# [cvxcovariance](http://www.cvxgrp.org/cov_pred_finance/book)

[![Coverage Status](https://coveralls.io/repos/github/cvxgrp/cov_pred_finance/badge.svg)](https://coveralls.io/github/cvxgrp/cov_pred_finance)
[![PyPI version](https://badge.fury.io/py/cvxcovariance.svg)](https://badge.fury.io/py/cvxcovariance)
![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)
[![Downloads](https://static.pepy.tech/personalized-badge/cvxcovariance?period=month&units=international_system&left_color=black&right_color=orange&left_text=PyPI%20downloads%20per%20month)](https://pepy.tech/project/cvxcovariance)

[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/cvxgrp/cov_pred_finance)

The `cvxcovariance` package
provides simple tools for creating an estimate $\hat\Sigma_t$ of the covariance
$\Sigma_t$ of the $n$-dimensional return vectors $r_t$, $t=1,2,\ldots$, based on
the observed returns $r_1, \ldots, r_{t-1}$. (Here $r_t$ is the return from
$t-1$ to $t$.) The covariance predictor $\hat\Sigma_t$ is generated by blending
$K$ different "expert" predictors $\hat\Sigma_t^{(1)},\ldots,\hat\Sigma_t^{(K)}$,
by solving a convex optimization problem at each time step.

For a detailed description of the methodology, see our manuscript
[A Simple Method for Predicting Covariance Matrices of Financial Returns](https://web.stanford.edu/~boyd/papers/cov_pred_finance.html)
(in particular Section 3).

In the simplest case the user provides a $T\times n$ pandas DataFrame
of returns $r_1,\ldots,r_T$ and $K$ half-life pairs, and gets back covariance
predictors for each time step. (The $K$ experts are computed as iterated
exponentially weighted moving average (IEWMA) predictors as described in
Section 2.5 of the [paper](https://web.stanford.edu/~boyd/papers/cov_pred_finance.html).)
In the more general case, the user provides the $K$ expert predictors
$\hat\Sigma_t^{(1)},\ldots,\hat\Sigma_t^{(K)}$, $t=1,\ldots,T$, and these are
blended together by solving the convex optimization problems. In either case
the result is returned as an iterator object over namedtuples:
`Result = namedtuple("Result", ["time", "mean", "covariance", "weights"])`.

Note: at time $t$ the user is provided with $\hat\Sigma_{t+1}$,
$\textit{i.e.}$, the covariance matrix for the next time step.
So `Result.covariance` returns the covariance prediction for `time+1`.

## Installation

To install the package, run the following command in the terminal:

```bash
pip install cvxcovariance
```

## Usage

There are two alternative ways to use the package. The first is to use the
`from_ewmas` function to create a combined multiple IEWMA (CM-IEWMA) predictor.
The second is to provide your own covariance experts, via dictionaries,
and pass them to the `from_sigmas` function. Both functions return an object
of the `_CovarianceCombination` class, which can be used to solve the covariance
combination problem.

### CM-IEWMA

The `from_ewmas` function takes as input a pandas DataFrame of
returns and the IEWMA half-life pairs (each pair consists of one half-life for
volatility estimation and one for correlation estimation), and returns an
iterator object that iterates over the CM-IEWMA covariance predictors defined
via namedtuples. Through the namedtuple you can access the
`time`, `mean`, `covariance`, and `weights` attributes. `time` is the timestamp.
`mean` is the estimated mean of the return at the $\textit{next}$ timestamp,
$\textit{i.e.}$ `time+1`, if the user wants to estimate the mean; per default
the mean is set to zero, which is a reasonable assumption for many financial
returns. `covariance` is the estimated covariance matrix for the $\textit{next}$
timestamp, $\textit{i.e.}$ `time+1`. `weights` are the $K$ weights attributed
to the experts. Here is an example:

```python
import pandas as pd
from cvx.covariance.combination import from_ewmas

# Load return data
returns = pd.read_csv("data/ff5_no_rf.csv", index_col=0, header=0, parse_dates=True).iloc[:1000]
n = returns.shape[1]

# Define half-life pairs for K=3 experts, (halflife_vola, halflife_cov)
halflife_pairs = [(10, 21), (21, 63), (63, 125)]

# Define the covariance combinator
combinator = from_ewmas(returns,
                        halflife_pairs,
                        min_periods_vola=n,  # min periods for volatility estimation
                        min_periods_cov=3 * n)  # min periods for correlation estimation

# Solve combination problem and loop through combination results to get predictors
covariance_predictors = {}
for predictor in combinator.solve(window=10):  # lookback window for optimization
    # From predictor we can access predictor.time, predictor.mean (=0 here),
    # predictor.covariance, and predictor.weights
    covariance_predictors[predictor.time] = predictor.covariance
```

Here `covariance_predictors[t]` is the covariance prediction for time $t+1$,
$\textit{i.e.}$, it is uses knowledge of $r_1,\ldots,r_t$.

### General covariance combination

The `from_sigmas` function takes as input a pandas DataFrame of
returns and a dictionary of covariance predictors `{key: {time:
sigma}`, where `key` is the key of an expert predictor and `{time:
sigma}` is the expert predictions. For example, here we combine two
EWMA covariance predictors from pandas:

```python
import pandas as pd
from cvx.covariance.combination import from_sigmas

# Load return data
returns = pd.read_csv("data/ff5_no_rf.csv", index_col=0,
                      header=0, parse_dates=True).iloc[:1000]
n = returns.shape[1]

# Define 21 and 63 day EWMAs as dictionaries (K=2 experts)
ewma21 = returns.ewm(halflife=21, min_periods=5 * n).cov().dropna()
expert1 = {time: ewma21.loc[time] for time in ewma21.index.get_level_values(0).unique()}
ewma63 = returns.ewm(halflife=63, min_periods=5 * n).cov().dropna()
expert2 = {time: ewma63.loc[time] for time in ewma63.index.get_level_values(0).unique()}

# Create expert dictionary
experts = {1: expert1, 2: expert2}

# Define the covariance combinator
combinator = from_sigmas(sigmas=experts, returns=returns)

# Solve combination problem and loop through combination results to get predictors
covariance_predictors = {}
for predictor in combinator.solve(window=10):
    # From predictor we can access predictor.time, predictor.mean (=0 here),
    # predictor.covariance, and predictor.weights
    covariance_predictors[predictor.time] = predictor.covariance
```

Here `covariance_predictors[t]` is the covariance prediction for time
$t+1$, $\textit{i.e.}$, it is uses knowledge of $r_1,\ldots,r_t$.

## Poetry

We assume you share already the love for [Poetry](https://python-poetry.org).
Once you have installed poetry you can perform

```bash
make install
```

to replicate the virtual environment we have defined in [pyproject.toml](pyproject.toml)
and locked in [poetry.lock](poetry.lock).

## Jupyter

We install [JupyterLab](https://jupyter.org) on fly within the aforementioned
virtual environment. Executing

```bash
make jupyter
```

will install and start the jupyter lab.

## Citing

If you want to reference our paper in your research,
please consider citing us by using the following BibTeX:

```BibTeX
@article{johansson2023covariance,
url = {http://dx.doi.org/10.1561/0800000047},
year = {2023},
volume = {12},
journal = {Foundations and Trends® in Econometrics},
title = {A Simple Method for Predicting Covariance Matrices of Financial Returns},
doi = {10.1561/0800000047},
issn = {1551-3076},
number = {4},
pages = {324-407},
author = {Kasper Johansson
and Mehmet G. Ogut
and Markus Pelger
and Thomas Schmelzer
and Stephen Boyd}
}
```