Financial data science andmachine learning
Ernie Chan, Ph.D.
QTS Capital Management, LLC.

About Us• Ernie is a principal of QTS CapitalManagement, LLC., a CPO/CTA.
• Ernie was a ML researcher at IBM, and quantat Morgan Stanley, Credit Suisse, etc.
• Detailed biography at www.epchan.com

Why ML now?
• Single-factor, linear quant models havedecaying alpha due to ease of replication.
• Advances in ML specifically address overfitting
issues.
• New tools in ML introduce more transparency,
less blackbox fitting.
• ML can be used for risk management andcapital allocation, not primary signalgenerator.

Traditional Quant vs ML
Traditional QuantML

Few predictors

Numerous predictors

Traditional data (prices, fundamentals,etc.)

Alternative data (news, credit cardtransactions, etc.)

LinearNonlinear

Intuitive, easy to replicate

Unintuitive, same data -> very different
modelsCan’t predict probability of successCan predict probability of success

Arbitrary capital allocation

Logical capital allocation

Harder to overfit

Easy to overfit

Hard to generate multiple backtests forstatistical assessment

Easy to generate multiple backtests forstatistical assessment

Research+implementation: Easy!

Research+implementation: Very laborintensive!

3 Steps• Financial data science– Find problems with data, and scrub them.
– Convert raw data into features.

• Machine learning– Use classification or regression techniques tomake predictions.

• Trading strategy construction– Use predictions as input to a trading strategy.
– Backtest various versions of strategy.

Data Is Nearly Everything
In the article
Data Challenges Are Halting AI Projects, IBM
Executive Says
Arvind Krishna, IBM’s senior vice president ofcloud and cognitive software, said about 80% of
the work with an AI project is collecting andpreparing data.
(www.wsj.com/articles/data-challenges-are-halting-ai-projectsibm-executive-says-11559035800)

Challenges of financial data science• Ever-changing company names and tickers• Dividend and split adjustment• Survivorship bias
• Look-ahead bias of earnings data
• Structural breaks in pre-processed alternativedata• Averaging categorical features

Changing Symbols• E.g. PCS -> TMUS, simple change.
• E.g. GOOG -> GOOG+GOOGL, split into 2classes of stocks.
– A common stock data provider has duplicated theprice history of GOOG prior to the split andprepended it to GOOGL’s history!

• Without a “Securities Master”, very hard tofind unique id of a stock, and to merge
different databases.

Survivorship Bias
• Easier to show data has bias than to “prove” itdoesn’t!
– Check for Enron, Worldcom, Lehman Brothers,Pets.com, Toys-R-Us.

• Only stock database that I know free ofsurvivorship bias is CRSP.com.

Look-ahead bias
• Many earnings announcement databases havelook-ahead bias.
– Actual announcement date is only known afterthe-fact.
– Backtesting a strategy should only use expected,
not actual, announcement date.
– Wall Street Horizon is the only earnings
announcement database without this bias.

Look-ahead bias
• Many fundamental stock database uses
“revised” data.
– E.g. Regular Compustat vs Point-in-time
Compustat.
– E.g. Sharadar SF1 database distinguishes between“As Reported” (AR) and “Most Recent” (MR) data.

• Revised data cannot be used for backtest.

Structural breaks• How does one check if alternative (e.g. newssentiment) data is sound?
– Check for structural breaks in statistics of datafeatures.
– E.g. daily min and max of sentiment score overtime.
– E.g. daily min and max of impact score over time.
– (Data from a startup news sentiment data vendor.)What ML models to use?
• Shallow vs deep ML models• Linear/Logistic regression is a “shallow” model.
– Simple (non-hierarchical) structure, little nonlinearity.

• Simple models often work best with few features.
• Deeper models work better when features arenumerous and their (nonlinear) interactions
important.
• Random forest is most commonly used deepermodel.

Random Forest (RF)• Random forest is an ensemble of classification (or regression)
trees.
• Tree is a method of select features one-by-one.
• After picking one predictor, we segment data (parent node)
into 2 subsets (child nodes) based on an inequality on thatpredictor.
• The predictor is picked based on minimizing the Gini’s
Diversity Index (GDI), in the responses within each child node.
– GDI=1 − 𝑝+2 − 𝑝−2– 𝑝+/− is the fraction of observations with positive/negativereturns.
– Perfect classifier gets GDI=0.

Random Forest• Repeat process on each child, until– Child size becomes too small; or– GDI in responses in the children are no smaller
than the parent’s.

Random Forest Regression• Is classification or regression more importantfor predicting returns?
– (Seehttps://twitter.com/lopezdeprado/status/1142815691151745025)

Metalabelling• Metalabelling is a general ML method to improveon a base model. (See Lopez de Prado.)• Good candidates for base model:– Shallow ML model.
– Time series model like ARIMA or GARCH.
– Simple factor model.
– Simple technical analysis/indicators.
– Fundamental and discretionary strategies.
– Your current trading strategy.

What is the meta label?
• The label (or metaLabel) is whether theprediction of the base model is correct.
– MetaLabel=+1(0) if base prediction correct(incorrect).
– MetaLabel=+1(0) if RT trade generated by basestrategy is profitable(unprofitable).

• Meta-model will also output probability that basemodel prediction is correct.
• If metaPrediction=0, or if metaPrediction=+1 butwith probability ~ 0.5, can refrain from taking
base trade.
• Probability can be used for capital allocation.

Example of metalabeling• Create your favorite technical indicators to use asfeatures– See https://github.com/bukosabino/ta– What about supplementary data such as implied
volatilities? Commodity, bond, FX indices?

• Create metaLabels, with base model = LR model
previously created.
• Train a RF model using these.
• A successful metalabel model:
http://www.quantsportal.com/does-metalabeling-add-to-signal-efficacy/From ML to Trading Strategy• Output from classification model=Prob(Up,Dn).
• Simple Strategy:– Buy and hold for one bar if Prob(Up) ≥ Threshold– Sell and hold for one bar if Prob(Dn) ≥ Threshold– Otherwise flat

• Performance metrics: CAGR, Sharpe Ratio,
maxDD, maxDDD, Calmar Ratio, Equity Curve

Capital weighting• Prob(Up, Dn) can also be used to decide tradesize.
• E.g. Weight (in $)=Prob(Up)-0.5– Negative weight means short position.
– If we have multiple assets s, can normalize
weights s.t. σ𝑠 𝑤𝑒𝑖𝑔ℎ𝑡𝑠 𝑠 = 1 .

Robust Backtesting• Even if backtest performance is computed on
test set, it is still one sample out of random
distribution of price paths.
– Good/bad performance due to chance?

• If strategy created by ML signals: try different
random seeds.
– Plot a histogram of resulting Sharpe ratios.

Robust Backtesting• If features are based on prices only, cansimulate multiple price paths for backtesting.
• Simulated prices (returns) can be based on– Sampling with replacement, or– Time series models• Linear: ARIMA+GARCH• Nonlinear: LSTM, GAN (Generative AdversarialNetwork)

CV Backtest• Ordinary CV generates only 1 backtest path.

Train

Forecasts

5678

Forecasts

9 10 11 12

9 10 11 12Test
Train

Forecasts

5678

9 10 11 125678

Test
1234

Test

Train

1234

Train

– E.g. 3 non-overlapping test folds, each with 4 rows
– E.g. Forecasts on t=1, 2, …, 12 are unique.
– Single OOS forecasted time series: t=1, 2, …, 12

CPCV* Backtest• CPCV generates multiple backtest paths.

*Combinatorial Purged CV, see AFML

Forecasts

1 4 8 125678
1234

Forecasts

1 4 8 11

9 10 11 12

9 10 11 125678

1 4 8 12Forecasts

123412345678

9 10 11 12– Allow test folds to overlap.
– E.g. test sets below overlap on t=1,4,8.
– Hence 3 different forecasts on 1,4,8.

Where to start• Books:– Chan: “Machine Trading: Deploying Computer
Algorithms to Conquer the Markets”– Murphy: “Machine Learning: A Probabilistic
Perspective”– López de Prado: “Advances in Financial Machine
Learning”• Websites:– www.quantresearch.org/– github.com/hudson-and-thames

Where to start?
• Python– Scikit-learn (all inclusive ML package)– LightGBM (random forests with boosting)– Keras (deep learning)• Matlab– Statistics and Machine Learning Toolbox– Deep Learning Toolbox

Where to start• Course:– “Lifecycle of Trading Strategy Development”• Online: epchan.com/workshops• In-person: London, Dec 9-11, www.globalmarketstraining.co.uk/datascience.html

Download my talks atwww.epchan.com