How to Generate Trading Strategies Automatically
By: Robert Maidla
Tags: Trading, Finance, Algorithmic Trading, Development, Investing

Coming back to the original goal, we could eventually set up a neural network to experiment with different rules and entry/exit settings to see which ones prove to be most profitable. The only components that would need to be hard-coded are the actual queries for RuleValues and settings for entries/exits, everything else can be combined or dynamically generated. What is more, a strategy is not limited to work on a single symbol, relying on a single timeframe.

Application layer

Now that we have our core domain models defined, let’s set up the piping from receiving a new data point to requesting an order.

First, we need to create and activate a strategy. Using the good practice of CQRS, we should use a command for that, which would initialize a new Strategy object (with the provided conditions) and would store that somewhere. For simplification, let’s use a singleton provider, which holds a collection of activated strategies. Holding the strategies in memory saves us a database trip as the active strategies will need to be accessed with every new data point. For resiliency, however, the strategies should be persisted in a database and preloaded on application startup into the StrategyProvider service. While the application is running, we can activate or deactivate a strategy whenever necessary. Upon the activation of a strategy, the application should also subscribe to the price feed of the underlying symbols.

The main inbound message that auto-trading mechanism is interested in is a new data point. This information reaches our service in the shape of events — more specifically, SymbolUpdatedEvents . The event contains information regarding the new data point (ask/bid for FX, trade price and volume for equities).

Then we‘d need to figure out which strategies are interested in this data point. For that, we need to inspect every entry and exit condition of each active strategy and go through all of the rules to see if any of the underlying dynamic values are dependent on the symbol. Sounds like a lot of looping considering that a single symbol can produce tens of individual data points per second. We could also determine the symbols of a strategy once, upon its creation. This, however, can be done only if the Symbols of the rules stay constant throughout the lifetime of a strategy. More on that later.

How the strategy should be assessed upon a suitable SymbolUpdatedEvent

Assessing strategies upon a new data point takes place in an AssessStrategies command. We get the active strategies from the injected StrategyProvider service, filter out the ones not interested in the symbol, and evaluate the conditions. Any condition which has a successful evaluation (meaning all the rules within the condition are valid) will lead to an execution request, whether an exit or an entry. This is done via corresponding commands who’ll delegate the tasks, the actual execution is not the auto-trader’s responsibility and is handled by another service within our system. The manner of condition evaluation deserves a closer inspection, however.

Naturally, we want the system to be as efficient and resilient as possible, thus we should evaluate all the conditions of interested strategies asynchronously and in parallel. But what if one of the entry rules state that the amount of opened orders should be less than 1 for example. Having an order open before the data point is processed, the entry condition would not receive a successful evaluation. However, if the same data point leads to a successful exit condition and thus an exit, it would make the entry condition valid. This might not be a huge problem if we get new data within milliseconds, but what if data is handled with daily intervals? The conclusion is that we should evaluate exit conditions before entries, while we can run them in parallel respectively across strategies. Here’s how the assessment of strategies looks like inside the command handler.

public async Task<Result> Handle(AssessStrategiesCommand command, CancellationToken cancellationToken)

{

// EXITS

var interestedStrategiesForExits = _strategyProvider.Strategies

.Where(strategy => strategy.ExitCalculationSymbols.Contains(command.DataPoint.Symbol));



var exitEvaluationResults = await Task

.WhenAll(interestedStrategiesForExits.Select(strategy => _exitEvaluator.Evaluate(strategy.ExitConditions)));



var exitDetails = exitEvaluationResults

.Where(result => result.IsSuccess)

.SelectMany(result => result.Details);



exitDetails

.ToList()

.ForEach(async details =>

await _commandProcessor.ExecuteCommand(new RequestExitCommand(command.Actor, details.Settings)));



// ENTRIES

var interestedStrategiesForEntries = _strategyProvider.Strategies

.Where(strategy => strategy.EntryCalculationSymbols.Contains(command.DataPoint.Symbol));



var entryEvaluationResults = await Task

.WhenAll(interestedStrategiesForEntries.Select(strategy => _entryEvaluator.Evaluate(strategy.EntryConditions)));



var entryDetails = entryEvaluationResults

.Where(result => result.IsSuccess)

.SelectMany(result => result.Details);



entryDetails

.ToList()

.ForEach(async details =>

await _commandProcessor.ExecuteCommand(new RequestEntryCommand(command.Actor, details.Settings)));



return Result.Success();

}

The Strategy class in the earlier example is missing the Entry - and ExitCalculationSymbols properties. Both of these are instances of the following CalculationSymbols class:

/// <summary>

/// Determines the symbols whose data points trigger calculations

/// </summary>

public class CalculationSymbols : ValueObject

{

public bool IsAny => Symbols.IsEmpty;

public ImmutableList<Symbol> Symbols { get; private set; }

public bool Contains(Symbol symbol) => Symbols.Contains(symbol) || IsAny;



public CalculationSymbols(IEnumerable<Symbol> symbols)

{

Symbols = symbols.ToImmutableList();

}



public CalculationSymbols(params Symbol[] symbols)

{

Symbols = symbols.ToImmutableList();

}



public static CalculationSymbols Any()

{

return new CalculationSymbols();

}



public static CalculationSymbols Merge(params CalculationSymbols[] calculationSymbols)

{

return new CalculationSymbols(calculationSymbols.SelectMany(x => x.Symbols).Distinct());

}



public static CalculationSymbols Merge(IEnumerable<CalculationSymbols> calculationSymbols)

{

return new CalculationSymbols(calculationSymbols.SelectMany(x => x.Symbols).Distinct());

}

}

Inside the Entry - and ExitEvaluator , we evaluate each condition separately. This extra wrapping keeps AssessStrategiesCommand handler simpler and enables us to package all the evaluated conditions into a single Response bundle for the strategy to act upon. If multiple conditions give positive evaluations, this is the place to assign weights or priorities to the executions.

Upon a successful evaluation, a Response object is returned with the execution details. The Details consist of information regarding all the rules validated (serving an informative purpose as to know which rules did exactly trigger an exit) and the actual execution settings (detailed in the Domain section).

The two evaluator mechanisms are nearly identical, but I have chosen to keep them separate and duplicated, as they might be needing separate optimization (for example, there is no need to evaluate an exit condition if the strategy has no opened orders in the first place).

public class EntryEvaluator : IEntryEvaluator

{

private readonly IConditionEvaluator _conditionEvaluator;



public EntryEvaluator(IConditionEvaluator conditionEvaluator)

{

_conditionEvaluator = conditionEvaluator;

}



public async Task<EntryEvaluationResult> Evaluate(IEnumerable<EntryCondition> conditions)

{

var details = await GetDetails(conditions).ToListAsync();



return details.Any() ? EntryEvaluationResult.Success(details) : EntryEvaluationResult.Failure();

}



private async IAsyncEnumerable<EntryDetails> GetDetails(IEnumerable<EntryCondition> conditions)

{

foreach (var condition in conditions)

{

var conditionEvaluationResult = await _conditionEvaluator.Evaluate(condition);

if (conditionEvaluationResult.IsSuccess)

yield return new EntryDetails(conditionEvaluationResult, condition.Settings);

}

}

}

Condition evaluator simply validates each rule within a given condition. It then assembles a ConditionEvaluationResult , which in turn contains RuleValidationResults . Whether a condition is successful or not is determined by whether all of the rules have had a successful internal comparison.

public class ConditionEvaluator : IConditionEvaluator

{

private readonly IRuleValidator _ruleValidator;



public ConditionEvaluator(IRuleValidator ruleValidator)

{

_ruleValidator = ruleValidator;

}



public async Task<ConditionEvaluationResult> Evaluate(Condition condition)

{

var ruleValidationResults = await Task.WhenAll(condition.Rules.Select(x => _ruleValidator.Validate(x)));

return new ConditionEvaluationResult(ruleValidationResults);

}

}

We’re nearing the core of a strategy — a comparison. Before that, we need to calculate the dynamic values within the rule. Remember, a rule has 2 values which will be compared based on the defined logical operator. At least one of the 2 values needs to be dynamic, otherwise, our comparison would always yield a constant response.

As a dynamic rule value is dependent on some sort of external data (or data series), we must account for the possibility that the query is unable to fetch all the necessary data to calculate the value. We wouldn’t want to make the comparison within the rule if one of the underlying values is calculated on incomplete data. Thus in such case, the CalculationQuery handler returns a failure, which in turn will automatically lead to an invalid rule validation.

public class RuleValidator : IRuleValidator

{

private readonly IQueryProcessor _queryProcessor;

private readonly IValueComparer _valueComparer;



public RuleValidator(IQueryProcessor queryProcessor, IValueComparer valueComparer)

{

_queryProcessor = queryProcessor;

_valueComparer = valueComparer;

}



public async Task<RuleValidationResult> Validate(Rule rule)

{

var ruleValues = await Task.WhenAll(new[]

{

GetRuleValue(rule.Value1),

GetRuleValue(rule.Value2)

});



var calculationErrors = ruleValues

.Where(x => x.IsFailure)

.Select(x => x.Error)

.ToArray();



if (calculationErrors.Any())

return new RuleValidationResult(calculationErrors);



var comparisonSucceeds = _valueComparer.Compare(ruleValues.First().Value, rule.LogicalOperator, ruleValues.Last().Value);

return comparisonSucceeds

? RuleValidationResult.Success()

: RuleValidationResult.Failure(ResultError.WithError("Value comparison did not succeed"));

}



private async Task<Result<decimal>> GetRuleValue(RuleValue ruleValue)

{

return ruleValue switch

{

ConstantRuleValue x => Result<decimal>.Success(x.Value),

DynamicRuleValue x => await _queryProcessor.ExecuteQuery(x.Query),

_ => throw new ArgumentOutOfRangeException()

};

}

}

DynamicRuleValue Query

In technical terms, getting the dynamic value is essentially a query. The query handler has to fetch the required data series and then implement a certain formula on the data. For that, we need to know the timeframe and the symbol of the data series. As the formula might rely on more than one symbol (when calculating ratios for example), I’ve wrapped the symbols in the previously referred CalculationSymbols class.

Calculations can consume a lot of memory, which is why it is unreasonable to redo the same calculations on an identical dataset. With possibly multiple strategies running simultaneously, we can easily find ourselves relying on similar underlying calculations. Let’s use a wrapper class called CalculationQuery , which can later be used to enforce cache logic for such occasions (or rather we can use the CalculationQueryHandler ).

public abstract class CalculationQuery<TResult> : Query<TResult>

{

public TimeFrame TimeFrame { get; }

public CalculationSymbols Symbols { get; }



protected CalculationQuery(TimeFrame timeFrame, CalculationSymbols symbols)

{

TimeFrame = timeFrame;

Symbols = symbols;

}

}

The actual formula of a calculation will be held within a concrete CalculationQuery handler. Here you can see how a failure is returned in case we don’t get sufficient data. Right now, only a simple moving average has been implemented. We can define countless indicators or any other calculations in form of CalculationQueries for the end developer to use. Here’s how a moving average query looks like:

public class MovingAverageCalculationQuery : CalculationQuery<Result<decimal>>

{

public AppliedPrice AppliedPrice { get; }

public DataSeriesIndex Index { get; }

public DataSeriesLength Length { get; }

public MAMethod Method { get; private set; }

public Symbol Symbol => Symbols.Symbols.Single();



public MovingAverageCalculationQuery(

TimeFrame timeFrame,

Symbol symbol,

AppliedPrice appliedPrice,

DataSeriesIndex index,

DataSeriesLength length,

MAMethod method)

: base(timeFrame, new CalculationSymbols(symbol))

{

AppliedPrice = appliedPrice;

Index = index;

Length = length;

Method = method;

}



internal class Handler : ICalculationQueryHandler<MovingAverageCalculationQuery, Result<decimal>>

{

private readonly IDataSeriesProvider _dataSeriesProvider;



public Handler(IDataSeriesProvider dataSeriesProvider)

{

_dataSeriesProvider = dataSeriesProvider;

}



public async Task<Result<decimal>> Handle(MovingAverageCalculationQuery query, CancellationToken cancellationToken)

{

var dataSeriesResult = await _dataSeriesProvider.Provide(query.Symbol, query.TimeFrame, query.AppliedPrice, query.Length, query.Index);

if (dataSeriesResult.IsFailure)

return Result<decimal>.Failure(dataSeriesResult.Error);



var dataSeries = dataSeriesResult.Value;

return query.Method switch

{

MAMethod.Simple => Result.Success(dataSeries.Values.Average()),

_ => throw new NotImplementedException()

};

}

}

}

Now the only thing left is to compare the 2 values. This is done in a ValueComparer service, which parses the logical operator type and compares the two values.

public class ValueComparer : IValueComparer

{

public bool Compare(decimal value1, LogicalOperator logicalOperator, decimal value2)

{

return logicalOperator switch

{

LogicalOperator.Equal => value1 == value2,

LogicalOperator.NotEqual => value1 != value2,

LogicalOperator.GreaterThan => value1 > value2,

LogicalOperator.GreaterThanOrEqual => value1 >= value2,

LogicalOperator.LesserThan => value1 < value2,

LogicalOperator.LesserThanOrEqual => value1 <= value2,

_ => throw new ArgumentOutOfRangeException()

};

}

}

Next steps

The current solution provides the core logic for implementing and simulating a technical trading strategy. There are, however, numerous matters and that require attention going forward.

Potential problems

As the system is asynchronous, it‘s important to ensure that whenever a new data point is reacted upon, it’s accounted for when fetching a data series as well. For example, if we get a 1 PM quote (which would start an hourly period) and we’re using an hourly interval for some of our queries, the data series provider must already have that quote accounted for in an hourly data series by the time the corresponding calculation query is being handled. For that reason, the event starting the workflow is called SymbolUpdatedEvent and not DataPointReceivedEvent (and the data assembly logic should account for that).

What if multiple entry/exit conditions are true simultaneously? What if executing one order within a strategy would immediately make another valid condition of the same strategy invalid (containing an opened orders amount rule for example). Conditions, upon successful evaluation, should carry weights or priorities to determine the execution order. It’s also possible that we’ll need to abandon parallelism when evaluating conditions.

The data should be meticulously processed to make sure that the times of the data points are transformed into the same timezone within the system. Furthermore, splits should be accounted for to get an adjusted price series. Finally, dividends and entry fees should be included in the final reports as well to give the most realistic projection of the results when backtesting.

Optimization

While the modeling is tilted towards an elegant solution rather than a high-frequency trading engine, I’m sure there are numerous ways to optimize the current solution, which are worth considering. Currently, we filtered out entire strategies based on the symbol of the new data point. What if inside the strategy there are 10 rules, of which only 1 is interested in the symbol. We should move the filtering closer to the actual CalculationQueries . Furthermore, we shouldn’t redo the calculations with each data point if we want to get a moving average on a daily data series for example.

Limitiations/Restrictions

The current solution works with technical strategies based on the price of the underlying symbols. This can lead to a potentially biased view, ideally, we’d want the symbols to be filtered for our strategies based on certain fundamental or technical criteria. Thus we’d need to be able to react to not only price data, but macroeconomic and fundamental figures as well as any form of alternative data. We could achieve that by different lifecycle events (similar to SymbolUpdatedEvents ) and evaluate only the conditions that are interested in the given data.

Another limitation is that the current CalculationQuery allows for calculations only on pure price data. We should include the form of the data series as well so that we could for example calculate a standard deviation not only on prices of a certain symbol but on any data we deem fit (the last 10 quarterly earnings of a given sector for example).

Summary

All in all, there are important matters that require tweaking the solution going forward. That being said, in the current form, the engine is ready for simulating technical strategies.

Along with an efficient engine, the goal was to provide a generic, flexible, and reusable solution for developing trading strategies. If various bits and pieces could be combined without the need to manually hard-code each strategy, we can eventually automate not only the evaluation of a strategy but the actual creation and analysis process.