Algo Trading for Dummies - Building a Custom Back-tester (Part 3)
While there are many simple backtesting libraries available, they can be quite complex to use effectively — requiring a lot of extra processing of data sets. It is sometimes worth coding a custom back-tester to suit your needs.
Building a back-tester is a fantastic conceptual exercise. Not only does this give you a deeper insight into orders and their interaction with the market, but it can also provide the framework for the order handling module of your trading bot.
Order Handling
One of the key pieces to an active trading strategy is the handling of more advanced orders types, such as trailing stops, automatically hedged positions or conditional orders.
For this you’ll want a separate module to manage the order logic before submitting to an exchange. You may even need a dedicated thread to actively manage orders once submitted, in case the platform itself doesn’t offer the necessary types.
Its best for the module to keep an internal representation of each position and its associated orders, which is then verified and amended as the orders are filled. This means you can run calculations against your positions without the need to constantly be querying the broker. It also allows you to easily convert the code for use in your back-tester, by simply altering the order fill checks to reference the historical data at each time step.
It may also be worth implementing order aggregation and splitting algorithms. For example, you may want a function to split a larger limit order across multiple price levels to hedge your bets on the optimal fill. Or, indeed, you might need a system to net together the orders of multiple simultaneous strategies.
Assumptions and Issues of Back-testing
Unless you’re using tick data and bid/ask snapshots to back-test against, there will always be a level of uncertainty in a simulated trade as to whether it would fill fully, at what price, and at what time. The period of each data point can also cause issues if its above the desired polling rate of the trading bot.
These uncertainties are lessened as the average holding period for each trade increased vs the resolution of your data, but is never fully eliminated. It is advised to always assume the worst case scenario in your simulation, as its better for a strategy to be over prepared than under.
For example, if a stop-loss order would have been triggered during the span of a bar, then you’d want to add some slippage to its trigger price and/or use the bar’s closing price. In reality, your are unlikely to get filled so unfavorably, but it’s impossible to tell without higher granularity data.
On top of this, it is impossible to simulate the effect of your order on the market movement itself. While this would be unlikely to have a noticeable effect on most strategies, if you’re using extremely short holding times on each trade or larger amounts of capital, it could certainly be a factor.
Designing an Efficient Back-tester
When calculating the next time step for an indicator, unless you’ve stored all relevant variables you will be recalculating a lot of information from the look-back period. This is unavoidable in a live system and, indeed, less of an issue, as you won’t be able to process data faster than it arrives. But you really don’t want to wait around longer than you have to for a simulation to complete.
The easiest and most efficient workaround is to calculate the full set of indicators over the whole dataset at start-up. These can then be indexed against their respective symbols and time stamps and saved for later. Even better, you could run a batch of back-tests in the same session without needing to recalculate the basic indicators between runs.
At each time you will then simply query the set of indexed indicators, construct the trading signals and push the orders to the order handling module, where the simulated positions are calculated along with their profit/ loss. You’ll also want to store the position and order fill information, either as a subscript to the back-tester or integrated directly into the position handling module.
Even Improving Your Back-tester
Back-testing is only as useful as the insight its statistics provide. Common review metrics include win/loss ratio, average profit/loss, average trade time, etc. However you may want to generate more insightful reports, such as position risk:reward ratios or an aggregate of price movement before and after each traded signal, which allows you to fine tune the algorithm.
Once the full framework has been designed, implemented and debugged should you start looking for ways to speed up and upgrade the inner loop of the back-tester (the order handling module). It is a lot easier to take a working program and make it faster than it is to take an overly optimized program and make it work.
By Matthew Tweed
Full position handling class framework:
import alpaca_trade_api as tradeapi
api = tradeapi.REST(key_id=<your key id>,secret_key=<your secret key>)
class positionHandler:
def __init__(self,startingBalance=10000,liveTrading=False):
self.cashBalance = startingBalance
self.livePositions = {} # Dictionary of currently open positions
self.openOrders = [] # List of open orders
self.positionHistory = [] # List of items [Symbol, new position size]
self.tradeHistory = [] # List of filled trades
self.liveTrading = liveTrading
def placeOrder(self, symbol, quantity, side, orderType, time_in_force, limit_price=None, stop_price=None, client_order_id=None):
if self.liveTrading:
returned = api.submit_order(symbol, qty, side, orderType, time_in_force, limit_price, stop_price, client_order_id)
self.tradeHistory.append(returned) # You'll probably want to make a simpler custom order dict format
else:
self.tradeHistory.append(<order Dict>)
if orderType == "market":
try:
if side == "buy":
fillPrice = data[symbol]["close"] # You'll need to make adjustments to the backtest fill price assumptions
self.livePositions[symbol][size] = self.livePositions[symbol][size] + quantity
self.cashBalance -= quantity * fillPrice
elif side == "sell":
fillPrice = data[symbol]["close"]
self.livePositions[symbol][size] = self.livePositions[symbol][size] - quantity
self.cashBalance += quantity * fillPrice
if self.livePositions[symbol][size] == 0:
del self.livePositions[symbol]
except:
self.livePositions[symbol] = {}
if side == "buy":
self.livePositions[symbol][size] = quantity
elif side == "sell":
self.livePositions[symbol][size] = -quantity
self.positionHistory.append([symbol,self.livePositions[symbol]])
else:
self.openOrders.append(<order Dict>) # You'll probably want to make a simpler custom order dict format
def processOpenOrders(self):
for order in self.openOrders:
symbol = order["symbol"]
if self.liveTrading:
returned = api.get_order(order["order_id"])
# Process the live order status into your storage format as necessary
else:
# Historical data input has to be adjusted for your own data pipeline setup
timeStepMin = data[symbol]["low"] # Reads the minimum trade price since last data point
timeStepMax = data[symbol]["high"] # Reads the maximum trade price since last data point
if order["orderType"] == "limit":
try:
if order["side"] == "buy" and order["limit"] > timeStepMin:
# You'll need to make adjustments to the backtest fill price assumptions
fillPrice = data[symbol]["close"]
self.livePositions[symbol][size] = self.livePositions[symbol][size] + quantity
self.cashBalance -= quantity * fillPrice
self.positionHistory.append([symbol,self.livePositions[symbol]])
elif order["side"] == "sell" and order["limit"] < timeStepMax:
fillPrice = data[symbol]["close"]
self.livePositions[symbol][size] = self.livePositions[symbol][size] - quantity
self.cashBalance += quantity * fillPrice
self.positionHistory.append([symbol,self.livePositions[symbol]])
except:
self.livePositions[symbol] = {}
if order["side"] == "buy" and order["limit"] > timeStepMin:
fillPrice = data[symbol]["close"]
self.livePositions[symbol][size] = quantity
self.cashBalance -= quantity * fillPrice
self.positionHistory.append([symbol,self.livePositions[symbol]])
elif order["side"] == "sell" and order["limit"] < timeStepMax:
fillPrice = data[symbol]["close"]
self.livePositions[symbol][size] = -quantity
self.cashBalance += quantity * fillPrice
self.positionHistory.append([symbol,self.livePositions[symbol]])
elif # Add processing for other required order types
def returnOpenPosition(self,symbol):
try:
return self.livePositions[symbol]
except:
return 0
){kind=link}