I have decided to document what metrics and process I use to determine a “good” strategy.
- Final Balance: The amount of money left after the testing
- PT/LT Ratio: The number of Profit trades over the number of Loss trades.
- Exp Profit: The Total Profit divided by the number of Profit trades.
- Exp Loss: The Total Loss divided by the number of Loss trades.
- Exp Ratio: The Exp Profit over the Exp Loss
- Profit Factor: The Total Profit over the Total Loss
- Min Equity: The lowest the equity ever went too.
- Max Fall in Equity:The largest amount the Equity fell by from the last high.
I believe 2, 5, and 6 should be over 1.00 with some flexibility on the Exp Ratio if the PT/LT Ratio is high (> 2.0). Also, I don’t have any confidence in statistics based off a small data sample so the number of trades should be at least 10, though the higher the better.
With special focus on the three metrics I mentioned above, I go through and try to pick the best two or three results for each similar set of parameters. By similar set of parameters, I mean significant parameter values are the same/similar. Sometimes there can be good strategies with large differences in significant parameters, so I would define these as a new set.
For example: If using MACD, we had lots of very good results for a Slow Period number of 25 and 26 but also for some reason the value of 17 produced good values, then I would define these as two different parameter sets and look at both more closely.
If there are many good sets, I then try to use the Min Equity and Max Fall in Equity numbers to filter the strategies that can lose lots of money out. After that if I’m still torn, I would pick the higher Exp Ratio.
Once I have a bunch of parameter sets I’m happy with, I then backtest them and take a look at the Equity/Balance curve. Ideally our strategies should have a nice smooth increasing curve. I don’t like seeing sharp drops or large dips – so if possible try to remove those sets as well.
After all that, I then backtest over the GFC period from 01/2008 – 12/2009. I’m not expecting the strategies to necessarily make money during this period, but rather not crash and burn. So if I have to choose between two strategies that were equally successful in the optimization results but one lost 80% equity in 2008 compared to one that lost 40% in 2008, the choice is clear.