My optimization metrics

I have decided to document what metrics and process I use to determine a “good” strategy.

  1. Final Balance:       The amount of money left after the testing
  2. PT/LT Ratio:          The number of Profit trades over the number of Loss trades.
  3. Exp Profit:             The Total Profit divided by the number of Profit trades.
  4. Exp Loss:               The Total Loss divided by the number of Loss trades.
  5. Exp Ratio:              The Exp Profit over the Exp Loss
  6. Profit Factor:         The Total Profit over the Total Loss
  7. Min Equity:            The lowest the equity ever went too.
  8. Max Fall in Equity:The largest amount the Equity fell by from the last high.

I believe 2, 5, and 6 should be over 1.00 with some flexibility on the Exp Ratio if the PT/LT Ratio is high (> 2.0). Also, I don’t have any confidence in statistics based off a small data sample so the number of trades should be at least 10, though the higher the better.

With special focus on the three metrics I mentioned above, I go through and try to pick the best two or three results for each similar set of parameters. By similar set of parameters, I mean significant parameter values are the same/similar. Sometimes there can be good strategies with large differences in significant parameters, so I would define these as a new set.

For example: If using MACD, we had lots of very good results for a Slow Period number of 25 and 26 but also for some reason the value of 17 produced good values, then I would define these as two different parameter sets and look at both more closely.

If there are many good sets, I then try to use the Min Equity and Max Fall in Equity numbers to filter the strategies that can lose lots of money out. After that if I’m still torn, I would pick the higher Exp Ratio.

Once I have a bunch of parameter sets I’m happy with, I then backtest them and take a look at the Equity/Balance curve. Ideally our strategies should have a nice smooth increasing curve. I don’t like seeing sharp drops or large dips – so if possible try to remove those sets as well.

After all that, I then backtest over the GFC period from 01/2008 – 12/2009. I’m not expecting the strategies to necessarily make money during this period, but rather not crash and burn. So if I have to choose between two strategies that were equally successful in the optimization results but one lost 80% equity in 2008 compared to one that lost 40% in 2008, the choice is clear.

Posted in Backtesting, Optimization, Research and tagged , , , .

Leave a Reply

Your email address will not be published. Required fields are marked *