The results below from 1 system that I ran through the optimizer throws aside the notion that your winners must always be bigger than your losers. It crunched all the scenarios varying the target from 10 to 150 ticks and the stop from 10 to 100 ticks, in increments of 5 ticks to find the highest net profit over past 6 months. Note that this is just an example and arguably, net profit isn't necessarily the best metric to optimize.
Posts
I wish I could back test my method that way, very cool.
ReplyDeleteQuestion, every system you've used since I started following along has back tested well in the past. Actual trading unfortunately gave you different results. Is it due to maybe the market conditions are different, so the system performs differently. Or are you not taking every signal, which might explain the difference?
It's mostly due to lack of conviction on my part which is partly due to not enough backtesting. Remember that in the past I would scroll through the charts and record every trade into Excel by hand so it might take weeks to backtest, say, 500 trades. And optimizing was mostly out of the question. Now I can backtest 1000s of trades in just a few minutes. This "should" increase my confidence so I won't stop trading a system just because of a seemingly "large" drawdown. But you are correct also in that some of my systems had some discretion built into them. I think I've learned to eliminate that monkey going forward!
ReplyDeleteCool... I'll keep my fingers crossed!
ReplyDeletejust want to remind you Mwinfrey on BigMike wrote many strategy http://www.bigmiketrading.com/ninjatrader-programming/5220-perrys-trading-strategy-development-21.html#post76101 also they have 'battle of the bot' thread that publishes many strategies. A perfect source for you to 'get' sample coding.
ReplyDeleteThanks Cory!! I hadn't seen that thread and will check it out. Sometimes I need to get out of my cocoon and find more useful stuff on the web. I did find very useful the programming video BigMike did awhile ago: http://www.bigmiketrading.com/ninjatrader/2512-video-tutorial-how-create-advanced-ninjatrader-strategy.html
ReplyDeleteI wish you the best but I am afraid you will be disappointed. Many traders don't realise that the more they bend the system to fit the data the more likely it is to collapse in real time trading. No one bit of data is going to replicate itself exactly again. On the internet, you can purchase many "profitable" systems... that are all curve fitted and will dramatically fail in real time trading. If a system is soundly based, it should work across several markets, use the same rules all the time and be simple with relatively few rules and parameters. Unfortunately, I have never found such a system so my trading is still 20% discretionary.
ReplyDeleteThe problem with most backtesting is that it assumes trading right through big economic news releases, it assumes that each entry is actually possible (i.e. "I would have entered" during that 2-second spike), and it assumes no slippage, and in your case also, no commissions. That's why I believe that while it can be interesting and offer insight to certain strategies, it is somewhat of an exercise in futility. Why did you abandon your old strategy, instead of tightening up your discretion?
ReplyDeleteFXEdge and kroz have some very valid points.
ReplyDeleteYesterday, I presented "keys to successful trading" at the TradersWorld Online Expo. One of my points was "Traditional backtests are useless."
In general, backtesting software is probably a trader's most dangerous tool. There are so many things that you can do that lead you down the wrong path it is not even funny. Here's a few:
1. Not including commissions and slippage. This is big.
2. If you optimized up until a few days, and you pick the "best" one, I can virtually guarantee it won't be the best (and may not even be profitable) going forward. I recommend either walkforward testing, or out of sample testing.
But, since you already optimized the last 6 months, just set that "best" method aside, and in a month look at it, and see how it would have done. You might be glad you did not trade with real money.
3. 6 months is not enough time for a backtest. 5 years minimum, preferably 10 years.
Good Luck! I wish you the best!
In addition, another problem I always had even with manual bar-by-bar backtesting, is that if you go back either a few years (financial crisis) or a decade (much less daily volume), the markets were moving differently...volatility was different, ADR was different, liquidity was different...so it doesn't even make sense to optimize a strategy that works well for market conditions that don't even exist anymore. That's why I believe that forward testing/refining a premise based on things that don't change -- namely fear and greed -- is the only way to go.
ReplyDeleteFXEdge, Kroz, Kevin - thanks, all great points that I'm very aware of. I've been around for awhile and have experienced over optimization and analysis paralysis in the past. Backtesting is a just a tool but a good one, imho. If a system didn't work in the past, there's not much chance it will work in the future. However if it DID work in the past, there's a greater chance of success in future.
ReplyDeleteRegarding slippage and commission - critical - agreed. I look at the Ninja reports and figure a net profit of ~$10/trade (1 contract) is around breakeven in general. And trading at news events also can be taken into account by downloading the trade report, sorting by time, and observing the affect of taking out all the trades that took place near 9am CST, for example.
Saving some data for out of sample testing is important too and/or forward testing as you suggest Kevin. Unfortunately 6 months is about the most tick data I have with IQ Feed. For minute based bars, I can go back a decade or more.
Tradestation is the same way - they only provide 6 months of tick data. There are sources of such data, but it is not cheap.
ReplyDeleteThat is the reason why I personally avoid tick based systems.
Sounds like you are on the right track. It will be neat to see you get back to making profits!