To Optimize or Not to Optimize?
If you have a computer, you can easily set up a search to find the “optimum” values for a system over historical data. The results can be truly astonishing. Imagine your profits if you could only have known ahead of time what the most profitable parameter combination was going to be. Therein lies the rub. The unfortunate fact is that parameters that work best on past data rarely provide similar performance in the future.
The term “optimization” is used rather loosely here to include all the activities affecting selection of parameter values in a trading system. We have already seen the difficulties of curve-fitting a model. You can also consider lower levels of optimization, in which you test variables over a broad range of values and markets, and try to select the one you like “best.” But the real issue is not whether a particular set is the best. It is whether you believe sufficiently in the system to trade it without deviations. The primary benefit of optimization may be that you improve your comfort level with a particular system.
The problem with system optimization is that past price patterns do not repeat exactly in the future. The same is true of intermarket relationships. Although broad relationships follow from historical data, there can be differences in the time-lags between events and the relative magnitudes of the effects.
You must also resolve other conflicts. For example, you must choose the period you will use to optimize your trading system values. As you will quickly discover, the values you choose depend on the length of the test period. You must also determine how often you will reoptimize your system in the future. You must then prescribe the time for which the optimized values are valid.
For example, you may decide to use 3 years of data to optimize the values and recalculate them after 3 months. Thus, one solution may be to reoptimize after 3 months on the latest 3 years of data available. This is equivalent to retraining your favorite neural net. If you do reoptimize, you must determine how to treat trades that may be open from the previous period or values of the trading system.
You must also decide if you want to use the same values of your system parameters on all markets. If not, you will have to optimize the system on each market separately. In that case, you must keep up a program of reoptimization and recalibration for each of your systems over every market that you trade. Is all this effort worth the trouble? The results of deterministic testing do not support any attempts at finding the “best” or optimized variables.
Consider the following test using actual deutsche mark futures contracts. The rollover dates are the twenty-first day of the month be-fore expiration. For simplicity, we will trade just one contract, allowing $100 for slippage and commissions, with a $1,500 initial money management stop. We will use a variation of the moving average crossover system, trading not the crossover, but a 5-day breakout in prices after the crossover. Thus, if the shorter moving average was above the longer moving average, then a 5-day breakout above the highs would trigger a long entry. Also included is a simple exit condition, ending the trade on the close of the twentieth day in the trade. One attractive feature of this arbitrary system is that the lengths of the short and long moving average can be optimized.
The calculations are simplified by fixing the length of the short average to a 3-day simple moving average of the close. The length of the longer simple moving average varies from 20 to 50 days, with an increment of 5 days. The test period was from November 14, 1983, through November 21, 1989. The performance of the various models was observed 3, 6, 9, and 12 months into the future. As Tables 3.6 and 3.7 show, there is no predicting how the model will do over a future period. The relative rankings change from period to period without any pattern or consistency.
We next test the hypothesis that if the optimization period were closer to the actual trading period, the predictions would be more reli-able. However, as Tables 3.8 and 3.9 show, there is again no way to predict what the model will do in the succeeding periods. This should be expected because there is no cause-and-effect relationship between our optimized model and market forces. Since we are merely fitting a model to past data, we are not capturing all the fundamental and psychological forces driving the market. Our poor ability to predict the future based only on past price data is not surprising.
Let us carry our argument one step forward. Because we do not capture any cause-and-effect relationships, optimization on one market should have little or no benefit for trading other markets. Indeed, as Table 3.10 shows, optimizing a system on one market (here the deutsche mark) does little to improve performance in other markets.
Any optimization exercise has many potential benefits. The first benefit is recognition of the type of market conditions under which the trading system is unprofitable. For any rules that you can construct, you can find market action that produces losses. This happens because the market triggers the signal, and then does just the opposite instead of following through.
The second benefit is verification of the general ideas underlying the model. For example, you can check to see if the model is profitable in trending markets or trendless markets. You have designed the rules to be profitable under certain market assumptions. The optimization exercise allows you to verify if your broad assumptions are correct.
A third benefit is understanding the effect of initial money management stops. You can quantify what level of initial stop allows you to capture the majority of potential profits. For example, if your stop is too wide, your losing trades will be relatively large. On the other hand, if your stop is too close to the starting position, you will be stopped out frequently. Your loss per trade will be small. However, the higher fre-quency of losing trades means your total drawdown could exceed a larger initial stop.
The biggest benefit of optimization is reinforcing your beliefs about a particular trading system. Ultimately, it is more important for you to implement the trading system exactly as planned. Hence, any testing you do that allows you to understand system performance and become more comfortable with its profit and loss characteristics will help you to execute it with greater confidence in actual trading.
The main point of this section is that you cannot assume your sys-tem is going to be as profitable in the future as it has been in the past. This raises the issue of how you control your risks to cope with uncer-tain future performance. The next section presents risk-control ideas.