Have you ever wonder how some traders win trading competitions; with staggering returns; sometimes in excess of 1000% in one single month.

the following is a simple technique. Variations of it does exists.

open two accounts; and run two strategies that are reverse of each other. i.e.:

strategy 1:

- buy 10 units of EURUSD

strategy 2:

- sell 10 units of EURUSD

hold all trades until end of contest; then liquidate.

one strategy will be losing; and the other winning. Of course there is a
*small* chance that both strategies will breakeven; but it is low
probability.

Strategy 1:

Strategy 2:

Majority of forex brokers encourage competitions using micro accounts;
as low as $500. Losing $500 in one account; to win a prize of $20,000 in
the other account; is not such a bad idea after all.

Moreover; entering trading contests with paper money; is free money for cheaters. losing $$$ on paper money in one
account; to win a price worth $50 ~ $500 is not bad.

i did a backtest of the above strategies in April 2012; strategy 1 lost 15%; and strategy 2 made 15%.

notice how the max profitable percentage of strategy2 was about 40%; at the beginning of April.

you could just liquidate your positions and stop trading at that time. and you are done for the month.

If you observe some winners in trade2win contests; you will see this
behavior. users stop trading suddenly after the equity curve reached a
high mark. and also; you will see that they only traded a position or
two.

If you have any other technique; please share it.

Disclaimer: the purpose of this thread is to expose the cheating
techniques; so you dont get duped into beleiving real life trading behave in the same way. Following this techinque may not improve your trading
career; despite you may end up winning prizes. many retail forex
brokers have weekly and monthly competitions; with prizes reaching as
high as $50K. now that you know about this secret; go for your life; and
start a career of collecting prizes from trading contests.

# Liquid Alpha

System Development Methods, Market Neutral Strategies, Automated Trading, Quantitative Methods, Research and Statistical Analysis

## Friday, May 11, 2012

## Thursday, May 10, 2012

### About this blog

Hi, this blog is a place where I like to share my knowledge, experiences; and a platform for collaboration with other system developers around the world.My focus is on automated trading strategies; based on quantitative methods. I trade all types of strategies; momentum, mean reversion, range bounds, and chart patterns.

I invite you to share your ideas and pose questions related to trading.

enjoy!

I can be reached at liquidalpha9 [at] gmail . com

I invite you to share your ideas and pose questions related to trading.

enjoy!

I can be reached at liquidalpha9 [at] gmail . com

### Data snooping bias, the evil of trading system development - Part 1

This blog and post started as a result of my recent discussion with Jez Liberty on his article on handling data mining bias. It is rather the most difficult topic in system development.

In this series of posts; am going to talk about my experience in handling data snooping bias (Also known as curve fitting bias, and data mining bias). In hope that we system developers learn more about this subject; and as a result benefit our trading career. I welcome posters to discuss their opinions and share thei rexperience.

The Problem

You have developed a new trading system, backtested it on recent data, and you were impressed with the 45% equity curve. what is the next step? certainly not trade it live until you proved you have an edge.

One of the biggest challenges of building mechanical systems; is whether the system you built has an edge; or simply is curve fitted.

Here are some tools you can use:

1- test it on out of sample data

2- forward test it on unseen data

3- robust statistical tests on the backtest performance to identify the robustness of the system

On part 1; i am going to talk about method (1) and method (2); and in the next post i will talk about the rather delicate topic; of robust statistics.

The Experiment

In this post; i am going to show a trading system that I developed; and walk you through the steps; to identify if the system is worthy of trading or not. In other words; if the system is curve fitted; or has a real edge.

The strategy is trend following on EUR/USD currency pair.

Backtest Result

The following is a backtest results are for two years (Jan 2010 till Nov 2011).

The total return was 177% with annual return of 98% and max drawdown of 12%. A sharpe ratio of 2.5.

at first glance; who wouldn't trade this system? the 45% degree angle of the equity curve is certainly appealing.

Out of Sample Test (OOS)

In reality OOS is not OOS. it is In-Sample (IS) testing; unless you are extremely disciplined and organized. why? because of hindsight.

Once you have tested your strategy in IS; then in OOS; and it failed in the latter. You went ahead and changed the rules; then tested again in IS then OOS until the OOS got good results.

Here is an example of this process:

1. You have in sample data set A, and out of sample data set B

2. Find a trading system that fits data set A, test it on data set B, it doesn’t work

3. Find or tweak the system to data set A, test it on data set B, it doesn’t work

4. Repeat step 3 for N times; and stop when you found the system that works on both data sets; A and B.

5. Et voila; you beleived you have found a system that tested well on OOS.

This process is essentially curve fitting. In practice; data set B is not OOS anymore. it is just an extension to data set A. You fooled yourself into beleiving your best strategy works in OOS.

You can only test your strategy one time only on OOS. Otherwise it became IS. You have now a hindsight; and hence you cannot claim later that your strategy worked in OOS.

as long as you only use OOS once; and only once for a particular strategy; then you have to go through forward testing before putting any money in it.

It is difficult psychologically to stick to the rules of IS / OOS. Here are some ways to get over this problem:

1. divide your data onto 3 parts, data set A, B and C.

2. develop your strategies using data set A. After you have found a 60 degrees equity curve,

3. test the strategy on data set B, if you still get a 60 degrees equity curve; or 30 degrees or what ever, prepare yourself for live trading the next day

4. do a dress rehearsal and do a final test on data set C, as a preparation for go live. If you get a 0 degree or less curve; back to step (1) above.

Forward Test

Forward testing is the mother of all tests. It is robust and guaranteed to work. Moreover; it addreses all sort of biases:

- curve fitting bias

- data mining bias

- look ahead bias

- survivalship bias

- software bugs

Experience suggest that forward testing of 2 months; or ~300 trades; is a must; in order to address the the biases (beside the curve fitting bias).

This robustness comes at a cost; both in time and efforts. If you were to test every strategy you develop in forward testing for a couple of months; it will take years and years before you find one profitbale strategy.

Also; it is easy to get carried out by other tasks and ignores the forward test system. You need to pay attention to it as close as to real life trading.

Back to our example above. Here is a 3 months forward test result of the above strategy:

The strategy went sideways for a couple of months; and then slide in the last 4 month.

In the next part of this post; i will discuss robust statistical tests; that saves you the hassle of forward testing.

If you have any thoughts on tackling this issue; please share them with us.

In this series of posts; am going to talk about my experience in handling data snooping bias (Also known as curve fitting bias, and data mining bias). In hope that we system developers learn more about this subject; and as a result benefit our trading career. I welcome posters to discuss their opinions and share thei rexperience.

The Problem

You have developed a new trading system, backtested it on recent data, and you were impressed with the 45% equity curve. what is the next step? certainly not trade it live until you proved you have an edge.

One of the biggest challenges of building mechanical systems; is whether the system you built has an edge; or simply is curve fitted.

Here are some tools you can use:

1- test it on out of sample data

2- forward test it on unseen data

3- robust statistical tests on the backtest performance to identify the robustness of the system

On part 1; i am going to talk about method (1) and method (2); and in the next post i will talk about the rather delicate topic; of robust statistics.

The Experiment

In this post; i am going to show a trading system that I developed; and walk you through the steps; to identify if the system is worthy of trading or not. In other words; if the system is curve fitted; or has a real edge.

The strategy is trend following on EUR/USD currency pair.

Backtest Result

The following is a backtest results are for two years (Jan 2010 till Nov 2011).

The total return was 177% with annual return of 98% and max drawdown of 12%. A sharpe ratio of 2.5.

at first glance; who wouldn't trade this system? the 45% degree angle of the equity curve is certainly appealing.

Out of Sample Test (OOS)

In reality OOS is not OOS. it is In-Sample (IS) testing; unless you are extremely disciplined and organized. why? because of hindsight.

Once you have tested your strategy in IS; then in OOS; and it failed in the latter. You went ahead and changed the rules; then tested again in IS then OOS until the OOS got good results.

Here is an example of this process:

1. You have in sample data set A, and out of sample data set B

2. Find a trading system that fits data set A, test it on data set B, it doesn’t work

3. Find or tweak the system to data set A, test it on data set B, it doesn’t work

4. Repeat step 3 for N times; and stop when you found the system that works on both data sets; A and B.

5. Et voila; you beleived you have found a system that tested well on OOS.

This process is essentially curve fitting. In practice; data set B is not OOS anymore. it is just an extension to data set A. You fooled yourself into beleiving your best strategy works in OOS.

You can only test your strategy one time only on OOS. Otherwise it became IS. You have now a hindsight; and hence you cannot claim later that your strategy worked in OOS.

as long as you only use OOS once; and only once for a particular strategy; then you have to go through forward testing before putting any money in it.

It is difficult psychologically to stick to the rules of IS / OOS. Here are some ways to get over this problem:

1. divide your data onto 3 parts, data set A, B and C.

2. develop your strategies using data set A. After you have found a 60 degrees equity curve,

3. test the strategy on data set B, if you still get a 60 degrees equity curve; or 30 degrees or what ever, prepare yourself for live trading the next day

4. do a dress rehearsal and do a final test on data set C, as a preparation for go live. If you get a 0 degree or less curve; back to step (1) above.

Forward Test

Forward testing is the mother of all tests. It is robust and guaranteed to work. Moreover; it addreses all sort of biases:

- curve fitting bias

- data mining bias

- look ahead bias

- survivalship bias

- software bugs

Experience suggest that forward testing of 2 months; or ~300 trades; is a must; in order to address the the biases (beside the curve fitting bias).

This robustness comes at a cost; both in time and efforts. If you were to test every strategy you develop in forward testing for a couple of months; it will take years and years before you find one profitbale strategy.

Also; it is easy to get carried out by other tasks and ignores the forward test system. You need to pay attention to it as close as to real life trading.

Back to our example above. Here is a 3 months forward test result of the above strategy:

The strategy went sideways for a couple of months; and then slide in the last 4 month.

In the next part of this post; i will discuss robust statistical tests; that saves you the hassle of forward testing.

If you have any thoughts on tackling this issue; please share them with us.

Subscribe to:
Posts (Atom)