EA Studio strategy generation workflow issues

[Daryn Stewart]

Hi Traders

Although a few of these topics have been covered in short in other threads, I thought I’d start a new thread as this is more related to the overall workflow in EA studio.

I have a lot to cover here but I will try and keep it as short as possible but the foundation of this is based on the teachings in Petko’s courses. I have been trading in demo environments for over a year now, however, with the introduction of EA Studio, it opens up a whole new world of possibilities and combinations of factors never considered before with creating strategies and I am still struggling to find an efficient workflow to manage this process as each step presents it own issues.

I will start with the generating strategies:

Firstly, the number of combinations of acceptance criteria is a bit overwhelming and in some instances counter productive. I know we all want winning strategies but setting a high win/loss ratio does not really mean you will have a great strategy because your losses could be higher than your wins as an example. So typically I would just set for 200 trades and a profit factor of 1.2 over 1 years data, otherwise the criteria becomes too strict and I do not get many strategies at all. I know we’re only looking for the good ones but more on this just now.

Typically I would generate strategies on premium data because I cannot download enough data from my broker to generate any strategies, unless I wait a few weeks for more data to become available.

For my own sanity, I have run a test on the EURUSD currency pair using the same generation settings but 4 different scenarios. Generator ran for 12hours. I have a high end PC in trading terms and latency to the server is ~20ms.

1. Generator Only

2. Generator + Monte Carlo

3. Generator + Normalisation

4. Generator + Normalisation + Monte Carlo

I chose the top 10 strategies and ran them on 3 demo accounts from 1 broker and additional demo account from another broker, and they were run as individual EAs and as Portfolio EAs.

There were quite a few differences in order data between individual EA’s vs Portfolio EA’s and running these on different accounts. The second broker was vastly different but I did not account for the account difference in account currency or currency specifications. Nevertheless, the results are all over the place, but generally speaking, Scenario 1 was the worst, scenario 2 and 3 performed better and scenario 4 had very mixed results.

So the question is to do any form of optimisation or not?

But with the strategies being generated on premium data and not being able to generate strategies on actual broker data, the only solution is to run it through the Validator to recalculate on the broker data and of course, the result are very different. This means I would need to optimise the strategies for my broker data? But now we run the risk of curve fitting again. Therefore, I use OOS to discard “over optimised” historic data and only focus on current market performance, as we’re only really concerned about performance in current market conditions. The issue here is that there is no easy way to focus on strategy specific OOS data when running a collection through the validator, without going into each strategy to check the OSS data. Some strategies that don’t look great on the complete back test, perform quite well in the OOS but all the visible generation stats are focused on the complete back test. Could this be a product development suggestion? 

Although you can set OSS acceptance criteria at the generation stage, some strategies don’t meet the OSS criteria until it has been optimised. I have run the same collection through the validator with different optimisation and robust testing methods and each produces a vastly different result. Same acceptance basic acceptance criteria, of course.

There also doesn’t seem to be any easy way to keep track of your EA’s because each time it goes through the validator or gets saved as an EA or in a Portfolio, the strategy ID and MagicNB changes, so tracking the live performance vs back test result is also a very daunting task, even with a platform like FXBlue. This also makes the change from Demo to Live very difficult because you cannot compare the two in the different environments, specifically in a Portfolio and a large numbers of individual EA’s are a nightmare to manage.

So with all of the combinations to choose from when generating strategies, the differences in execution between single EA’s & Portfolios, broker data differences, execution challenges, risk of curve fitting with different broker data and anything else mentioned above, how do you actually know if you have a good strategy or not?

As you can see, I have ran numerous tests but I seem to be going in circles trying to find a decent and efficient workflow. I must say, I have learnt a lot through Petko’s courses and the EA studio software and I open to learning more, so any comments or suggestions the these problems or my workflow will be greatly appreciated.

 

 

[Alan Northam]

Hi Daryn,

You are doing good work on your analysis to figure out the best workflow that works for you. I have been through a similar effort in determining my workflow. We all need to determine our own workflows as what works for me may not work for you. So, keep up the good work.

Note: I also use the premium data to generate my strategies. I then use my broker account with as much historical data possible to further evaluate the strategies just like you are doing.

My one recommendation would be to do your workflow on just one account. The problem with using several accounts is they all use different servers so the historical data will be somewhat different between all these servers and can influence your workflow analysis results which can cause confusion. Once you have determined your workflow download the historical data from a different demo account and see how it work with that different account.

Note: I generate strategies for each of my trading accounts by using the historical data from each account.

Alan,

[Daryn Stewart]

Hi Alan

Thanks for your reply and insights. The difference in server data does make sense because the result from 3 demo accounts with the same broker only yielded slight differences in most cases.

I think this part of the process should solve a major portion of my problems but with a lot of focus going into different data sets from different brokers, I do think the platform needs a stronger focus on OOS results to navigate through strategy performance in current market conditions on a specific broker to make sure we pick good strategies based on our broker data.

After generating your strategies with premium data and recalculating with broker data, do you optimise/normalise the strategies on your broker data? Or do you do all the optimisation/robustness testing at the generation stage with premium data and then simply check the performance on broker data afterwards? I still have quite a big question mark on the optimisation part of strategies generation in general.

Is there a way to save symbol specifications from multiple brokers on the premium dataset to ensure strategies are generated with broker settings from the start? From what I can see, the premium data only allows one customisation per currency pair in the symbol settings.

[Alan Northam]

Hi Daryn,

As to the difference with historical data between accounts, I have noticed a large difference in using the same EA on my Oanda demo account and my Oanda live account. So, even with the same brokerage it is possible to see significant differences between accounts. Of course this difference may be smaller or larger with different brokerages. So while you are seeing small differences between different accounts with the same brokerage this is not always going to be the same with different brokerages.

I do not optimize strategies. I do use Monte Carlo robustness.

As to saving symbol specifications from multiple brokers on the premium dataset: After customizing the premium data for a specific brokers specifications download the settings in the Tools>Settings tab for each broker. This will however, also download all the settings you have made in EA Studio. So when you upload the broker settings back into EA Studio be aware it will also change all the settings in EA studio. This won’t be a problem if you are going to use all the same settings except for the custom premium data settings. If you are going to use different strategy settings for each broker then you will also need to download the settings saved as different strategies. I hope this makes sense.

Alan,

[Author]

Posts