Different results from broker data & premium data

[Tanya Jay]

Hello everyone!

Has anyone gotten different results from broker data and premium data?

I generated EAs using H1 broker data in the past 5 years (because I can get 5 years of data of H1, but only 1.5 years of M15). And then, I used Validator to run these EAs with the same exact acceptance criteria using premium data with my broker settings (add $7 commission fees, change timezone GMT+3).

Out of 204 collections:

Premium data = 89 (43.63 %) Passed validation
Broker data = 201 (98.53 %) Passed validation

Side note:
– On data horizon, I used data start date 05/11/2017 and end date 05/11/2022
– Acceptance criteria: 20% OOS, PF 1.2, min trade 300
– I tested it without Monte Carlo
– My broker is IC markers

Why so much difference between premium data and my broker data? Anyone here experienced this?

Cheers,
Tanya

[Samuel Jackson]

Hi Tanya,

As you had sent me you collection I was able to check this quickly for you. First of all I changed the dates to 11/05/2017 and 11/05/2022 (since 05/11/2022 is a few months into the future so I assumed a typo?).

I have run it using IC Broker data and Premium data including Monte Carlo and there are 121 strategies passing for premium data validation and 176 for IC broker data.

This is totally fine, it is likely that many only marginally passed the acceptance criteria for IC and only marginally failed for IC data. Also plenty could have been overfit to the data used to generate (IC?) and therefor aren’t performing as well on different broker data which is a good robustness test and best to lose those strategies anyway.

You can also simply load the full 204 collection into EA studio and open up each strategy individually and see how the chart looks when you change between the data sources and remove the ones that differ too much, should give similar results. Probably about an hours work for 204 strategies but a good exercise.

All is well, these results seem very reasonable to me. The broker data wont be hugely different but it can be different enough to get these results.

[Author]

Posts