DATA COLLECTION; Updates to EAS/FSB upgrades

[Anonymous]

Good day Petko. I’m sorry to add more questions even though the previous ones still have not been answered, but here are some further questions I wanted to ask you.

1. DATA COLLECTION

a. May I ask for the export to csv script?

b. Discrepancy of data downloaded from demo and live account coming from the same broker and account type.

I noticed that when I downloaded the data from my demo account both the m1 and m5 timeframes have 65,000 bars. However, for my live account, also for the m1 and m5 timeframes, it was only roughly 43,000 and 64,000 bars respectively. What might have caused this to happen?

c. You mentioned that for generating strategies using the m1 timeframe, 300,000 bars is the ideal number of bars that need to be used. However, when I exported the data from my broker it was only 65,000 bars (of which you said is the maximum that the EA Studio Data Export script could do).

Given the fact that 65,000 bars does not meet the 300,000 bars criteria for generating strategies on the m1 timeframe, how do I remedy this problem? I’m guessing the logical answer is to manually add the missing historical data into the most recent ones that were collected, which presupposes that I have previously collected price data in the past already (of which is something that I still have not done as of the moment). Can you help me out on this?

Also, related to my question regarding the manual addition of the most recent data with THE previous ones, I noticed in your first video that the file that you used by which you added the above-mentioned data was in CSV format. So this begs me to ask… How can you use this document (having CSV format) by which you added the recent and previous data together and use it with EA Studio since the format that is required is not CSV, but instead, JSON? Did you convert it back to json before uploading to EA studio?

2. Updates to EAS/FSB upgrades

a. Can you update the course with the most recent upgrades of the EAS/FSB, such as knowing how to use the Walk Forward and OOS Monitor for EAS?

b. Acceptance criteria under the Generator settings in EAS

In the acceptance criteria…

i. There is no longer the option to use the “maximum ambiguous bars.” What is the alternative to this?

ii. There are already three parts: complete backtest, in sample, and out of sample. Can you update us on how to use this? Before, the acceptance criteria was more general, instead of being divided into three.

I’m sorry for taking some of your time. Looking forward to hear from you.

Best,

Jay-R

[Anonymous]

Hey Jay-R,

You are very welcome to ask any questions. I just do not have the physical time to answer all in time but I do my best to answer within 12 hours.

The 65k is limit from the MetaTrader when you first install the platform. You can remove it. Go to tools – options – charts. Enter 999999999999 to the two boxes available.

After that press the home key again.

When you remove this limitation you will be able to collect more bars. Until you collect it you can use more the M15 and H1 time frames.

I collect data from FSB Pro using the csv files. But with EA Studio I just export the data every time from MetaTrader where it actually saves.

By the way, you can save the data from your Meta Trader as well. You can go to File – Open data Folder – history and select your broker’s server. Copy those hst files. You can do this every one month or two, to make sure you will not lose it.

After that, you can use the scripts to export the data. This way you will be keeping the Historical data in Meta Trader.

Yes, I am planning to update the course. The videos are actually published in the EA Studio Updates topic here in the Forum.

The alternative of max ambiguous bars is the backtest quality in %.

Yes, the acceptance criteria are divided which improved a lot the software. If you use the OOS function, you can filter with acceptance criteria the In Sample part and the Out of Sample parts. This way you will have strategies that are just profitable at the end :)

Basically you can eliminate the strategies that are losing in the Out of Sample. It is a great method by the way. You can learn more about it in this video:

 

[Anonymous]

Thanks @Petko. And thanks also for referring me to this forum. I’m sorry for not replying sooner, it took me a while to finish reading the comments to avoid asking questions that have already been asked before. I do have some additional questions based on what you’ve answered regarding data collection, and other matters related to it.

1. Regarding the data collection, wherein you mentioned…

“By the way, you can save the data from your Meta Trader as well. You can go to File – Open data Folder – history and select your broker’s server. Copy those hst files. You can do this every one month or two, to make sure you will not lose it.”

…how do I actually make use of those hst files that I will be saving? Do I need to convert the hst file to either json or csv for them to be usable?

2. As mentioned in the comments in the forum, one of the best ways to collect data is to simply open the MT4. Suppose I want to collect m15, m30, h1, and h4 timeframes for EURUSD, does this mean that I have to open all charts having those timeframes in EURUSD for me to gather the data? Or would it be sufficient if I’ll just open any one of those timeframes for EURUSD, say, the h4, and still be able to gather the data for the other timeframes?

3. I have noticed that gathering data from my live account, which has been existing for a few months now, is very slow compared to my demo account? Is this normal?

When I say slow I mean it took me more than an hour to gather m1, m5, m15, m30, h1, h4, d, w and m timeframes from XAUUSD. Since it was so slow, I decided to ask you I just thought maybe I’m doing something wrong.

Similar experience with EURUSD, USDJPY, and USDCAD.

[Anonymous]

Hello Jay – R,

I have asked Petko for all of that in the beginning in the 1:1 class so I will afford to answer:

1. When you save the .hst files you make sure you are not losing the data. It is a way to collect it. For example, if you change your computer, laptop, VPS and you install the MT4 again, you will lose the Historical data you collected till the moment. If you have the .hst files you can place them in the path described from Petko, and the Historical data will show in the new platform that you install.

2. You need to open one chart for EURUSD, and switch from M1 to MN, and on every timeframe, press the Home key. After that leave it on M1 (because there the ticks are coming). Before I export data, I switch quickly the time frames to make sure the new data is added.

3. I do not understand what exactly is slow? When you press the Home Key it loads slow, or when you place the script?

Hope this answers your questions without the 3rd one.

Kind regards,

[Anonymous]

Glad to hear from you, Haliffa.

1. It’s clearer now =)

2. I see. And after switching the timeframes before exporting the data, it is no longer needed to press the home key again for every single timeframe?

3. I meant when pressing the Home Key, it was so slow. Is this normal when loading data from the live account? It was very fast with my demo.

Those were very good answers. Thanks Haliffa.

Best,

Jay-R

[Anonymous]

Hello Jay-R,

Haliffa gave you great answers! 100% correct.

I have noticed with the time that it is possible to load slowly the chart for two reasons. One is if your computer is very very slow (I do not believe yours is since on Demo it loads fast) and second is from the server of the broker. Could be that they load the data slowly, the server is far from you…anything like that, and in most cases, we can not understand that.

Feel free to ask questions.

It is important to get started properly with the Historical data. That is the first important steps.

Kind regards,

[Anonymous]

Hi Petko… I’ve tried downloading data from my live account through VPS and it’s faster now. Maybe the slowness might indeed be related to what you mentioned regarding the server location. Thanks for pointing that out.

[Anonymous]

Glad to hear that Jay-R. Sometimes we really do not know what might be the reason and looking for an alternative might help.

 

[Author]

Posts