Walk Forward optimization Course

[cgargiulo70]

Hi,

I just terminated the course. I learned how to remove the EAs that are not in profitable phase and leave just the ones that have a good profit factor.

My question is this:

The EAs that I remove from the Portfolio Expert can be optimized to be recovered or I have necessarily constrained to put in the trash?

If the answer is Yes, could you illustrate me the step aimed to recover these EAs?

Thanks a lot,

Claudio Gargiulo

[Petko Aleksandrov]

Hello Claudio,

I get your point. But I do not do it. Simply because I avoid the optimization totally. The risk to over-optimize the strategy is huge. Does not worth the risk to do it. You better use new strategies or portfolio EAs. That is why I launch new ones every month actually.

The other thing to be done is to keep testing all the strategies in a Demo account, and if you see that any of the strategies is back with a good profit factor, you can enable it in the live account by removing the “//” from the Portfolio Expert.

Kind regards,

[cgargiulo70]

Hi Petko,

 

I’ve seen that the only times you try to recover a strategy through optimization is when you make use of FSB Pro. You never do with EA Studio.

Could you please explain why?

Thanks a lot, best

Claudio

[Petko Aleksandrov]

Hey Claudio,

Actually, I do, many of the EAs that are included in the courses I optimize them carefully with EA Studio. For example, the 10 USDCAD EAs, the 10 EURUSD EAs, the 10 GBPUSD EAs,  the new 10 GOLD EAs that are coming up next week…and maybe I forget some. Right, the basic algo course.

Just with some, I decided to launch new ones ( EA Studio is fast enough so I can do it).

Cheers,

[Simon]

Hi Petko,

 

I have a question about the Walk Forward optimization tool..

 

My understanding is that the Walk Forward tool is for finding out if a strategy performs well on OOS data, and that if it does, then it is likely to be a robust strategy, ready for demo testing (possibly after some Monte Carlo testing as well), am I correct in this?

 

In video 8: “Walk Forward analysis”, from what I can gather, the Walk Forward tool is being used to try to over-optimize a strategy, to see if the original strategy is already over-optimized or not?

If the strategy is better after all the segments of the Walk Forward optimization have been calculated, then the original strategy is not over-optimized, and therefore the preferable one over the “better” optimized strategy?

And vice-versa; If the strategy is worse after all the segments of the Walk Forward optimization have been calculated, then the original strategy is  over-optimized, and therefore the not the preferable one over the “worse” optimized strategy?

Or have I got it really mixed up?…

Thanks,

Simon

[Andi]

Hey Simon,

you got it pretty correct!

If the Walk Forward finishes with a better equity line this gives us the idea that the original strategy is not over-optimized.

And at the end, it does a complete backtest with the final parameters. If the complete backtest is a better strategy, we can use it. We have simply a better strategy and it is not over-optimized one. Because of the optimizer work just for the last segment, not for the whole period. The backtest is on the whole period.

After that walk forward optimization course from Petko, this thing got clearer for me 🙂

[Simon]

Hi Andi,

 

Thanks for your response!

I’m afraid I’m still a bit confused, particularly with how Walk Forward behaves in the “Optimization” section of the Reactor, and the “Robustness Testing” section.

 

Walk Forward optimization

So if a strategy passes the “Walk Forward optimization” stage of the Reactor, it is basically the same strategy that arrived into this section, but “Walk Forward optimization” performed optimization on the strategy, found better parameters, and made a “better” strategy, BUT, the strategy that has passed through “Walk Forward optimization” and has been validated, is actually the ORIGINAL one, which has now been shown to not be over-optimized??

Why would there be any need to then backtest the complete period of data again, if we are keeping the original strategy?

If “Walk Forward optimization” performs optimization on the original strategy , and can only come up with a worse strategy,  would the worse strategy pass validation?, or would both strategies be discarded, being that the original one was over-optimized, and the “worse” one was likely to be too, given that the “Walk Forward optimization” is designed to over-optimize?

Additionally, there is the option to be “In Sample” for “Walk Forward optimization”,  I would think that we would always want to be “Out Of Sample” for this part of the process, am I wrong?

 

Walk Forward validation

I’m not sure how this is different to “Walk Forward optimization”, other than it is actually for optimizing strategies on purpose?

You use the Optimization options in this section to optimize – how do we know we are not over-optimizing? How is this testing the robustness of the strategy?

And regarding the final backtest, is this where “Walk Forward validation” tweaks the parameters as it goes through the segments, and the parameter settings that are the result of the final segment being processed are then tested over the whole period of data again? The results of this final backtest deciding whether the strategy is valid or not?

Is the best use of this section to streamline a strategy, and then hope it passes Monte Carlo?

 

I hope I’ve asked these questions clearly, it’s almost as hard knowing what questions to ask, as it is figuring this out!

 

Thanks,

Simon

 

[Petko Aleksandrov]

Hey Simon,

Andi gave you a pretty decent answer but let me bring a bit of clarification:

The Walk Forward Optimization changes the strategies. The complete backtest with the last parameters is different from the original strategy. Simply because it uses the last parameters. And it will take those last parameters only if they are better than the parameters in the original strategy.

The Walk Forward Validation does not change the strategy. If the strategy passes when in the Reactor, the same strategy will continue, with the same parameters. Here we are looking for over-optimization. If the Walk forward validation fails to create better segments by optimizing each one, it means that the original strategy might be over-optimized. Simply, the Walk Forward can not get a better result.

I hope that makes it clearer. If not, let me know.

Cheers,

[Simon]

Thank you Petko and Andi for helping me on this.

So, simply:

We want “Walk Forward optimization” to change the strategy that arrives into it and make it better by optimizing it, although it may produce an over-optimized strategy.

and

We want “Walk Forward validation” to try to improve the parameters of the strategy that arrives into it, and if it can’t, then we consider the strategy to be over-optimized. If it does find better parameters, they are NOT applied to the strategy, and the strategy that arrived in “Walk Forward validation” is validated as not over-optimized.

Thanks,

Simon

[Petko Aleksandrov]

Hello Simon,

The Walk Forward Optimization does the optimization segment by segment, so it is unlikely to over-optimize the strategies. No like the Optimizer that works on the complete Historical data. That is the huge difference.

It takes the parameters from the last segment and tests the complete period with these parameters. They are optimized just for the recent Historical data(the recent market conditions). If they do well for the whole period, it will take them.

[Simon]

Thanks for the clarification!

[Sharapova Sisi]

Good questions here, Simon. I needed those answers as well 🙂

Thanks, Petko! You are always very helpful!

[Simon]

Hi Petko,

I have just completed the whole course (perhaps I should do that before I ask any questions!), and discovered in video 12 – Building the Portfolio Experts – that MT5 is not supported for Portfolio Experts.

I think I have mentioned in other posts that I have to use MT5 from Pepperstone because that seems to be the only way I can get enough bars of historical data.

I hear that there may be a solution regarding historical data on the way soon, but in the event that this still does not solve my data problems, I may still have to stick with MT5 for data for the time being.

However, I really like the idea of portfolio experts, and would like to become familiar with them. I am wondering if the experts that you kindly provide with the course are compatible with all brokers/servers? Or must we use the broker and server from which you sourced the data for building the experts?

Cheers,

Simon

[james444]

Hey Simon,

Very smart questions here and points. These are things I have been stragleing with as well.

What I know for sure is that the Portfolios are for MT4 exclusively. This is because MT5 with most of the brokers do not support hedging, and trading with the portfolio EAs is hedging indeed.

Personally I am trading the EAs from the walk forward optimization course on my broker (different from what Petko uses) and I do well the last 3 months.

The idea is that we remove the strategies that do not perform well, right? So even those are different strategies from what Petko would be removing, still I will remain with the proftaible strategies from my broker.

Petkos courses are really amazing and I always ask him after I complete the course 🙂

Kind regards,

James

[nedelchev_1992]

Hi Petko,

I have some question about the Walk Forward Validation/Optimization. The case is: I run the reactor with some acceptance criteria and Walk Forward Validation only. After that I check the strategies one by one and run the single Walk Forward process and I see that not all strategies full-fill all 3 criteria (Segments validation, Common Acceptance Criteria and Is strategy better). Why this is the case? Why in the reactor they show successful passed validation?

Second question. When we run the a single Walk Forward test for a particular strategy, should we EDIT the strategy only if all 3 criteria are valid/green?

Third question. If we EDIT a strategy with the Walk Forward Optimization and run it once again with Walk Forward Optimization, it is very likely that the optimization will produce much worse results. Does it mean that our EDITED strategy now is over-optimized?

Thanks a lot

Nedko

[Petko Aleksandrov]

Hey Nedko,

Glad to hear from you.

Your questions are very logical. But before I answer them, I would ask you to watch that video:

After that, if you still can not understand it, ask me again, and I will answer. It is not that I don’t want to write the answers, but I believe that by watching the video, you will understand even better how the Walk Forward works. Because it is visual, this is why I do inline courses, which is the easiest way to learn it all.

Please, watch it, and let me know if something left unclear.

[nedelchev_1992]

Hi Petko,

I watched the video very carefully and I want to double check if I understood the Walk Forward process correctly.

1. In the reactor, a strategy to pass Walk Forward Validation is needed only to pass the segments validation.
2. In the reactor, a strategy to pass Walk Forward Optimization is needed to pass all 3 criteria (Segments validation, Common Acceptance Criteria and Is strategy better).
3. If we choose to EDIT the strategy after successful Walk Forward Optimization (all 3 criteria are green), we expect to have a new not over-optimized strategy, because the EDITED parameters from the Optimization are based on 70% in-of-sample optimization based on the last segment and are tested on 30% out-of-sample. Additionally the Common Acceptance Criteria in-and-out-of-sample are applied on the whole back-test, which additionally speaks for not over-optimized strategy.

Please correct me if I am wrong. I am looking forward to see your comments.

Best,

Nedko

[Petko Aleksandrov]

Hello Nedko,

Glad to hear from you again.

Yes, I think you got it all.

Just keep in mind that the Walk Forward optimization changes the strategy, and the Walk Forward validation doesn’t change it. It just validates it.

Also, when you have run one time the Walk Forward Optimization in the reactor, you are seeing in the collection the strategies that passed it. And if you run the test after that it will fail, because it does it over the new strategies(that already passed). That was a question in your previous post I believe.

Cheers,

[Bart Meijrink]

That is the next course I am looking at 🙂 I don’t understand the walk forward and I believe that this course will help me understand it.

 

[Author]

Posts