- This topic has 59 replies, 9 voices, and was last updated 5 months, 3 weeks ago by
.
-
Posts
-
It sounds like you are doing well to me btw, like you say theres perhaps just a few subtleties missing I think.
Plenty to chew threw from last post but small additional point is check what your acceptance is in the Monte Carlo, I am taking a complete guess that you are running indicators and backtesting date tests and that your acceptance criteria for the Monte Carlo is set to profit factor > 1??
If this is the case for example, I expect you might get a similar amount of strategies passing if you reduce the 90% to 80% and increase the profit factor to 1.1 or even 1.2 which may be worth a try, just a side suggestion/food for thought comment though 🙂
Also my approach is generally similar and I have mixed results so I don’t claim to have everything figured out at all (but the harder I work the more things improve). I have been at it a while though and certainly tried and pondered lots for sure though so hopefully some of what I have said helps give you some ideas to move forward 🙂
One last point also, is that we are discussing obviously just one pair and timeframe and it could just be a bad month for that pair, so definitely a good idea to do some backtesting tests on the spare months day for say three well balanced pairs and them sum the results of the combined portfolio. Say EURGBP, EURJPY and GBPJPY for example.
ONE last thing Matthew (Things keep popping into my head haha). Have a good read of some of the other forum threads even if they are a bit old, there’s lots of great information especially coming from Petko (It’s all here it just takes a bit of time and practice to assimilate)
‘what acceptance criteria do you use’ is a really good one for example.
Ah hold on it IS a big deal that you are generating IS, definitely change that to be 30% OOS. You have changed the strategies by normalizing them but it definitely still takes away from the value of the OOS check considerably.
Thanks Again for your reply, Samuel.
I’m very interested in Petko’s response as well.
1. I have chosen to generate using the complete sample of 1.2 because I want good quality strategies to start with. The 30% OOS normalization allows me to optimize the strategy.
2. the 1.2 in sample filter and out of sample filter ensures that I have only strategies that are performing similarly during the optimization period and after the optimization period. I have chosen to do this to prevent over-optimized strategies and to hopefully get consistent future profits. (no success yet sadly. However, I’m not giving up.)
I have tried this method of holding back a month of data and filtering PF>1.2 Minimum count of trades >10. Then tested those bots and found that when they are all combined that they were not profitable as well.
I’d really like to try your suggested method to forward test strategies for a week and then pick only those for the remainder of the month of data.
I had thought of this but was limited due to my strategies are all generated based on the H1 time frame. I’m having an issue doing this as the Minimum data bars are floored at 300 under the data horizon tab. 300 / 24 = 12.5 trading days. NASDAQ only trades on Monday through Friday, so I have to use 2 weeks and 3 days in a forward test. BTCUSD is always open, I only have to wait 13 calendar days. In both cases that is longer than the 1 week.
Is there a way that I don’t know about to lower the floor in data horizon on minimum data bars to 120(5 days x 24 bars) for NASDAQ and 168(7 days x 24 bars) for BTCUSD?
Yes My monte Carlo boxes checked are “Randomize indicator parameters” and “Randomize backtest starting bar” Validition is Minimum profit is >= 0 or in other words profit factor >= 1.00 and 90% is considered acceptable.
I’ll try the minimum profit factor 1.1 at the 80% method and see what that does for me.
Thanks for the suggestion.
Thanks for the suggestion to diversify the risk! I started testing forex pairs at the beginning of my use with ea studio and had horrible results. Maybe I should go back and try again now that I’ve been using ea studio more.
Okay, I will test this method of 30% OOS for the generator as well!
Another thought I had is sharing my normalization parameters and strategy properties settings to see if you or Petko or any other experienced user might have some suggestions on if I’m doing something wrong.
I’m using strategy properties of always using SL and TP on NASDAQ and BTCUSD.
My NASDAQ settings are at 10,000 min and 50,000 max for both SL and TP.
BTCUSD I’ve tested 300,000 min to 600,000 max for both SL and TP (here I find it easy to find strategies, however, the system gets high into profit and then doesn’t close the trades once they get into profit because the TP is so far away and the market just reverses)
BTCUSD I’d like to test 150,000 min to 300,000 max for both SL and TP. The problem here is I find it difficult to find strategies. The 14-period average true range of the D1 chart is around 250,000 ish right now. So that means that on an average day I’ll see about 250,000 pips of movement in a single direction and the hope is the strategy catches somewhere at the beginning of that 250,000 and continues for the remainder of the day and into the next day to close at the TP. So trades would be open for usually a max of 2 days unless it’s a fluctuating market.
Optimization settings:
Remove Take Profit is unchecked everything else is checked.
Out of Sample 30% OOS.
Numeric Value steps 20 steps for NASDAQ and 40 steps for BTCUSD
Search best Net Balance
Thanks again for any suggestions you might have for me to try.
Hi Matthew,
The optimization settings seem fine to me, of course you can try experimenting with changing search search best Net balance to perhaps R/D or Maximum drawdown for example to help for the rules of FTMO challenge but for now if stick with what you have rather than changing too much at once.
Ah yeah, I have the same issue so yeah can’t check 1 week but can do 2 weeks as I mostly favour H1 timeframe also currently.
Additional detail with this is that EA Studio won’t make any trades in the first 100 bars of data, so if you want to check 2 weeks which is 240 bars for H1 then set the bars to 340 bars, you can check this is the journal and also will see the setting in the tools page. So if you put 300 bars in for H1 you are actually checking about 12 days.
One other suggestion is dont change everything at once but rather make a change and try to get a feel with some tests of how that single change has affected the results. Obviously more time consuming to start but in the long run I think you will win faster.
Hopefully your results start improving soon, let us know 🙂
Yeah my original thought re generating for complete but then normalizing for 30% was as you say, optimization has changed the strategy and then you check it’s still okay OOS.
But if you want more bars for generation then I think just set the bars higher, but keeping the OOS completely reserved for a final check so that it hasn’t been used at any point in the strategy development is the way to get the most out of this robustness check.
I’d like to hear how many months of data Petko recommends. I’ve always heard having more time for more trades is more robust, however, having too much of a time period reduces potential profits. That’s why I stuck with 13 months and 300 trades.
My new game plan for testing is going 15 months backwards to 2 months backwards.
300 count trades.
PF >= 1.2 in sample and out of sample.
normalize 30% OOS (all boxes checked except for remove take profit.)
Monte Carlo 80% 1.1 profit factor last 2 boxes checked.
I’ll generate a pool of about 1000 strategies to run through the validator from 2 months to 1 month ago.
All the strategies that had PF >=1.2 will be downloaded into a new collection. (This will be my last step before a mock demo)
Then I’ll run the new collection through the validator from 1 month ago to half a month ago.
all the strategies that had PF >=1.2 will be downloaded into a new collection.
At this point, if it was real-time I’d consider the strategy acceptable to go on a demo account (it’s going to be a “live” demo account.)
I’ll run it through the validator one last time to see if it was profitable from half a month ago to today. (This would be the point if it was a real account that I’d repeat the process and swap out the bots for new ones)
I’d like to find a way to make consistent profits every month with this and I’m gonna get there!
All please let me know if you have any suggestions for me and I’ll let you know how this testing goes.
That sounds like a pretty good plan forward Matthew. I’d say increase the number of bars to around 35000 for the H1 timeframe though which is about 5 years. Maybe try the same for the M30 or M15 timeframe also for 4-5 years.
15 months is too short for H1 period.
Also do this using several pairs as discussed. This is also going to be important in limiting your account drawdown.
Keeping track of everything in a spreadsheet is a really good idea also, not just a quick sheet but something that you will maintain for months.
“I’d like to find a way to make consistent profits every month with this and I’m gonna get there!” With this attitude you WILL get there!! Keep pushing, every time you try something and if it doesn’t work you will learn something and develop deeper insight and this will lead to success over time. Fall 7 times get up 8 as they say – I’ve fallen a hell of a lot more times than 7 but I never stop getting up;-).
Enjoy the challenge 🙂 Looking forward to hearing how you get on.
Hey Matthew,
Your plan looks really promising. Please, keep us updated.
Regarding the data, it depends on a few things. First, if you use a too big number of bars, the program will get slower. At the same time, if you use a small range of data, you will get to over-optimized strategies. But if you use a min count of trades that will eliminate the issue.
Something in the middle works best.
OK…NO I have a question. Should I do the analyze every day? Or Every second day? Probably the weekly is not that what is needed to be successful.
I did it yesterday evening. From 99 EA’s I have already nic data. Monthly performance I was able to choose 10 EA’s with over 10 trades with over 100 EUR net profit and profit factor over 2.00 which is really good. For weekly performace there were EA’s with over 3 trades Profit factor over 1.4 and net profitover 50EUR….well as top 5 there were 4 EA’s as EUR/USD which I lowered to 2 best of them that makes top 3 of 99 EA’s The top 3 right now #87543211 / #87513216 / #86879152
From Ilans EA’sthe conditions are little bit lower because there a lot of H1 EA’sBut still monthly I have over 100EUR net profit over 2.00 Profit factor and over 5 trades.
Weekle at least 3 Trades and PF over 1.4 but here I will take in consideration all EA’s with any profit. Finally I took out 3 of them #48095358 / #51327790 / #48162246
From your Gold EA’s there was no EA with over 1.4 Profit factor so I could not take them considering the fact that also with 0.01 lot thery have still huge SL level which could lower my account down by over 30% per trade if the SL were hit so this will have to wait for far mo data as I have this GOLD demo running for only 2 weeks
FInally I woke up this morning and I found on SHADOW demo closed profit with 5EUR …As I set this accout to my starting 500EUR account this made over night 1%……WHY?
Why on demo is all working great and makes profit…I had this same system on live account and made over 35% losses……
Currently I am learning far mor strategies also for manual trading….also scalping……I set another demo account for this…..so now I am running 7 demo accounts in one computer.
One is 99 EA’s from you….another is weekle EA’s from Ilan…third one is GOLD EA’s from you. Another is my Shadow with top EA’s another is for Manual swing trading and scalping Another 2 I have for my own EA’s created manually from my own strategies in ETASOFT FOREX GENERATOR……
As you can see I really want to make trading my job because I see here a far more financial possibilities and freedom as at my Work where I am too busy to make this in that scale I would like to. Tharefore I am heading to automatic “ROBOT” trading which can trade in time when I am asleep or at work. There are days when I am simply too tired from work to make any trading at evening. So hopefully I will be able to manage this somehow so that I could finally do trading as full time business.
So guys if you read my notes an management don’t hesitate to tell me If I am doing something wrong and what I could make better. You are free to tell me everything because critical notes are those which make us better.
Thank you
Hey Jaroslav,
Great to hear form you 🙂
It definitely sounds like things are improving a lot which is great!
Your first question is around the timings. I personally think the more the better if you have time to update the EAs so if you can make it a quick and easy daily task to do then why not. Its important to tweak your system around your current schedule and make it manageable. BUT how often is necessary is dependent on the timeframe and frequency of how often the EAs trade. For example if your EA’s are H1 and M30 and typically opening trades 1-2x a week then changing a couple of times a week is likely to work well, but if you have M15 and M5 that are trading say 3 or more times a week typically then I would probably favor updating every day or two.
What broker are you using? Demo and live should be very similar so to check this for yourself why don’t you try the following:
1 – Upload the live data and demo data to your EA studio account and then see how the results compare when you switch from using the live to demo data. Also you can check with Mt4 back tester.
2 – If you want to convince yourself of how live and demo compare then why not pick a few single EA’s each for a different pair and trade them each with the smallest lot size on both the demo and live account to compare the results. Of course this could cost you a small amount if they lose but it really shouldn’t be a costly lesson (You could even cheapen the lesson by reducing the stop loss and profit taking from typical max=100 to 50 when generating EA)
Definitely a smart move to run more than one demo at once, just make sure to track everything properly so you learn from all the things you are trying but will hugely speed up the learning.
Keep doing what you are doing. I really think you are going in the right direction, just keep doing what you are doing and learning from the results.
I can understand wanting to get onto live as soon as possible but once you have sufficiently consistent demo results then this will translate to good live results. Whenever you go live, to start you could also put the same EA’s on a demo initially to further build your confidence in this “live ~= demo” equation.
You must be logged in to reply to this topic. Login here