2Settimana
Very cool! But I have a few questions:
What are the performance figures of the backtest, including trading costs?
How do you get around the problem of overfitting?
Is there an overarching economic logic to your model beyond the pure backtest results?
What are the performance figures of the backtest, including trading costs?
How do you get around the problem of overfitting?
Is there an overarching economic logic to your model beyond the pure backtest results?
•
33
•
2Settimana
@Epi Overfitting was indeed a big issue in the beginning, especially for parameter variation... I ran the backtests for Bayesian-Optimization over smaller time periods and then wanted to combine the results via Weighted Average --> saw that the algorithm overoptimizes the parameters for the time periods and then other time periods with the same parameters don't work at all... Solution now is actually the optimization over the entire period as training data minus test data whereby the parameters must exist for different bear/bull markets... The parameters found are then varied slightly +- and the effect on drawdown is observed (so that an "optimal spike" is not found and the same deviations do not work at all). I am currently at 35% drawdown +- 4% depending on the variation, which is ok for me, but maybe I will increase the deductions for drawdown to get closer to the 30 range...
For the signal forecast itself, the LSTM is particularly susceptible to overfitting, the current remedy against overfitting training/test data is above all the ensemble agreement of the 3 models, as RF and GB are less susceptible to overfitting... But the ensemble is the next point I would like to look at more closely.
As a technician, I had to google the topic of "higher-level economic logic" for a moment, but I would describe the whole thing as a kind of extended fear-greed --> bull market starts to falter --> switch from TQQQ to SQQQ, bear market starts to falter --> switch from SQQQ to TQQQ, side market or uncertain model confidence = cash. But yes, the model depends a lot on error-free backtests that I had to recalculate manually for a long time (or the respective TQQQ/SQQQ positions)
For the signal forecast itself, the LSTM is particularly susceptible to overfitting, the current remedy against overfitting training/test data is above all the ensemble agreement of the 3 models, as RF and GB are less susceptible to overfitting... But the ensemble is the next point I would like to look at more closely.
As a technician, I had to google the topic of "higher-level economic logic" for a moment, but I would describe the whole thing as a kind of extended fear-greed --> bull market starts to falter --> switch from TQQQ to SQQQ, bear market starts to falter --> switch from SQQQ to TQQQ, side market or uncertain model confidence = cash. But yes, the model depends a lot on error-free backtests that I had to recalculate manually for a long time (or the respective TQQQ/SQQQ positions)
•
33
•
2Settimana
@InternetExplorer What performance p.a. did your model achieve in the backtest and did you also work with dropout or noise to test the robustness of the model?
•
11
•1Settimana
I am currently at 105% pa with TQQQ and SQQQ at 35% dropdown over the entire period (Learn+Test), I have also tested a variant without SQQQ, which was slightly better than the SMA TQQQ + Cash Model, I would have to look up the exact values. Dropout/Noise are currently not included, but is a good input... Would generally like to split test/learn data better, but ea is a bit difficult due to the long bull market the last few years (as the model is less challenged there)... If I do Test-Data from 2020-2025 (where Bull/Bear would be included), Learndata 2011-2020 is relatively short in relation...
•
22
•