OVERFITTING_FAILED
Hi,
I am repeatedly receiving the OVERFITTING_FAILED flag on submission, despite my model strictly following the architectural principles outlined in demo_engineered.py and COMPETITION.md.
My current implementation uses only bounded cross-sectional ranks (csrank) and rolling means/stds as features, a standard regression (MSE) objective with light regularization (min_data_in_leaf=50, max_depth=4, 100 rounds), raw target values with no clipping or normalization, and strict de-meaning only in predict() with no FWL neutralization, Winsorization, or output ranking. All logic is contained within the Predictor subclass.
This architecture mirrors the EngineeredXGBoostPredictor in demo.ipynb (which passes the N=100 permutation test) but substitutes XGBoost for LightGBM with equivalent regularization. My local validation Sharpe is stable (~0.04–0.05) and IC is ~0.019, yet every submission triggers OVERFITTING_FAILED.
I would appreciate any directional guidance on whether this is more likely caused by LightGBM’s min_data_in_leaf behaving differently than XGBoost’s min_child_weight under label shuffling, cross-feature rank spreads being flagged as leakage, or another aspect of the proprietary test I may be misunderstanding.
Thank you for your time and for organizing this competition.
0 Replies
No replies yet. Be the first to reply!
Sign in to reply.