AlphaNova
2

OVERFITTING_FAILED

Md_Khairul_Islam's avatarMd_Khairul_Islam
1mo agocompetition-5

Hi, I am repeatedly receiving the OVERFITTING_FAILED flag on submission, despite my model strictly following the architectural principles outlined in demo_engineered.py and COMPETITION.md.

My current implementation uses only bounded cross-sectional ranks (csrank) and rolling means/stds as features, a standard regression (MSE) objective with light regularization (min_data_in_leaf=50, max_depth=4, 100 rounds), raw target values with no clipping or normalization, and strict de-meaning only in predict() with no FWL neutralization, Winsorization, or output ranking. All logic is contained within the Predictor subclass.

This architecture mirrors the EngineeredXGBoostPredictor in demo.ipynb (which passes the N=100 permutation test) but substitutes XGBoost for LightGBM with equivalent regularization. My local validation Sharpe is stable (~0.04–0.05) and IC is ~0.019, yet every submission triggers OVERFITTING_FAILED.

I would appreciate any directional guidance on whether this is more likely caused by LightGBM’s min_data_in_leaf behaving differently than XGBoost’s min_child_weight under label shuffling, cross-feature rank spreads being flagged as leakage, or another aspect of the proprietary test I may be misunderstanding.

Thank you for your time and for organizing this competition.

4 Replies

1
Nonius's avatarNonius1mo ago

Hi Md_Khairul_Islam,

We’ll take a look and revert .

Best

Marc

1
Md_Khairul_Islam's avatarMd_Khairul_Islam1mo ago

Thank you so much. Please let me know.

0
Nonius's avatarNonius1mo ago

Hi Md_Khairul_Islam,

We did the analysis and will send you an email on our findings.

Best

Marc

0
Md_Khairul_Islam's avatarMd_Khairul_Islam1mo ago

Thank you so much.

Sign in to reply.