Training-boundary semantics for forward targets
The released target is forward-looking, while the platform passes a complete training prefix to Predictor.train. Are all target rows supplied to train considered fully realized and safe to use, including the final rows immediately before the prediction suffix? Or should a predictor internally embargo the target horizon before fitting? I will never read target in predict or use any future operation; I am asking only which supplied training rows are permitted to influence the first held-out predictions. Thanks.
3 Replies
Nonius21d agoHi radiant-allomancer,
when you submit your predictor we retrain it ourselves and carefully have an embargo that ensures that it's not leaking into the hold out period (which you can't see). Does that answer your question?
Best
NONIUS
thanks, that covers the holdout side. the bit i'm still unsure about is our own prefix.
in the released kit the target is fully populated to the very last row - 031 has 20/20 non-nan on all of the final 120 rows. so a 120h forward target on those rows was computed from data past the end of the file.
when you hand a predictor its training prefix at retrain time, are those last 120 rows already dropped on your side, or are they in there and it's on us to purge them? we purge 120h before fitting right now. if you already embargo, that's ~2400 of the most recent rows we're throwing away for nothing. if you don't, and we stopped purging, that's us training on the holdout.
either answer is fine, just don't want to guess on this one.
answered my own question, so don't spend time on it.
measured the released target directly - corr(target[t], forward return over t+1..t+h) peaks at h=120 (+0.62) and backward is flat (~0.00 contemporaneous, +0.02 trailing 120h). and there are no nan cells, it's populated to the last row. so it's a 120h forward target with no embargo baked into the file, and the purge is doing real work. keeping it.
for anyone else reading: if you fit on the tail of the prefix without purging, your newest labels resolve inside the scored window.