status on signals as of about 1 hour ago

Hi everyone,

so as you know when you submit a signal, there is a pipeline of tests we do on the backend. Here are some stats on the status counts so far.

Outcome	Your term	Count	% of all
OK	passed ✅	90	52.9%
OVERFITTING_FAILED	overfit	42	24.7%
CANT_RUN	failed to run	24	14.1%
CODE_OUTSIDE_CLASS	code outside class	8	4.7%
NOT_DEMEANED	(not de-meaned)	4	2.4%
PEEK_THE_FUTURE	peeking into future	2	1.2%

Not much we can do about things that are flagged as overfit, or writing code outside of class, not de-meaned or data leakage, but on "FAILED TO RUN", that's a catch all that groups together various outcomes, one of which is it's too slow during our overfitting test. So one thing we've done is increase the "time out" on the overfitting test run. so, if you've a submission that was flagged as "FAILED TO RUN" and locally you were able to run it with our runner script, feel free to rename the submissioin and re-submit. and if it still doesn't pass feel free to reach out to me.