How to select final submission for out-of-sample scoring?

[newbee]

Also how many final submission will be allowed for the out-of-sample scoring? Thanks.

[enzo]

Just one, we will release the parameters selection soon!

[fun-robert]

@enzo Any updates on this? How do we select the submission that goes on for OOS testing?

[enzo]

It will be released this week.

[cruncher-jean]

@newbee

Also how

@newbee @fun-robert Out-of-Sample Submission Selection - ADIA Lab Market Prediction Competition

[bigfish]

Please allow the selection of two or three submissions for the final round of the competition.

Given the temporality of the dataset, only one submission/idea may not give the best results.

If each account could pick two or three submissions, their success in the completion would be less based on chance and more based on their ideas.

[selected-ilia]

I totally support the suggestion to submit two solutions for OOS. It is a pretty common practice in many competitions

[resonant-poincare]

Hi @selected-ilia and @bigfish:

Increasing the number of selected submissions for the OOS scoring to two or three

Hey there, bigfish! It seems that you’re interested in assessing an average of predictions from a group of models. Multi-submission would be equivalent to this, BUT with the advantage of more computational cost. As people had to build complex models with the computational cost limitation in mind, it would be unfair to modify this so late into the competition. Please explore different ensemble techniques that are compatible with the computational resources provided.

[selected-ilia]

Multi-submission would be equivalent to assessing an average of predictions
No, it wouldn’t Multi-submission is equivalent to assessing the prediction with the maximum OOS score, which is impossible to create as one submission of ensembles. So it is not a question of computational cost, rather of evaluating not one, but two ideas independently.

[enzo]

@selected-ilia Please answer in the tread itself and not report the answers here.