Which model will be running Out-of-Sample (OOS)

Competition ADIA Lab
1

Hey ADIA team ,

I just managed to run my first submission in Cloud and obtain a score in leaderboard.

I know my model will be running Out-of-Sample in later stage.
My question is:
If I submitted multiple models , which model will be selected for Out-of-Sample phase ?
Is it the model with highest score ? Or the model with most recent submission date ? Or I need to manually select one model for Out-of-Sample phase.

Thanks
YX

2

Hi @notorious-yuxin,

It’s ADIA Lab team :slight_smile: behind this dataset.

→ The model with the most recent submission date

AM

1 Like
3

what is meant by Out-of-Sample here?

4

Competition phases and format

This competition is focused on forecasting and has two phases. The first is the submission phase where participants can submit and test their models. The second phase, which is automatic, involves running the models against unobserved live market data.

Submission phase - 12 weeks:

In the first phase, participants are required to submit either a Python notebook (.ipynb) or Python script (.py) file. This file should contain the necessary code to build, load, or update their models trained on the data. The code will be executed by the CrunchDAO platform for every submission, to obtain predictions on unseen data. Participants can either use static models, trained only once on the initial training set, or dynamic models that update or retrain themselves on the unseen data, as explained further in the documentation.

Out-of-Sample phase - 12 weeks:

In the second phase, also called Out-of-Sample (OOS), the participant’s code will be automatically run by the platform on live market data and evaluated. In this phase, the participants won’t be able to modify their code.