OOS submission dudes

Competition ADIA Lab
1

Hi!

I have two question regarding the OOS period:

  1. How many times a week will the code be executed?

  2. What is the maximun quota of hours we have per week in the OOS period?

Thanks in advance

2

Once a week, with 3 date each time.

16h per weeks.
Giving your script an average of ~5 hours per date.

2 Likes
3

Hi @enzo

What you just said contradict what is written online:

Out-of-Sample Phase

  • Your code needs to cover 1-time cross-section in under 16 hours.

  • CPU or GPU ≤ 16 hours run-time/date

  • RAM ≤ 120Gb

Would be good to clarify which is the correct statement.

The online information is unclear… that would mean that one cross-section is equal to 3 time ids?

4

Thing changed and I forgot to update the documentation sorry.
I will do it now.

The correct is:

Out-of-Sample Phase

  • Your code needs to cover 1-time cross-section in under 16 hours.
  • CPU or GPU ≤ 16 hours of run-time per week
  • RAM ≤ 120Gb
  • 3 dates per week
1 Like
5

Okay thanks,

Still… To be sure:

  • 1-time cross section is equal to 1 time id in the dataset (or 3)?
  • The maximum amount of times we can call the train function is once per time id (i.e, 3 times a week) or only once per week (i.e, once every three time ids)?
6

1
@resonant-poincare could you confirm?

It will depends on what train frequency you set.
If you used 1, you will be trained on every date.
If you select 5, you might have a Run without any train.

The train frequency will be preserved between Runs (here is an example for a train frequency of 2):

  • Run 1:
    • moon 301, train=false
    • moon 302, train=true
    • moon 303, train=false
  • Run 2:
    • moon 304, train=true
    • moon 305, train=false
    • moon 306, train=true
  • Run 2:
    • moon 307, train=false
    • moon 308, train=true
    • moon 309, train=false
1 Like
7

Okay, so that makes sense for the train frequency.

So since the max CPU time is 16h per week, if I want to train on all dates, within a week/run, then I need on average less than 4 hours of compute per time_id/moon.

In that sense, the following point is not as important:

" Your code needs to cover 1-time cross-section in under 16 hours."

Thanks for the quick clarifications

8

@enzo Thanks!

9

Hey @salty-francisco

I made a mistake and I updated the documentation about the out-of-sample runtime quota:

Its not 16h per 3 dates, but 23h for the whole 36 dates now.
Very sorry about that.

1 Like
10

Thanks for notifying @enzo ! Still it’s long enough :slightly_smiling_face:

11

Is it possible to reset the daily submission count given the change to the time limits and deadline?

12

Done :slight_smile:

1 Like
13

Hi apologies this might be a dumb ques, but where can we see the OOS performance for our model? When I go to the ‘My Submission’ tab I see nothing under the ‘Out-of-Sample Runs’ table

14

Nothing has been published yet, that why.