Current score is from how many time periods

The current LB score is from how many time periods. When we submit it to the cloud, we can see it starts from moon=268 up to moon=299 (32 time periods).

1 moon != 1 time period, I am asking because It’s confusing as it’s mentioned in the problem description that there will be 12 weeks to be evaluated in the Submission phase.

Does this mean that the Out-of-Sample phase ( 12 weeks ) will also include 32 new time periods (from moon=300 to moon=331)?

Please clarify if it’s correct understanding

Hi @assistant-tarique

Sorry for late response…
I think there is a confusion.

Submission phase

The submission phase is from May 16th, 2023, 05.00 PM CET to August 16th, 2023 04:59 PM CET which is 12 weeks.

In the submission phase you are scored on the date 269 to the date 299. The mean of your scores is not taking into account the first five dates though as they are available for you to test through the cli crunch test

Out-Of-Sample (OOS) phase

The OOS phase is from August 16th, 2023 04:59 PM CET to November 16th, 2023 11:59 PM CET which is also 12 weeks.

In the OOS phase you will be scored on the date 301 to the date 335; So 35 date in total.
Each week of the OOS phase the leaderboard will be updated with 3 new date (2 for the first week). Your mean score each week will be an average of your computed score (eg. date 301 to date 305 included on the second week leaderboard).

I hope this answers your questions.

If it’s not clear enough don’t hesitate to ask, I’ll reply faster this time :wink:

1 Like

@enzo @xgilbert

Sorry, me again. I apologize if I am being slow to understand :slight_smile:

However, there seems to be conflicting information across the posts about whether there are 35, or 36 ids in OOS.

Namely moon 300 is causing confusion. Which week is it part of…?

From the above post by @xgilbert it is not mentioned.

However, in the following post: Save X_test at inference time for use in train - #7 by enzo

moon 300 is mentioned.

Would be great to have a clarification of which moons are in the first 2 weeks:

case 1:
week 1 = 300
week 2 = 301, 302

case 2:
week 1 = 300, 301, 302

case 3 (no moon 300):
week 1 = 301, 302, 303

case 4 (no moon 300):
week 1 = 301, 302
week 2 = 303, 304, 305

Many thanks!

Its case 2:

But because of some issues, we have to include moon 300, but the score will be taken into account.

week train moons infer moon taken into account
1 0-298 300
1 0-299 301
1 0-300 302
2 0-301 303
2 0-302 304
2 0-303 305
3 0-304 306
3 0-305 307
3 0-306 308
12 0-331 333
12 0-332 334
12 0-333 335

ok, now I see why sometimes it was mentioned, and sometimes not.

I think you meant to write “will not be taken into account” :slight_smile:

thank you

Yeah sorry for the confusion :slight_smile: