Results of Timed Out Runs

Dear Enzo,

Is it possible to see the results of timed out runs? It would speed up the research process as I wait for additional run time.

Best wishes,

Emil

Hi Emil,

I can’t find a timeout in your recent runs?
But I suggest you try to:

  • get rid (or silence) the warnings.
  • use tqdm on an outer loop instead of an inner loop, see this example.

I gave some extra quota just in case.

1 Like

@enzo Hi, is it possible to get some extra quota ? I’m in timeout for a few submissions and even 15h quota returns a timeout. It should run based on my local estimation. Could you add me a few hours (just for this week)?

Another question: According to the sample notebook, we have to predict one row by one row (due to yield instruction) because dataset we iterate is just a single time series each time, right? So parallelization on time series batch is not possible. Would it be possible that you increase dataset batch size to more than 1? In fact my code, with parallelization, locally runs in 15-20 minutes for 10000 rows, but on inference it times out after 15h. Previous models that runs 5-10 minutes for 10000 rows (locally) takes 4 to 6 hours on submission.

Thanks a lot.

I increased your quota, but unfortunately we cannot allow more than one by one.

This approach is volontary to avoid potential usage of next dataset to predict the previous ones.

It is also a cruncher’s job to optimize his code to fit within the quota.

Understood.

Thanks it worked.

Hi Enzo,

I have made the same mistake in my previous run (tqdm inside the inner loop), it will probably run out of time. Can I please get some extra run time just for this week?

Best wishes,

Emil

Hi Emil,

You already have more quota, and a new week has already started, so your quota already reset!

1 Like