Runs not starting and Bad endpoint - ADIA Lab Real Time

Competition ADIA Lab
1

Hi There,

None of my runs today get past the “Created Task” step or begin running, they seem to get stuck as “Pending”. Is the server down?
Additionally on Monday I had a run successfully complete, but I get a “Bad Prediction” result from your endpoint. I ran the exact code a second time and it scored fine. Do you know what the issue was there?

Thanks,
Dylan

2

Hello Dylan,

Unfortunately, the situation is out of our direct control. It seems that GCP is experiencing an increase in demand for GPUs.

Your run has been put in a queue and will eventually run, but for now, you can only wait.
Of course, you will only be billed for the actual duration, not the waiting time.

I just made an announcement on Discord.
I am sorry about the situation, especially since it’s the last week of the competition.

3

Regarding the bad prediction, I was actually redeploying the platform at that time. You just got unlucky.

I recomputed it, and you received a score of 64.35% on the Run #119314.

4

Thanks Enzo, two questions:

  1. Since I can’t squeeze in my last couple runs before the rest, as the service is down, would you consider adding my unused time to my quota for next week (the 15h resetting today)?
  2. Similarly for the bad prediction, I ran the submission, so it used my quota but then came back with no score. Would you consider adding back the time of that run to my quota for next week?

Thanks!
Dylan

5

I will increase your quota since the problem in 2. was genuinely ours.

However, I cannot increase your quota for 1. because the issue affects everyone, sorry.

6

Ok, thank you for you help

7

@enzo are GPU runs down again? Looks like they aren’t starting on my end

8

Murphy’s law, as I post the question the runs started working! So no issues now.

1 Like
9

Hello Dylan,

You actually got lucky!

There is still a shortage on the GPU :pensive_face:.

10

Runner died 30min in :frowning:

11

Really sorry about all of this.

We asked GCP to unlock another region for us, but unfortunately, it will take some time.
We hope to have it ready by the weekend.