So, this is my second time participating in a crunch hacathon. I had one doubt the way the runs are scored. I submitted 4 models on saturday (Means 4 completely different models). However, the leaderboard only shows the best and the last run metrics. I want to know following things:
Which submission corresponds to the best score in the metric?
Second, is it possible to get metrics for all the models I submitted or do i need to submit each model on Monday to get the results?
You can find the exact score for each run on its detail page.
Unfortunately, we don’t have a “summary” view that displays all the scores at once.
The system doesn’t keep track of it internally. You will have to find it yourself.
All of the Run are scored on Monday, so you should be able to look at them all.
Hello!
This is also my first time participant n a crunch hacathon. I’m wondering why my run was finished sussessfully but the score is still unknow.
I started the scoring mechanism, but got distracted.
I just published them.
Thank you for your help! I also confirm whether the score of the submission would only be announced every Monday?
Yes that correct.
Scores are released at what we call checkpoint, which happen every Monday at 6 p.m. UTC.
Thank you for the information. I am currently trying to push my submission using the Crunch CLI, but I encountered the following error:
Crunch not found.
The competition may be over or the server is not correctly configured.
If you think that is an error, please contact an administrator.
I would like to ask whether this message appears because the submission is currently only being scored at checkpoints, or if there might be an issue on my side (e.g., configuration or competition access).
Could you please advise on how to resolve this issue?
Thank you very much for your help.
Hi ding-yang-wang,
The system was stuck and our monitoring did not reported the problem.
Apologies for the delay, it is now fixed!