Hi, looks like the ranking is based on the ‘best’ Spearman’s correlation. Can you please clarify the ‘Best’? the best of what? For each model, we can only select one submission for checkpoint, right?
Thank you in advance!
We rank based on the best prediction score. Where best is the highest spearman score.
I’m still confused. The highest spearman score among all the submissions, or among 8 samples, or among all the cells and genes or what? Can you please further clarify it? Thank you!
The score is a mean across all samples, which is the weighted mean across all cells.
The best value is your highest score (the mean of means).
Sorry, what I still don’t understand is why some of the ‘last subm. Spearman’ are NOT the same of the ‘Best’. Is the ‘Best’ the best among all submission history? Does ‘last’ mean ‘latest’ submission? For each model, we can only select one submission for checkpoint, right? Can you please explain using this example , the current Rank 2 (Best = 39.067, last subm. Spearman = 29.906)? What is the ‘Best’ here? Thank you very much!
Normally the leaderboard everytime some is scored while on the public leaderboard.
That mean the “last submission” is really the last submission they sent and run!
Since the broad competitions does not have a live public leaderboard, the “last submission” refers to the last submission before the checkpoint.