I don’t think this is a good plan:
To validate the most promising gene panels, we will select up to 500 genes for experimental evaluation. This selection will occur via two routes:
Route 1: the top performers from Crunch 2 who also participate in Crunch 3 will have up to 50 of their highest-ranked genes included.
Route 2: the top performers from Crunch 3, determined by peer review and expert evaluations, will also have up to 50 of their top genes included.
because you could have an unranked participant who suggests genes which are actually the most relevant genes, are not on another participant’s panel, but are not used because the participant is unranked.
A participant’s panel that includes unused genes that are actually relevant could be scored lower than a less performant panel by the mechanics of the scoring function because any unused gene will be NA going into the scoring function.
This is unfair to participants that didn’t rank on Crunch 1 or Crunch 2 but otherwise have good insight.
A fair and reliable approach would be to run multiple gene sensitivity assays if necessary to get the expression of all genes mentioned in any participant’s panel.
Please also share with us the differential gene expression measure to be used in the final scoring process.