Broad Crunch 3 gene panel selection

Competition Broad Institute Crunch #3
1

I don’t think this is a good plan:

To validate the most promising gene panels, we will select up to 500 genes for experimental evaluation. This selection will occur via two routes:

Route 1: the top performers from Crunch 2 who also participate in Crunch 3 will have up to 50 of their highest-ranked genes included.

Route 2: the top performers from Crunch 3, determined by peer review and expert evaluations, will also have up to 50 of their top genes included.

because you could have an unranked participant who suggests genes which are actually the most relevant genes, are not on another participant’s panel, but are not used because the participant is unranked.

A participant’s panel that includes unused genes that are actually relevant could be scored lower than a less performant panel by the mechanics of the scoring function because any unused gene will be NA going into the scoring function.

This is unfair to participants that didn’t rank on Crunch 1 or Crunch 2 but otherwise have good insight.

A fair and reliable approach would be to run multiple gene sensitivity assays if necessary to get the expression of all genes mentioned in any participant’s panel.

Please also share with us the differential gene expression measure to be used in the final scoring process.

2

The broad team answered:

Crunch3 is to evaluate the quality of the entire gene panel design. We may do some gene expression analysis for the top performers’ gene list before finalizing the gene panel. At the current stage, we didn’t plan to run differential gene expression measures for all genes in any participant’s panel.

3

What do you mean by “gene panel design”. Basically we are sorting our predicted gene expressions for dysplasia/nondysplasia by abs log fold change. There is no design in that, just a calculation, which we take the top 500 genes off the list. Or I guess that’s the default design. Is the idea that some people might apply some more insight about the biology, chemistry or signal chain in the selection? I am just doing the logFC abs sort.