# Submission format-- CSV or notebook?

**URL:** https://forum.crunchdao.com/t/submission-format-csv-or-notebook/575
**Category:** (ended) Broad Institute Crunch #1
**Created:** [November 2, 2024, 3:05am UTC](https://forum.crunchdao.com/t/submission-format-csv-or-notebook/575 "2024-11-02T03:05:22Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![dizzy-rachel](https://forum.crunchdao.com/user_avatar/forum.crunchdao.com/dizzy-rachel/32/359_2.png) [@dizzy-rachel](https://forum.crunchdao.com/u/dizzy-rachel)
#### Post date: [November 2, 2024, 3:05am UTC](https://forum.crunchdao.com/t/submission-format-csv-or-notebook/575/1 "2024-11-02T03:05:22Z")

</div>

For the Broad Autoimmune competition, are the submissions expected to be in the format of a CSV or as a model in a notebook? Some of the documentation says CSV (such as on this page: [Broad Institute Autoimmune Disease | CrunchDAO Docs V3](https://docs.crunchdao.com/competitions/competitions/broad-institute-autoimmune-disease)), but the dummy submission is a notebook with train and infer methods ([Submit a Notebook - Broad Institute Autoimmune Disease Competition - CrunchDAO](https://hub.crunchdao.com/competitions/broad-1/submit/via/notebook)) .

---

<div class="post-metadata">

### Author: ![combative-lusheng](https://forum.crunchdao.com/user_avatar/forum.crunchdao.com/combative-lusheng/32/374_2.png) [@combative-lusheng](https://forum.crunchdao.com/u/combative-lusheng)
#### Post date: [November 2, 2024, 5:22pm UTC](https://forum.crunchdao.com/t/submission-format-csv-or-notebook/575/2 "2024-11-02T17:22:06Z")

</div>

I also want to ask how to use the random-submission.ipynb. If the host would like to show demo case will be very helpful.

---

<div class="post-metadata">

### Author: ![cruncher-abde](https://forum.crunchdao.com/user_avatar/forum.crunchdao.com/cruncher-abde/32/298_2.png) [@cruncher-abde](https://forum.crunchdao.com/u/cruncher-abde)
#### Post date: [November 5, 2024, 4:42pm UTC](https://forum.crunchdao.com/t/submission-format-csv-or-notebook/575/3 "2024-11-05T16:42:17Z")

</div>

Hello,

We have corrected the discrepancy in the document and the notebook; the correct expected format is the one provided by the notebook. Sorry for the confusion

---

<div class="post-metadata">

### Author: ![spicy-questo](https://forum.crunchdao.com/user_avatar/forum.crunchdao.com/spicy-questo/32/243_2.png) [@spicy-questo](https://forum.crunchdao.com/u/spicy-questo)
#### Post date: [November 24, 2024, 5:56pm UTC](https://forum.crunchdao.com/t/submission-format-csv-or-notebook/575/4 "2024-11-24T17:56:37Z")

</div>

Having pipeline, which is crops and featurize images , all in all from 8 zarr standard datasets it delivers ~60gb+ , how much capacity we have in cloud, is it going to handle such size?

---

<div class="post-metadata">

### Author: ![spicy-questo](https://forum.crunchdao.com/user_avatar/forum.crunchdao.com/spicy-questo/32/243_2.png) [@spicy-questo](https://forum.crunchdao.com/u/spicy-questo)
#### Post date: [November 25, 2024, 5:52am UTC](https://forum.crunchdao.com/t/submission-format-csv-or-notebook/575/5 "2024-11-25T05:52:13Z")

</div>

alright,if we need only predict on validation and test, the train data might be cut out, that way we can avoid to allocation of excessive space in cloud.

---

<div class="post-metadata">

### Author: ![enzo](https://forum.crunchdao.com/user_avatar/forum.crunchdao.com/enzo/32/15_2.png) [@enzo](https://forum.crunchdao.com/u/enzo)
#### Post date: [November 25, 2024, 10:36am UTC](https://forum.crunchdao.com/t/submission-format-csv-or-notebook/575/6 "2024-11-25T10:36:09Z")

</div>

The large dataset is not available in the Runner.

You can read about resource limitations in the documentation: [Resources Limit | CrunchDAO Docs V3](https://docs.crunchdao.com/competitions/resources-limit#runs)

---

<div class="post-metadata">

### Author: ![spicy-questo](https://forum.crunchdao.com/user_avatar/forum.crunchdao.com/spicy-questo/32/243_2.png) [@spicy-questo](https://forum.crunchdao.com/u/spicy-questo)
#### Post date: [November 25, 2024, 11:22am UTC](https://forum.crunchdao.com/t/submission-format-csv-or-notebook/575/7 "2024-11-25T11:22:41Z")

</div>

i currently use Standard Dataset only, because the large dataset is excessive to my model  
 ![image](https://forum.crunchdao.com/uploads/default/original/1X/668728a929458653e0d96a72c9e0cdaec20e895a.png)  
i checked the sizes of folders:  
the 1.1 gb is the size of initial DC1.zarr  
13 gb is the size of postprocessed DC1.zarr images where is most share is belong to centroid cropped images and additional features.  
I’m gonna try to reduce it by throwing out the train data and leaving only the validation and test

---

<div class="post-metadata">

### Author: ![enzo](https://forum.crunchdao.com/user_avatar/forum.crunchdao.com/enzo/32/15_2.png) [@enzo](https://forum.crunchdao.com/u/enzo)
#### Post date: [November 25, 2024, 11:40am UTC](https://forum.crunchdao.com/t/submission-format-csv-or-notebook/575/8 "2024-11-25T11:40:39Z")

</div>

The GPU Runner has a disk size of 100GB.  
If you encounter disk size problems, contact me as soon as possible and I will increase it.  
But be careful as your model cannot be larger than 10gb.

---

<div class="post-metadata">

### Author: ![spicy-questo](https://forum.crunchdao.com/user_avatar/forum.crunchdao.com/spicy-questo/32/243_2.png) [@spicy-questo](https://forum.crunchdao.com/u/spicy-questo)
#### Post date: [November 28, 2024, 11:18am UTC](https://forum.crunchdao.com/t/submission-format-csv-or-notebook/575/9 "2024-11-28T11:18:33Z")

</div>

does rm command in infer() function will work?  
or it need sudo, ! or % as precursor?  
rm -r ‘/tmp/DC1.zarr’  
!rm -r ‘/tmp/DC1.zarr’  
%rm -r ‘/tmp/DC1.zarr’

---

<div class="post-metadata">

### Author: ![enzo](https://forum.crunchdao.com/user_avatar/forum.crunchdao.com/enzo/32/15_2.png) [@enzo](https://forum.crunchdao.com/u/enzo)
#### Post date: [November 28, 2024, 11:35am UTC](https://forum.crunchdao.com/t/submission-format-csv-or-notebook/575/10 "2024-11-28T11:35:48Z")

</div>

```python
import shutil

shutil.rmtree("/tmp/DC1.zarr")

```

or

```python
import os

os.system("rm -r '/tmp/DC1.zarr'")

```
