ADIA Lab real time questions

Competition ADIA Lab
1

A few questions:

  1. Running crunch test locally for the real time challenge, crunch-cli 11.10.0 began failing today with: TypeError: LocalRunnerContext.execute() got an unexpected keyword argument ‘log_limits’ raised inside the runner.py that the CLI fetches from competitions master. The same workspace passed at 8am and failed at 3pm the same day, so runner.py seems to have changed? Which crunch-cli version should I be on, and does the submission-side runner use the same runner.py as the local test?
  2. My submission ships training code under training_code/ as suggested in the documentation. The push uploads one file per HTTP request, so lots of small files took hours. Now I ship the bulk as a single .tar.gz inside training_code/, with all manifests/repository path/per-file index readable. Is a single archive acceptable for the training-code requirement, or do you require loose files? Also is it acceptable to provide training data producing script, rather than uploading the data itself?
2

  1. Upgrade the crunch-cli locally to restore functionalities: pip install --upgrade crunch-cli
    Don’t worry about the cloud environment, it will always be the latest version.

  2. Code need to be provided as raw .py files. We are planning to add a parallelized upload soon, but its not ready yet.
    Yes, providing the scripts capable of reproducing the data is enough.

3

When you say code must be provided as raw .py files, does that mean only the .py sources must be loose, and non-code files (JSON receipts, manifests, exported model .txt/.npz, etc) may stay inside an archive, or must every file under training_code/ be loose?

4

All files must be loose.

5

I promise we will work on parallel upload soon :folded_hands:.

6

ok thanks, please let me know when! Uploading everything loose right now can take hours

7

@enzo do you think the parallel upload will be ready in time to use before the deadline on Thurs? (ie tomorrow/Wed)
Also I’m guessing I need to reload my current selected submission? as it’s not compliant with the loose files requirement you indicated.

8

I am currently working on it.

If you already submitted something that works, keep it as is.
Its better you use the quota for improving your model instead of wasting it because some file were zipped.

9

I think either the new CLI is quicker, or my ISP is being friendlier to the upload :rofl:
Either way upload times are now running <1h

10

I am still working on it, I haven’t published anything yet haha!

To tell you a bit more, the new version will cache uploaded files.
If you haven’t updated a file locally, it can be reused directly without uploading it again.
It significantly speeds up everything by not using the network at all!

(It’s not true parallelism yet. This will come later. Sorry!)