No space left on device

Competition Broad Institute Crunch #1
1

I am getting a no space left on device error even though i am removing the files in the tmp directory. Can you please look into this

2

I am unable to take a look right now.

Could you please give me a Run ID?

3

RUN ID #31266 - This is the runid

4

The error is very weird:

filename = ‘/tmp/patches/xxx’, file descriptor = 17, errno = 28, error message = ‘No space left on device’, buf = 0x1220dcb8, total write size = 96, bytes this sub-write = 96, bytes actually written = 18446744073709551615, offset = 0)

Especially: bytes actually written = 18446744073709551615
Which is equivalent to 17179869184 Gb…

5

GPU Runs already have access to 80Gb.
If you really need more, I can increase, but the bug is likely due to:

  • your code
  • or library version mismatch
6

I am removing the temporary files that i am creating can you confirm that the files are indeed removed because that is the issue also i ran this code on my local and i dont write such huge files i will look at my code again but if you can confirm that i am removig my tmp files that would be great

7

I cannot really confirm that for you, but what you can do is this:

import os
os.system("ls /tmp")

It will list the content in the console so that you can visually confirm that the directory is empty.

8

is it possible to double the space on the device to 160 GB

9

I made a mistake and misunderstood the big number, it is actually an unsigned long max value. So likely a -1 if signed. Meaning nothing has even been wrote yet.

I increased the size to 300 gb for GPU Runs only.