Repository: DataBiosphere/toil

Issue:
No documentation for the FileID.size attribute appears on ReadTheDocs
I have a job that needs to use some files from the file store. I initially tried with with just 1 GB of disk, but that's not enough to download the files that it wants to work with.

I'm trying to rewrite the job to measure how big the files it wants to use are, and to re-queue itself with the appropriate amount of disk required, so it can actually download them. But the file store doesn't seem to expose a file size query. All you can do with a file in the file store is read it---and I think even if you read it streaming, your job is liable to fail if the file is bigger than the amount of disk available on the node you are running on.

The file store needs to expose file sizes, so that jobs can know how much disk they need to ask for in order to use particular files.

┆Issue is synchronized with this [Jira Story](https://ucsc-cgl.atlassian.net/browse/TOIL-296)
┆Issue Number: TOIL-296



Discussion:
Use the `size` attribute. The filestore methods should all return a FileID object, which contains both the ID string and a size.
There's no mention of this feature on http://toil.readthedocs.io/en/3.15.0/developingWorkflows/toilAPIFilestore.html and I can't find a page on ReadTheDocs for the FileID type.

So this looks like a documentation problem.
Yeah, it really should be documented somewhere.

Resolving patch:
diff --git a/src/toil/fileStore.py b/src/toil/fileStore.py
--- a/src/toil/fileStore.py
+++ b/src/toil/fileStore.py
@@ -1838,9 +1838,10 @@ def shutdown(cls, dir_):
 
 class FileID(str):
     """
-    A class to wrap the job store file id returned by writeGlobalFile and any attributes we may want
-    to add to it.
+    A small wrapper around Python's builtin string class. It is used to represent a file's ID in the file store, and
+    has a size attribute that is the file's size in bytes. This object is returned by importFile and writeGlobalFile.
     """
+
     def __new__(cls, fileStoreID, *args):
         return super(FileID, cls).__new__(cls, fileStoreID)
 
diff --git a/src/toil/jobStores/abstractJobStore.py b/src/toil/jobStores/abstractJobStore.py
--- a/src/toil/jobStores/abstractJobStore.py
+++ b/src/toil/jobStores/abstractJobStore.py
@@ -288,7 +288,7 @@ def importFile(self, srcUrl, sharedFileName=None):
         :param str sharedFileName: Optional name to assign to the imported file within the job store
 
         :return: The jobStoreFileId of the imported file or None if sharedFileName was given
-        :rtype: FileID or None
+        :rtype: toil.fileStore.FileID or None
         """
         # Note that the helper method _importFile is used to read from the source and write to
         # destination (which is the current job store in this case). To implement any
@@ -313,7 +313,7 @@ def _importFile(self, otherCls, url, sharedFileName=None):
         :param str sharedFileName: Optional name to assign to the imported file within the job store
 
         :return The jobStoreFileId of imported file or None if sharedFileName was given
-        :rtype: FileID or None
+        :rtype: toil.fileStore.FileID or None
         """
         if sharedFileName is None:
             with self.writeFileStream() as (writable, jobStoreFileID):


Test patch: