CellProfiler/CellProfiler — Issue #29: JIRA [IMG-1531] HDF5 file creation weirdness: File misnaming + locking
Author: thouis
Created: 2011-11-23 20:38:23+00:00
State: open
Labels: current

This issue was noticed when looking at this forum post: http://cellprofiler.org/forum/viewtopic.php?f=16&t=1463

I have noticed the appearance of these files as well, even if I specify "Do not write measurements". However, if an .h5 file must always be created, can it be cleaned up at the end if "Do not write..." is selected?

The forum poster is running headless, so I'm not sure what recourse he has.

However, if I specify "HDF5" as the measurement format, and allow overwriting, I notice the following:

(1) The output filename will be shown as "DefaultOut.h5" rather than the usual "DefaultOUT.h5" (note the case change)

(2) Running the pipeline with the output filename in (1) will create a "DefaultOUT" file (no extension but it's an HDF5 file)

(3) Changing the output filename to "DefaultOUT.h5" will create a DefaultOUT.h5 file on the run, as it should. But if I run it again, it get this error on the 1st image set:

----------------------------------------------

Traceback (most recent call last):

File "C:\Trunk\CellProfiler\cellprofiler\gui\pipelinecontroller.py", line 754, in on_analyze_images

m = cpm.Measurements(filename = output_path)

File "C:\Trunk\CellProfiler\cellprofiler\measurements.py", line 156, in __init__

self.hdf5_dict = HDF5Dict(filename, is_temporary = is_temporary)

File "C:\Trunk\CellProfiler\cellprofiler\utilities\hdf5_dict.py", line 115, in __init__

self.hdf5_file = h5py.File(self.filename, 'w')

File "C:\Python27\lib\site-packages\h5py\_hl\files.py", line 150, in __init__

fid = make_fid(name, mode, fapl)

File "C:\Python27\lib\site-packages\h5py\_hl\files.py", line 51, in make_fid

fid = h5f.create(name, h5f.ACC_TRUNC, fapl=plist)

File "h5f.pyx", line 89, in h5py.h5f.create (h5py\h5f.c:1762)

IOError: unable to create file (File accessability: Unable to open file)

Original reporter: mbray
Jira link: https://jira.broadinstitute.org/browse/IMG-1531
Fix version: Current
Assignee: leek

**Comment from leek on Fri, 16 Sep 2011 11:49:21 -0400:**
I answered the post. The files are needed to keep the measurements from

consuming memory. They should be deleted at the end of the run.

On 9/16/2011 11:18 AM, Mark-Anthony Bray (JIRA) wrote:

> HDF5 file creation weirdness: File misnaming + locking

> ------------------------------------------------------

>

> Key: IMG-1531

> URL: https://jira.broadinstitute.org/browse/IMG-1531

> Project: Imaging

> Issue Type: Bug

> Components: CellProfiler

> Affects Versions: Current

> Reporter: Mark-Anthony Bray

> Assignee: Lee Kamentsky

> Priority: Major

> Fix For: Current

>

>

> This issue was noticed when looking at this forum post: http://cellprofiler.org/forum/viewtopic.php?f=16&t=1463

>

> I have noticed the appearance of these files as well, even if I specify "Do not write measurements". However, if an .h5 file must always be created, can it be cleaned up at the end if "Do not write..." is selected?

>

> The forum poster is running headless, so I'm not sure what recourse he has.

>

> However, if I specify "HDF5" as the measurement format, and allow overwriting, I notice the following:

> (1) The output filename will be shown as "DefaultOut.h5" rather than the usual "DefaultOUT.h5" (note the case change)

> (2) Running the pipeline with the output filename in (1) will create a "DefaultOUT" file (no extension but it's an HDF5 file)

> (3) Changing the output filename to "DefaultOUT.h5" will create a DefaultOUT.h5 file on the run, as it should. But if I run it again, it get this error on the 1st image set:

> ----------------------------------------------

> Traceback (most recent call last):

> File "C:\Trunk\CellProfiler\cellprofiler\gui\pipelinecontroller.py", line 754, in on_analyze_images

> m = cpm.Measurements(filename = output_path)

> File "C:\Trunk\CellProfiler\cellprofiler\measurements.py", line 156, in __init__

> self.hdf5_dict = HDF5Dict(filename, is_temporary = is_temporary)

> File "C:\Trunk\CellProfiler\cellprofiler\utilities\hdf5_dict.py", line 115, in __init__

> self.hdf5_file = h5py.File(self.filename, 'w')

> File "C:\Python27\lib\site-packages\h5py\_hl\files.py", line 150, in __init__

> fid = make_fid(name, mode, fapl)

> File "C:\Python27\lib\site-packages\h5py\_hl\files.py", line 51, in make_fid

> fid = h5f.create(name, h5f.ACC_TRUNC, fapl=plist)

> File "h5f.pyx", line 89, in h5py.h5f.create (h5py\h5f.c:1762)

> IOError: unable to create file (File accessability: Unable to open file)

>

> --

> This message is automatically generated by JIRA.

> For more information on JIRA, see: http://www.atlassian.com/software/jira