wangd/SciDB-HDF5 — pull request #5 by sfridellap4 (2013-12-09 23:16:23+00:00)
Title: Updates for compatibility with SciDB 13.11 and supporting multiple instances.

Hi Daniel ---
My name is Steve Fridella and I work at Paradigm4. Recently one of my tasks has been to update your HDF5 loader to build/work with the latest 13.11 SciDB release. Take a look at the changes and please consider pulling them into your repo. Here are some details:

1) Some internal APIs (DBArray) had changed which were preventing the loader from successfully building. (see change 55c08b8).

2) The loader was previously designed to directly copy data from HDF5 slabs into SciDB internal chunk buffers. This resulted in the chunks being in "raw" format. The "raw" format is deprecated, so some special code needs to be called to alert the system that it is working with it. (see change 11e819b)

3) In order to use the standard "RLE" chunk format, it is necessary to go through the standard chunk iterator interface to put data into SciDB chunks. (see change 1f3ca09)

4) The loader wasn't playing nicely with multiple instances and data replication. To fix this, I changed the 'loadhdf' function so that it produced a temporary array as output, instead of writing directly to a DBArray. In order to make the loaded array persistent the user simply needs to wrap the result of 'loadhdf' inside a 'store()' operator. 'store()' takes care of data replication and distribution among instances using the standard SciDB infrastructure. (See change 60321da)

Let me know if you have any questions or comments about the code, or anything SciDB related...
regards,
Steve