biocore/biom-format — pull request #386 by josenavas (2014-05-14 19:42:49+00:00)
Title: Hdf5 subset round 2

Fixes #291

@wasade @ElBrogrammer this is the easier/simpler way that I could find to perform the hdf5 subsetting. However, it has a performance impact. Here is the results of a pathological case in which we load the entire table:

```
timeit -n10 t = Table.from_hdf5(h5py.File('./10000x10000x0.100_bench_hdf5.biom.gz'))
10 loops, best of 3: 226 ms per loop

timeit -n10 t = Table.from_hdf5_subset(h5py.File('./10000x10000x0.100_bench_hdf5.biom.gz'))
10 loops, best of 3: 11.3 s per loop
```

Since the performance drops down, when I merged these two functions (I coded them separately so I can test them) I added an if statement that in case that no subsetting is needed, it loads the biom table using the old code.

If you have a better idea on how can this be done let me know, but I don't think we can avoid this overhead.