Issue 1:
Original ticket http://projects.scipy.org/numpy/ticket/1445 on 2010-04-01 by trac user Onanymous, assigned to unknown.

Just like it is implemented in gnuplot, different datasets are separated by two empty lines, when reading the file I can choose which dataset to read.

Probably another parameter, like loadtxt(fname, dataset=0).

Meanwhile there is no easy way to do this, especially when datasets have different shapes

Issue 2:
Original ticket http://projects.scipy.org/numpy/ticket/1107 on 2009-05-13 by trac user ctenney, assigned to unknown.

Problem:

I wanted to be able to easily read multiple blocks of array data into Numpy from a single text file (e.g. xyz positions of a collection of atoms at different times). I was unable to find an existing tool to accomplish this.

Solution:

I modified the loadtxt function in numpy/lib/io.py (diff pasted below).

Summary:

I added two new parameter options: 'numcols' and 'datablocks'. The defaults were chosen to reproduce previous behavior.

'numcols' is an optional int that defaults to None. If set, 'numcols' specifies the number of columns in a valid data line. This is useful for filtering out non-comment header info (particularly from files with multiple blocks of data).

'datablocks' is a boolean that defaults to False. If True, data blocks within the file are assumed to be separated by two or more non-data lines (e.g. blank, comment, or invalid 'numcols'). An additional leading dimension is added to each returned array if two or more data blocks are present. (e.g. If X = np.loadtxt(..., datablocks=True), then X[0] is the first block, X[1] the second, et cetera; and X.ravel returns the same result as np.loadtxt(..., datablocks=False).) The result is unchanged for single-block data sets.

Notes:

I successfully ran the changes against numpy/lib/tests/test_io.py.

When faced with multi-block datasets, use of either or both of these options makes plotting in matplotlib almost as easy as plotting in gnuplot, which is particularly handy for data exploration via ipython -pylab.

diff for numpy/lib/io.py for v1.3.0:

332,333c332
< skiprows=0, usecols=None, unpack=False,
< numcols=None, datablocks=False):

---
> skiprows=0, usecols=None, unpack=False):
368,379d366
< numcols : int
< If set, 'numcols' specifies the number of columns in a valid data line.
< This is useful for filtering out non-comment header info (particularly
< from files with multiple blocks of data).
< datablocks : bool
< If True, data blocks within the file are assumed to be separated by two
< or more non-data lines (e.g. blank, comment, or invalid 'numcols'). An
< additional leading dimension is added to each returned array if two or
< more data blocks are present. (e.g. If X = np.loadtxt(...,
< datablocks=True), then X[0] is the first block, X[1] the second, et
< cetera; and X.ravel returns the same result as np.loadtxt(...,
< datablocks=False).) The result is unchanged for single-block data sets.
471,472d457
< if numcols and len(first_vals) != numcols:
< first_vals = None
495,497d479
< X.append([])
< block = 0
< blanks = 0
501d482
< blanks += 1
503,509d483
< elif numcols and len(vals) != numcols:
< blanks += 1
< continue
< elif datablocks and blanks >= 2:
< block += 1
< X.append([])
< blanks = 0
515,516c489
< X[block].append(tuple([conv(val) for (conv, val) in zip(converters, vals)]))
<

---
> X.append(tuple([conv(val) for (conv, val) in zip(converters, vals)]))
536c509
< return np.rollaxis(X, -1, 0)

---
> return X.T