Made modifications so it can tackle the whole data folder (loading an ndarray caused it to run out of memory).

Will need to improve it's speed however, as an overnight run did not finish.