This area hurts my brain, and the repeated calls to np.concatenate will be a nightmare for performance. Don't we know the final size of the array? If so, allocate it from the start and write to the appropriate part. If not, then use a list and append, and a single call to np.concatenate at the end.