It it because X_indices[startptr: endptr] are not continuous and hence accessing different terms in R and storing it as a numpy array slows things down?