Remove indexes some more

This commit especially focusses on removing m_idx and r_ptr.