updates to the internal link processing + index all the things