Rename detection slow

I have a diff essentially comparing the entire history of a repo, which ends up being a bunch of added files with only one deleted file. There are some binary files in there, some approaching 1MB in size but the majority are text.

Using rename detection my benchmark takes ~7.5s to complete and if I comment out find_similar it drops to ~1.5s. I spoke to @arrbee a little about this so he has a little more background on the issue as well.