optimize repo crawler