Adds crawler link rules to config json