Only count a reference as unresolved if it's used.