tweak neighbor cache data structure and simplify code

this is a preparation to implement request timeout.
no functional changes yet.