Freeze user vector for neighbor finding

The user to find neighbors for can be frozen, so that its norm is cached.