Only observed chunks trigger neighbor generation