Remove call to `key?` in `LazyAttributeHash#[]`

Performance improvement, as well as improved code clarity
LazyAttributeHash is private
Remove needless call to `key?` when building attributes

This appears to be a performance hotspot, see #17655.