Check block and entity valid on inventory click too. Should prevent any
timing attacks.