All places we check isBinary also need to check isSimpleValue