convert ips to bytesRef if loaded from fieldcache currently there is no clean