Unified way of getting an underlying CLMem object.