File size: 2,020 Bytes
5cea285 def228a 5cea285 40dfdf6 f01feac 40dfdf6 8769afc 40dfdf6 8769afc 40dfdf6 8769afc 40dfdf6 3d61f46 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 |
---
license: apache-2.0
language:
- en
- de
- es
- fr
- it
- ja
- ko
- pl
- ru
- tr
- zh
- ar
---
<h1 align="center">UForm</h1>
<h3 align="center">
Multi-Modal Inference Library<br/>
For Semantic Search Applications<br/>
</h3>
---
UForm is a Multi-Modal Modal Inference package, designed to encode Multi-Lingual Texts, Images, and, soon, Audio, Video, and Documents, into a shared vector space!
This is the repository of [English](https://huggingface.co/unum-cloud/uform-vl-english/tree/main) and [multilingual](https://huggingface.co/unum-cloud/uform-vl-multilingual) UForm models converted to CoreML MLProgram format.
Currently, only __unimodal__ parts of models are converted.
## Description
Each model is separated into two parts: `image-encoder` and `text-encoder`:
* English image-encoder: [english.image-encoder.mlpackage](https://huggingface.co/unum-cloud/uform-coreml/blob/main/english.image-encoder.mlpackage.zip)
* English text-encoder: [english.text-encoder.mlpackage](https://huggingface.co/unum-cloud/uform-coreml/blob/main/english.text-encoder.mlpackage.zip)
* Multilingual image-encoder: [multilingual.image-encoder.mlpackage](https://huggingface.co/unum-cloud/uform-coreml/blob/main/multilingual.image-encoder.mlpackage.zip)
* Multilingual text-encoder: [multilingual.text-encoder.mlpackage](https://huggingface.co/unum-cloud/uform-coreml/blob/main/multilingual.text-encoder.mlpackage.zip)
Each checkpoint is a zip archive with an MLProgram of the corresponding encoder.
Text encoders have the following input fields:
* `input_ids`: int32
* `attention_mask`: int32
and support flexible batch size.
Image encoders has a single input field `image`: float32 and support only batch of single image (due to CoreML bug).
Both encoders return:
* `features`: float32
* `embeddings`: float32
If you want to convert a model with other parameters (i.e fp16 precision or other batch size range), you can use [convert.py](https://huggingface.co/unum-cloud/uform-coreml/blob/main/convert_model.py).
|