michael-guenther
commited on
add info for access token, trouble shooting, and usage with sentence-transformers
Browse files
README.md
CHANGED
@@ -2685,7 +2685,25 @@ embeddings = F.normalize(embeddings, p=2, dim=1)
|
|
2685 |
</p>
|
2686 |
</details>
|
2687 |
|
2688 |
-
You can use Jina Embedding models directly from transformers package
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2689 |
```python
|
2690 |
!pip install transformers
|
2691 |
from transformers import AutoModel
|
@@ -2706,11 +2724,34 @@ embeddings = model.encode(
|
|
2706 |
)
|
2707 |
```
|
2708 |
|
2709 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2710 |
|
2711 |
1. _Managed SaaS_: Get started with a free key on Jina AI's [Embedding API](https://jina.ai/embeddings/).
|
2712 |
2. _Private and high-performance deployment_: Get started by picking from our suite of models and deploy them on [AWS Sagemaker](https://aws.amazon.com/marketplace/seller-profile?id=seller-stch2ludm6vgy).
|
2713 |
|
|
|
2714 |
## Use Jina Embeddings for RAG
|
2715 |
|
2716 |
According to the latest blog post from [LLamaIndex](https://blog.llamaindex.ai/boosting-rag-picking-the-best-embedding-reranker-models-42d079022e83),
|
@@ -2726,6 +2767,28 @@ According to the latest blog post from [LLamaIndex](https://blog.llamaindex.ai/b
|
|
2726 |
2. Multimodal embedding models enable Multimodal RAG applications.
|
2727 |
3. High-performt rerankers.
|
2728 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2729 |
## Contact
|
2730 |
|
2731 |
Join our [Discord community](https://discord.jina.ai) and chat with other community members about ideas.
|
|
|
2685 |
</p>
|
2686 |
</details>
|
2687 |
|
2688 |
+
You can use Jina Embedding models directly from transformers package.
|
2689 |
+
|
2690 |
+
First, you need to make sure that you are logged into huggingface. You can either use the huggingface-cli tool (after installing the `transformers` package) and pass your [hugginface access token](https://huggingface.co/docs/hub/security-tokens):
|
2691 |
+
```bash
|
2692 |
+
huggingface-cli login
|
2693 |
+
```
|
2694 |
+
Alternatively, you can provide the access token as an environment variable in the shell:
|
2695 |
+
```bash
|
2696 |
+
export HF_TOKEN="<your token here>"
|
2697 |
+
```
|
2698 |
+
or in Python:
|
2699 |
+
```python
|
2700 |
+
import os
|
2701 |
+
|
2702 |
+
os.environ['HF_TOKEN'] = "<your token here>"
|
2703 |
+
```
|
2704 |
+
|
2705 |
+
Then, you can use load and use the model via the `AutoModel` class:
|
2706 |
+
|
2707 |
```python
|
2708 |
!pip install transformers
|
2709 |
from transformers import AutoModel
|
|
|
2724 |
)
|
2725 |
```
|
2726 |
|
2727 |
+
Using the its latest release (v2.3.0) sentence-transformers also supports Jina embeddings (Please make sure that you are logged into huggingface as well):
|
2728 |
+
|
2729 |
+
```python
|
2730 |
+
!pip install -U sentence-transformers
|
2731 |
+
from sentence_transformers import SentenceTransformer
|
2732 |
+
from sentence_transformers.util import cos_sim
|
2733 |
+
|
2734 |
+
model = SentenceTransformer(
|
2735 |
+
"jinaai/jina-embeddings-v2-base-de", # switch to en/zh for English or Chinese
|
2736 |
+
trust_remote_code=True
|
2737 |
+
)
|
2738 |
+
|
2739 |
+
# control your input sequence length up to 8192
|
2740 |
+
model.max_seq_length = 1024
|
2741 |
+
|
2742 |
+
embeddings = model.encode([
|
2743 |
+
'How is the weather today?',
|
2744 |
+
'Wie ist das Wetter heute?'
|
2745 |
+
])
|
2746 |
+
print(cos_sim(embeddings[0], embeddings[1]))
|
2747 |
+
```
|
2748 |
+
|
2749 |
+
## Alternatives to Using Transformers (or SentencTransformers) Package
|
2750 |
|
2751 |
1. _Managed SaaS_: Get started with a free key on Jina AI's [Embedding API](https://jina.ai/embeddings/).
|
2752 |
2. _Private and high-performance deployment_: Get started by picking from our suite of models and deploy them on [AWS Sagemaker](https://aws.amazon.com/marketplace/seller-profile?id=seller-stch2ludm6vgy).
|
2753 |
|
2754 |
+
|
2755 |
## Use Jina Embeddings for RAG
|
2756 |
|
2757 |
According to the latest blog post from [LLamaIndex](https://blog.llamaindex.ai/boosting-rag-picking-the-best-embedding-reranker-models-42d079022e83),
|
|
|
2767 |
2. Multimodal embedding models enable Multimodal RAG applications.
|
2768 |
3. High-performt rerankers.
|
2769 |
|
2770 |
+
## Trouble Shooting
|
2771 |
+
|
2772 |
+
**Loading of Model Code failed**
|
2773 |
+
|
2774 |
+
If you forgot to pass the `trust_remote_code=True` flag when calling `AutoModel.from_pretrained` or initializing the model via the `SentenceTransformer` class, you will receive an error that the model weights could not be initialized.
|
2775 |
+
This is caused by tranformers falling back to creating a default BERT model, instead of a jina-embedding model:
|
2776 |
+
|
2777 |
+
```bash
|
2778 |
+
Some weights of the model checkpoint at jinaai/jina-embeddings-v2-base-en were not used when initializing BertModel: ['encoder.layer.2.mlp.layernorm.weight', 'encoder.layer.3.mlp.layernorm.weight', 'encoder.layer.10.mlp.wo.bias', 'encoder.layer.5.mlp.wo.bias', 'encoder.layer.2.mlp.layernorm.bias', 'encoder.layer.1.mlp.gated_layers.weight', 'encoder.layer.5.mlp.gated_layers.weight', 'encoder.layer.8.mlp.layernorm.bias', ...
|
2779 |
+
```
|
2780 |
+
|
2781 |
+
|
2782 |
+
**User is not logged into Huggingface**
|
2783 |
+
|
2784 |
+
The model is only availabe under [gated access](https://huggingface.co/docs/hub/models-gated).
|
2785 |
+
This means you need to be logged into huggingface load load it.
|
2786 |
+
If you receive the following error, you need to provide an access token, either by using the huggingface-cli or providing the token via an environment variable as described above:
|
2787 |
+
```bash
|
2788 |
+
OSError: jinaai/jina-embeddings-v2-base-en is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
|
2789 |
+
If this is a private repository, make sure to pass a token having permission to this repo with `use_auth_token` or log in with `huggingface-cli login` and pass `use_auth_token=True`.
|
2790 |
+
```
|
2791 |
+
|
2792 |
## Contact
|
2793 |
|
2794 |
Join our [Discord community](https://discord.jina.ai) and chat with other community members about ideas.
|