michael-guenther commited on
Commit
7597002
1 Parent(s): 69d4370

add info for access token, trouble shooting, and usage with sentence-transformers

Browse files
Files changed (1) hide show
  1. README.md +65 -2
README.md CHANGED
@@ -2685,7 +2685,25 @@ embeddings = F.normalize(embeddings, p=2, dim=1)
2685
  </p>
2686
  </details>
2687
 
2688
- You can use Jina Embedding models directly from transformers package:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2689
  ```python
2690
  !pip install transformers
2691
  from transformers import AutoModel
@@ -2706,11 +2724,34 @@ embeddings = model.encode(
2706
  )
2707
  ```
2708
 
2709
- ## Alternatives to Using Transformers Package
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2710
 
2711
  1. _Managed SaaS_: Get started with a free key on Jina AI's [Embedding API](https://jina.ai/embeddings/).
2712
  2. _Private and high-performance deployment_: Get started by picking from our suite of models and deploy them on [AWS Sagemaker](https://aws.amazon.com/marketplace/seller-profile?id=seller-stch2ludm6vgy).
2713
 
 
2714
  ## Use Jina Embeddings for RAG
2715
 
2716
  According to the latest blog post from [LLamaIndex](https://blog.llamaindex.ai/boosting-rag-picking-the-best-embedding-reranker-models-42d079022e83),
@@ -2726,6 +2767,28 @@ According to the latest blog post from [LLamaIndex](https://blog.llamaindex.ai/b
2726
  2. Multimodal embedding models enable Multimodal RAG applications.
2727
  3. High-performt rerankers.
2728
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2729
  ## Contact
2730
 
2731
  Join our [Discord community](https://discord.jina.ai) and chat with other community members about ideas.
 
2685
  </p>
2686
  </details>
2687
 
2688
+ You can use Jina Embedding models directly from transformers package.
2689
+
2690
+ First, you need to make sure that you are logged into huggingface. You can either use the huggingface-cli tool (after installing the `transformers` package) and pass your [hugginface access token](https://huggingface.co/docs/hub/security-tokens):
2691
+ ```bash
2692
+ huggingface-cli login
2693
+ ```
2694
+ Alternatively, you can provide the access token as an environment variable in the shell:
2695
+ ```bash
2696
+ export HF_TOKEN="<your token here>"
2697
+ ```
2698
+ or in Python:
2699
+ ```python
2700
+ import os
2701
+
2702
+ os.environ['HF_TOKEN'] = "<your token here>"
2703
+ ```
2704
+
2705
+ Then, you can use load and use the model via the `AutoModel` class:
2706
+
2707
  ```python
2708
  !pip install transformers
2709
  from transformers import AutoModel
 
2724
  )
2725
  ```
2726
 
2727
+ Using the its latest release (v2.3.0) sentence-transformers also supports Jina embeddings (Please make sure that you are logged into huggingface as well):
2728
+
2729
+ ```python
2730
+ !pip install -U sentence-transformers
2731
+ from sentence_transformers import SentenceTransformer
2732
+ from sentence_transformers.util import cos_sim
2733
+
2734
+ model = SentenceTransformer(
2735
+ "jinaai/jina-embeddings-v2-base-de", # switch to en/zh for English or Chinese
2736
+ trust_remote_code=True
2737
+ )
2738
+
2739
+ # control your input sequence length up to 8192
2740
+ model.max_seq_length = 1024
2741
+
2742
+ embeddings = model.encode([
2743
+ 'How is the weather today?',
2744
+ 'Wie ist das Wetter heute?'
2745
+ ])
2746
+ print(cos_sim(embeddings[0], embeddings[1]))
2747
+ ```
2748
+
2749
+ ## Alternatives to Using Transformers (or SentencTransformers) Package
2750
 
2751
  1. _Managed SaaS_: Get started with a free key on Jina AI's [Embedding API](https://jina.ai/embeddings/).
2752
  2. _Private and high-performance deployment_: Get started by picking from our suite of models and deploy them on [AWS Sagemaker](https://aws.amazon.com/marketplace/seller-profile?id=seller-stch2ludm6vgy).
2753
 
2754
+
2755
  ## Use Jina Embeddings for RAG
2756
 
2757
  According to the latest blog post from [LLamaIndex](https://blog.llamaindex.ai/boosting-rag-picking-the-best-embedding-reranker-models-42d079022e83),
 
2767
  2. Multimodal embedding models enable Multimodal RAG applications.
2768
  3. High-performt rerankers.
2769
 
2770
+ ## Trouble Shooting
2771
+
2772
+ **Loading of Model Code failed**
2773
+
2774
+ If you forgot to pass the `trust_remote_code=True` flag when calling `AutoModel.from_pretrained` or initializing the model via the `SentenceTransformer` class, you will receive an error that the model weights could not be initialized.
2775
+ This is caused by tranformers falling back to creating a default BERT model, instead of a jina-embedding model:
2776
+
2777
+ ```bash
2778
+ Some weights of the model checkpoint at jinaai/jina-embeddings-v2-base-en were not used when initializing BertModel: ['encoder.layer.2.mlp.layernorm.weight', 'encoder.layer.3.mlp.layernorm.weight', 'encoder.layer.10.mlp.wo.bias', 'encoder.layer.5.mlp.wo.bias', 'encoder.layer.2.mlp.layernorm.bias', 'encoder.layer.1.mlp.gated_layers.weight', 'encoder.layer.5.mlp.gated_layers.weight', 'encoder.layer.8.mlp.layernorm.bias', ...
2779
+ ```
2780
+
2781
+
2782
+ **User is not logged into Huggingface**
2783
+
2784
+ The model is only availabe under [gated access](https://huggingface.co/docs/hub/models-gated).
2785
+ This means you need to be logged into huggingface load load it.
2786
+ If you receive the following error, you need to provide an access token, either by using the huggingface-cli or providing the token via an environment variable as described above:
2787
+ ```bash
2788
+ OSError: jinaai/jina-embeddings-v2-base-en is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
2789
+ If this is a private repository, make sure to pass a token having permission to this repo with `use_auth_token` or log in with `huggingface-cli login` and pass `use_auth_token=True`.
2790
+ ```
2791
+
2792
  ## Contact
2793
 
2794
  Join our [Discord community](https://discord.jina.ai) and chat with other community members about ideas.