Repository: confident-ai/deepeval
Issue: trying to use deepseek as custom llm
Root cause: Configuration Error

**Describe the bug**
i am trying to use deepseek as following and i get an error as api key is wrong and ask me to get direct for openai to get key although i have define the base url of deepseek itself


**Expected behavior**
use the base url of deepseek and work.

**Screenshots**

![Image](https://github.com/user-attachments/assets/721d1043-585f-4424-89eb-098406d83785)

![Image](https://github.com/user-attachments/assets/3c7e62c9-0260-48dd-9087-601e953325a7)

![Image](https://github.com/user-attachments/assets/47ab27f5-5c73-475d-91cb-9f3005aa1787)

**Desktop (please complete the following information):**
 - OS: windows
 - Browser chrome


Hey @Sara-Hossny i believe it is because context construction config also involves an LLM. You need to supply the model there as well: https://docs.confident-ai.com/docs/synthesizer-generate-from-docs#customize-context-construction

Can you try it and let me know how it goes?

 it reaches 67 % and give me the same error 
✨ 🚀 ✨ Loading Documents: 100%|██████████| 1/1 [00:00<00:00, 141.85it/s]









✨ 🧩 ✨ Generating Contexts:   0%|          | 0/9 [00:00<?, ?it/s]








✨ 🧩 ✨ Generating Contexts:  67%|██████▋   | 6/9 [00:02<00:01,  2.86it/s]
---------------------------------------------------------------------------
AuthenticationError                       Traceback (most recent call last)
<ipython-input-46-02ca2500cc6c> in <cell line: 13>()
     11 
     12 # Generate goldens from documents
---> 13 dataset.generate_goldens_from_docs(
     14     document_paths=["/kaggle/input/1973_oil_crisis.txt"],
     15     context_construction_config=ContextConstructionConfig(chunk_size=230,critic_model=deepseek_model),

/usr/local/lib/python3.10/dist-packages/deepeval/dataset/dataset.py in generate_goldens_from_docs(self, document_paths, include_expected_output, max_goldens_per_context, context_construction_config, synthesizer)
    725 
    726         self.goldens.extend(
--> 727             synthesizer.generate_goldens_from_docs(
    728                 document_paths=document_paths,
    729                 include_expected_output=include_expected_output,

/usr/local/lib/python3.10/dist-packages/deepeval/synthesizer/synthesizer.py in generate_goldens_from_docs(self, document_paths, include_expected_output, max_goldens_per_context, context_construction_config, _send_data)
    120         if self.async_mode:
    121             loop = get_or_create_event_loop()
--> 122             goldens = loop.run_until_complete(
    123                 self.a_generate_goldens_from_docs(
    124                     document_paths=document_paths,

/usr/local/lib/python3.10/dist-packages/nest_asyncio.py in run_until_complete(self, future)
     96                 raise RuntimeError(
     97                     'Event loop stopped before Future completed.')
---> 98             return f.result()
     99 
    100     def _run_once(self):

/usr/lib/python3.10/asyncio/futures.py in result(self)
    199         self.__log_traceback = False
    200         if self._exception is not None:
--> 201             raise self._exception.with_traceback(self._exception_tb)
    202         return self._result
    203 

/usr/lib/python3.10/asyncio/tasks.py in __step(***failed resolving arguments***)
    232                 result = coro.send(None)
    233             else:
--> 234                 result = coro.throw(exc)
    235         except StopIteration as exc:
    236             if self._must_cancel:

/usr/local/lib/python3.10/dist-packages/deepeval/synthesizer/synthesizer.py in a_generate_goldens_from_docs(self, document_paths, include_expected_output, max_goldens_per_context, context_construction_config)
    202 
    203         contexts, source_files, context_scores = (
--> 204             await self.context_generator.a_generate_contexts(
    205                 num_context_per_document=context_construction_config.max_contexts_per_document,
    206                 max_context_size=context_construction_config.max_context_length,

/usr/local/lib/python3.10/dist-packages/deepeval/synthesizer/chunking/context_generator.py in a_generate_contexts(self, num_context_per_document, max_context_size)
    199             for path, collection in self.source_files_to_collections_map.items()
    200         ]
--> 201         results = await asyncio.gather(*tasks)
    202         for path, contexts_per_doc, scores_per_doc, num_chunks in results:
    203             contexts.extend(contexts_per_doc)

/usr/lib/python3.10/asyncio/tasks.py in __wakeup(self, future)
    302     def __wakeup(self, future):
    303         try:
--> 304             future.result()
    305         except BaseException as exc:
    306             # This may also be a cancellation.

/usr/lib/python3.10/asyncio/tasks.py in __step(***failed resolving arguments***)
    230                 # We use the `send` method directly, because coroutines
    231                 # don't have `__iter__` and `__next__` methods.
--> 232                 result = coro.send(None)
    233             else:
    234                 result = coro.throw(exc)

/usr/local/lib/python3.10/dist-packages/deepeval/synthesizer/chunking/context_generator.py in _a_process_document_async(self, path, collection, num_context_per_document, max_context_size, p_bar)
    220         min_num_context = min(num_context_per_document, num_chunks)
    221         contexts_per_doc, scores_per_doc = (
--> 222             await self._a_get_n_random_contexts_per_doc(
    223                 path=path,
    224                 n_contexts_per_doc=min_num_context,

/usr/local/lib/python3.10/dist-packages/deepeval/synthesizer/chunking/context_generator.py in _a_get_n_random_contexts_per_doc(self, path, n_contexts_per_doc, context_size, similarity_threshold, p_bar)
    325             # Query for similar chunks
    326             similar_chunks = collection.query(
--> 327                 self.embedder.embed_text(random_chunk),
    328                 n_results=min(context_size, collection.count()),
    329             )

/usr/local/lib/python3.10/dist-packages/deepeval/models/openai_embedding_model.py in embed_text(self, text)
     78     def embed_text(self, text: str) -> List[float]:
     79         embedding_model = self.load_model()
---> 80         return embedding_model.embed_query(text)
     81 
     82     def embed_texts(self, texts: List[str]) -> List[List[float]]:

/usr/local/lib/python3.10/dist-packages/langchain_openai/embeddings/base.py in embed_query(self, text)
    627             Embedding for the text.
    628         """
--> 629         return self.embed_documents([text])[0]
    630 
    631     async def aembed_query(self, text: str) -> List[float]:

/usr/local/lib/python3.10/dist-packages/langchain_openai/embeddings/base.py in embed_documents(self, texts, chunk_size)
    586         #       than the maximum context and use length-safe embedding function.
    587         engine = cast(str, self.deployment)
--> 588         return self._get_len_safe_embeddings(texts, engine=engine)
    589 
    590     async def aembed_documents(

/usr/local/lib/python3.10/dist-packages/langchain_openai/embeddings/base.py in _get_len_safe_embeddings(self, texts, engine, chunk_size)
    481         batched_embeddings: List[List[float]] = []
    482         for i in _iter:
--> 483             response = self.client.create(
    484                 input=tokens[i : i + _chunk_size], **self._invocation_params
    485             )

/usr/local/lib/python3.10/dist-packages/openai/resources/embeddings.py in create(self, input, model, dimensions, encoding_format, user, extra_headers, extra_query, extra_body, timeout)
    123             return obj
    124 
--> 125         return self._post(
    126             "/embeddings",
    127             body=maybe_transform(params, embedding_create_params.EmbeddingCreateParams),

/usr/local/lib/python3.10/dist-packages/openai/_base_client.py in post(self, path, cast_to, body, options, files, stream, stream_cls)
   1281             method="post", url=path, json_data=body, files=to_httpx_files(files), **options
   1282         )
-> 1283         return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
   1284 
   1285     def patch(

/usr/local/lib/python3.10/dist-packages/openai/_base_client.py in request(self, cast_to, options, remaining_retries, stream, stream_cls)
    958             retries_taken = 0
    959 
--> 960         return self._request(
    961             cast_to=cast_to,
    962             options=options,

/usr/local/lib/python3.10/dist-packages/openai/_base_client.py in _request(self, cast_to, options, retries_taken, stream, stream_cls)
   1062 
   1063             log.debug("Re-raising status error")
-> 1064             raise self._make_status_error_from_response(err.response) from None
   1065 
   1066         return self._process_response(

AuthenticationError: Error code: 401 - {'error': {'message': 'Incorrect API key provided: sk-d32da***********************45d0. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}

@Sara-Hossny can you try the latest version?

I  have tried with the latest version now and i found another error AssertionError: n_contexts_per_doc must be a positive integer. even I make chunk_size small or let it with the default 
I let embedding to be default with open ai embedding 

@kritinv is working on this, its a known problem. Does this happen consistently without fail?

@Sara-Hossny can you show your code because we're not able to reproduce it. Be best if you can share your docs as well

i have tried it again and it didn't reproduce with me ,thanks for your
assistance

On Sat, 25 Jan 2025 at 08:18, Jeffrey Ip ***@***.***> wrote:

> @Sara-Hossny <https://github.com/Sara-Hossny> can you show your code
> because we're not able to reproduce it. Be best if you can share your docs
> as well
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/confident-ai/deepeval/issues/1306#issuecomment-2613804816>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AYZVBBTACCKKJT6DTBXHBB32MMUC5AVCNFSM6AAAAABVWZ4CCOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMJTHAYDIOBRGY>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>


@Sara-Hossny no problem, let me know if the problem comes up again!