Repository: confident-ai/deepeval
Issue: ValueError: Evaluation LLM outputted an invalid JSON. Please use a better evaluation model--- while trying to run synthesizer with Ollama model
Root cause: Interface Contract Mismatch

while trying to tun the synthesizer for generate_goldens_from_docs as per the documentation https://docs.confident-ai.com/docs/evaluation-datasets-synthetic-data#creating-an-synthesizer, facing an issue with Ollama ( phi,llama 3.1 etc..)
Where as it is working for azure.

Traceback (most recent call last):
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\deepeval\synthesizer\synthesizer.py", line 117, in generate_synthetic_inputs
    **res: SyntheticDataList = self.model.generate(
TypeError: generate() got an unexpected keyword argument 'schema'**

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\deepeval\metrics\utils.py", line 153, in trimAndLoadJson
    return json.loads(jsonStr)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\json\decoder.py", line 340, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 1 column 42 (char 41)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\uvicorn\protocols\http\httptools_impl.py", line 401, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 70, in __call__
    return await self.app(scope, receive, send)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\fastapi\applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\starlette\applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\starlette\middleware\errors.py", line 186, in __call__
    raise exc
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\starlette\middleware\errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\starlette\middleware\cors.py", line 93, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\starlette\middleware\cors.py", line 148, in simple_response
    await self.app(scope, receive, send)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\starlette\middleware\exceptions.py", line 65, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\starlette\_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\starlette\_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\starlette\routing.py", line 754, in __call__
    await self.middleware_stack(scope, receive, send)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\starlette\routing.py", line 774, in app  
    await route.handle(scope, receive, send)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\starlette\routing.py", line 295, in handle
    await self.app(scope, receive, send)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\starlette\routing.py", line 77, in app   
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\starlette\_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\starlette\_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\starlette\routing.py", line 74, in app   
    response = await f(request)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\fastapi\routing.py", line 278, in app    
    raw_response = await run_endpoint_function(
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\fastapi\routing.py", line 193, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\starlette\concurrency.py", line 42, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
  File "C:\Users\sowvinna1\OneDrive - Publicis Groupe\SGAI\minervaa-deepeval-synthesizer-api\src\main\python\app\main.py", line 40, in synthesize_data
    dataset = synthesize(synthesize_request)
  File "C:\Users\sowvinna1\OneDrive - Publicis Groupe\SGAI\minervaa-deepeval-synthesizer-api\src\main\python\services\data_synthesizer.py", line 47, in synthesize
    dataset = dataset_service.generate_dataset_from_docs(
  File "C:\Users\sowvinna1\OneDrive - Publicis Groupe\SGAI\minervaa-deepeval-synthesizer-api\src\main\python\deepeval_agent\datasets.py", line 82, in generate_dataset_from_docs
    dataset.generate_goldens_from_docs(
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\deepeval\dataset\dataset.py", line 536, in generate_goldens_from_docs
    synthesizer.generate_goldens_from_docs(
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\deepeval\synthesizer\synthesizer.py", line 1025, in generate_goldens_from_docs
    return self.generate_goldens(
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\deepeval\synthesizer\synthesizer.py", line 844, in generate_goldens
    synthetic_data = self.generate_synthetic_inputs(prompt)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\deepeval\synthesizer\synthesizer.py", line 123, in generate_synthetic_inputs
    data = trimAndLoadJson(res, self)
  File "c:\users\sowvinna1\appdata\local\programs\python\python39\lib\site-packages\deepeval\metrics\utils.py", line 158, in trimAndLoadJson
    raise ValueError(error_str)
**ValueError: Evaluation LLM outputted an invalid JSON. Please use a better evaluation model.**


code snippet:
![image](https://github.com/user-attachments/assets/2c5b1db7-3757-470c-86b9-82060567baba)
this is inside generate_synthetic_inputs() function, in deepeval.synthesizer

hey @vsowmya105 did you follow this guide? https://docs.confident-ai.com/docs/guides-using-custom-llms

> hey [@vsowmya105](https://github.com/vsowmya105) did you follow this guide? [docs.confident-ai.com/docs/guides-using-custom-llms](https://docs.confident-ai.com/docs/guides-using-custom-llms)

now being here: https://deepeval.com/guides/guides-using-custom-llms#json-confinement-for-custom-llms