Repository: run-llama/llama_index
Pull request: [FIX] faulty imports when using PydanticProgramMode.LM_FORMAT_ENFORCER
# Description

- `PydanticProgramMode.LM_FORMAT_ENFORCER` needs integration `llama-index-program-lmformatenforcer` but when trying to lazy import it from core, it tries to load from ... core :)
- Also fixed an incorrect import for llama_utils

Fixes #11351 

## Type of Change

Please delete options that are not relevant.

- [x] Bug fix (non-breaking change which fixes an issue)

# How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

- [x] I stared at the code and made sure it makes sense


Problem statement:
[Bug]: ModuleNotFoundError: No module named 'llama_index.core.program.lmformatenforcer_program'
### Bug Description

Weird i faced this error,  unable to find a way to resolve. Please help thank you

```
llama-index==0.10.12
llama-index-llms-huggingface==0.1.3
llama-index-program-lmformatenforcer==0.1.2
```

### Version

0.10.12

### Steps to Reproduce

```
llm = HuggingFaceLLM(
    model_name="HuggingFaceH4/zephyr-7b-beta",
    tokenizer_name="HuggingFaceH4/zephyr-7b-beta",
    query_wrapper_prompt=PromptTemplate("
\n
\n
\n{query_str}
\n
\n"),
    context_window=6000,
    max_new_tokens=1000,
    model_kwargs={"quantization_config": quantization_config},
    # tokenizer_kwargs={},
    generate_kwargs={"temperature": 0.7, "top_k": 50, "top_p": 0.95},
    messages_to_prompt=messages_to_prompt,
    device_map="auto",
    pydantic_program_mode = PydanticProgramMode.LM_FORMAT_ENFORCER
)
```

### Relevant Logs/Tracbacks

_No response_

Hints from issue discussion:
_🤖_
<!-- Greeting -->
Hey there @GxTeo! Good to see you back. Running into another quirky corner of the code, I see? Let's dive in and untangle this one together.

<!-- Answer -->
Based on the error message you're encountering, it seems like the module 'llama_index.core.program.lmformatenforcer_program' cannot be found. However, based on the provided context, it seems like the module you're trying to import is not 'llama_index.core.program.lmformatenforcer_program', but rather 'lmformatenforcer'.

The 'lmformatenforcer' module is imported within the `__init__` method of the `LMFormatEnforcerPydanticProgram` class:

```py
try:
    import lmformatenforcer
except ImportError as e:
    raise ImportError(
        "lm-format-enforcer package not found."
        "please run `pip install lm-format-enforcer`"
    ) from e
```

This code is located in the [llama-index-integrations/program/llama-index-program-lmformatenforcer/llama_index/program/lmformatenforcer/base.py](https://github.com/run-llama/llama_index/blob/main/llama-index-integrations/program/llama-index-program-lmformatenforcer/llama_index/program/lmformatenforcer/base.py) file. If the 'lmformatenforcer' module cannot be found, an ImportError is raised with the message "lm-format-enforcer package not found. please run `pip install lm-format-enforcer`". This suggests that the 'lmformatenforcer' package may not be installed in your environment.

To resolve this issue, you can try installing the 'lm-format-enforcer' package by running the following command in your terminal:

```sh
pip install lm-format-enforcer
```

If you're still encountering the ModuleNotFoundError after installing the 'lm-format-enforcer' package, it's possible that there may be an issue with your Python environment or the way your project is structured. In that case, you may need to provide more information about your setup for further assistance.

Regarding your question about the 'llama_index.core.program.lmformatenforcer_program' module, it is a part of the LlamaIndex repository. This is indicated by the 'import_path' under the '[tool.llamahub]' section in the pyproject.toml file. The 'import_path' is set to "llama_index.program.lmformatenforcer", which suggests that the module is included within the LlamaIndex repository.

Additionally, the 'llama-index-core' is listed under the '[tool.poetry.dependencies]' section, which means it is a dependency for the project and is installed as part of the project setup.

```toml
[tool.llamahub]
import_path = "llama_index.program.lmformatenforcer"

[tool.poetry.dependencies]
llama-index-core = "^0.10.1"
```

This information can be found in the [llama-index-integrations/program/llama-index-program-lmformatenforcer/pyproject.toml](https://github.com/run-llama/llama_index/blob/main/llama-index-integrations/program/llama-index-program-lmformatenforcer/pyproject.toml) file.

## Sources
- [llama-index-integrations/program/llama-index-program-lmformatenforcer/llama_index/program/lmformatenforcer/base.py](https://github.com/run-llama/llama_index/blob/main/llama-index-integrations/program/llama-index-program-lmformatenforcer/llama_index/program/lmformatenforcer/base.py)
- [llama-index-integrations/program/llama-index-program-lmformatenforcer/pyproject.toml](https://github.com/run-llama/llama_index/blob/main/llama-index-integrations/program/llama-index-program-lmformatenforcer/pyproject.toml)

<!-- Dosu Comment Footer -->
<details open>
<summary>

#### About Dosu
</summary>

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.
</details>
Ah this one slipped thru the cracks. There's a faulty import in `core` here that needs to be fixed. Will submit a PR shortly.
@nerdai thank you! Can you help me with a different question though? Currently i want to use the `pydantic program mode` with the `LM_FORMAT_ENFORCER` and I facing an issue which I can't get my head around.
```
llm = HuggingFaceLLM(
    model_name="TheBloke/zephyr-7B-beta-GPTQ",
    tokenizer_name="TheBloke/zephyr-7B-beta-GPTQ",
    # query_wrapper_prompt=PromptTemplate("
\n
\n
\n{query_str}
\n
\n"),
    context_window=6000,
    max_new_tokens=1000,
    # tokenizer_kwargs={},
    generate_kwargs={"temperature": 0.7, "top_k": 50, "top_p": 0.95},
    messages_to_prompt=messages_to_prompt,
    device_map="auto",
    pydantic_program_mode = PydanticProgramMode.LM_FORMAT_ENFORCER
)
```

I understand that the default summary template for `TreeSummarize` is the `SelectorPromptTemplate`, which have the attribute `default_template`, and i have been getting the error, `AttributeError: 'SelectorPromptTemplate' object has no attribute 'template'`.  

Without the `LM_FORMAT_ENFORCER`, i didn't have the error.

```
DEFAULT_TREE_SUMMARIZE_PROMPT_SEL = SelectorPromptTemplate(
    default_template=DEFAULT_TREE_SUMMARIZE_PROMPT,
    conditionals=[(is_chat_model, CHAT_TREE_SUMMARIZE_PROMPT)],
)
```

This is how i am getting output for reference:

```
class Zephyr_ResumeScreener:
    def __init__(self, job_role:str, job_descriptions: List[str], job_requirements: List[str], llm: Optional[LLM] = None) -> None:
        Settings.llm = llm
        self.llm = llm
        self.synthesizer = TreeSummarize(output_cls=ResumeScreenerDecision, llm=self.llm)
        self.job_descriptions_str = _format_descriptions_str(job_descriptions)
        self.job_requirements_str = _format_requirements_str(job_requirements)
        self.job_role = job_role

        self.query = QUERY_TEMPLATE.format(
            job_role = self.job_role, job_description=self.job_descriptions_str, job_requirements=self.job_requirements_str
        )

    def run(self, resume_path: str) -> Any:
        docs =SimpleDirectoryReader(input_files=resume_path, filename_as_id=True).load_data()
        output = self.synthesizer.synthesize(
            query=self.query,
            nodes=[NodeWithScore(node=doc, score=1.0) for doc in docs],
        )
        return output.response
```

Commit message: fix faulty imports

Patch to review:
diff --git a/llama-index-core/llama_index/core/llms/utils.py b/llama-index-core/llama_index/core/llms/utils.py
index ecb388cd18..1c87dd38ca 100644
--- a/llama-index-core/llama_index/core/llms/utils.py
+++ b/llama-index-core/llama_index/core/llms/utils.py
@@ -64,7 +64,7 @@ def resolve_llm(
                 "llm must start with str 'local' or of type LLM or BaseLanguageModel"
             )
         try:
-            from llama_index.llms.llama.utils import (
+            from llama_index.llms.llama_cpp.llama_utils import (
                 completion_to_prompt,
                 messages_to_prompt,
             )  # pants: no-infer-dep
diff --git a/llama-index-core/llama_index/core/program/utils.py b/llama-index-core/llama_index/core/program/utils.py
index fa60e5734b..050e2e63ea 100644
--- a/llama-index-core/llama_index/core/program/utils.py
+++ b/llama-index-core/llama_index/core/program/utils.py
@@ -90,9 +90,15 @@ def get_program_for_llm(
             **kwargs,
         )
     elif pydantic_program_mode == PydanticProgramMode.LM_FORMAT_ENFORCER:
-        from llama_index.core.program.lmformatenforcer_program import (
-            LMFormatEnforcerPydanticProgram,
-        )
+        try:
+            from llama_index.program.lmformatenforcer import (
+                LMFormatEnforcerPydanticProgram,
+            )
+        except ImportError:
+            raise ImportError(
+                "This mode requires the `llama-index-program-lmformatenforcer package. Please"
+                " install it by running `pip install llama-index-program-lmformatenforcer`."
+            )
 
         return LMFormatEnforcerPydanticProgram.from_defaults(
             output_cls=output_cls,


Code review comments:
diff_hunk: @@ -90,9 +90,15 @@ def get_program_for_llm(
             **kwargs,
         )
     elif pydantic_program_mode == PydanticProgramMode.LM_FORMAT_ENFORCER:
-        from llama_index.core.program.lmformatenforcer_program import (
-            LMFormatEnforcerPydanticProgram,
-        )
+        try:
+            from llama_index.program.lmformatenforcer import (
+                LMFormatEnforcerPydanticProgram,
+            )
path: llama-index-core/llama_index/core/program/utils.py
text: @user1:
Can we add the `  # pants: no-infer-dep` comment at the end of this import? (Similar to other 3rd party imports in core)

This helps avoid pants detecting this as a dependency when testing, making the venv smaller for core

@author:
sorry just seeing this now. on it!

@author:
that did the trick - all checks have passed now :)