Repository: stanfordnlp/dspy
Pull request #1682 (2024-10-23 19:52:29)

Fix error in Evaluate with display_table=True with outputs that cannot be converted to dict
If I write a DSPy module that returns an object that is not coercable to `dict`, I cannot currently call `Evaluate()` on the module with `display_table=True` because `display_table()` assumes that all outputs are dictionaries. This PR fixes the issue and adds a test.


Patch:
diff --git a/docs/docs/deep-dive/modules/assertions.md b/docs/docs/deep-dive/modules/assertions.md
deleted file mode 100644
index c0ba89df..00000000
--- a/docs/docs/deep-dive/modules/assertions.md
+++ /dev/null
@@ -1,267 +0,0 @@
----
-sidebar_position: 2
----
-
-# DSPy Assertions 
-
-## Introduction
-
-Language models (LMs) have transformed how we interact with machine learning, offering vast capabilities in natural language understanding and generation. However, ensuring these models adhere to domain-specific constraints remains a challenge. Despite the growth of techniques like fine-tuning or “prompt engineering”, these approaches are extremely tedious and rely on heavy, manual hand-waving to guide the LMs in adhering to specific constraints. Even DSPy's modularity of programming prompting pipelines lacks mechanisms to effectively and automatically enforce these constraints. 
-
-To address this, we introduce DSPy Assertions, a feature within the DSPy framework designed to automate the enforcement of computational constraints on LMs. DSPy Assertions empower developers to guide LMs towards desired outcomes with minimal manual intervention, enhancing the reliability, predictability, and correctness of LM outputs.
-
-### dspy.Assert and dspy.Suggest API    
-
-We introduce two primary constructs within DSPy Assertions:
-
-- **`dspy.Assert`**:
-  - **Parameters**: 
-    - `constraint (bool)`: Outcome of Python-defined boolean validation check.
-    - `msg (Optional[str])`: User-defined error message providing feedback or correction guidance.
-    - `backtrack (Optional[module])`: Specifies target module for retry attempts upon constraint failure. The default backtracking module is the last module before the assertion.
-  - **Behavior**: Initiates retry  upon failure, dynamically adjusting the pipeline's execution. If failures persist, it halts execution and raises a `dspy.AssertionError`.
-
-- **`dspy.Suggest`**:
-  - **Parameters**: Similar to `dspy.Assert`.
-  - **Behavior**: Encourages self-refinement through retries without enforcing hard stops. Logs failures after maximum backtracking attempts and continues execution.
-
-- **dspy.Assert vs. Python Assertions**: Unlike conventional Python `assert` statements that terminate the program upon failure, `dspy.Assert` conducts a sophisticated retry mechanism, allowing the pipeline to adjust. 
-
-Specifically, when a constraint is not met:
-
-- Backtracking Mechanism: An under-the-hood backtracking is initiated, offering the model a chance to self-refine and proceed, which is done through
-- Dynamic Signature Modification: internally modifying your DSPy program’s Signature by adding the following fields:
-    - Past Output: your model's past output that did not pass the validation_fn
-    - Instruction: your user-defined feedback message on what went wrong and what possibly to fix
-
-If the error continues past the `max_backtracking_attempts`, then `dspy.Assert` will halt the pipeline execution, altering you with an `dspy.AssertionError`. This ensures your program doesn't continue executing with “bad” LM behavior and immediately highlights sample failure outputs for user assessment. 
-
-- **dspy.Suggest vs. dspy.Assert**: `dspy.Suggest` on the other hand offers a softer approach. It maintains the same retry backtracking as `dspy.Assert` but instead serves as a gentle nudger. If the model outputs cannot pass the model constraints after the `max_backtracking_attempts`, `dspy.Suggest` will log the persistent failure and continue execution of the program on the rest of the data. This ensures the LM pipeline works in a "best-effort" manner without halting execution. 
-
-- **`dspy.Suggest`** are best utilized as "helpers" during the evaluation phase, offering guidance and potential corrections without halting the pipeline.
-- **`dspy.Assert`** are recommended during the development stage as "checkers" to ensure the LM behaves as expected, providing a robust mechanism for identifying and addressing errors early in the development cycle.
-
-
-## Use Case: Including Assertions in DSPy Programs
-
-We start with using an example of a multi-hop QA SimplifiedBaleen pipeline as defined in the intro walkthrough. 
-
-```python
-class SimplifiedBaleen(dspy.Module):
-    def __init__(self, passages_per_hop=2, max_hops=2):
-        super().__init__()
-
-        self.generate_query = [dspy.ChainOfThought(GenerateSearchQuery) for _ in range(max_hops)]
-        self.retrieve = dspy.Retrieve(k=passages_per_hop)
-        self.generate_answer = dspy.ChainOfThought(GenerateAnswer)
-        self.max_hops = max_hops
-
-    def forward(self, question):
-        context = []
-        prev_queries = [question]
-
-        for hop in range(self.max_hops):
-            query = self.generate_query[hop](context=context, question=question).query
-            prev_queries.append(query)
-            passages = self.retrieve(query).passages
-            context = deduplicate(context + passages)
-        
-        pred = self.generate_answer(context=context, question=question)
-        pred = dspy.Prediction(context=context, answer=pred.answer)
-        return pred
-
-baleen = SimplifiedBaleen()
-
-baleen(question = "Which award did Gary Zukav's first book receive?")
-```
-
-To include DSPy Assertions, we simply define our validation functions and declare our assertions following the respective model generation. 
-
-For this use case, suppose we want to impose the following constraints:
-    1. Length - each query should be less than 100 characters
-    2. Uniqueness - each generated query should differ from previously-generated queries. 
-    
-We can define these validation checks as boolean functions:
-
-```python
-#simplistic boolean check for query length
-len(query) <= 100
-
-#Python function for validating distinct queries
-def validate_query_distinction_local(previous_queries, query):
-    """check if query is distinct from previous queries"""
-    if previous_queries == []:
-        return True
-    if dspy.evaluate.answer_exact_match_str(query, previous_queries, frac=0.8):
-        return False
-    return True
-```
-
-We can declare these validation checks through `dspy.Suggest` statements (as we want to test the program in a best-effort demonstration). We want to keep these after the query generation `query = self.generate_query[hop](context=context, question=question).query`.
-
-```python
-dspy.Suggest(
-    len(query) <= 100,
-    "Query should be short and less than 100 characters",
-    target_module=self.generate_query
-)
-
-dspy.Suggest(
-    validate_query_distinction_local(prev_queries, query),
-    "Query should be distinct from: "
-    + "; ".join(f"{i+1}) {q}" for i, q in enumerate(prev_queries)),
-    target_module=self.generate_query
-)
-```
-
-It is recommended to define a program with assertions separately than your original program if you are doing comparative evaluation for the effect of assertions. If not, feel free to set Assertions away!
-
-Let's take a look at how the SimplifiedBaleen program will look with Assertions included:
-
-```python
-class SimplifiedBaleenAssertions(dspy.Module):
-    def __init__(self, passages_per_hop=2, max_hops=2):
-        super().__init__()
-        self.generate_query = [dspy.ChainOfThought(GenerateSearchQuery) for _ in range(max_hops)]
-        self.retrieve = dspy.Retrieve(k=passages_per_hop)
-        self.generate_answer = dspy.ChainOfThought(GenerateAnswer)
-        self.max_hops = max_hops
-
-    def forward(self, question):
-        context = []
-        prev_queries = [question]
-
-        for hop in range(self.max_hops):
-            query = self.generate_query[hop](context=context, question=question).query
-
-            dspy.Suggest(
-                len(query) <= 100,
-                "Query should be short and less than 100 characters",
-                target_module=self.generate_query
-            )
-
-            dspy.Suggest(
-                validate_query_distinction_local(prev_queries, query),
-                "Query should be distinct from: "
-                + "; ".join(f"{i+1}) {q}" for i, q in enumerate(prev_queries)),
-                target_module=self.generate_query
-            )
-
-            prev_queries.append(query)
-            passages = self.retrieve(query).passages
-            context = deduplicate(context + passages)
-        
-        if all_queries_distinct(prev_queries):
-            self.passed_suggestions += 1
-
-        pred = self.generate_answer(context=context, question=question)
-        pred = dspy.Prediction(context=context, answer=pred.answer)
-        return pred
-```
-
-Now calling programs with DSPy Assertions requires one last step, and that is transforming the program to wrap it with internal assertions backtracking and Retry logic. 
-
-```python
-from dspy.primitives.assertions import assert_transform_module, backtrack_handler
-
-baleen_with_assertions = assert_transform_module(SimplifiedBaleenAssertions(), backtrack_handler)
-
-# backtrack_handler is parameterized over a few settings for the backtracking mechanism
-# To change the number of max retry attempts, you can do
-baleen_with_assertions_retry_once = assert_transform_module(SimplifiedBaleenAssertions(), 
-    functools.partial(backtrack_handler, max_backtracks=1))
-```
-
-Alternatively, you can also directly call `activate_assertions` on the program with `dspy.Assert/Suggest` statements using the default backtracking mechanism (`max_backtracks=2`):
-
-```python
-baleen_with_assertions = SimplifiedBaleenAssertions().activate_assertions()
-```
-
-Now let's take a look at the internal LM backtracking by inspecting the history of the LM query generations. Here we see that when a query fails to pass the validation check of being less than 100 characters, its internal `GenerateSearchQuery` signature is dynamically modified during the backtracking+Retry process to include the past query and the corresponding user-defined instruction: `"Query should be short and less than 100 characters"`.
-
-
-```text
-Write a simple search query that will help answer a complex question.
-
----
-
-Follow the following format.
-
-Context: may contain relevant facts
-
-Question: ${question}
-
-Reasoning: Let's think step by step in order to ${produce the query}. We ...
-
-Query: ${query}
-
----
-
-Context:
-[1] «Kerry Condon | Kerry Condon (born 4 January 1983) is [...]»
-[2] «Corona Riccardo | Corona Riccardo (c. 1878October 15, 1917) was [...]»
-
-Question: Who acted in the shot film The Shore and is also the youngest actress ever to play Ophelia in a Royal Shakespeare Company production of "Hamlet." ?
-
-Reasoning: Let's think step by step in order to find the answer to this question. First, we need to identify the actress who played Ophelia in a Royal Shakespeare Company production of "Hamlet." Then, we need to find out if this actress also acted in the short film "The Shore."
-
-Query: "actress who played Ophelia in Royal Shakespeare Company production of Hamlet" + "actress in short film The Shore"
-
-
-
-Write a simple search query that will help answer a complex question.
-
----
-
-Follow the following format.
-
-Context: may contain relevant facts
-
-Question: ${question}
-
-Past Query: past output with errors
-
-Instructions: Some instructions you must satisfy
-
-Query: ${query}
-
----
-
-Context:
-[1] «Kerry Condon | Kerry Condon (born 4 January 1983) is an Irish television and film actress, best known for her role as Octavia of the Julii in the HBO/BBC series "Rome," as Stacey Ehrmantraut in AMC's "Better Call Saul" and as the voice of F.R.I.D.A.Y. in various films in the Marvel Cinematic Universe. She is also the youngest actress ever to play Ophelia in a Royal Shakespeare Company production of "Hamlet."»
-[2] «Corona Riccardo | Corona Riccardo (c. 1878October 15, 1917) was an Italian born American actress who had a brief Broadway stage career before leaving to become a wife and mother. Born in Naples she came to acting in 1894 playing a Mexican girl in a play at the Empire Theatre. Wilson Barrett engaged her for a role in his play "The Sign of the Cross" which he took on tour of the United States. Riccardo played the role of Ancaria and later played Berenice in the same play. Robert B. Mantell in 1898 who struck by her beauty also cast her in two Shakespeare plays, "Romeo and Juliet" and "Othello". Author Lewis Strang writing in 1899 said Riccardo was the most promising actress in America at the time. Towards the end of 1898 Mantell chose her for another Shakespeare part, Ophelia im Hamlet. Afterwards she was due to join Augustin Daly's Theatre Company but Daly died in 1899. In 1899 she gained her biggest fame by playing Iras in the first stage production of Ben-Hur.»
-
-Question: Who acted in the shot film The Shore and is also the youngest actress ever to play Ophelia in a Royal Shakespeare Company production of "Hamlet." ?
-
-Past Query: "actress who played Ophelia in Royal Shakespeare Company production of Hamlet" + "actress in short film The Shore"
-
-Instructions: Query should be short and less than 100 characters
-
-Query: "actress Ophelia RSC Hamlet" + "actress The Shore"
-
-```
-
-
-## Assertion-Driven Optimizations
-
-DSPy Assertions work with optimizations that DSPy offers, particularly with `BootstrapFewShotWithRandomSearch`, including the following settings:
-
-- Compilation with Assertions
-    This includes assertion-driven example bootstrapping and counterexample bootstrapping during compilation. The teacher model for bootstrapping few-shot demonstrations can make use of DSPy Assertions to offer robust bootstrapped examples for the student model to learn from during inference. In this setting, the student model does not perform assertion aware optimizations (backtracking and retry) during inference.
-- Compilation + Inference with Assertions
-    -This includes assertion-driven optimizations in both compilation and inference. Now the teacher model offers assertion-driven examples but the student can further optimize with assertions of its own during inference time. 
-```python
-teleprompter = BootstrapFewShotWithRandomSearch(
-    metric=validate_context_and_answer_and_hops,
-    max_bootstrapped_demos=max_bootstrapped_demos,
-    num_candidate_programs=6,
-)
-
-#Compilation with Assertions
-compiled_with_assertions_baleen = teleprompter.compile(student = baleen, teacher = baleen_with_assertions, trainset = trainset, valset = devset)
-
-#Compilation + Inference with Assertions
-compiled_baleen_with_assertions = teleprompter.compile(student=baleen_with_assertions, teacher = baleen_with_assertions, trainset=trainset, valset=devset)
-
-```
\ No newline at end of file
diff --git a/docs/docs/deep-dive/optimizers/_category_.json b/docs/docs/deep-dive/optimizers/_category_.json
index c7be633a..35f07a1e 100644
--- a/docs/docs/deep-dive/optimizers/_category_.json
+++ b/docs/docs/deep-dive/optimizers/_category_.json
@@ -1,8 +1,8 @@
 {
-    "label": "Teleprompters",
+    "label": "Optimizers",
     "position": 7,
     "link": {
       "type": "generated-index",
-      "description": "Teleprompters are powerful optimizers (included in DSPy) that can learn to bootstrap and select effective prompts for the modules of any program. (The \"tele-\" in the name means \"at a distance\", i.e., automatic prompting at a distance.)"
+      "description": "Optimizers are powerful constructs (included in DSPy) that can learn to bootstrap and select effective prompts for the modules of any program."
     }
 }
\ No newline at end of file
diff --git a/docs/docs/deep-dive/signature/understanding-signatures.md b/docs/docs/deep-dive/signature/understanding-signatures.md
index 0c14fd35..c3c02881 100644
--- a/docs/docs/deep-dive/signature/understanding-signatures.md
+++ b/docs/docs/deep-dive/signature/understanding-signatures.md
@@ -2,8 +2,6 @@
 sidebar_position: 1
 ---
 
-import AuthorDetails from '@site/src/components/AuthorDetails';
-
 # Understanding Signatures
 
 A DSPy Signature is the most basic form of task description which simply requires inputs and outputs and optionally, a small description about them and the task too.
@@ -119,5 +117,3 @@ As you can see, the instruction is more well-defined by our task's instruction i
 ![Class Based Prompt Creation](./img/class_based_prompt_creation.png)
 
 ***
-
-<AuthorDetails name="Herumb Shandilya"/>
\ No newline at end of file
diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml
index 5491bd69..ff0f9bfd 100644
--- a/docs/mkdocs.yml
+++ b/docs/mkdocs.yml
@@ -7,12 +7,11 @@ edit_uri: edit/main/docs/docs/
 docs_dir: 'docs/'
 nav:
   - About DSPy: intro.md
-  - Docs:
+  - Getting Started:
     - Quick Start:
       - Installation: quick-start/installation.md
       - Getting Started I: quick-start/getting-started-01.md
       - Getting Started II: quick-start/getting-started-02.md
-
     - Building Blocks:
       - DSPy in 8 Steps: building-blocks/solving_your_task.md
       - Language Models: building-blocks/1-language_models.md
@@ -29,7 +28,7 @@ nav:
     - Summarization: tutorials/summarization.md
     - Additional Resources: tutorials/other_tutorial.md
     - Examples: tutorials/examples.md
-  - Deep Dive:
+  - Concepts:
     - Data Handling:
       - Examples in DSPy: deep-dive/data-handling/examples.md
       - Utilizing Built-in Datasets: deep-dive/data-handling/built-in-datasets.md
@@ -47,6 +46,15 @@ nav:
       - Assertions: deep-dive/modules/assertions.md
       - Retreive: deep-dive/modules/retrieve.md
       - Modules Guide: deep-dive/modules/guide.md
+    - Optimizers (formerly Teleprompters):
+      - LabeledFewShot: deep-dive/optimizers/LabeledFewShot.md
+      - BootstrapFewShot: deep-dive/optimizers/bootstrap-fewshot.md
+      - BootstrapFinetune: deep-dive/optimizers/BootstrapFinetune.md
+      - Ensemble: deep-dive/optimizers/Ensemble.md
+      - BFRS: deep-dive/optimizers/bfrs.md
+      - CoPro: deep-dive/optimizers/copro.md
+      - MIProV2: deep-dive/optimizers/miprov2.md
+    - Assertions: deep-dive/assertions.md
     - Language Model Clients:
       - Local Language Model Clients:
         - HFClientTGI: deep-dive/language_model_clients/lm_local_models/HFClientTGI.md
@@ -72,15 +80,6 @@ nav:
       - WatsonDiscovery: deep-dive/retrieval_models_clients/WatsonDiscovery.md
       - WeaviateRM: deep-dive/retrieval_models_clients/WeaviateRM.md
       - YouRM: deep-dive/retrieval_models_clients/YouRM.md
-    - Assertions: deep-dive/assertions.md
-    - Teleprompters:
-      - LabeledFewShot: deep-dive/optimizers/LabeledFewShot.md
-      - BootstrapFewShot: deep-dive/optimizers/bootstrap-fewshot.md
-      - BootstrapFinetune: deep-dive/optimizers/BootstrapFinetune.md
-      - Ensemble: deep-dive/optimizers/Ensemble.md
-      - BFRS: deep-dive/optimizers/bfrs.md
-      - CoPro: deep-dive/optimizers/copro.md
-      - MIProV2: deep-dive/optimizers/miprov2.md
   - FAQ: faqs.md
   - Cheatsheet: cheatsheet.md
   - DSPy Usecases: dspy-usecases.md
@@ -129,12 +128,98 @@ plugins:
   - redirects:
       redirect_maps:
         'index.md': 'intro.md'
+        # Intro page redirects
+        'docs/intro.md': 'intro.md'
+        'docs/intro/index.md': 'intro.md'
+
+        
+        # Quick Start redirects
+        'docs/quick-start/installation.md': 'quick-start/installation.md'
+        'docs/quick-start/getting-started-01.md': 'quick-start/getting-started-01.md'
+        'docs/quick-start/getting-started-02.md': 'quick-start/getting-started-02.md'
+        
+        # Building Blocks redirects
+        'docs/building-blocks/solving_your_task.md': 'building-blocks/solving_your_task.md'
+        'docs/building-blocks/1-language_models.md': 'building-blocks/1-language_models.md'
+        'docs/building-blocks/2-signatures.md': 'building-blocks/2-signatures.md'
+        'docs/building-blocks/3-modules.md': 'building-blocks/3-modules.md'
+        'docs/building-blocks/4-data.md': 'building-blocks/4-data.md'
+        'docs/building-blocks/5-metrics.md': 'building-blocks/5-metrics.md'
+        'docs/building-blocks/6-optimizers.md': 'building-blocks/6-optimizers.md'
+        'docs/building-blocks/7-assertions.md': 'building-blocks/7-assertions.md'
+        'docs/building-blocks/8-typed_predictors.md': 'building-blocks/8-typed_predictors.md'
+        
+        # Tutorials redirects
         'docs/tutorials/rag.md': 'tutorials/rag.md'
         'docs/tutorials/simplified-baleen.md': 'tutorials/simplified-baleen.md'
         'docs/tutorials/summarization.md': 'tutorials/summarization.md'
         'docs/tutorials/other_tutorial.md': 'tutorials/other_tutorial.md'
         'docs/tutorials/examples.md': 'tutorials/examples.md'
         
+        # Deep Dive redirects
+        # Data Handling
+        'docs/deep-dive/data-handling/examples.md': 'deep-dive/data-handling/examples.md'
+        'docs/deep-dive/data-handling/built-in-datasets.md': 'deep-dive/data-handling/built-in-datasets.md'
+        'docs/deep-dive/data-handling/loading-custom-data.md': 'deep-dive/data-handling/loading-custom-data.md'
+        
+        # Signatures
+        'docs/deep-dive/signature/understanding-signatures.md': 'deep-dive/signature/understanding-signatures.md'
+        'docs/deep-dive/signature/executing-signatures.md': 'deep-dive/signature/executing-signatures.md'
+        
+        # Modules
+        'docs/deep-dive/modules/predict.md': 'deep-dive/modules/predict.md'
+        'docs/deep-dive/modules/chain-of-thought.md': 'deep-dive/modules/chain-of-thought.md'
+        'docs/deep-dive/modules/chain-of-thought-with-hint.md': 'deep-dive/modules/chain-of-thought-with-hint.md'
+        'docs/deep-dive/modules/react.md': 'deep-dive/modules/react.md'
+        'docs/deep-dive/modules/multi-chain-comparison.md': 'deep-dive/modules/multi-chain-comparison.md'
+        'docs/deep-dive/modules/program-of-thought.md': 'deep-dive/modules/program-of-thought.md'
+        'docs/deep-dive/modules/assertions.md': 'building-blocks/7-assertions.md'
+        'deep-dive/modules/assertions.md': 'building-blocks/7-assertions.md'
+        'docs/deep-dive/modules/retrieve.md': 'deep-dive/modules/retrieve.md'
+        'docs/deep-dive/modules/guide.md': 'deep-dive/modules/guide.md'
+        
+        # Language Model Clients
+        'docs/deep-dive/language_model_clients/lm_local_models/HFClientTGI.md': 'deep-dive/language_model_clients/lm_local_models/HFClientTGI.md'
+        'docs/deep-dive/language_model_clients/lm_local_models/HFClientVLLM.md': 'deep-dive/language_model_clients/lm_local_models/HFClientVLLM.md'
+        'docs/deep-dive/language_model_clients/lm_local_models/MLC.md': 'deep-dive/language_model_clients/lm_local_models/MLC.md'
+        'docs/deep-dive/language_model_clients/lm_local_models/Ollama.md': 'deep-dive/language_model_clients/lm_local_models/Ollama.md'
+        'docs/deep-dive/language_model_clients/lm_local_models/LlamaCpp.md': 'deep-dive/language_model_clients/lm_local_models/LlamaCpp.md'
+        'docs/deep-dive/language_model_clients/lm_local_models/TensorRTLLM.md': 'deep-dive/language_model_clients/lm_local_models/TensorRTLLM.md'
+        
+        # Retrieval Model Clients
+        'docs/deep-dive/retrieval_models_clients/Azure.md': 'deep-dive/retrieval_models_clients/Azure.md'
+        'docs/deep-dive/retrieval_models_clients/ChromadbRM.md': 'deep-dive/retrieval_models_clients/ChromadbRM.md'
+        'docs/deep-dive/retrieval_models_clients/ColBERTv2.md': 'deep-dive/retrieval_models_clients/ColBERTv2.md'
+        'docs/deep-dive/retrieval_models_clients/custom-rm-client.md': 'deep-dive/retrieval_models_clients/custom-rm-client.md'
+        'docs/deep-dive/retrieval_models_clients/DatabricksRM.md': 'deep-dive/retrieval_models_clients/DatabricksRM.md'
+        'docs/deep-dive/retrieval_models_clients/FaissRM.md': 'deep-dive/retrieval_models_clients/FaissRM.md'
+        'docs/deep-dive/retrieval_models_clients/LancedbRM.md': 'deep-dive/retrieval_models_clients/LancedbRM.md'
+        'docs/deep-dive/retrieval_models_clients/MilvusRM.md': 'deep-dive/retrieval_models_clients/MilvusRM.md'
+        'docs/deep-dive/retrieval_models_clients/MyScaleRM.md': 'deep-dive/retrieval_models_clients/MyScaleRM.md'
+        'docs/deep-dive/retrieval_models_clients/Neo4jRM.md': 'deep-dive/retrieval_models_clients/Neo4jRM.md'
+        'docs/deep-dive/retrieval_models_clients/QdrantRM.md': 'deep-dive/retrieval_models_clients/QdrantRM.md'
+        'docs/deep-dive/retrieval_models_clients/RAGatouilleRM.md': 'deep-dive/retrieval_models_clients/RAGatouilleRM.md'
+        'docs/deep-dive/retrieval_models_clients/SnowflakeRM.md': 'deep-dive/retrieval_models_clients/SnowflakeRM.md'
+        'docs/deep-dive/retrieval_models_clients/WatsonDiscovery.md': 'deep-dive/retrieval_models_clients/WatsonDiscovery.md'
+        'docs/deep-dive/retrieval_models_clients/WeaviateRM.md': 'deep-dive/retrieval_models_clients/WeaviateRM.md'
+        'docs/deep-dive/retrieval_models_clients/YouRM.md': 'deep-dive/retrieval_models_clients/YouRM.md'
+        
+        # Assertions
+        'docs/deep-dive/assertions.md': 'deep-dive/assertions.md'
+        
+        # Teleprompters
+        'docs/deep-dive/optimizers/LabeledFewShot.md': 'deep-dive/optimizers/LabeledFewShot.md'
+        'docs/deep-dive/optimizers/bootstrap-fewshot.md': 'deep-dive/optimizers/bootstrap-fewshot.md'
+        'docs/deep-dive/optimizers/BootstrapFinetune.md': 'deep-dive/optimizers/BootstrapFinetune.md'
+        'docs/deep-dive/optimizers/Ensemble.md': 'deep-dive/optimizers/Ensemble.md'
+        'docs/deep-dive/optimizers/bfrs.md': 'deep-dive/optimizers/bfrs.md'
+        'docs/deep-dive/optimizers/copro.md': 'deep-dive/optimizers/copro.md'
+        'docs/deep-dive/optimizers/miprov2.md': 'deep-dive/optimizers/miprov2.md'
+        
+        # Additional pages redirects
+        'docs/faqs.md': 'faqs.md'
+        'docs/cheatsheet.md': 'cheatsheet.md'
+        'docs/dspy-usecases.md': 'dspy-usecases.md'
 
 extra:
   social:
diff --git a/dspy/evaluate/evaluate.py b/dspy/evaluate/evaluate.py
index 64f8f8cc..4ca43ad3 100644
--- a/dspy/evaluate/evaluate.py
+++ b/dspy/evaluate/evaluate.py
@@ -226,8 +226,20 @@ class Evaluate:
         if return_outputs:  # Handle the return_outputs logic
             results = [(example, prediction, score) for _, example, prediction, score in predicted_devset]
 
+        def prediction_is_dictlike(prediction):
+            try:
+                dict(prediction)
+                return True
+            except Exception:
+                return False
+
         data = [
-            merge_dicts(example, prediction) | {"correct": score} for _, example, prediction, score in predicted_devset
+            (
+                merge_dicts(example, prediction) | {"correct": score}
+                if prediction_is_dictlike(prediction)
+                else dict(example) | {"prediction": prediction, "correct": score}
+            )
+            for _, example, prediction, score in predicted_devset
         ]
 
         result_df = pd.DataFrame(data)


Test patch:
diff --git a/tests/evaluate/test_evaluate.py b/tests/evaluate/test_evaluate.py
index a8552c7a..7615c70b 100644
--- a/tests/evaluate/test_evaluate.py
+++ b/tests/evaluate/test_evaluate.py
@@ -8,6 +8,7 @@ import dsp
 import dspy
 from dspy.evaluate.evaluate import Evaluate
 from dspy.evaluate.metrics import answer_exact_match
+from dspy.functional import TypedPredictor
 from dspy.predict import Predict
 from dspy.utils.dummies import DummyLM
 
@@ -120,14 +121,38 @@ def test_evaluate_call_bad():
     assert score == 0.0
 
 
+@pytest.mark.parametrize(
+    "program_with_example",
+    [
+        (Predict("question -> answer"), new_example("What is 1+1?", "2")),
+        (
+            # Create a program that extracts entities from text and returns them as a list,
+            # rather than returning a Predictor() wrapper. This is done intentionally to test
+            # the case where the program does not output a dictionary-like object because
+            # Evaluate() has failed for this case in the past
+            lambda text: TypedPredictor("text: str -> entities: List[str]")(text=text).entities,
+            dspy.Example(text="United States", entities=["United States"]).with_inputs("text"),
+        ),
+    ],
+)
 @pytest.mark.parametrize("display_table", [True, False, 1])
 @pytest.mark.parametrize("is_in_ipython_notebook_environment", [True, False])
-def test_evaluate_display_table(display_table, is_in_ipython_notebook_environment, capfd):
-    devset = [new_example("What is 1+1?", "2")]
-    program = Predict("question -> answer")
+def test_evaluate_display_table(program_with_example, display_table, is_in_ipython_notebook_environment, capfd):
+    program, example = program_with_example
+    example_input = next(iter(example.inputs().values()))
+    example_output = {key: value for key, value in example.toDict().items() if key not in example.inputs()}
+
+    dspy.settings.configure(
+        lm=DummyLM(
+            {
+                example_input: example_output,
+            }
+        )
+    )
+
     ev = Evaluate(
-        devset=devset,
-        metric=answer_exact_match,
+        devset=[example],
+        metric=lambda example, pred, **kwargs: example == pred,
         display_table=display_table,
     )
     assert ev.display_table == display_table
@@ -140,4 +165,5 @@ def test_evaluate_display_table(display_table, is_in_ipython_notebook_environmen
         if not is_in_ipython_notebook_environment and display_table:
             # In console environments where IPython is not available, the table should be printed
             # to the console
-            assert "What is 1+1?" in out
+            example_input = next(iter(example.inputs().values()))
+            assert example_input in out