Repository: stanfordnlp/dspy
Pull request #8370 (2025-06-12 23:50:01)

Add a tutorial on customizing modules
We've been using custom modules everywhere in the doc, but we don't have a dedicated doc to walk through users how to write a custom module. This lightweight tutorial covers this missing piece. 


Patch:
diff --git a/docs/docs/learn/programming/modules.md b/docs/docs/learn/programming/modules.md
index 98ea6c87..a52f2bc4 100644
--- a/docs/docs/learn/programming/modules.md
+++ b/docs/docs/learn/programming/modules.md
@@ -245,6 +245,13 @@ class Hop(dspy.Module):
         return dspy.Prediction(notes=notes, titles=list(set(titles)))
 ```
 
+Then you can create a instance of the custom module class `Hop`, then invoke it by the `__call__` method:
+
+```
+hop = Hop()
+print(hop(claim="Stephen Curry is the best 3 pointer shooter ever in the human history"))
+```
+
 ## How do I track LM usage?
 
 !!! note "Version Requirement"
diff --git a/docs/docs/tutorials/build_ai_program/index.md b/docs/docs/tutorials/build_ai_program/index.md
index b04a44b4..729b9fd1 100644
--- a/docs/docs/tutorials/build_ai_program/index.md
+++ b/docs/docs/tutorials/build_ai_program/index.md
@@ -1,4 +1,5 @@
-- [Build AI Agents with DSPy](/tutorials/customer_service_agent/)
+- [Building AI Agents with DSPy](/tutorials/customer_service_agent/)
+- [Building AI Applications by Customizing DSPy Modules](/tutorials/custom_module/)
 - [Retrieval-Augmented Generation (RAG)](/tutorials/rag/)
 - [Building RAG as Agent](/tutorials/agents/)
 - [Entity Extraction](/tutorials/entity_extraction/)
diff --git a/docs/docs/tutorials/custom_module/index.ipynb b/docs/docs/tutorials/custom_module/index.ipynb
new file mode 100644
index 00000000..6fd08fdb
--- /dev/null
+++ b/docs/docs/tutorials/custom_module/index.ipynb
@@ -0,0 +1,197 @@
+{
+  "cells": [
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "1cR_pjqz1AsF"
+      },
+      "source": [
+        "# Building AI Applications by Customizing DSPy Modules\n",
+        "\n",
+        "In this guide, we will walk you through how to build a GenAI application by customizing `dspy.Module`.\n",
+        "\n",
+        "A [DSPy module](https://dspy.ai/learn/programming/modules/) is the building block for DSPy programs.\n",
+        "\n",
+        "- Each built-in module abstracts a prompting technique (like chain of thought or ReAct). Crucially, they are generalized to handle any signature.\n",
+        "\n",
+        "- A DSPy module has learnable parameters (i.e., the little pieces comprising the prompt and the LM weights) and can be invoked (called) to process inputs and return outputs.\n",
+        "\n",
+        "- Multiple modules can be composed into bigger modules (programs). DSPy modules are inspired directly by NN modules in PyTorch, but applied to LM programs.\n",
+        "\n",
+        "Although you can build a DSPy program without implementing a custom module, we highly recommend putting your logic with a custom module so that you can use other DSPy features, like DSPy optimizer or MLflow DSPy tracing."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "KBYjBQtv3Cn5"
+      },
+      "source": [
+        "Before getting started, make sure you have DSPy installed:\n",
+        "\n",
+        "```\n",
+        "!pip install dspy\n",
+        "```"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "reQSTM8a8qMf"
+      },
+      "source": [
+        "## Customize DSPy Module\n",
+        "\n",
+        "You can implement custom prompting logic and integrate external tools or services by customizing a DSPy module. To achieve this, subclass from `dspy.Module` and implement the following two key methods:\n",
+        "\n",
+        "- `__init__`: This is the constructor, where you define the attributes and sub-modules of your program.\n",
+        "- `forward`: This method contains the core logic of your DSPy program.\n",
+        "\n",
+        "Within the `forward()` method, you are not limited to calling only other DSPy modules; you can also integrate any standard Python functions, such as those for interacting with Langchain/Agno agents, MCP tools, database handlers, and more.\n",
+        "\n",
+        "The basic structure for a custom DSPy module looks like this:\n",
+        "\n",
+        "```python\n",
+        "class MyProgram(dspy.Module):\n",
+        "    \n",
+        "    def __init__(self, ...):\n",
+        "        # Define attributes and sub-modules here\n",
+        "        {constructor_code}\n",
+        "\n",
+        "    def forward(self, input_name1, input_name2, ...):\n",
+        "        # Implement your program's logic here\n",
+        "        {custom_logic_code}\n",
+        "```"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "DziTWwT8_TrY"
+      },
+      "source": [
+        "Let's illustrate this with a practical code example. We will build a simple Retrieval-Augmented Generation (RAG) application with mulitple stages:\n",
+        "\n",
+        "1.  **Query Generation:** Generate a suitable query based on the user's question to retrieve relevant context.\n",
+        "2.  **Context Retrieval:** Fetch context using the generated query.\n",
+        "3.  **Answer Generation:** Produce a final answer based on the retrieved context and the original question.\n",
+        "\n",
+        "The code implementation for this multi-stage program is shown below."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 3,
+      "metadata": {
+        "id": "lAoV5_v7YlvN"
+      },
+      "outputs": [],
+      "source": [
+        "import dspy\n",
+        "\n",
+        "class QueryGenerator(dspy.Signature):\n",
+        "    \"\"\"Generate a query based on question to fetch relevant context\"\"\"\n",
+        "    question: str = dspy.InputField()\n",
+        "    query: str = dspy.OutputField()\n",
+        "\n",
+        "def search_wikipedia(query: str) -> list[str]:\n",
+        "    \"\"\"Query ColBERT endpoint, which is a knowledge source based on wikipedia data\"\"\"\n",
+        "    results = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts')(query, k=1)\n",
+        "    return [x[\"text\"] for x in results]\n",
+        "\n",
+        "class RAG(dspy.Module):\n",
+        "    def __init__(self):\n",
+        "        self.query_generator = dspy.Predict(QueryGenerator)\n",
+        "        self.answer_generator = dspy.ChainOfThought(\"question,context->answer\")\n",
+        "\n",
+        "    def forward(self, question, **kwargs):\n",
+        "        query = self.query_generator(question=question).query\n",
+        "        context = search_wikipedia(query)[0]\n",
+        "        return self.answer_generator(question=question, context=context).answer"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Let's take a look at the `forward` method. We first send the question to `self.query_generator`, which is a `dspy.Predict`, to get the query for context retrieving. Then we use the query to call ColBERT and keep the first context retrieved. Finally, we send the question and context into `self.answer_generator`, which is a `dspy.ChainOfThought` to generate the final answer."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "FBq_4e8NamwY"
+      },
+      "source": [
+        "Next, we'll create an instance of our `RAG` module to run the program.\n",
+        "\n",
+        "**Important:** When invoking a custom DSPy module, you should use the module instance directly (which calls the `__call__` method internally), rather than calling the `forward()` method explicitly. The `__call__` method handles necessary internal processing before executing the `forward` logic."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 7,
+      "metadata": {
+        "colab": {
+          "base_uri": "https://localhost:8080/"
+        },
+        "id": "ZR7xcFSTa596",
+        "outputId": "f3427754-8a16-48fe-c540-8c9f31d9a30d"
+      },
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "The question of whether LeBron James is the basketball GOAT is subjective and depends on personal opinions. Many consider him one of the greatest due to his achievements and impact on the game, but others may argue for different players like Michael Jordan.\n"
+          ]
+        }
+      ],
+      "source": [
+        "import os\n",
+        "\n",
+        "os.environ[\"OPENAI_API_KEY\"] = \"{your_openai_api_key}\"\n",
+        "\n",
+        "dspy.configure(lm=dspy.LM(\"openai/gpt-4o-mini\"))\n",
+        "rag = RAG()\n",
+        "print(rag(question=\"Is Lebron James the basketball GOAT?\"))"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "That's it! In summary, to build your GenAI applications, we just put the custom logic into the `forward()` method, then create a module instance and call the instance itself."
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {
+        "id": "aYAYc-Hg39ri"
+      },
+      "source": [
+        "## Why Customizing Module?\n",
+        "\n",
+        "DSPy is a lightweight authoring and optimization framework, and our focus is to resolve the mess of prompt engineering by transforming prompting (string in, string out) LLM into programming LLM (structured inputs in, structured outputs out) for robust AI system.\n",
+        "\n",
+        "While we provide pre-built modules which have custom prompting logic like `dspy.ChainOfThought` for reasoning, `dspy.ReAct` for tool calling agent to facilitate building your AI applications, we don't aim at standardizing how you build agents.\n",
+        "\n",
+        "In DSPy, your application logic simply goes to the `forward` method of your custom Module, which doesn't have any constraint as long as you are writing python code. With this layout, DSPy is easy to migrate to from other frameworks or vanilla SDK usage, and easy to migrate off because essentially it's just python code.\n"
+      ]
+    }
+  ],
+  "metadata": {
+    "colab": {
+      "provenance": []
+    },
+    "kernelspec": {
+      "display_name": "Python 3",
+      "name": "python3"
+    },
+    "language_info": {
+      "name": "python"
+    }
+  },
+  "nbformat": 4,
+  "nbformat_minor": 0
+}
diff --git a/docs/docs/tutorials/index.md b/docs/docs/tutorials/index.md
index 91f012fb..f253c924 100644
--- a/docs/docs/tutorials/index.md
+++ b/docs/docs/tutorials/index.md
@@ -13,9 +13,10 @@ Welcome to DSPy tutorials! We've organized our tutorials into three main categor
 
 
 - Build AI Programs with DSPy
+    - [Building AI Agents with DSPy](/tutorials/customer_service_agent/)
+    - [Building AI Applications by Customizing DSPy Modules](/tutorials/custom_module/)
     - [Retrieval-Augmented Generation (RAG)](/tutorials/rag/)
     - [Building RAG as Agent](/tutorials/agents/)
-    - [Build AI Agents with DSPy](/tutorials/customer_service_agent/)
     - [Entity Extraction](/tutorials/entity_extraction/)
     - [Classification](/tutorials/classification/)
     - [Multi-Hop RAG](/tutorials/multihop_search/)
diff --git a/docs/docs/tutorials/observability/index.md b/docs/docs/tutorials/observability/index.md
index 6293739e..1e338936 100644
--- a/docs/docs/tutorials/observability/index.md
+++ b/docs/docs/tutorials/observability/index.md
@@ -203,6 +203,8 @@ Sometimes, you may want to implement a custom logging solution. For instance, yo
 |`on_lm_start` / `on_lm_end` | Triggered when a `dspy.LM` subclass is invoked. |
 |`on_adapter_format_start` / `on_adapter_format_end`| Triggered when a `dspy.Adapter` subclass formats the input prompt. |
 |`on_adapter_parse_start` / `on_adapter_parse_end`| Triggered when a `dspy.Adapter` subclass postprocess the output text from an LM. |
+|`on_tool_start` / `on_tool_end` | Triggered when a `dspy.Tool` subclass is invoked. |
+|`on_evaluate_start` / `on_evaluate_end` | Triggered when a `dspy.Evaluate` instance is invoked. |
 
 Here's an example of custom callback that logs the intermediate steps of a ReAct agent:
 
diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml
index 556d2fd0..af1405ca 100644
--- a/docs/mkdocs.yml
+++ b/docs/mkdocs.yml
@@ -27,7 +27,8 @@ nav:
         - Tutorials Overview: tutorials/index.md
         - Build AI Programs with DSPy:
             - Overview: tutorials/build_ai_program/index.md
-            - Build AI Agents with DSPy: tutorials/customer_service_agent/index.ipynb
+            - Building AI Agents with DSPy: tutorials/customer_service_agent/index.ipynb
+            - Building AI Applications by Customizing DSPy Modules: tutorials/custom_module/index.ipynb
             - Retrieval-Augmented Generation (RAG): tutorials/rag/index.ipynb
             - Building RAG as Agent: tutorials/agents/index.ipynb
             - Entity Extraction: tutorials/entity_extraction/index.ipynb
diff --git a/dspy/predict/predict.py b/dspy/predict/predict.py
index 14b44f0d..50db41a9 100644
--- a/dspy/predict/predict.py
+++ b/dspy/predict/predict.py
@@ -99,7 +99,22 @@ class Predict(Module, Parameter):
 
         # Get the right LM to use.
         lm = kwargs.pop("lm", self.lm) or settings.lm
-        assert isinstance(lm, BaseLM), "No LM is loaded."
+
+        if lm is None:
+            raise ValueError(
+                "No LM is loaded. Please configure the LM using `dspy.configure(lm=dspy.LM(...))`. e.g, "
+                "`dspy.configure(lm=dspy.LM('openai/gpt-4o-mini'))`"
+            )
+
+        if isinstance(lm, str):
+            # Many users mistakenly use `dspy.configure(lm="openai/gpt-4o-mini")` instead of
+            # `dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))`, so we are providing a specific error message.
+            raise ValueError(
+                f"LM must be an instance of `dspy.BaseLM`, not a string. Instead of using a string like "
+                f"'dspy.configure(lm=\"{lm}\")', please configure the LM like 'dspy.configure(lm=dspy.LM(\"{lm}\"))'"
+            )
+        elif not isinstance(lm, BaseLM):
+            raise ValueError(f"LM must be an instance of `dspy.BaseLM`, not {type(lm)}. Received `lm={lm}`.")
 
         # If temperature is unset or <=0.15, and n > 1, set temperature to 0.7 to keep randomness.
         temperature = config.get("temperature") or lm.kwargs.get("temperature")
diff --git a/dspy/primitives/program.py b/dspy/primitives/program.py
index e306591e..d59c83af 100644
--- a/dspy/primitives/program.py
+++ b/dspy/primitives/program.py
@@ -1,3 +1,5 @@
+import inspect
+import logging
 from typing import Optional
 
 import magicattr
@@ -9,6 +11,7 @@ from dspy.utils.callback import with_callbacks
 from dspy.utils.inspect_history import pretty_print_history
 from dspy.utils.usage_tracker import track_usage
 
+logger = logging.getLogger(__name__)
 
 class ProgramMeta(type):
     """Metaclass ensuring every ``dspy.Module`` instance is properly initialised."""
@@ -155,6 +158,20 @@ class Module(BaseModule, metaclass=ProgramMeta):
             results = parallel_executor.forward(exec_pairs)
             return results
 
+    def __getattribute__(self, name):
+        attr = super().__getattribute__(name)
+
+        if name == 'forward' and callable(attr):
+            # Check if forward iscalled through __call__
+            stack = inspect.stack()
+            called_via_call = len(stack) > 1 and stack[1].function == '__call__'
+
+            if not called_via_call:
+                logger.warning(f"Calling {self.__class__.__name__}.forward() directly is discouraged. "
+                               f"Use {self.__class__.__name__}() instead.")
+
+        return attr
+
 
 def set_attribute_by_name(obj, name, value):
     magicattr.set(obj, name, value)


Test patch:
diff --git a/tests/predict/test_predict.py b/tests/predict/test_predict.py
index c1aea972..e60cd14e 100644
--- a/tests/predict/test_predict.py
+++ b/tests/predict/test_predict.py
@@ -582,6 +582,28 @@ def test_positional_arguments():
     )
 
 
+def test_error_message_on_invalid_lm_setup():
+    # No LM is loaded.
+    with pytest.raises(ValueError, match="No LM is loaded"):
+        Predict("question -> answer")(question="Why did a chicken cross the kitchen?")
+
+    # LM is a string.
+    dspy.configure(lm="openai/gpt-4o-mini")
+    with pytest.raises(ValueError) as e:
+        Predict("question -> answer")(question="Why did a chicken cross the kitchen?")
+
+    assert "LM must be an instance of `dspy.BaseLM`, not a string." in str(e.value)
+
+    def dummy_lm():
+        pass
+
+    # LM is not an instance of dspy.BaseLM.
+    dspy.configure(lm=dummy_lm)
+    with pytest.raises(ValueError) as e:
+        Predict("question -> answer")(question="Why did a chicken cross the kitchen?")
+    assert "LM must be an instance of `dspy.BaseLM`, not <class 'function'>." in str(e.value)
+
+
 @pytest.mark.parametrize("adapter_type", ["chat", "json"])
 def test_field_constraints(adapter_type):
     class SpyLM(dspy.LM):
diff --git a/tests/primitives/test_module.py b/tests/primitives/test_module.py
index 54414683..023f92c3 100644
--- a/tests/primitives/test_module.py
+++ b/tests/primitives/test_module.py
@@ -489,3 +489,25 @@ async def test_module_history_async():
         assert len(program.history) == 2
         assert len(program.cot.history) == 3
         assert len(program.cot.predict.history) == 3
+
+
+def test_forward_direct_call_warning(capsys):
+    class TestModule(dspy.Module):
+        def forward(self, x):
+            return x
+
+    module = TestModule()
+    module.forward("test")
+    captured = capsys.readouterr()
+    assert "Calling TestModule.forward() directly is discouraged" in captured.err
+
+
+def test_forward_through_call_no_warning(capsys):
+    class TestModule(dspy.Module):
+        def forward(self, x):
+            return x
+
+    module = TestModule()
+    module(x="test")
+    captured = capsys.readouterr()
+    assert "Calling TestModule.forward() directly is discouraged" not in captured.err