Repository: python-telegram-bot/python-telegram-bot

Issue:
Make it easier to handle webhooks manually, by providing a convience setup method for dispatcher
Sometimes you need not just webhooks from telegram, but from other services too. It would be nice if we could integrate this nicely into the library, without forcing users to not use Updater (which means handling threading, jobqueue starting, and of cause webhooks themselves).

As this is still very much an advanced feature, I think a good way of doing this could be to allow passing a custom `WebhookAppClass` to `Updater._start_webhook` somehow.

https://github.com/python-telegram-bot/python-telegram-bot/blob/eca0ccf6b376699ffcc7816d4db7e91b205a4fdd/telegram/ext/updater.py#L357-L365

For example to also receive webhooks from github one could write a custom GithubWebhookHandler (instance of `tornado.web.RequestHandler` that perhaps turns incomming data into a `GithubUpdate` type that a `TypeHandler` can then handle) and then a [WebhookAppClass](https://github.com/python-telegram-bot/python-telegram-bot/blob/master/telegram/utils/webhookhandler.py#L71-L79) like:
``` python
class WebhookAppClass(tornado.web.Application):
    def __init__(self, webhook_path, bot, update_queue):
        handlers = [
            (
                r'/github/?',
                GithubWebhookHandler,
                {'update_queue': update_queue}
            ),
            (
                r"{0}/?".format(webhook_path),
                telegram.utils.webhookhandler.WebhookHandler,
                {'bot': bot, 'update_queue': update_queue}
            )
        ]
        super().__init__(handlers)
```

The only way this is currently possible to do is something like
```
telegram.utils.webhookhandler.WebhookAppClass = WebhookAppClass
```
But as that line needs to come before any `telegram` imports, it makes it quite messy.


Discussion:
So after discussing this a lot in our developer chat, we have decided to go about it differently.
Instead of the above, we are gonna provide a method that sets up and configures a Dispatcher, including setting up a JobQueue (if desired), an update queue, a Request instance, and of cause a Bot.
This makes it a lot easier to integrate with other webhook solutions (and therefore adding additional services), as configuring Dispatcher was the difficult job before.
I take it it was meant more as a joke, but the factory/builder pattern might indeed come in handy for the purpose of creating all these instances with their specific settings and the option to start threads immediately or delayed.

This way, we can refactor our existing code base to use a cleaner approach while also exposing it to our users.
> So after discussing this a lot in our developer chat, we have decided to go about it differently.
> Instead of the above, we are gonna provide a method that sets up and configures a Dispatcher, including setting up a JobQueue (if desired), an update queue, a Request instance, and of cause a Bot.
> This makes it a lot easier to integrate with other webhook solutions (and therefore adding additional services), as configuring Dispatcher was the difficult job before.

So is this ready yet?.. If not what options are available at the moment
Catching up on this old issue, I had the following idea:

`tornade.web.Application` already has the `add_handlers` method, so we can leverage that. Sketch of my idea:

* Provide a `PTBRequestHandler` (or whatever naming) along the lines

```python
class PTBRequestHandler(tornado.web.RequestHandler):
     update_queue: CassVar[Queue] = None

     @classmethod
     def set_update_queue(cls, uq):
        cls.update_queue = uq
```

* make the `app` currently set here:

https://github.com/python-telegram-bot/python-telegram-bot/blob/9ddb361f761abd34b9cbc03a15251a84a9abdb11/telegram/ext/updater.py#L519

  a instance attribute
* add `Updater.add_webhook_handlers(handlers: List[PTBRequestHandler])` as wrapper around `self.app.add_handlers` that also takes care of calling `PTBRequestHandler.set_update_queue`.

That way users can implement their own handler and just call `self.update_queue.put(update)`. Alternatively one could skip the part with `PTBRequestHandler` and make the user manually pass `updater.update_queue` at the right spot in `Updater.add_webhook_handlers`

We can still provide a `Dispatcher.build()` (or whatever) class-method that provides all missing optional args of `__init__`. Can indeed be useful for customized webhook solutions, where `Updater` isn't used.

Resolving patch:
diff --git a/examples/contexttypesbot.py b/examples/contexttypesbot.py
--- a/examples/contexttypesbot.py
+++ b/examples/contexttypesbot.py
@@ -57,8 +57,8 @@ def __init__(self) -> None:
 class CustomContext(CallbackContext[ExtBot, dict, ChatData, dict]):
     """Custom class for context."""
 
-    def __init__(self, application: Application):
-        super().__init__(application=application)
+    def __init__(self, application: Application, chat_id: int = None, user_id: int = None):
+        super().__init__(application=application, chat_id=chat_id, user_id=user_id)
         self._message_id: Optional[int] = None
 
     @property
diff --git a/examples/customwebhookbot.py b/examples/customwebhookbot.py
new file mode 100644
--- /dev/null
+++ b/examples/customwebhookbot.py
@@ -0,0 +1,192 @@
+#!/usr/bin/env python
+# This program is dedicated to the public domain under the CC0 license.
+# pylint: disable=import-error,wrong-import-position
+"""
+Simple example of a bot that uses a custom webhook setup and handles custom updates.
+For the custom webhook setup, the libraries `starlette` and `uvicorn` are used. Please install
+them as `pip install starlette~=0.20.0 uvicorn~=0.17.0`.
+Note that any other `asyncio` based web server framework can be used for a custom webhook setup
+just as well.
+
+Usage:
+Set bot token, url, admin chat_id and port at the start of the `main` function.
+You may also need to change the `listen` value in the uvicorn configuration to match your setup.
+Press Ctrl-C on the command line or send a signal to the process to stop the bot.
+"""
+import asyncio
+import html
+import logging
+from dataclasses import dataclass
+from http import HTTPStatus
+
+import uvicorn
+from starlette.applications import Starlette
+from starlette.requests import Request
+from starlette.responses import PlainTextResponse, Response
+from starlette.routing import Route
+
+from telegram import __version__ as TG_VER
+
+try:
+    from telegram import __version_info__
+except ImportError:
+    __version_info__ = (0, 0, 0, 0, 0)  # type: ignore[assignment]
+
+if __version_info__ < (20, 0, 0, "alpha", 1):
+    raise RuntimeError(
+        f"This example is not compatible with your current PTB version {TG_VER}. To view the "
+        f"{TG_VER} version of this example, "
+        f"visit https://github.com/python-telegram-bot/python-telegram-bot/tree/v{TG_VER}/examples"
+    )
+
+from telegram import Update
+from telegram.constants import ParseMode
+from telegram.ext import (
+    Application,
+    CallbackContext,
+    CommandHandler,
+    ContextTypes,
+    ExtBot,
+    TypeHandler,
+)
+
+# Enable logging
+logging.basicConfig(
+    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s", level=logging.INFO
+)
+logger = logging.getLogger(__name__)
+
+
+@dataclass
+class WebhookUpdate:
+    """Simple dataclass to wrap a custom update type"""
+
+    user_id: int
+    payload: str
+
+
+class CustomContext(CallbackContext[ExtBot, dict, dict, dict]):
+    """
+    Custom CallbackContext class that makes `user_data` available for updates of type
+    `WebhookUpdate`.
+    """
+
+    @classmethod
+    def from_update(
+        cls,
+        update: object,
+        application: "Application",
+    ) -> "CustomContext":
+        if isinstance(update, WebhookUpdate):
+            return cls(application=application, user_id=update.user_id)
+        return super().from_update(update, application)
+
+
+async def start(update: Update, context: CustomContext) -> None:
+    """Display a message with instructions on how to use this bot."""
+    url = context.bot_data["url"]
+    payload_url = html.escape(f"{url}/submitpayload?user_id=<your user id>&payload=<payload>")
+    text = (
+        f"To check if the bot is still running, call <code>{url}/healthcheck</code>.\n\n"
+        f"To post a custom update, call <code>{payload_url}</code>."
+    )
+    await update.message.reply_html(text=text)
+
+
+async def webhook_update(update: WebhookUpdate, context: CustomContext) -> None:
+    """Callback that handles the custom updates."""
+    chat_member = await context.bot.get_chat_member(chat_id=update.user_id, user_id=update.user_id)
+    payloads = context.user_data.setdefault("payloads", [])
+    payloads.append(update.payload)
+    combined_payloads = "</code>\n• <code>".join(payloads)
+    text = (
+        f"The user {chat_member.user.mention_html()} has sent a new payload. "
+        f"So far they have sent the following payloads: \n\n• <code>{combined_payloads}</code>"
+    )
+    await context.bot.send_message(
+        chat_id=context.bot_data["admin_chat_id"], text=text, parse_mode=ParseMode.HTML
+    )
+
+
+async def main() -> None:
+    """Set up the application and a custom webserver."""
+    url = "https://domain.tld"
+    admin_chat_id = 123456
+    port = 8000
+
+    context_types = ContextTypes(context=CustomContext)
+    # Here we set updater to None because we want our custom webhook server to handle the updates
+    # and hence we don't need an Updater instance
+    application = (
+        Application.builder().token("TOKEN").updater(None).context_types(context_types).build()
+    )
+    # save the values in `bot_data` such that we may easily access them in the callbacks
+    application.bot_data["url"] = url
+    application.bot_data["admin_chat_id"] = admin_chat_id
+
+    # register handlers
+    application.add_handler(CommandHandler("start", start))
+    application.add_handler(TypeHandler(type=WebhookUpdate, callback=webhook_update))
+
+    # Pass webhook settings to telegram
+    await application.bot.set_webhook(url=f"{url}/telegram")
+
+    # Set up webserver
+    async def telegram(request: Request) -> Response:
+        """Handle incoming Telegram updates by putting them into the `update_queue`"""
+        await application.update_queue.put(
+            Update.de_json(data=await request.json(), bot=application.bot)
+        )
+        return Response()
+
+    async def custom_updates(request: Request) -> PlainTextResponse:
+        """
+        Handle incoming webhook updates by also putting them into the `update_queue` if
+        the required parameters were passed correctly.
+        """
+        try:
+            user_id = int(request.query_params["user_id"])
+            payload = request.query_params["payload"]
+        except KeyError:
+            return PlainTextResponse(
+                status_code=HTTPStatus.BAD_REQUEST,
+                content="Please pass both `user_id` and `payload` as query parameters.",
+            )
+        except ValueError:
+            return PlainTextResponse(
+                status_code=HTTPStatus.BAD_REQUEST,
+                content="The `user_id` must be a string!",
+            )
+
+        await application.update_queue.put(WebhookUpdate(user_id=user_id, payload=payload))
+        return PlainTextResponse("Thank you for the submission! It's being forwarded.")
+
+    async def health(_: Request) -> PlainTextResponse:
+        """For the health endpoint, reply with a simple plain text message."""
+        return PlainTextResponse(content="The bot is still running fine :)")
+
+    starlette_app = Starlette(
+        routes=[
+            Route("/telegram", telegram, methods=["POST"]),
+            Route("/healthcheck", health, methods=["GET"]),
+            Route("/submitpayload", custom_updates, methods=["POST", "GET"]),
+        ]
+    )
+    webserver = uvicorn.Server(
+        config=uvicorn.Config(
+            app=starlette_app,
+            port=port,
+            use_colors=False,
+            host="127.0.0.1",
+        )
+    )
+
+    # Run application and webserver together
+    async with application:
+        await application.start()
+        await webserver.serve()
+        await application.stop()
+
+
+if __name__ == "__main__":
+    asyncio.run(main())
diff --git a/telegram/ext/_callbackcontext.py b/telegram/ext/_callbackcontext.py
--- a/telegram/ext/_callbackcontext.py
+++ b/telegram/ext/_callbackcontext.py
@@ -18,18 +18,7 @@
 # along with this program.  If not, see [http://www.gnu.org/licenses/].
 # pylint: disable=no-self-use
 """This module contains the CallbackContext class."""
-from typing import (
-    TYPE_CHECKING,
-    Coroutine,
-    Dict,
-    Generic,
-    List,
-    Match,
-    NoReturn,
-    Optional,
-    Tuple,
-    Type,
-)
+from typing import TYPE_CHECKING, Coroutine, Dict, Generic, List, Match, NoReturn, Optional, Type
 
 from telegram._callbackquery import CallbackQuery
 from telegram._update import Update
@@ -83,6 +72,14 @@ class CallbackContext(Generic[BT, UD, CD, BD]):
     Args:
         application (:class:`telegram.ext.Application`): The application associated with this
             context.
+        chat_id (:obj:`int`, optional): The ID of the chat associated with this object. Used
+            to provide :attr:`chat_data`.
+
+            .. versionadded:: 20.0
+        user_id (:obj:`int`, optional): The ID of the user associated with this object. Used
+            to provide :attr:`user_data`.
+
+            .. versionadded:: 20.0
 
     Attributes:
         coroutine (:term:`coroutine function`): Optional. Only present in error handlers if the
@@ -109,8 +106,8 @@ class CallbackContext(Generic[BT, UD, CD, BD]):
 
     __slots__ = (
         "_application",
-        "_chat_id_and_data",
-        "_user_id_and_data",
+        "_chat_id",
+        "_user_id",
         "args",
         "matches",
         "error",
@@ -119,10 +116,15 @@ class CallbackContext(Generic[BT, UD, CD, BD]):
         "__dict__",
     )
 
-    def __init__(self: "CCT", application: "Application[BT, CCT, UD, CD, BD, Any]"):
+    def __init__(
+        self: "CCT",
+        application: "Application[BT, CCT, UD, CD, BD, Any]",
+        chat_id: int = None,
+        user_id: int = None,
+    ):
         self._application = application
-        self._chat_id_and_data: Optional[Tuple[int, CD]] = None
-        self._user_id_and_data: Optional[Tuple[int, UD]] = None
+        self._chat_id = chat_id
+        self._user_id = user_id
         self.args: Optional[List[str]] = None
         self.matches: Optional[List[Match]] = None
         self.error: Optional[Exception] = None
@@ -159,8 +161,8 @@ def chat_data(self) -> Optional[CD]:
             <https://github.com/python-telegram-bot/python-telegram-bot/wiki/
             Storing-bot,-user-and-chat-related-data#chat-migration>`_.
         """
-        if self._chat_id_and_data:
-            return self._chat_id_and_data[1]
+        if self._chat_id is not None:
+            return self._application.chat_data[self._chat_id]
         return None
 
     @chat_data.setter
@@ -175,8 +177,8 @@ def user_data(self) -> Optional[UD]:
         For each update from the same user it will be the same :obj:`ContextTypes.user_data`.
         Defaults to :obj:`dict`.
         """
-        if self._user_id_and_data:
-            return self._user_id_and_data[1]
+        if self._user_id is not None:
+            return self._application.user_data[self._user_id]
         return None
 
     @user_data.setter
@@ -200,16 +202,14 @@ async def refresh_data(self) -> None:
         if self.application.persistence:
             if self.application.persistence.store_data.bot_data:
                 await self.application.persistence.refresh_bot_data(self.bot_data)
-            if (
-                self.application.persistence.store_data.chat_data
-                and self._chat_id_and_data is not None
-            ):
-                await self.application.persistence.refresh_chat_data(*self._chat_id_and_data)
-            if (
-                self.application.persistence.store_data.user_data
-                and self._user_id_and_data is not None
-            ):
-                await self.application.persistence.refresh_user_data(*self._user_id_and_data)
+            if self.application.persistence.store_data.chat_data and self._chat_id is not None:
+                await self.application.persistence.refresh_chat_data(
+                    chat_id=self._chat_id, chat_data=self.chat_data
+                )
+            if self.application.persistence.store_data.user_data and self._user_id is not None:
+                await self.application.persistence.refresh_user_data(
+                    user_id=self._user_id, user_data=self.user_data
+                )
 
     def drop_callback_data(self, callback_query: CallbackQuery) -> None:
         """
@@ -264,6 +264,12 @@ def from_error(
                 context.
             job (:class:`telegram.ext.Job`, optional): The job associated with the error.
 
+                .. versionadded:: 20.0
+            coroutine (:term:`coroutine function`, optional): The coroutine function associated
+                with this error if the error was caused by a coroutine run with
+                :meth:`Application.create_task` or a handler callback with
+                :attr:`block=False <BaseHandler.block>`.
+
                 .. versionadded:: 20.0
 
         Returns:
@@ -295,23 +301,15 @@ def from_update(
         Returns:
             :class:`telegram.ext.CallbackContext`
         """
-        self = cls(application)  # type: ignore
-
-        if update is not None and isinstance(update, Update):
+        if isinstance(update, Update):
             chat = update.effective_chat
             user = update.effective_user
 
-            if chat:
-                self._chat_id_and_data = (
-                    chat.id,
-                    application.chat_data[chat.id],
-                )
-            if user:
-                self._user_id_and_data = (
-                    user.id,
-                    application.user_data[user.id],
-                )
-        return self
+            chat_id = chat.id if chat else None
+            user_id = user.id if user else None
+
+            return cls(application, chat_id=chat_id, user_id=user_id)  # type: ignore
+        return cls(application)  # type: ignore
 
     @classmethod
     def from_job(
@@ -333,19 +331,8 @@ def from_job(
         Returns:
             :class:`telegram.ext.CallbackContext`
         """
-        self = cls(application)  # type: ignore
+        self = cls(application, chat_id=job.chat_id, user_id=job.user_id)  # type: ignore
         self.job = job
-
-        if job.chat_id:
-            self._chat_id_and_data = (
-                job.chat_id,
-                application.chat_data[job.chat_id],
-            )
-        if job.user_id:
-            self._user_id_and_data = (
-                job.user_id,
-                application.user_data[job.user_id],
-            )
         return self
 
     def update(self, data: Dict[str, object]) -> None:


Test patch: