Repository: python-telegram-bot/python-telegram-bot

Issue:
Usage of InlineKeyboardButton raises BadRequest when given callback_data of 64 characters if the characters are above \x80
### Steps to reproduce
1. Run the code below
```  python
bot = Bot(TOKEN)
callback_data = ''.join(chr(i) for i in range(128, 128+64))
reply_markup = InlineKeyboardMarkup([[InlineKeyboardButton('Test', callback_data=callback_data)]])
bot.send_message(chat_id=CHAT_ID, text="text", reply_markup=reply_markup)
```

### Expected behaviour
The code should send a message to `CHAT_ID `with the text "text" and a button "Test" with the callback_data set to `\x80\x81\x82` and so on up to `\xbf`.

### Actual behaviour
`telegram.error.BadRequest: Button_data_invalid`

Note that the code below works as expected:
```python
bot = Bot(TOKEN)

callback_data = bytes(bytearray(j for j in range(128, 128+64)))

bot._request._request_wrapper('POST', '{0}/sendMessage'.format(bot.base_url),
                              body=b'{"chat_id": "CHAT_ID", "text": "test", "reply_markup": "{\\"inline_keyboard\\": [[{\\"text\\": \\"Test\\", \\"callback_data\\": \\"' + callback_data + b'\\"}]]}"}',
                              headers={'Content-Type': 'application/json'})
```

This means that it is indeed possible to send bytes `\x80` through `\xbf` as the callback_data of a button.
The problem originates from the from

https://github.com/python-telegram-bot/python-telegram-bot/blob/9d99660ba95b103b3e1dc80414a5ce2fd805260b/telegram/utils/request.py#L323

where we encode all the data (including the callback_data) using utf-8, which mangles all characters above \x80 (128). This means that `\x80` through `\xbf` actually turns into 128 bytes instead of the expected 64 (and therefore the BadRequest)

### Bug?
Is this a bug we even really care about?
I figured I just might as well report it.
One optimal fix would be to allow `bytes` as the input to InlineKeyboardButton, that would then be excluded from the .encode('utf-8') but I wouldn't even know how to begin implementing that.
We could also just ignore it - I highly doubt this bug really effects anyone tbh.


### Configuration
**Operating System:**
Windows 10 version 1803 build 17134.285

**Version of Python, python-telegram-bot & dependencies:**
```
python-telegram-bot 11.1.0
certifi 2018.08.24
future 0.16.0
Python 3.7.0 (default, Jun 28 2018, 08:04:48) [MSC v.1912 64 bit (AMD64)]
```
*NOTE* actually using `master` branch.

### Logs
With
```python
import http.client
http.client.HTTPConnection.debuglevel = 5
```
so we can see the data we send.

```
send: b'POST /botTOKEN/sendMessage HTTP/1.1\r\nHost: api.telegram.org\r\nAccept-Encoding: identity\r\nContent-Length: 579\r\nContent-Type: application/json\r\nconnection: keep-alive\r\nuser-agent: Python Telegram Bot (https://github.com/python-telegram-bot/python-telegram-bot)\r\n\r\n'
send: b'{"chat_id": "CHAT_ID", "text": "text", "reply_markup": "{\\"inline_keyboard\\": [[{\\"text\\": \\"Test\\", \\"callback_data\\": \\"\\\\u0080\\\\u0081\\\\u0082\\\\u0083\\\\u0084\\\\u0085\\\\u0086\\\\u0087\\\\u0088\\\\u0089\\\\u008a\\\\u008b\\\\u008c\\\\u008d\\\\u008e\\\\u008f\\\\u0090\\\\u0091\\\\u0092\\\\u0093\\\\u0094\\\\u0095\\\\u0096\\\\u0097\\\\u0098\\\\u0099\\\\u009a\\\\u009b\\\\u009c\\\\u009d\\\\u009e\\\\u009f\\\\u00a0\\\\u00a1\\\\u00a2\\\\u00a3\\\\u00a4\\\\u00a5\\\\u00a6\\\\u00a7\\\\u00a8\\\\u00a9\\\\u00aa\\\\u00ab\\\\u00ac\\\\u00ad\\\\u00ae\\\\u00af\\\\u00b0\\\\u00b1\\\\u00b2\\\\u00b3\\\\u00b4\\\\u00b5\\\\u00b6\\\\u00b7\\\\u00b8\\\\u00b9\\\\u00ba\\\\u00bb\\\\u00bc\\\\u00bd\\\\u00be\\\\u00bf\\"}]]}"}'

Traceback (most recent call last):
*SNIP*
  File "C:/Users/bomja_000/.PyCharm2018.2/config/scratches/scratch_22.py", line 77, in <module>
    bot.send_message(chat_id=95205500, text="text", reply_markup=reply_markup)
  File "C:\Anaconda3\envs\githubbot\lib\site-packages\telegram\bot.py", line 65, in decorator
    result = func(self, *args, **kwargs)
  File "C:\Anaconda3\envs\githubbot\lib\site-packages\telegram\bot.py", line 90, in decorator
    result = self._request.post(url, data, timeout=kwargs.get('timeout'))
  File "C:\Anaconda3\envs\githubbot\lib\site-packages\telegram\utils\request.py", line 325, in post
    headers={'Content-Type': 'application/json'})
  File "C:\Anaconda3\envs\githubbot\lib\site-packages\telegram\utils\request.py", line 239, in _request_wrapper
    raise BadRequest(message)
telegram.error.BadRequest: Button_data_invalid
```
An attempt to fix the docs page
![image](https://user-images.githubusercontent.com/6885864/45088400-4cc84d80-b122-11e8-9f5f-bf58f35278ab.png)


Discussion:
After discussing in internal developer group, we have decided to "fix" this by improving documentation about in InlineKeyboardButton.
This is because telegram specifically states that all requests should be made using UTF-8 encoding, which explicitly encodes \x80 into two bytes \xc2\x80. That telegram happens to support non-valid utf-8 isn't really something we can count on. Also implementing an actual fix for this would be really difficult, since we don't just gotta support sending these odd sequences of bytes, but also receive them, and that is stupidly difficult due to how json and string encodings work.
You can move those words back to the previous line, just indent the second line another level.
@alexbagirov We'd love you to fix this, but as @nmlorg said, there is a better way to do it :)

Resolving patch:
diff --git a/telegram/ext/conversationhandler.py b/telegram/ext/conversationhandler.py
--- a/telegram/ext/conversationhandler.py
+++ b/telegram/ext/conversationhandler.py
@@ -57,11 +57,6 @@ class ConversationHandler(Handler):
     a regular text message is expected. You could use this for a ``/cancel`` command or to let the
     user know their message was not recognized.
 
-    The fourth, optional collection of handlers, a ``list`` named :attr:`timed_out_behavior` is
-    used if the wait for ``run_async`` takes longer than defined in :attr:`run_async_timeout`.
-    For example, you can let the user know that they should wait for a bit before they can
-    continue.
-
     To change the state of conversation, the callback function of a handler must return the new
     state after responding to the user. If it does not return anything (returning ``None`` by
     default), the state will not change. If an entry point callback function returns None,
@@ -86,7 +81,7 @@ class ConversationHandler(Handler):
             ID.
         conversation_timeout (:obj:`float`|:obj:`datetime.timedelta`): Optional. When this handler
             is inactive more than this timeout (in seconds), it will be automatically ended. If
-            this value is 0 (default), there will be no timeout. when it's triggered. The last
+            this value is 0 (default), there will be no timeout. When it's triggered, the last
             received update will be handled by ALL the handler's who's `check_update` method
             returns True that are in the state :attr:`ConversationHandler.TIMEOUT`.
         name (:obj:`str`): Optional. The name for this conversationhandler. Required for
diff --git a/telegram/loginurl.py b/telegram/loginurl.py
--- a/telegram/loginurl.py
+++ b/telegram/loginurl.py
@@ -26,7 +26,8 @@ class LoginUrl(TelegramObject):
     authorize a user. Serves as a great replacement for the Telegram Login Widget when the user is
     coming from Telegram. All the user needs to do is tap/click a button and confirm that they want
     to log in. Telegram apps support these buttons as of version 5.7.
-    Sample bot: @discussbot
+
+    Sample bot: `@discussbot <https://t.me/dicussbot>`_
 
     Attributes:
         url (:obj:`str`): An HTTP URL to be opened with user authorization data.
diff --git a/telegram/message.py b/telegram/message.py
--- a/telegram/message.py
+++ b/telegram/message.py
@@ -865,7 +865,7 @@ def parse_entity(self, entity):
 
         Args:
             entity (:class:`telegram.MessageEntity`): The entity to extract the text from. It must
-            be an entity that belongs to this message.
+                be an entity that belongs to this message.
 
         Returns:
             :obj:`str`: The text of the given entity
@@ -890,7 +890,7 @@ def parse_caption_entity(self, entity):
 
         Args:
             entity (:class:`telegram.MessageEntity`): The entity to extract the text from. It must
-            be an entity that belongs to this message.
+                be an entity that belongs to this message.
 
         Returns:
             :obj:`str`: The text of the given entity


Test patch: