Setting encoding of UTF-8 for some websites with non-UTF-8 encoding change response.text to be gibberish

When making a request to http://www.qq.com (or other pages like it), the encoding is returned as GB-2312. Looking at the response.text with the debugger, everything looks fine - correct characters.
![image](https://user-images.githubusercontent.com/3903412/86387117-5d549b00-bc9b-11ea-8151-7c54f6239818.png)

However, setting the response.encoding to UTF-8 causes the text to become garbled, gibberish:
![image](https://user-images.githubusercontent.com/3903412/86387167-6e9da780-bc9b-11ea-8088-babb5c9edbc9.png)

I am thinking it might be that the document is actually utf-8 encoded, but the content-type and document all say GB-2312, so the decoder gets confused.

## Expected Result

Setting response.encoding to 'utf-8' should not cause text to become gibberish.

## Actual Result

Text becomes gibberish.

## Reproduction Steps

```python
import requests

response = \
    requests.get(
        'http://www.qq.com',
        headers={
            'Accept-Language': 'en-US, en;q=0.5',
            'Accept-Charset':  'utf-8',
        },
        timeout=10,
        verify=False,
    )

# If we have an error, raise an exception
response.raise_for_status()

# Looking at response.text, it looks fine right now
print(
    response.text[0:200]
)

# It is GB2312, convert to UTF-8
if response.encoding != 'utf-8':
    response.encoding = 'utf-8'

# Looking at response.text, it is garbled now
print(
    response.text[0:200]
)
```

Output looks like:
![image](https://user-images.githubusercontent.com/3903412/86387695-377bc600-bc9c-11ea-9ccc-f1cdccda407c.png)

and
![image](https://user-images.githubusercontent.com/3903412/86387713-3ea2d400-bc9c-11ea-9821-b4aef481b04f.png)

## System Information

    $ python -m requests.help

```
{
  "chardet": {
    "version": "3.0.4"
  },
  "cryptography": {
    "version": ""
  },
  "idna": {
    "version": "2.8"
  },
  "implementation": {
    "name": "CPython",
    "version": "3.8.2"
  },
  "platform": {
    "release": "5.4.0-39-generic",
    "system": "Linux"
  },
  "pyOpenSSL": {
    "openssl_version": "",
    "version": null
  },
  "requests": {
    "version": "2.22.0"
  },
  "system_ssl": {
    "version": "1010106f"
  },
  "urllib3": {
    "version": "1.25.8"
  },
  "using_pyopenssl": false
}
```

According to https://requests.readthedocs.io/en/master/user/advanced/#encodings, the encodings are extracted from the headers, and if it cannot be found, it uses `chardet` as a fallback. When the encoding is in the body, it seems it is just ignored.
I am trying out this one: https://w3lib.readthedocs.io/en/latest/w3lib.html#w3lib.encoding.html_body_declared_encoding
You can figure out the encoding with that one, and then run `response.encoding = 'my-body-encoding'`. After that, `response.text` should be correct.

I think that requests should include something similar. There are many cases where the encoding is in the body.

There is a related discussion on here: https://github.com/psf/requests/issues/2086

Thanks @cdvv7788, I will look into it, and update if the workaround helps me.

This isn't a bug with requests. If you run `response.content.decode('utf-8')` you will get the same results. Not everything can be coerced into `utf-8`.

> I think that requests should include something similar. There are many cases where the encoding is in the body.

It did and it didn't work well enough because websites lie. There are utilities for this in the requests-toolbelt if I remember correctly. Regardless, these won't be added back as the project is in feature freeze.

> It did and it didn't work well enough because websites lie

`chardet` gets it wrong too, and it is still there.

I know that it is not a bug, and a change in behavior is unlikely. The problem is that when this bites you, takes a while to figure why `requests` is basically guessing what the encoding could be. Either a better effort in that direction, or not trying to guess the encoding at all would help with the ambiguity. The middle ground is not that good in this case.

@cdvv7788 please read the issue closely. They're setting `utf-8` if the encoding isn't already utf8

```py
# It is GB2312, convert to UTF-8
if response.encoding != 'utf-8':
    response.encoding = 'utf-8'
```

Hi @advance512,

As was pointed out above, this isn't a defect with Requests. Setting an encoding for a response won't convert the bytes to that encoding, it will simply try to decode it with the given codec. In this case, you're asserting the response is utf-8 which isn't correct, leading to gibberish. You'll either need to convert the bytes to the correct encoding, or use the one given by the webserver.