Response 400 while using Requests package in Django

I want to scrap Myntra using Django Framework but when I put my code in Django project I get 400 error. The same code runs without error when I run it on the same file even in Django. Means calling the MyntraScraperClass in MyntraScraper.py file.

Here is my project directory
```
BackendController
    -MyntraScraper.py
myDjangoApp
    -views.py
```

Inside views.py, there is a function where I am calling my MyntraScraperClass
```
def tst(request):
    ------------
```

The same code that is MyntraScraperClass runs error free when I call it on 
```
BackendController
    -MyntraScraper.py
```

Here is my code:
```
import requests, json
from bs4 import BeautifulSoup
import os, ssl
import certifi
import urllib3

class MyntraScraperClass:

    def __init__(self,url):
        self.url=url


    def myntra(self):
        mt={}

        http = urllib3.PoolManager(
                cert_reqs='CERT_REQUIRED',
                ca_certs=certifi.where())

        if (not os.environ.get('PYTHONHTTPSVERIFY', '') and getattr(ssl, '_create_unverified_context', None)):
            ssl._create_default_https_context = ssl._create_unverified_context

        proxy = {'http': '-------'}

        headers = {'User-Agent' : 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36'}
        for x in range(0,5):
            print(x)
            try:
                s = requests.Session()
                res = s.get(self.url, headers=headers, verify=True, proxies=proxy)
                print(res)
                soup = BeautifulSoup(res.text, "lxml")
                print(soup)
                if "Access Denied" in soup.title:
                    continue
                break
            except requests.exceptions.ProxyError:
                continue
            except Exception as e:
                print(e)
                mt['error']=e


        script = None
        for s in soup.find_all("script"):
            if 'pdpData' in s.text:
                script = s.get_text(strip=True)
                break
            else:
                continue

        mt['data'] =json.loads(script[script.index('{'):])
        return mt

```

I am calling MyntraScraperClass in my Django Views from BackendController directory. 
```
def tst(request):
    url = request.GET.get('url')
    from BackendController.MyntraScraper import  MyntraScraperClass
    
    obj1 = MyntraScraperClass(url)
    kk = obj1.myntra()
    print(kk)
    return JsonResponse(kk)
```


The response is

```
<Response [400]>
```

I am getting this in soup

```
<html><head>
<title>Invalid URL</title>
</head><body>
<h1>Invalid URL</h1>
The requested URL "[no URL]", is invalid.<p>
Reference #9.6b722c31.1596891375.7bfc4ae
</p></body></html>
```

Had the same issue. I couldn't solved it.

Hi there! Thanks for opening this issue. Unfortunately, it seems this is a request for help instead of a report of a defect in the project. Please use [StackOverflow](https://stackoverflow.com) for general usage questions instead and only report defects here.

@sigmavirus24 My way of describing the problem is different but the thing is that it is actually an issue with Requests.

I believe the URL you're passing to Requests isn't a URL and that's why you're getting an invalid URL error. Thus this isn't a bug

> I believe the URL you're passing to Requests isn't a URL and that's why you're getting an invalid URL error. Thus this isn't a bug

No, as I described above that it is working when I run the same code on same file. Also, I have tested the URL by passing it directly in the class and then just call that class method in views.py . Still, It throws the same error

```
The requested URL "[no URL]", is invalid.<p>
```

With what very little you've provided, all I can come back to is the bit above. That looks like something isn't working as you expect it to. Without more information and evidence of what you've tried, I remain confident this isn't a bug. Especially since there isn't a bug I know about that causes a 400 response from Django.

@sigmavirus24  If you know something that causes error then please check this [Stackoverflow](https://stackoverflow.com/questions/63315807/response-400-while-using-requests-package-in-django). It will be very helpful if you could help me to get out from this problem. 

From StackOverflow, you said:

> The same code I try with URLLIB but it is not working for me.

So again, I'll reiterate that this isn't a requests bug if it doesn't work for urllib either.

@sigmavirus24  I have already moved my Amazon and Flipkart scraper from Requests to urllib because of the same reason. Only I am not able to code a perfect urllib scraper for Myntra. So, I wanted to resolve this issue asap. If you want proof then I can make a video on this issue. Also, you can check this issue on TeamViewer or AnyDesk. I am happy to share everything you need to resolve this issue except project repo. If you want all the packages list that I am using in my Django projects then you can check below.


`
amqp==2.5.2
asn1crypto==0.24.0
awesome-slugify==1.6.5
beautifulsoup4==4.8.0
billiard==3.6.1.0
bitly-api==0.3
celery==4.4.0
certifi==2019.6.16
cffi==1.12.3
chardet==3.0.4
cryptography==2.7
DateTime==4.3
defusedxml==0.6.0
Django==2.2.4
django-allauth==0.40.0
django-appconf==1.0.3
django-celery-beat==1.5.0
django-celery-results==1.1.2
django-cors-headers==3.4.0
django-crontab==0.7.1
django-rest-auth==0.9.5
django-timezone-field==4.0
django-webpush==0.3.1
djangorestframework==3.10.2
djangorestframework-jwt==1.11.0
djangorestframework-simplejwt==4.3.0
http-ece==1.1.0
idna==2.8
importlib-metadata==1.5.0
kombu==4.6.7
lxml==4.5.2
money-parser==0.0.1
numpy==1.17.0
oauthlib==3.1.0
pandas==0.25.0
Pillow==6.2.1
pinax-notifications==5.0.3
psycopg2==2.8.3
psycopg2-binary==2.8.5
py-vapid==1.7.0
pycparser==2.19
pycrypto==2.6.1
pyfcm==1.4.7
PyJWT==1.7.1
pyOpenSSL==19.0.0
python-crontab==2.4.0
python-dateutil==2.8.0
python-memcached==1.59
python-settings==0.2
python3-openid==3.1.0
pytz==2019.2
pywebpush==1.9.4
redis==3.5.3
regex==2020.7.14
requests==2.24.0
requests-oauthlib==1.2.0
simplejson==3.16.0
six==1.12.0
soupsieve==1.9.3
sqlparse==0.3.0
Unidecode==0.4.21
urllib3==1.25.3
vine==1.3.0
zipp==2.1.0
zope.interface==4.6.0
`