Requests module in Python: 📭 How to use requests in Python ?
Tags: programming, webdev, python, opensource
Published: 2021-11-01 09:12:28

## Getting Started with Requests

👉 Every web developer knows HTTP methods and almost their usage. So, how we can handle those methods using python ? Of course one of the efficient way to use those methods is **requests module**.

📌 The requests module is quite common used module in python. In this essay we are going to investigate what **requests** is and how to use with most common HTTP methods.

To begin using the **requests** module it should be installed before. To do this, run the following code:

```
pip install requests
```
After installing the module it is time to code ! Make sure the module is imported before code. 

```python
import requests
```

That's it! Now we are ready to use **requests**. 🛠 
Let's take a look at what we are able to do using **requests**.

###  The GET Request
One of the most two common HTTP methods *GET* is used for reach out the data from specific resource. Python provides us *get()* method that takes data resource as argument in **requests** module. 

To be able to do that we will make GET request to *http://api.open-notify.org/*. This API page has been created as an open source project for the purpose of information so, you can find something on this page about the ISS and astronauts in the space right now.
Alright, let's code !

```python
   def req():
       response = requests.get("http://api.open-notify.org/")
       print(response)
```

Running out the **req()** function in the *main* method we will have our the first request in this manner.

```python
   def __name__ == "__main__":
       req()
```
After run the code above you probably are going to get the following response

```
Output:
<Response [200]>
```
As you can see in the output the *get()* method returned us response by status code 200. That means the request was successful to resource.
 
📌 Note that there will more info associated with *HTTP Status Codes* later.

Alright we have a response and its class and we know that all the content is stored in response variable so that we need to access them now. To do this there are some way that we can use. So why don't we try some of them ⁉️

Before getting content of response we are going to check status whether successful or not. It might be useful to make decision on the code.

```python
   def status():
       if response.status_code == 200:
           return True
       return False
```

If the output is *True* then the request was successful. 
‼️ However, note that you can not check out the all status codes one by one. Instead of that you can use **raise_for_status()** function that will do necessary process for you.

```python
   def status():
       try:
           response = requests.get("http://api.open-notify.org/asdasd")
           response.raise_for_status()
       except Exception as e:
           print(f'Error occurred: {e}') 
```
If an exception occurs you would probably get an *HTTPError* in the output. 

So, after checking out the response we can focus on the content of response variable. The content of response variable that you got from the GET method may contain valuable information called as payload. To see the content we are going to use **content** by response variable. 

```python
   def content(url="http://api.open-notify.org/astros.json"):
       try:
           response = requests.get(url)
           content = response.content
           print(content)
       except Exception as e:
           print(f'Error occurred: {e}') 
```
```
Output:
b'{"message": "success", "number": 10, "people": [{"craft": "ISS", "name": "Mark Vande Hei"}, {"craft": "ISS", "name": "Oleg Novitskiy"}, {"craft": "ISS", "name": "Pyotr Dubrov"}, {"craft": "ISS", "name": "Thomas Pesquet"}, {"craft": "ISS", "name": "Megan McArthur"}, {"craft": "ISS", "name": "Shane Kimbrough"}, {"craft": "ISS", "name": "Akihiko Hoshide"}, {"craft": "ISS", "name": "Anton Shkaplerov"}, {"craft": "ISS", "name": "Klim Shipenko"}, {"craft": "ISS", "name": "Yulia Pereslid"}]}'
```
This function returns us raw byte value of response payload like output above. The output inform us about astronauts in the space right now. 

We usually want to get in different form like json, string etc. To convert the response into string just enough using the **text** function.

```python
   def convert_string(url="http://api.open-notify.org/astros.json"):
    
       try:
           response = requests.get(url)
           string_content = response.text
           print(f"The content of response => {string_content}")
       except Exception as e:
           print(f'Error occurred: {e}') 
```
The output will be into string form like this :
```
The content of response => {"message": "success", "number"...
```
If you take a look at the content of response you are going to see it is like *JSON*. However, how can we access as JSON format? There is an easy way to get content by JSON so **requests** module provide us its own *json()* function to convert content of response to JSON format.

```python
   def convert_json(url="http://api.open-notify.org/astros.json"):
    
       try:
           response = requests.get(url)
           json_content = response.json()
           print(f"The content of response => {json_content}")
           print(type(json_content))

       except Exception as e:
           print(f'Error occurred: {e}') 
```
Almost the same output but not in the text form in this time. If you check out the type of *json_content* variable in the code above you will get <class 'dict'>.

You can also use **pprint** module instead of print function while you write the JSON content. You would get the output like following.

```
The content of response => {'message': 'success',
 'number': 10,
 'people': [{'craft': 'ISS', 'name': 'Mark Vande Hei'},
            {'craft': 'ISS', 'name': 'Oleg Novitskiy'},
            {'craft': 'ISS', 'name': 'Pyotr Dubrov'},
            {'craft': 'ISS', 'name': 'Thomas Pesquet'},
            {'craft': 'ISS', 'name': 'Megan McArthur'},
            {'craft': 'ISS', 'name': 'Shane Kimbrough'},
            {'craft': 'ISS', 'name': 'Akihiko Hoshide'},
            {'craft': 'ISS', 'name': 'Anton Shkaplerov'},
            {'craft': 'ISS', 'name': 'Klim Shipenko'},
            {'craft': 'ISS', 'name': 'Yulia Pereslid'}]}
```

#### Working with Headers 🕵🏼

To get more information about the response that you got there is need to access *headers* of the response. To view those headers python **requests** module provides us *headers* function. Take a look at this how to implement *headers* on response of get request.

```python
   def get_headers(url="http://api.open-notify.org/astros.json"):

       try:
           response = requests.get(url)
           headers = response.headers
           print("Headers: ", end="\n\n")
           pprint(dict(headers))
        
       except Exception as e:
           print(f'Error occurred: {e}') 
```

```
Output:
Headers:

{'Connection': 'keep-alive',
 'Content-Length': '505',
 'Content-Type': 'application/json',
 'Date': 'Mon, 18 Oct 2021 08:39:23 GMT',
 'Server': 'nginx/1.10.3',
 'access-control-allow-origin': '*'}
```
A lot of details we can get by the headers as you can see in the output above. To access one of them we can also use python dictionary properties. For instance in the code above we can check the server or whatever you want using this *headers.get('Server')*. You probably are going to obtain that response *nginx/1.10.3*.  

#### Query Parameters

Sometimes we may want to filter in the response.*Query parameters* can be used to customize the response of GET method. To do this the parameters that we have determined should be passed to *params* in **get()** function. 

For instance;

```python
   def query_params(url="http://api.open-notify.org/iss-pass.json"):
       params = {
           "lat":"25",
           "lon":"35"
           }
       try:
           response = requests.get(url, params=params)
           json_content = response.json()
           print(f"Estimate of ISS location according to parameter => ", end="")
           pprint(json_content)
       except Exception as e:
           print(f'Error occurred: {e}') 
```
```
Output:
Estimate of ISS location according to parameter => {'message': 'success',
 'request': {'altitude': 100,
             'datetime': 1634224449,
             'latitude': 25.0,
             'longitude': 35.0,
             'passes': 5},
 'response': [{'duration': 613, 'risetime': 1634270566},
              {'duration': 590, 'risetime': 1634276384},
              {'duration': 502, 'risetime': 1634300139},
              {'duration': 648, 'risetime': 1634305891},
              {'duration': 193, 'risetime': 1634311894}]}
```
The code above yields us an estimation about the ISS according to given parameter's data. When you make GET request to this url without query parameter then you would get a response message that it is must.

If you want to get information about current location of ISS then make GET request using this url: *http://api.open-notify.org/iss-now.json*.

What if we want to approach by OOP to estimate ISS location ⁉️⁉️⁉️

```python
   class EstimateISS:

         def __init__(self, url="http://api.open-notify.org/iss-pass.json"):
        
            self.lat = int(input("latitude: "))
            self.lon = int(input("longitude: "))
            self.url = url

        def estimate_request(self):
        
            params = {
                "lat": self.lat,
                "lon": self.lon
                }

            try:
                response = requests.get(self.url, params=params)
                self.json_content = response.json()
                return self.json_content

            except Exception as e:
                return f'Error occurred: {e}'

        def __repr__(self):
            return f"{pprint(self.estimate_request())}"
```
Have a look at the code above that what it does. When the code is run firstly gets latitude and longitude in **__init__** function and moves to other *key* function, **estimate_request()**. The both parameters and other processes occur in this function. If no error, it estimation about ISS location according to given values

```
latitude: 45
longitude: 30


{'message': 'success',
 'request': {'altitude': 100,
             'datetime': 1634908665,
             'latitude': 45.0,
             'longitude': 30.0,
             'passes': 5},
 'response': [{'duration': 600, 'risetime': 1634956714},
              {'duration': 655, 'risetime': 1634962480},
              {'duration': 627, 'risetime': 1634968328},
              {'duration': 635, 'risetime': 1634974172},
              {'duration': 652, 'risetime': 1634979986}]}
```
### Working on POST Method
One of the another most famous method is **POST**. To be able to use this method we need a body that gets data its inside. The body gets dictionary, a list of tuple, byte or object according to corresponding function parameters. Firstly, we are going to use that site *https://httpbin.org/* to post method. 

```python
   def post_data(url="https://httpbin.org/post"):
       data = {"key": "value"}
       response = requests.post(url, data=data)
       print(response)
```
```
   Response [200]
```

We got the status code 200 in the response above. So if we need to access response headers is there any header in the response that returned in post method ? 
Of course, we can demonstrate the all headers like what we did for get. 

```python
   def post_data(url="https://httpbin.org/post"):
       data = {"key": "value"}
       response = requests.post(url, data=data)
       headers = response.headers
       pprint(dict(headers))
```
```
{'Access-Control-Allow-Credentials': 'true',
 'Access-Control-Allow-Origin': '*',
 'Connection': 'keep-alive',
 'Content-Length': '479',
 'Content-Type': 'application/json',
 'Date': 'Mon, 25 Oct 2021 11:02:35 GMT',
 'Server': 'gunicorn/19.9.0'}
```
In the output above we can see the headers for post method we sent. Note that those processes are related to web requests and responses so that we can build up communication by *JSON* data. To use that in the post method just pass data as **json** parameter. 

```python
   def post_json_data(url="https://httpbin.org/post"):
       data = {"key": "value"}
       response = requests.post(url, json=data)
       headers = response.headers
       content = response.content
       dec = content.decode("utf-8")
       json_cont = json.loads(dec)
       pprint(json_cont)
```

In the code above we have changed the *data* by *json* and run again. Let's look at the data inside of the response content.

```
{'args': {},
 'data': '{"key": "value"}',
 'files': {},
 'form': {},
 'headers': {'Accept': '*/*',
             'Accept-Encoding': 'gzip, deflate',
             'Content-Length': '16',
             'Content-Type': 'application/json',
             'Host': 'httpbin.org',
             'User-Agent': 'python-requests/2.26.0',
             'X-Amzn-Trace-Id': 'Root=1-6177e4a2-25a616ba208403692f718992'},
 'json': {'key': 'value'},
 'origin': '176.236.37.52',
 'url': 'https://httpbin.org/post'}
```
Pay attention to those points **data** and **json** inside of the response content. When you look at the *data* and *json* keys you are going to see a dictionary that we have sent. 
So far so good ‼️ 

Alright, as we said before this is related to web process so, why we don't write a mini web server and make request to that 🤔 ⁉️

Ok then let's do it ‼️

To be able to do that we'll need some modules one of them is **flask**. Firstly we will use flask to get requests and return responses. I thought it is easy way in python for now. In the another one sharing we are going to use flask with its all details.

First of all we need to import some packages from *flask* module.
```python
   from flask import Flask, request
```
Now, we can run out the flask and use its properties. 

```python
   app = Flask(__name__)

   @app.route("/test/test_get", methods=["POST", "GET"])
   def get():
       if request.method == "GET":
          return jsonify({"message": "Success!"})
       return
```
To run out flask app we need to add and run this code in main method below.

```python
   if __name__ == "__main__":
      app.run(debug=True)
```
After running out the server you will get an output about it is running. 
```
 * Serving Flask app 'test_server' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: on
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 735-221-534
 * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
```
Now, it is time to make request to our test server. Alright we need to move on other python file and write following.

```python
   def test_get():
       try: 
        response = requests.get("http://127.0.0.1:5000/test/test_get")
        if response.status_code == 200:
           print("Success!")
      except Exception as e:
           print(e)
```
Run the code above in a different python file you are going to get **Success!** if you don't have an exception. Apart of that if you go to browser by this url you would get that response *message: Success!*. 

Let's do something to make post request to the test server. Go back to the file that have *flask* and write another function called as *post()* like following.

```python
   @app.route("/test/test_post", methods=["POST", "GET"])
   def post():
       try:
           post_data = request.json
           if request.method == "POST":
              return post_data
           return
       except Exception as e:
          return e
```
Now we need a function to be able to send json data to the server via post request. Ok let's do it !

```python
   def test_post():
       header = {"Content-Type":"application/json",}
       url = "http://127.0.0.1:5000/test/test_post"
       post_data = {
               "message": "this message from post request"
           }
       json_data = json.dumps(post_data)
       try:
           response = requests.post(url=url,headers=header, data=json_data)
           return_content = json.loads(response.content)
           if response.status_code == 200:
               print("Success!")
               print("Returned data: ", return_content)
            
       except Exception as e:
           print(e)
```

The server gets request that sent from **test_post()** function and returns the content of request. When we run out the both file the outputs will be like following. Firstly the server return us status code and the function prints out message according to this status code.
```
127.0.0.1 - - [28/Oct/2021 10:45:47] "POST /test/test_post HTTP/1.1" 200 -
``` 

```
Success!
Returned data:  {'message': 'this message from post request'}
```
So far so good ! 
You can also diversify using other HTTP methods like PATCH, PUT, DELETE. We are going to write the last instance using one of them. 

This time we got PUT method to use with function. Suppose we have list database holding dictionary data in the test server and we are going to change something on the data using PUT. Let's code it! 

```python
   @app.route("/test/test_put", methods=["POST", "PUT"])
   def put():

       _db = [
           {"Id": "1",
            "user": "fake",
            "pass": "fake"}
           ]    
       try:
           put_data = request.json
           if request.method == "PUT":
              for _ in _db:
                  if _["Id"] == put_data["Id"]:
                     _["user"] = put_data["user"]
                     _["pass"] = put_data["pass"]
           print(_db)
           return put_data
       except Exception as e:
           return e
``` 
In the code above *_db* created like database and db updated in try block according to *Id* using incoming data.

```python
   def test_put():
       header = {"Content-Type":"application/json",}
       url = "http://127.0.0.1:5000/test/test_put"
       put_data = {
               "Id": "1",
               "user": "fakeuser",
               "pass": "fakepass"
           }
       json_data = json.dumps(put_data)
       try:
           response = requests.put(url=url,headers=header, data=json_data)
           return_content = json.loads(response.content)
           if response.status_code == 200:
               print("Success!")
               print("User Updated by: ", return_content["user"])
            
       except Exception as e:
           print(e)
```
When we run the code files in order, outputs will be like following.

```
Success!
User Updated by:  fakeuser
```
And the server outputs:

```
[{'Id': '1', 'user': 'fakeuser', 'pass': 'fakepass'}]
127.0.0.1 - - [01/Nov/2021 09:30:45] "PUT /test/test_put HTTP/1.1" 200 -
```
Server side also prints out the updated *_db* and status code 200. We are be able to create new record into the database using POST method and delete it again using DELETE method. Let us build up delete function to delete record that already created.

```python
   def test_delete():
       header = {"Content-Type":"application/json",}
       url = "http://127.0.0.1:5000/test/test_delete"
       delete_data = {
               "Id": "1",
               "user": "fakeuser",
               "pass": "fakepass"
           }
       json_data = json.dumps(delete_data)
       try:
           response = requests.delete(url=url,headers=header, data=json_data)
           if response.status_code == 200:
               print("Success!")
            
       except Exception as e:
           print(e)
```
Now let's write the server side.
```python
   @app.route("/test/test_delete", methods=["POST", "DELETE"])
   def delete():
       _db = [
           {"Id": "1",
            "user": "fake",
            "pass": "fake"}
           ]    
       try:
           delete_data = request.json
           if request.method == "DELETE":
               for _ in _db:
                   if _["Id"] == delete_data["Id"]:
                       _db.remove(_)
           print(_db)
           return delete_data
       except Exception as e:
           return e
```
As the last, server prints empty db if status code 200. 
```
[]
```

Thank you so much for reading the essay. The new one will come soon...
Good day !