Generate HTTP Request
Published: 2021-10-11 05:27:46

---
title: Generate HTTP Request
published: true
date: 2021-10-10 18:33:27 UTC
tags: 
canonical_url: https://kracekumar.com/post/print_http_request/
---
<img src="https://upload.wikimedia.org/wikipedia/commons/thumb/5/5b/HTTP_logo.svg/2880px-HTTP_logo.svg.png">



[HTTPie](https://httpie.io/cli) is a command-line utility for making HTTP requests with more straightforward syntax(controversial, I agree). The interesting feature is `--offline` flag which prints HTTP raw request text. The client sends the HTTP request to the server, and the server responds to the request. It's an alternate to [curl](https://curl.se/).



## HTTP Syntax



[HTTP Flow and syntrax from Wikipedia.](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol)

>  A client sends *request messages* to the server, which consist of
>
>  - a request line, consisting of the case-sensitive request method, a [space](https://en.wikipedia.org/wiki/Space_(punctuation)), the request target, another space, the protocol version, a [carriage return](https://en.wikipedia.org/wiki/Carriage_return), and a [line feed](https://en.wikipedia.org/wiki/Line_feed) (e.g. *GET /images/logo.png HTTP/1.1*)
>  - zero or more [request header fields](https://en.wikipedia.org/wiki/HTTP_request_header_field), each consisting of the case-insensitive field name, a colon, optional leading [whitespace](https://en.wikipedia.org/wiki/Whitespace_(computer_science)), the field value, and optional trailing whitespace (e.g. *Accept-Language: en*), and ending with a carriage return and a line feed.
>  - an empty line, consisting of a carriage return and a line feed;
>  - an optional [message body](https://en.wikipedia.org/wiki/HTTP_message_body).
>  - In the HTTP/1.1 protocol, all header fields except *Host* are optional.
>  - A request line containing only the path name is accepted by servers to maintain compatibility with HTTP clients before the HTTP/1.0 specification in [RFC](https://en.wikipedia.org/wiki/RFC_(identifier)) [1945](https://datatracker.ietf.org/doc/html/rfc1945).

Throughout the post, I'll use `--offline` feature to understand how the HTTP request structure looks for educational purposes.



## 🉑 Accept only JSON response data 🉑



- HTTPie uses `:` to separate header key and values in the terminal. `Accept:application/json`.

- `Accept` [header tells the server, the client accepts only specific MIME types, here `JSON`. ](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept)

```bash
$http --offline httpbin.org/get Accept:application/json
GET /get HTTP/1.1
Accept: application/json
Accept-Encoding: gzip, deflate
Connection: keep-alive
Host: httpbin.org
User-Agent: HTTPie/2.5.0
# Sample request sent to the server
$ http httpbin.org/get Accept:application/json
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 310
Content-Type: application/json
Date: Sun, 10 Oct 2021 07:21:40 GMT
Server: gunicorn/19.9.0

{
    "args": {},
    "headers": {
        "Accept": "application/json",
        "Accept-Encoding": "gzip, deflate",
        "Host": "httpbin.org",
        "User-Agent": "HTTPie/2.5.0",
        "X-Amzn-Trace-Id": "Root=1-61629484-3b25a3631e2a89bf60f2600e"
    },
    "origin": "xxx.xxx.xxx.xxx",
    "url": "http://httpbin.org/get"
}
```

- You can pass more than one value in the `Accept` header seperated by comma.

- The response from the server contains `Content-Type` as JSON. The server can choose to ignore the `Accept` header.



## 📚 Request the Tamil language version of duckduckgo 📚

- `Accept-Language` [header instructs the web server to deliver particluar language version of the page.](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Language)
- Here `Accept-Language` is set to `ta`. `ta` is [ISO 639-1 code.](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes)

```bash
$http --offline  https://duckduckgo.com Accept-Language:ta
GET / HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: ta
Connection: keep-alive
Host: duckduckgo.com
User-Agent: HTTPie/2.5.0

$http  https://duckduckgo.com Accept-Language:ta
HTTP/1.1 200 OK
Cache-Control: no-cache
Connection: keep-alive
Content-Encoding: gzip
Content-Security-Policy: default-src 'none' ; connect-src  https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/ https://spreadprivacy.com/ https://duck.co ; manifest-src  https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/ https://spreadprivacy.com/ ; media-src  https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/ https://spreadprivacy.com/ ; script-src blob:  https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/ https://spreadprivacy.com/ 'unsafe-inline' 'unsafe-eval' ; font-src data:  https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/ https://spreadprivacy.com/ ; img-src data:  https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/ https://spreadprivacy.com/ ; style-src  https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/ https://spreadprivacy.com/ 'unsafe-inline' ; object-src 'none' ; worker-src blob: ; child-src blob:  https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/ https://spreadprivacy.com/ ; frame-src blob:  https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/ https://spreadprivacy.com/ ; form-action  https://duckduckgo.com https://*.duckduckgo.com https://3g2upl4pq6kufc4m.onion/ https://duckduckgogg42xjoc72x3sjasowoarfbgcmvfimaftt6twagswzczad.onion/ https://spreadprivacy.com/ https://duck.co ; frame-ancestors 'self' ; base-uri 'self' ; block-all-mixed-content ;
Content-Type: text/html; charset=UTF-8
Date: Sat, 09 Oct 2021 21:44:33 GMT
ETag: W/"6161a338-16b8"
Expect-CT: max-age=0
Expires: Sat, 09 Oct 2021 21:44:32 GMT
Permissions-Policy: interest-cohort=()
Referrer-Policy: origin
Server: nginx
Strict-Transport-Security: max-age=31536000
Transfer-Encoding: chunked
Vary: Accept-Encoding
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1;mode=block

<!DOCTYPE html>
...
<body id="pg-index" class="page-index body--home">
	<script type="text/javascript" src="/tl5.js"></script>
<script type="text/javascript" src="/lib/l123.js"></script>
<script type="text/javascript" src="/locale/ta_IN/duckduckgo50.js"></script>
<script type="text/javascript" src="/util/u588.js"></script>
<script type="text/javascript" src="/d3012.js"></script>
...
<noscript>
    <div class="tag-home">
        <div class="tag-home__wrapper">
            <div class="tag-home__item">
                Privacy, simplified&period;
                <span class="hide--screen-xs"><a href="/about" class="tag-home__link"> மேலும் கற்க</a></span>
            </div>
        </div>
    </div>
</noscript>
...
</html>
```

- The header can carry more than one value separated by a comma.
- When more than one value is present, an extra parameter `q=0.5` represents the weightage among the values. Example: `Accept-Language: ta,fr;q=0.50`.
- One of the link description is in Tamil, `மேலும் கற்க`.

## 🔑 Post Authorization token as part of login 🔑

- `Authorization` [header can be used to provide credentials to authenticate the user](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Authorization).
- Example: `Authorization:"Bearer my-dear-darkness"`.
- Because of space between Bearer and the value quotes is mandatory. 

```bash
$http --offline POST httpbin.org/auth Authorization:"Bearer my-dear-darkness"
POST /auth HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Authorization: Bearer my-dear-darkness
Connection: keep-alive
Content-Length: 0
Host: httpbin.org
User-Agent: HTTPie/2.5.0

```



## 🍪 Send Cookie as part of a request 🍪

- The server can send HTTP Cookie as part of the response, and the client can send the Cookies in subsequent requests.
- `Cookie` [header carries the cookies previously sent by the user.](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Authorization)
- Semi-colon separates multiple cookie pairs.
- Example: `"name=not-unique;value=23"`

```bash
$http --offline http://pie.dev/cookies Cookie:"name=not-unique;value=23"
GET /cookies HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Cookie: name=not-unique;value=23
Host: dev.pie
User-Agent: HTTPie/2.5.0
$http http://pie.dev/cookies Cookie:"name=not-unique;value=23"
HTTP/1.1 200 OK
CF-Cache-Status: DYNAMIC
CF-RAY: 69be1b984c743c0c-BLR
Connection: keep-alive
Content-Encoding: gzip
Content-Type: application/json
Date: Sun, 10 Oct 2021 07:24:13 GMT
NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=2Zt9wQcKQYStY5dPukoEXFwn7K6pKNaVTiKljZ5h9oL3mcQZj0khYq8Kmzo8PmQinPncStZeHASetQqHzCODe0wbrljPEIJxCWGdRWMbry9rWOG%2FBheDvJs7"}],"group":"cf-nel","max_age":604800}
Server: cloudflare
Transfer-Encoding: chunked
access-control-allow-credentials: true
access-control-allow-origin: *
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400, h3-28=":443"; ma=86400, h3-27=":443"; ma=86400

{
    "cookies": {
        "name": "not-unique",
        "value": "23"
    }
}
```



## ✉️ Send JSON Request ✉️

- HTTPie supports sending form-encoded values or JSON values. `=` sign indiciates JSON key-value pair.
- `=` is useful for primitive values like `number, string, null, boolean`.
- Example: `lang-py`

```bash
$http --offline PUT http://pie.dev/put lang=py version=3.10
PUT /put HTTP/1.1
Accept: application/json, */*;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 33
Content-Type: application/json
Host: pie.dev
User-Agent: HTTPie/2.5.0

{
    "lang": "py",
    "version": "3.10"
}
$HTTP/1.1 200 OK
CF-Cache-Status: DYNAMIC
CF-RAY: 69be1e017fb53c12-BLR
Connection: keep-alive
Content-Encoding: gzip
Content-Type: application/json
Date: Sun, 10 Oct 2021 07:25:52 GMT
NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=GMW5PQHMwjxpVUeDG7uWo6M%2FiKLMzmQLd1e5BIG3AXljmuQgwCP9nzrFPdaidR2wL14eisfiViDhumDHpepgNB6yIrsVKXRybHa5tRqmH7lUDoQdRqCK1Ijg"}],"group":"cf-nel","max_age":604800}
Server: cloudflare
Transfer-Encoding: chunked
access-control-allow-credentials: true
access-control-allow-origin: *
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400, h3-28=":443"; ma=86400, h3-27=":443"; ma=86400

{
    "args": {},
    "data": "{\"lang\": \"py\", \"version\": \"3.10\"}",
    "files": {},
    "form": {},
    "headers": {
        "Accept": "application/json, */*;q=0.5",
        "Accept-Encoding": "gzip",
        "Cdn-Loop": "cloudflare",
        "Cf-Connecting-Ip": "49.207.222.139",
        "Cf-Ipcountry": "IN",
        "Cf-Ray": "69be1e017fb53c12-FRA",
        "Cf-Visitor": "{\"scheme\":\"http\"}",
        "Connection": "Keep-Alive",
        "Content-Length": "33",
        "Content-Type": "application/json",
        "Host": "pie.dev",
        "User-Agent": "HTTPie/2.5.0"
    },
    "json": {
        "lang": "py",
        "version": "3.10"
    },
    "origin": "xxx.xxx.xxx.xxxx",
    "url": "http://pie.dev/put"
}

```



## ✉️ Send non-primitive JSON values ✉️

- `:=` means the JSON value is non-primitve values like `array/list and dictionary`.
- Single quote carries the value.
- Example: `os:='["GNU/Linux", "Mac OSX"]'`.

```bash
$http --offline  PUT http://pie.dev/put lang=py version=3.10 os:='["GNU/Linux", "Mac OSX"]'
PUT /put HTTP/1.1
Accept: application/json, */*;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 65
Content-Type: application/json
Host: pie.dev
User-Agent: HTTPie/2.5.0

{
    "lang": "py",
    "os": [
        "GNU/Linux",
        "Mac OSX"
    ],
    "version": "3.10"
}
$http  PUT http://pie.dev/put lang=py version=3.10 os:='["GNU/Linux", "Mac OSX"]'
HTTP/1.1 200 OK
CF-Cache-Status: DYNAMIC
CF-RAY: 69be224ce97c3c0c-BLR
Connection: keep-alive
Content-Encoding: gzip
Content-Type: application/json
Date: Sun, 10 Oct 2021 07:28:48 GMT
NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=dgY0R%2FqomBbumKs0IVB7GjPR2chhHJ9wVKAzW33IOnvZ%2Fs4l2LYnvKKeXo6Xhd162AYKGzyIrK4pNrYdH8SEs1NvGmYfJqEDIPmfUOELqpC6HK9iP1zIENa0"}],"group":"cf-nel","max_age":604800}
Server: cloudflare
Transfer-Encoding: chunked
access-control-allow-credentials: true
access-control-allow-origin: *
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400, h3-28=":443"; ma=86400, h3-27=":443"; ma=86400

{
    "args": {},
    "data": "{\"lang\": \"py\", \"version\": \"3.10\", \"os\": [\"GNU/Linux\", \"Mac OSX\"]}",
    "files": {},
    "form": {},
    "headers": {
        "Accept": "application/json, */*;q=0.5",
        "Accept-Encoding": "gzip",
        "Cdn-Loop": "cloudflare",
        "Cf-Connecting-Ip": "49.207.222.139",
        "Cf-Ipcountry": "IN",
        "Cf-Ray": "69be224ce97c3c0c-FRA",
        "Cf-Visitor": "{\"scheme\":\"http\"}",
        "Connection": "Keep-Alive",
        "Content-Length": "65",
        "Content-Type": "application/json",
        "Host": "pie.dev",
        "User-Agent": "HTTPie/2.5.0"
    },
    "json": {
        "lang": "py",
        "os": [
            "GNU/Linux",
            "Mac OSX"
        ],
        "version": "3.10"
    },
    "origin": "xx.xxx.xxx.xxx",
    "url": "http://pie.dev/put"
}

```



## 📤 Upload files 📤



- `-f` flag indicates the data is form-encoded values.
- `@` symbol indicates the value is a file. 
- Example: `cv@hello-world.txt`. `cv` form field name.

```bash
$cat hello-world.txt
hello-world
$http --offline -f POST pie.dev/post name='Krace' cv@hello-world.txt
POST /post HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 277
Content-Type: multipart/form-data; boundary=183f70d3da41432d95bcd839e2cc20e2
Host: pie.dev
User-Agent: HTTPie/2.5.0

--183f70d3da41432d95bcd839e2cc20e2
Content-Disposition: form-data; name="name"

Krace
--183f70d3da41432d95bcd839e2cc20e2
Content-Disposition: form-data; name="cv"; filename="hello-world.txt"
Content-Type: text/plain

hello-world

--183f70d3da41432d95bcd839e2cc20e2--
$ http -f POST pie.dev/post name='Krace' cv@hello-world.txt
HTTP/1.1 200 OK
CF-Cache-Status: DYNAMIC
CF-RAY: 69be2564a8fc3c0c-BLR
Connection: keep-alive
Content-Encoding: gzip
Content-Type: application/json
Date: Sun, 10 Oct 2021 07:30:54 GMT
NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=yNaTSZJ7ouYH%2FSJZw6LIR5vNl6KNTFeDKF1u8V60abb3ClKLzdOj0zkchcIAWqTvZDbxXm5MnffDkCLdqviMQuAo7DqFA2GH%2Bm%2FiZ6sH90oOr0HyFGAuS4Gp"}],"group":"cf-nel","max_age":604800}
Server: cloudflare
Transfer-Encoding: chunked
access-control-allow-credentials: true
access-control-allow-origin: *
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400, h3-28=":443"; ma=86400, h3-27=":443"; ma=86400

{
    "args": {},
    "data": "",
    "files": {
        "cv": "hello-world\n"
    },
    "form": {
        "name": "Krace"
    },
    "headers": {
        "Accept": "*/*",
        "Accept-Encoding": "gzip",
        "Cdn-Loop": "cloudflare",
        "Cf-Connecting-Ip": "49.207.222.139",
        "Cf-Ipcountry": "IN",
        "Cf-Ray": "69be2564a8fc3c0c-FRA",
        "Cf-Visitor": "{\"scheme\":\"http\"}",
        "Connection": "Keep-Alive",
        "Content-Length": "277",
        "Content-Type": "multipart/form-data; boundary=252c6fcd1dcc40e09de54958660d672d",
        "Host": "pie.dev",
        "User-Agent": "HTTPie/2.5.0"
    },
    "json": null,
    "origin": "xxx.xxx.xxx.xxxx",
    "url": "http://pie.dev/post"
}

```



[Explore more in HTTPie website.](https://httpie.io/)

# 🐈 How to verify the the generated request works? 🐈



[Netcat](https://en.wikipedia.org/wiki/Netcat) is a utility for sending and receiving data in network connection using TCP or UDP. Netcat take hostname, port, and body as arguments and sends it to the server and displays the response.

```bash
$http --offline  PUT http://httpbin.org/put lang=py version=3.10 os:='["GNU/Linux", "Mac OSX"]' | nc httpbin.org 80
HTTP/1.1 200 OK
Date: Sun, 10 Oct 2021 08:27:58 GMT
Content-Type: application/json
Content-Length: 631
Connection: keep-alive
Server: gunicorn/19.9.0
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true

{
  "args": {},
  "data": "{\"lang\": \"py\", \"version\": \"3.10\", \"os\": [\"GNU/Linux\", \"Mac OSX\"]}",
  "files": {},
  "form": {},
  "headers": {
    "Accept": "application/json, */*;q=0.5",
    "Accept-Encoding": "gzip, deflate",
    "Content-Length": "65",
    "Content-Type": "application/json",
    "Host": "httpbin.org",
    "User-Agent": "HTTPie/2.5.0",
    "X-Amzn-Trace-Id": "Root=1-6162a40e-34b9a83f40868b4a73e8fa09"
  },
  "json": {
    "lang": "py",
    "os": [
      "GNU/Linux",
      "Mac OSX"
    ],
    "version": "3.10"
  },
  "origin": "xxx.xxx.xxx.xxx",
  "url": "http://httpbin.org/put"
}
```



Netcat doesn't support encrypted network connections.

<blockquote class="twitter-tweet" data-dnt="true" data-theme="dark"><p lang="en" dir="ltr">❓How to understand what HTTP request the client sends to the server?<br>💡Use HTTPie `--offline` feature to print the HTTP request.<br><br>Details on how to send various HTTPie options in the blog post - <a href="https://t.co/mIEG2FpBkP">https://t.co/mIEG2FpBkP</a><a href="https://twitter.com/hashtag/TIL?src=hash&amp;ref_src=twsrc%5Etfw">#TIL</a> <a href="https://twitter.com/hashtag/Python?src=hash&amp;ref_src=twsrc%5Etfw">#Python</a> <a href="https://twitter.com/hashtag/CLI?src=hash&amp;ref_src=twsrc%5Etfw">#CLI</a> <a href="https://t.co/kBvfc9QQAt">pic.twitter.com/kBvfc9QQAt</a></p>&mdash; kracekumar || கிரேஸ்குமார் (@kracetheking) <a href="https://twitter.com/kracetheking/status/1447289495234105345?ref_src=twsrc%5Etfw">October 10, 2021</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

# References

- HTTPie - https://httpie.io/
- HTTPBin - http://httpbin.org
- NetCat - https://en.wikipedia.org/wiki/Netcat
- HTTP MDN Docs: https://developer.mozilla.org/en-US/docs/Web/HTTP
- HTTP Header Image: https://upload.wikimedia.org/wikipedia/commons/thumb/5/5b/HTTP_logo.svg/2880px-HTTP_logo.svg.png