curlでhttpヘッダをごにょごにょする

httpのレスポンスヘッダに項目追加される案件をやったので、これを確認する手順を調べた。

# 1.curl
 curlを使うと、リクエストヘッダ、レスポンスヘッダを見ることができる。

```bash
$ curl --help
Usage: curl [options...] <url>
Options: (H) means HTTP/HTTPS only, (F) means FTP only
....
 -D, --dump-header FILE  Write the received headers to FILE
 -L, --location      Follow redirects (H)
     --location-trusted  Like '--location', and send auth to other hosts (H)
 -o, --output FILE   Write to FILE instead of stdout
 -v, --verbose       Make the operation more talkative
 -i, --include       Include protocol headers in the output (H/F)
```

## curlコマンド例
- -v で、リクエストヘッダ、レスポンスヘッダが標準エラーに出力されれるので、 2> HEADER.txt でリダイレクトして、ファイルに出力する。
- -o BODY.html で```<html>〜</html>```をファイルに出力する。
- -i ヘッダを出力ファイルに含める。

```
curl -L http://www.sample.com -v -o BODY.html 2> HEADER.txt
```

## 結果：リクエスト、レスポンスのやりとり
- * : curl自体の出力のようです。
- > : リクエスト
- < : レスポンス

```text:HEADER.txt
* Rebuilt URL to: www.nttdata.co.jp/
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 210.227.41.5...
* TCP_NODELAY set
* Connected to www.nttdata.co.jp (210.227.41.5) port 80 (#0)
> GET / HTTP/1.1
> Host: www.nttdata.co.jp
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< Date: Sun, 02 Jun 2019 05:10:20 GMT
< Server: Apache
< Location: http://www.nttdata.com/jp/ja/
< Content-Length: 237
< Connection: close
< Content-Type: text/html; charset=iso-8859-1
< Set-Cookie: server-20480-%3Fmntdlb01%3Fsfarm-web_ap=ECBBGDAKFAAA; Path=/
< 
  0   237    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Closing connection 0
* Issue another request to this URL: 'http://www.nttdata.com/jp/ja/'
*   Trying 2600:140b:b000:181::1962...
* TCP_NODELAY set
* Connected to www.nttdata.com (2600:140b:b000:181::1962) port 80 (#1)
> GET /jp/ja/ HTTP/1.1
> Host: www.nttdata.com
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< Server: AkamaiGHost
< Content-Length: 0
< Location: https://www.nttdata.com/jp/ja/
< Expires: Sun, 02 Jun 2019 05:10:20 GMT
< Cache-Control: max-age=0, no-cache, no-store
< Pragma: no-cache
< Date: Sun, 02 Jun 2019 05:10:20 GMT
< Connection: keep-alive
< 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Connection #1 to host www.nttdata.com left intact
* Issue another request to this URL: 'https://www.nttdata.com/jp/ja/'
*   Trying 2600:140b:b000:181::1962...
* TCP_NODELAY set
* Connected to www.nttdata.com (2600:140b:b000:181::1962) port 443 (#2)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [221 bytes data]
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [108 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [2725 bytes data]
.... httpsの認証MSGは割愛 .....
*  SSL certificate verify ok.
> GET /jp/ja/ HTTP/1.1
> Host: www.nttdata.com
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
< Server: Microsoft-IIS/8.5
< X-Frame-Options: SAMEORIGIN
< Content-Length: 62605
< Expires: Sun, 02 Jun 2019 05:10:21 GMT
< Cache-Control: max-age=0, no-cache, no-store
< Pragma: no-cache
< Date: Sun, 02 Jun 2019 05:10:21 GMT
< Connection: keep-alive
< Set-Cookie: ASP.NET_SessionId=oaainwvmb0ucpqa15cwqtlbn; path=/; HttpOnly
< Set-Cookie: SC_ANALYTICS_GLOBAL_COOKIE=66b3e38b1f5f478794fcb222791bb555|False; expires=Wed, 30-May-2029 05:10:20 GMT; path=/; HttpOnly
< Set-Cookie: GlobalGeoipRedirection=0; expires=Tue, 02-Jul-2019 05:10:21 GMT; path=/
< Set-Cookie: StayonGlobal=0; expires=Tue, 02-Jul-2019 05:10:21 GMT; path=/
< 
{ [1479 bytes data]
100 62605  100 62605    0     0  85161      0 --:--:-- --:--:-- --:--:-- 94284
* Connection #2 to host www.nttdata.com left intact
```
## 結果：html本体
```html:BODY.html
<!DOCTYPE html>
<html lang="">
<head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <meta name="format-detection" content="telephone=no">
    <link rel="shortcut icon" type="image/png" href="/assets/NTTDataJapan/css/i/favicon.png" />
    <link rel="apple-touch-icon-precomposed" type="image/png" href="/assets/NTTDataJapan/css/i/apple-touch-icon-precomposed.png">
....
<title>NTTデータ公式サイト</title>
<meta name="keywords" content=""/>
<meta name="description" content="NTTデータのWebサイトです。NTTデータがご提供し
ているサービスやソリューション、企業情報をご紹介しています。"/>
<meta name="google-site-verification" content="VIoqwbIi3FYycYilvfYVFscme98AO2mV10D1RXdU8P4" />
....
</head>
<body>
    <div class="emea-ntt">
            <div class="left-main-menu primary-menu-component slider">
        <div class="submenu-holder">
....
</body>
</html>
```
# 2.curlでtraceする。おすすめ。
http通信全体がトレース取得される。
[参考](https://qiita.com/takc923/items/1b508bb370c78b7a9d44)

```$ curl -L  www.nttdata.co.jp --trace-ascii  trace.log -o body.html```
 trace.logにトレースログが出力され、body.htmlに```<html>...</html>```が出力される。



# 3.telnetで８０番ポートを叩く
　telnetへの入力は緑、応答は白。

## port80でtelnet
<font color="LightGreen"><pre>
$ telnet www.nttdata.co.jp 80
</pre></font>

```
Trying 210.227.41.5...
Connected to www.nttdata.co.jp.
Escape character is '^]'.
```
## GET(/)
<font color="LightGreen"><pre>
GET / HTTP/1.1
Host: www.nttdata.co.jp
Accept: */*
（空エンター)
</font>

## 帰ってきたレスポンス
```
HTTP/1.1 301 Moved Permanently
Date: Sun, 02 Jun 2019 05:29:38 GMT
Server: Apache
Location: http://www.nttdata.com/jp/ja/
Content-Length: 237
Connection: close
Content-Type: text/html; charset=iso-8859-1
Set-Cookie: server-20480-%3Fmntdlb01%3Fsfarm-web_ap=EBBBGDAKFAAA; Path=/

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="http://www.nttdata.com/jp/ja/">here</a>.</p>
</body></html>
Connection closed by foreign host.
```

## 別のロケーションへ再度リクエスト
```HTTP/1.1 301 Moved Permanently```、と移動されたと応答があったので、そこへリクエスト。

<font color="LightGreen"><pre>
$ telnet www.nttdata.co.jp 80
</pre></font>

```
Trying 210.227.41.5...
Connected to www.nttdata.co.jp.
Escape character is '^]'.
```

## get(/jp/ja/)

<font color="LightGreen"><pre>
GET /jp/ja/ HTTP/1.1
Host: www.nttdata.com
Accept: */*
(空エンター)
</pre></font>

## 帰ってきたレスポンス。```HTTP/1.1 200 OK```となり、表示される。

```
HTTP/1.1 200 OK
Date: Sun, 02 Jun 2019 05:31:43 GMT
Server: Apache
Last-Modified: Mon, 25 Feb 2019 07:15:45 GMT
Accept-Ranges: bytes
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
Set-Cookie: server-20480-%3Fmntdlb01%3Fsfarm-web_ap=EBBBGDAKFAAA; Path=/

172
<!doctype html>
<!--[if lt IE 7]><html class="no-js ie6 oldie" lang="ja"><![endif]-->
<!--[if IE 7]><html class="no-js ie7 oldie" lang="ja"><![endif]-->
<!--[if IE 8]><html class="no-js ie8 oldie" lang="ja"><![endif]-->
<!--[if gt IE 8]><!--><html class="no-js" lang="ja"><!--<![endif]-->
<head>
<meta charset="utf-8">
<title>NTTデータ公式サイト</title>
.....
</body>
</html>

0

Connection closed by foreign host.

```