Repository: sharkdp/bat (language: rust)

Issue:
Support for multiple syntaxes in a single file
I have a bunch of files like this. They're incoming web requests that I've captured to a file:

```
POST /foo/bar HTTP/1.1
X-Forwarded-For: 54.187.216.72
Content-Length: 3077
Cache-Control: no-cache
X-Forwarded-Proto: https
Content-Type: application/json; charset=utf-8
Host: 7261363d4b2c.ngrok.io
User-Agent: Stripe/1.0 (+https://stripe.com/docs/webhooks)
Accept: */*; q=0.5, application/xml
Accept-Encoding: gzip

{
  "id": "blahblahblahblah",
  "object": "event",
  "api_version": "2020-03-02",
  "created": 1626790174,
  "data": {
  ...
```

and it would be Dead Handy to be able to highlight the two sections independently, both the HTTP headers and the JSON body. Obviously this would require a manually specified `-l` and probably also telling bat what the boundary between the two languages is, something like this perhaps:

`bat -l http+json --boundary='\r\n\r\n' filename`


Discussion / hints:
You may find that highlighting it as YAML will give reasonable results. I personally don't think we will complicate the parser or public API/CLI by adding an argument for this use case when we already support custom syntax definitions. I will see if I can make one for you when I get some spare time. Probably I can tweak some prior work: https://github.com/kylebebak/Requester/commit/b6db9c02a86b30c50709bd6cb42bccdb5680fd31
![image](https://user-images.githubusercontent.com/11882719/126826176-772e1ed5-443d-4e66-b21c-e587b03382ef.png)

<details><summary>I have created a custom syntax definition for this - it currently supports highlighting of json, xml and html request/response bodies. I wonder if we would want to include this in `bat` by default - I'm guessing not because we normally have a 15k package control download requirement, and this seems fairly niche, though there are some Sublime Text packages for making HTTP requests which do have over 15k downloads, especially collectively. Well, I may need to create a separate git repository for it if I decide I want to maintain it separately.
Anyway, to use it, expand this "summary/details" to copy/paste the syntax definition into a `.sublime-syntax` file, and follow <a href="https://github.com/sharkdp/bat#adding-new-syntaxes--language-definitions">the instructions in the readme</a> to make it available to <b>bat</b>.
</summary>

```yaml
%YAML 1.2
---
# See http://www.sublimetext.com/docs/3/syntax.html
scope: text.http-request-response
name: HTTP Request and Response
variables:
  content_type_sep: (?=;|$)
contexts:
  main:
    - match: (?:GET|HEAD|POST|PUT|DELETE)
      scope: meta.http-verb.http-request-response keyword.operator.word.http-request-response
    - match: \bHTTP/\d\.\d\b
      scope: constant.language.protocol-version.http-request-response
      push: maybe_response
    - include: url_escape_char
    - match: /
      scope: meta.url punctuation.separator.path.directory.http-request-response
    - match: \?
      scope: punctuation.separator.sequence.querystring.http-request-response
      push: querystring
    - match: '[^&\s?/]+'
      scope: meta.url.http-request-response
    - match: $
      set: http-headers

  maybe_response:
    - match: $
      pop: true
    - match: (?=[123])
      set: [info, response_code]
    - match: (?=4)
      set: [warning, response_code]
    - match: (?=5)
      set: [error, response_code]
    - include: response_code

  warning:
    - meta_scope: markup.warning.log
    - match: ''
      pop: true

  error:
    - meta_scope: markup.error.log
    - match: ''
      pop: true

  info:
    - meta_scope: markup.info.log
    - match: ''
      pop: true

  response_code:
    - match: \d+(\.\d+)?
      scope: meta.http-status-code.http-request-response meta.number.integer.decimal.http-request-response constant.numeric.value.http-request-response
    - match: \S.*$
      scope: meta.http-response-status-text.http-request-response string.unquoted.http-request-response
      pop: true

  querystring:
    - meta_content_scope: meta.querystring.http-request-response
    - match: '[^&\s=]+'
      scope: variable.parameter.http-request-response
    - match: \&
      scope: keyword.operator.querystring.and.http-request-response
    - match: =
      scope: keyword.operator.querystring.equals.http-request-response
      push:
        - match: '[^&\s%]+'
          scope: string.unquoted.value.http-request-response
        - include: url_escape_char
        - match: ''
          pop: true
    - match: ''
      pop: true

  url_escape_char:
    - match: (%)\h{2}
      scope: meta.url constant.character.escape.url.html
      captures:
        1: punctuation.definition.escape.html

  content-type:
    - meta_content_scope: meta.headers.http-request-response
    - match: (application/json|text/json){{content_type_sep}}
      set: [content-type-json, header-value]
    - match: (application/xml|text/xml){{content_type_sep}}
      set: [content-type-xml, header-value]
    - match: (application/xhtml+xml|text/html){{content_type_sep}}
      set: [content-type-html, header-value]

  http-headers:
    - meta_content_scope: meta.headers.http-request-response
    - match: (?i)^(Content-Type)(:)\s*
      captures:
        1: keyword.other.http-header.key.http-request-response
        2: punctuation.separator.key-value.http-request-response
      set: content-type
    - match: ([^:]+)(:)\s*(\d+)$
      captures:
        1: keyword.other.http-header.key.http-request-response
        2: punctuation.separator.key-value.http-request-response
        3: meta.number.integer.decimal.http-request-response constant.numeric.value.http-request-response
    - match: ([^:]+)(:)\s*
      captures:
        1: keyword.other.http-header.key.http-request-response
        2: punctuation.separator.key-value.http-request-response
      push: header-value
    - match: ^$\n
      set: scope:text.plain

  header-value:
    - meta_scope: string.unquoted.http-header.value.http-request-response
      pop: true
    - match: '[;,]'
      scope: punctuation.separator.mapping.key-value.http-request-response
    - match: \b([-\w]+)(=)
      captures:
        1: variable.parameter.http-request-response
        2: keyword.operator.assignment.http-request-response
      push: possible-number
    - match: $
      pop: true

  possible-number:
    - match: \b\d+(?!\.)\b
      scope: meta.number.integer.decimal.http-request-response constant.numeric.value.http-request-response
      pop: true
    - match: ''
      pop: true

  content-type-json:
    - meta_content_scope: meta.headers.http-request-response
    - match: ^$\n
      set: [response-body, embed_json]
    - include: http-headers

  content-type-xml:
    - meta_content_scope: meta.headers.http-request-response
    - match: ^$\n
      set: [response-body, embed_xml]
    - include: http-headers

  content-type-html:
    - meta_content_scope: meta.headers.http-request-response
    - match: ^$\n
      set: [response-body, embed_html]
    - include: http-headers

  response-body:
    - meta_content_scope: meta.response-body.http-request-response

  embed_json:
    - match: ''
      embed: scope:source.json
      escape: (?!)

  embed_xml:
    - match: ''
      embed: scope:text.xml
      escape: (?!)

  embed_html:
    - match: ''
      embed: scope:text.html.basic
      escape: (?!)

```

</details>
@keith-hall Awesome, I love it. I think the downloads policy is not always a great deciding factor due to the difference of tools: Maybe there are a lot of people who want to *view* HTTP responses (using `bat`), but probably only very few people who want to *edit* HTTP responses (in Sublime Text).

Having this in a separate git repository would be great, but if you don't want to, I'm fine with maintaining it in the `bat` repository.

Edit: I just now realized that this can also be used for HTTP *requests*, where it might make a bit more sense to edit in Sublime Text... anyway.
That's wonderful, thank you!

One minor issue - it doesn't show up in the output from `bat -L` but works just fine when I `bat -l 'HTTP Request and Response' ...`.
> One minor issue - it doesn't show up in the output from `bat -L` but works just fine when I `bat -l 'HTTP Request and Response' ...`.

Thanks for letting me know. It seems that it will only show up if it has some file extensions associated with it. We could add `http` as one to get it to appear in the list.
Yep, that fixes it.
I have decided that I am interested in maintaining this syntax, it's quite fun to work on :)

I have created a repository for it here: https://github.com/keith-hall/http-request-response-syntax

I will add a PR to `bat` soon :tm: to add it as a submodule for syntax assets unless there are any objections :)

Fix patch:
diff --git a/.gitmodules b/.gitmodules
index dbb6dc9c17..5780f3510e 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -221,3 +221,6 @@
 [submodule "assets/syntaxes/02_Extra/Groff"]
 	path = assets/syntaxes/02_Extra/Groff
 	url = https://github.com/carsonoid/sublime_man_page_support
+[submodule "assets/syntaxes/02_Extra/http-request-response"]
+	path = assets/syntaxes/02_Extra/http-request-response
+	url = https://github.com/keith-hall/http-request-response-syntax.git
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 79481d8709..25045be390 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -14,6 +14,7 @@
 ## Syntaxes
 
 - Groff, see #1685 (@scop)
+- HTTP Requests and Responses, see #1748 (@keith-hall)
 
 ## New themes
 
diff --git a/assets/syntaxes/02_Extra/http-request-response b/assets/syntaxes/02_Extra/http-request-response
new file mode 160000
index 0000000000..f58bffe483
--- /dev/null
+++ b/assets/syntaxes/02_Extra/http-request-response
@@ -0,0 +1,1 @@
+Subproject commit f58bffe48314f1ef350f555649e08445a23edf4c
diff --git a/doc/assets.md b/doc/assets.md
index d30d3ab80f..ee65710117 100644
--- a/doc/assets.md
+++ b/doc/assets.md
@@ -9,7 +9,9 @@ in the `.sublime-syntax` format.
 **Important:** Before proceeding, verify that the syntax you wish to add meets the [criteria for inclusion](#Criteria-for-inclusion-of-new-syntaxes).
 
 1. Find a Sublime Text syntax for the given language, preferably in a separate Git repository
-   which can be included as a submodule (under `assets/syntaxes`).
+   which can be included as a submodule (under `assets/syntaxes`) using
+   `git submodule add <https github link> ./assets/syntaxes/02_Extra/<repo name>`, replacing
+   the contents of the angle brackets as appropriate.
 
 2. If the Sublime Text syntax is only available as a `.tmLanguage` file, open the file in
    Sublime Text and convert it to a `.sublime-syntax` file via *Tools* -> *Developer* ->
@@ -26,7 +28,8 @@ in the `.sublime-syntax` format.
 6. Add a syntax test for the new language. See [below](#Syntax-tests) for details.
 
 7. If you send a pull request with your changes, please do *not* include the changed `syntaxes.bin`
-   file. A new binary cache file will be created once before every new release of `bat`.
+   file. A new binary cache file will be created once before every new release of `bat`. This
+   avoids bloating the repository size unnecessarily.
 
 ### Syntax tests
 


Test patch:
diff --git a/tests/syntax-tests/highlighted/http-request-response/example.http b/tests/syntax-tests/highlighted/http-request-response/example.http
new file mode 100644
index 0000000000..0553706238
--- /dev/null
+++ b/tests/syntax-tests/highlighted/http-request-response/example.http
@@ -0,0 +1,44 @@
+[38;2;249;38;114mPOST[0m[38;2;248;248;242m [0m[38;2;248;248;242m/[0m[38;2;248;248;242mfoo[0m[38;2;248;248;242m/[0m[38;2;248;248;242mbar[0m[38;2;248;248;242m?[0m[3;38;2;253;151;31mid[0m[38;2;249;38;114m=[0m[38;2;230;219;116m4[0m[38;2;249;38;114m&[0m[3;38;2;253;151;31mx[0m[38;2;249;38;114m=[0m[38;2;230;219;116my[0m[38;2;190;132;255m%[0m[38;2;190;132;255m20[0m[38;2;230;219;116mz[0m[38;2;248;248;242m [0m[38;2;190;132;255mHTTP/1.1[0m
+[38;2;249;38;114mX-Forwarded-For[0m[38;2;230;219;116m:[0m[38;2;230;219;116m 127.0.0.1[0m
+[38;2;249;38;114mContent-Length[0m[38;2;248;248;242m:[0m[38;2;248;248;242m [0m[38;2;190;132;255m124[0m
+[38;2;249;38;114mCache-Control[0m[38;2;230;219;116m:[0m[38;2;230;219;116m no-cache[0m
+[38;2;249;38;114mX-Forwarded-Proto[0m[38;2;230;219;116m:[0m[38;2;230;219;116m https[0m
+[38;2;249;38;114mContent-Type[0m[38;2;248;248;242m:[0m[38;2;248;248;242m [0m[38;2;230;219;116mapplication/json[0m[38;2;230;219;116m;[0m[38;2;230;219;116m [0m[3;38;2;253;151;31mcharset[0m[38;2;249;38;114m=[0m[38;2;230;219;116mutf-8[0m
+[38;2;249;38;114mHost[0m[38;2;230;219;116m:[0m[38;2;230;219;116m example.com[0m
+[38;2;249;38;114mAccept[0m[38;2;230;219;116m:[0m[38;2;230;219;116m */*[0m[38;2;230;219;116m;[0m[38;2;230;219;116m [0m[3;38;2;253;151;31mq[0m[38;2;249;38;114m=[0m[38;2;230;219;116m0.5[0m[38;2;230;219;116m,[0m[38;2;230;219;116m application/xml[0m
+[38;2;249;38;114mAccept-Encoding[0m[38;2;230;219;116m:[0m[38;2;230;219;116m gzip[0m
+
+[38;2;248;248;242m{[0m
+[38;2;248;248;242m  [0m[38;2;253;151;31m"[0m[38;2;253;151;31mid[0m[38;2;253;151;31m"[0m[38;2;248;248;242m:[0m[38;2;248;248;242m [0m[38;2;230;219;116m"[0m[38;2;230;219;116mblahblahblahblah[0m[38;2;230;219;116m"[0m[38;2;248;248;242m,[0m
+[38;2;248;248;242m  [0m[38;2;253;151;31m"[0m[38;2;253;151;31mobject[0m[38;2;253;151;31m"[0m[38;2;248;248;242m:[0m[38;2;248;248;242m [0m[38;2;230;219;116m"[0m[38;2;230;219;116mevent[0m[38;2;230;219;116m"[0m[38;2;248;248;242m,[0m
+[38;2;248;248;242m  [0m[38;2;253;151;31m"[0m[38;2;253;151;31mapi_version[0m[38;2;253;151;31m"[0m[38;2;248;248;242m:[0m[38;2;248;248;242m [0m[38;2;230;219;116m"[0m[38;2;230;219;116m2020-03-02[0m[38;2;230;219;116m"[0m[38;2;248;248;242m,[0m
+[38;2;248;248;242m  [0m[38;2;253;151;31m"[0m[38;2;253;151;31mcreated[0m[38;2;253;151;31m"[0m[38;2;248;248;242m:[0m[38;2;248;248;242m [0m[38;2;190;132;255m1626790174[0m[38;2;248;248;242m,[0m
+[38;2;248;248;242m  [0m[38;2;253;151;31m"[0m[38;2;253;151;31mdata[0m[38;2;253;151;31m"[0m[38;2;248;248;242m:[0m[38;2;248;248;242m [0m[38;2;248;248;242m{[0m
+[38;2;248;248;242m  [0m[38;2;248;248;242m}[0m
+[38;2;248;248;242m}[0m
+
+
+[38;2;190;132;255mHTTP/1.1[0m[38;2;248;248;242m [0m[38;2;190;132;255m200[0m[38;2;248;248;242m [0m[38;2;230;219;116mOK[0m
+[38;2;249;38;114mServer[0m[38;2;230;219;116m:[0m[38;2;230;219;116m nginx[0m
+[38;2;249;38;114mDate[0m[38;2;230;219;116m:[0m[38;2;230;219;116m Fri[0m[38;2;230;219;116m,[0m[38;2;230;219;116m 23 Jul 2021 10:15:12 GMT[0m
+[38;2;249;38;114mContent-Type[0m[38;2;248;248;242m:[0m[38;2;248;248;242m [0m[38;2;230;219;116mtext/html[0m[38;2;230;219;116m;[0m[38;2;230;219;116m [0m[3;38;2;253;151;31mcharset[0m[38;2;249;38;114m=[0m[38;2;230;219;116mutf-8[0m
+[38;2;249;38;114mTransfer-Encoding[0m[38;2;230;219;116m:[0m[38;2;230;219;116m chunked[0m
+[38;2;249;38;114mConnection[0m[38;2;230;219;116m:[0m[38;2;230;219;116m keep-alive[0m
+[38;2;249;38;114mVary[0m[38;2;230;219;116m:[0m[38;2;230;219;116m Accept-Encoding[0m
+[38;2;249;38;114mCache-Control[0m[38;2;230;219;116m:[0m[38;2;230;219;116m private[0m[38;2;230;219;116m;[0m[38;2;230;219;116m [0m[3;38;2;253;151;31mmax-age[0m[38;2;249;38;114m=[0m[38;2;190;132;255m0[0m
+[38;2;249;38;114mX-Frame-Options[0m[38;2;230;219;116m:[0m[38;2;230;219;116m DENY[0m
+[38;2;249;38;114mX-Content-Type-Options[0m[38;2;230;219;116m:[0m[38;2;230;219;116m nosniff[0m
+[38;2;249;38;114mX-XSS-Protection[0m[38;2;230;219;116m:[0m[38;2;230;219;116m 1[0m[38;2;230;219;116m;[0m[38;2;230;219;116m [0m[3;38;2;253;151;31mmode[0m[38;2;249;38;114m=[0m[38;2;230;219;116mblock[0m
+[38;2;249;38;114mReferrer-Policy[0m[38;2;230;219;116m:[0m[38;2;230;219;116m origin[0m
+[38;2;249;38;114mStrict-Transport-Security[0m[38;2;230;219;116m:[0m[38;2;230;219;116m [0m[3;38;2;253;151;31mmax-age[0m[38;2;249;38;114m=[0m[38;2;190;132;255m31556900[0m
+
+[38;2;255;255;255m<![0m[38;2;249;38;114mDOCTYPE[0m[38;2;248;248;242m [0m[38;2;190;132;255mhtml[0m[38;2;255;255;255m>[0m
+[38;2;255;255;255m<[0m[38;2;249;38;114mhtml[0m[38;2;255;255;255m>[0m
+[38;2;255;255;255m<[0m[38;2;249;38;114mhead[0m[38;2;255;255;255m>[0m
+[38;2;248;248;242m  [0m[38;2;255;255;255m<[0m[38;2;249;38;114mmeta[0m[38;2;166;226;46m [0m[38;2;166;226;46mcharset[0m[38;2;166;226;46m=[0m[38;2;255;255;255m"[0m[38;2;230;219;116mutf-8[0m[38;2;255;255;255m"[0m[38;2;255;255;255m>[0m
+[38;2;248;248;242m  [0m[38;2;255;255;255m<[0m[38;2;249;38;114mtitle[0m[38;2;255;255;255m>[0m[38;2;248;248;242mHello World[0m[38;2;255;255;255m</[0m[38;2;249;38;114mtitle[0m[38;2;255;255;255m>[0m
+[38;2;255;255;255m</[0m[38;2;249;38;114mhead[0m[38;2;255;255;255m>[0m
+[38;2;255;255;255m<[0m[38;2;249;38;114mbody[0m[38;2;255;255;255m>[0m
+
+[38;2;255;255;255m</[0m[38;2;249;38;114mbody[0m[38;2;255;255;255m>[0m
+[38;2;255;255;255m</[0m[38;2;249;38;114mhtml[0m[38;2;255;255;255m>[0m
diff --git a/tests/syntax-tests/source/http-request-response/example.http b/tests/syntax-tests/source/http-request-response/example.http
new file mode 100644
index 0000000000..26ec1395a0
--- /dev/null
+++ b/tests/syntax-tests/source/http-request-response/example.http
@@ -0,0 +1,44 @@
+POST /foo/bar?id=4&x=y%20z HTTP/1.1
+X-Forwarded-For: 127.0.0.1
+Content-Length: 124
+Cache-Control: no-cache
+X-Forwarded-Proto: https
+Content-Type: application/json; charset=utf-8
+Host: example.com
+Accept: */*; q=0.5, application/xml
+Accept-Encoding: gzip
+
+{
+  "id": "blahblahblahblah",
+  "object": "event",
+  "api_version": "2020-03-02",
+  "created": 1626790174,
+  "data": {
+  }
+}
+
+
+HTTP/1.1 200 OK
+Server: nginx
+Date: Fri, 23 Jul 2021 10:15:12 GMT
+Content-Type: text/html; charset=utf-8
+Transfer-Encoding: chunked
+Connection: keep-alive
+Vary: Accept-Encoding
+Cache-Control: private; max-age=0
+X-Frame-Options: DENY
+X-Content-Type-Options: nosniff
+X-XSS-Protection: 1; mode=block
+Referrer-Policy: origin
+Strict-Transport-Security: max-age=31556900
+
+<!DOCTYPE html>
+<html>
+<head>
+  <meta charset="utf-8">
+  <title>Hello World</title>
+</head>
+<body>
+
+</body>
+</html>