Repository: haraka/Haraka (2026-03-24T00:23:20Z)

Issue:
txn.add_header not appending headers correctly
**Describe the bug**

````
Return-Path: <wordpress@simerson.net>
Delivered-To: matt@simerson.net
Received: from mail.theartfarm.com ([172.16.15.9])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	by dovecot with LMTPS
	id T4IPN8a/TGKnLwAAkzG9Ng
	(envelope-from <wordpress@simerson.net>)
	for <matt@simerson.net>; Tue, 05 Apr 2022 15:16:38 -0700
Received: (Haraka outbound); Tue, 05 Apr 2022 13:16:38 -0900
Authentication-Results: mail.theartfarm.com;
	spf=pass smtp.mailfrom=simerson.net
Received-SPF: Pass (mail.theartfarm.com: domain of simerson.net designates 172.16.16.8 as permitted sender) receiver=mail.theartfarm.com; identity=mailfrom; client-ip=66.128.51.165 helo=wordpress; envelope-from=<wordpress@simerson.net>
Received: from wordpress (wordpress [172.16.16.8])
	by mail.theartfarm.com (Haraka/2.8.28) with SMTP id 3A70D3B0-F413-46A2-98A9-BB2B5C9969F6.1
	envelope-from <wordpress@simerson.net>;
	Tue, 05 Apr 2022 13:16:37 -0900
Received: by wordpress (sSMTP sendmail emulation); Tue, 05 Apr 2022 15:16:37 -0700
To: matt@simerson.net
Subject: =?us-ascii?Q?[Simerson_Family_Network]_Your_site_has_updated?=
 =?us-ascii?Q?_to_WordPress_5.9.3?=
Date: Tue, 5 Apr 2022 22:16:37 +0000
From: WordPress <wordpress@simerson.net>
Message-ID: <T6rttBg1or2g1s0rHBIEhN6TyAisFwIFFWisnQShp0@simerson.net>
X-Mailer: PHPMailer 6.5.3 (https://github.com/PHPMailer/PHPMailer)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8

Howdy! Your site at https://simerson.net has been updated automatically to WordPress 5.9.3.

No further action is needed on your part. For more on version 5.9.3, see the About WordPress screen:
https://simerson.net/wp-admin/about.php

If you experience any issues or need support, the volunteers in the WordPress.org support forums may be able to help.
https://wordpress.org/support/forums/

You also have some plugins or themes with updates available. Update them now:
https://simerson.net/wp-admin/network/

The WordPress Team

X-Spam-Level: ***
X-Spam-Status: No, score=3.4 required=5.0 autolearn=no autolearn_force=no
X-Spam-DCC: Etherboy: dcc 1002; Body=1 Fuz1=1 Fuz2=1
X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on spamassassin
X-Spam-Tests: BAYES_50,FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,SPF_PASS,
	T_SCC_BODY_TEXT_LINE,URI_WPADMIN
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;
	d=simerson.net; s=mar2013;
	h=from:subject:date:message-id:to:mime-version;
	bh=kvqv5W+7H5vwIPWJM7oY6wjspD+nUOyK+wng53dalro=;
	b=W+b87su+vtB57Z8HT1UsN1H7+hBpKPHN/6C3KwsuEwffTfUHpmaT9zAEUjsQ2oKzGZf6AEkX3j
	Z/+tQFmICmV6OKw8HQpfmdACsIY7bzeP8f+8hyuo2/mFA//pH2ZT2tiKZQV8sa46fww+RP5SMkTq
	3RvBoKNygdOLoQhGPexYCBRSmM4FB4GesoyTNIsFoKGpqvrLEG08U+0wDmFKpWgSJ1XJrw5BcM+o
	q92cM9oPxFz3Fd7uUY6wbiodbae+SEZH1YhDmRgDWLLVmef5CssY2QpuzWbMKE6ba9zzFGqRcqFr
	3SJapCwv0FZgJ633+Fcn9San8HSk2iAq2WsQcZqw==

````


**Expected behavior**

Appended headers belong in the header section, not at the end of the message body.

**Observed behavior**

Sometimes, appended headers aren't put in the right spot. This is atypical behavior which I see on occasion.

**System Info:**

Please report your OS, Node version, and Haraka version by running this shell script on your Haraka server and replacing this section with the output.

Haraka | Haraka.js — Version: 2.8.28
 --- | :--- 
Node | v14.18.1
OS | FreeBSD haraka 13.0-RELEASE-p7 FreeBSD 13.0-RELEASE-p7 #0: Mon Jan 31 18:24:03 UTC 2022     root@amd64-builder.daemonology.net:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64
openssl | OpenSSL 1.1.1h-freebsd  24 Aug 2021



Discussion:

Patch:
diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md
index 702379702..cf3066772 100644
--- a/.github/copilot-instructions.md
+++ b/.github/copilot-instructions.md
@@ -14,9 +14,9 @@
 - When updating files, add `node:` prefixes to any Node.js built-in `require()` calls that lack them (e.g. `require('fs')` → `require('node:fs')`)
 
 # Test
-- `npm test`                          # run all tests
-./run_tests                           # Same as npm test
-./run_tests test/plugins/bounce.js    # Single test file
+npm test                            # run all tests
+./run_tests                         # Same as npm test
+./run_tests test/plugins/bounce.js  # Single test file
 
 # Lint & format
 npm run lint                          # ESLint check
diff --git a/.prettierignore b/.prettierignore
index 221d6f3cb..db1315f8d 100644
--- a/.prettierignore
+++ b/.prettierignore
@@ -2,3 +2,4 @@ docs/*.md
 Changes.md
 CONTRIBUTORS.md
 Plugins.md
+.github/copilot-instructions.md
diff --git a/.release b/.release
index 737a8876d..a35dbfed2 160000
--- a/.release
+++ b/.release
@@ -1,1 +1,1 @@
-Subproject commit 737a8876d7b102e14ac02c4b4d1cae2d1598b3fd
+Subproject commit a35dbfed2af659b56171af58aeb0d53966041b3a
diff --git a/Changes.md b/Changes.md
index d8f48fc27..1366c66de 100644
--- a/Changes.md
+++ b/Changes.md
@@ -4,6 +4,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/).
 
 ### Unreleased
 
+- fix(txn): more robust handling of broken messages #3535
 - deps(all): bump versions
 - doc(Plugins): added more sections
 - test: fixes for test-fixtures 1.4 & local plugins that do require('./')
diff --git a/package.json b/package.json
index 9d9eaaa13..9e329f555 100644
--- a/package.json
+++ b/package.json
@@ -101,8 +101,9 @@
     "prettier": "npx prettier . --check",
     "prettier:fix": "npx prettier . --write --log-level=warn",
     "test": "npx mocha --exit --timeout=4000 test test/outbound test/plugins/auth test/plugins/queue test/plugins",
-    "versions": "npx dependency-version-checker check",
-    "versions:fix": "npx dependency-version-checker update"
+    "versions": "npx npm-dep-mgr check",
+    "versions:fix": "npx npm-dep-mgr update",
+    "test:coverage": "npx c8 --reporter=text --reporter=text-summary npm test"
   },
   "prettier": {
     "singleQuote": true,
diff --git a/plugins.js b/plugins.js
index 1ba9fef45..91efcdc12 100644
--- a/plugins.js
+++ b/plugins.js
@@ -157,9 +157,9 @@ class Plugin {
             }
 
             for (const ext of [`${module}.js`, module]) {
-              if (fs.existsSync(path.join(__dirname, ext))) {
-                return require(module);
-              }
+                if (fs.existsSync(path.join(__dirname, ext))) {
+                    return require(module)
+                }
             }
 
             return require(path.join(path.dirname(this.plugin_path), module))
diff --git a/transaction.js b/transaction.js
index 8ada5c032..c1cebf8c8 100644
--- a/transaction.js
+++ b/transaction.js
@@ -133,7 +133,7 @@ class Transaction {
         // is this the end of headers line?
         if (this.header_pos === 0 && (line[0] === 0x0a || (line[0] === 0x0d && line[1] === 0x0a))) {
             this.header.parse(this.header_lines)
-            this.header_pos = this.header_lines.length
+            this.header_pos = this.header.header_list ? this.header.header_list.length : -1
             this.found_hb_sep = true
             if (this.parse_body) this.ensure_body()
         } else if (this.header_pos === 0) {
@@ -169,13 +169,23 @@ class Transaction {
             }
             const body_lines = this.header_lines.splice(header_pos + 1)
             this.header.parse(this.header_lines)
-            this.header_pos = header_pos
+            this.header_pos = this.header.header_list ? this.header.header_list.length : -1
+            this.found_hb_sep = true
+
+            // We MUST add an empty line to the message_stream so that it knows
+            // where the headers end. Otherwise it skips everything.
+            if (!this.discard_data) this.message_stream.add_line(Buffer.from('\r\n'))
+
             if (this.parse_body) {
                 this.ensure_body()
                 for (const bodyLine of body_lines) {
                     this.body.parse_more(bodyLine)
                 }
             }
+
+            for (const bodyLine of body_lines) {
+                if (!this.discard_data) this.message_stream.add_line(Buffer.from(`${bodyLine}\n`))
+            }
         }
         if (this.header_pos && this.parse_body) {
             const line = this.add_dot_stuffing_and_ensure_crlf_newlines(this.body.parse_end())
@@ -195,22 +205,21 @@ class Transaction {
 
     add_header(key, value) {
         this.header.add_end(key, value)
-        if (this.header_pos > 0) this.reset_headers()
+        this.reset_headers()
     }
 
     add_leading_header(key, value) {
         this.header.add(key, value)
-        if (this.header_pos > 0) this.reset_headers()
+        this.reset_headers()
     }
 
     reset_headers() {
-        const header_lines = this.header.lines()
-        this.header_pos = header_lines.length
+        this.header_pos = this.header.header_list ? this.header.header_list.length : -1
     }
 
     remove_header(key) {
         this.header.remove(key)
-        if (this.header_pos > 0) this.reset_headers()
+        this.reset_headers()
     }
 
     attachment_hooks(start, data, end) {


Test patch:
diff --git a/test/transaction.js b/test/transaction.js
index de61670cb..b4ba62627 100644
--- a/test/transaction.js
+++ b/test/transaction.js
@@ -369,9 +369,60 @@ describe('transaction', () => {
         it('end_data handles email with no blank line separator', (done) => {
             this.transaction.add_data('Subject: test\n')
             this.transaction.add_data('From: a@b.com\n')
+            this.transaction.add_data('Body line 1\n')
             this.transaction.end_data(() => {
                 assert.ok(this.transaction.header, 'header exists')
-                done()
+                assert.equal(this.transaction.header.get('Subject').trim(), 'test')
+                assert.equal(this.transaction.header.get('From').trim(), 'a@b.com')
+
+                this.transaction.message_stream.get_data((output) => {
+                    const str = output.toString()
+                    assert.ok(str.includes('Subject: test'), 'Subject in output')
+                    assert.ok(str.includes('Body line 1'), 'Body in output')
+                    done()
+                })
+            })
+        })
+    })
+
+    describe('late header additions', () => {
+        beforeEach(_set_up)
+
+        it('adds late header to busted email in correct position', (done) => {
+            this.transaction.add_data('Subject: Test\r\n')
+            this.transaction.add_data('From: user@example.com\r\n')
+            this.transaction.add_data('Body line 1\r\n')
+
+            this.transaction.end_data(() => {
+                this.transaction.add_header('X-Late', 'true')
+
+                this.transaction.message_stream.get_data((output) => {
+                    const str = output.toString()
+                    assert.ok(str.includes('X-Late: true'), 'Late header present')
+                    const lateIdx = str.indexOf('X-Late: true')
+                    const bodyIdx = str.indexOf('Body line 1')
+                    assert.ok(lateIdx < bodyIdx, 'Late header before body')
+                    done()
+                })
+            })
+        })
+
+        it('adds late header to clean email in correct position', (done) => {
+            this.transaction.add_data('Subject: Clean\r\n')
+            this.transaction.add_data('\r\n')
+            this.transaction.add_data('Body line 1\r\n')
+
+            this.transaction.end_data(() => {
+                this.transaction.add_header('X-Late', 'true')
+
+                this.transaction.message_stream.get_data((output) => {
+                    const str = output.toString()
+                    assert.ok(str.includes('X-Late: true'), 'Late header present')
+                    const lateIdx = str.indexOf('X-Late: true')
+                    const bodyIdx = str.indexOf('Body line 1')
+                    assert.ok(lateIdx < bodyIdx, 'Late header before body')
+                    done()
+                })
             })
         })
     })