<PAGE>

                                                                    EXHIBIT 10.4

                  THIRD AMENDED AND RESTATED EXCHANGE AGREEMENT

      THIS THIRD AMENDED AND RESTATED EXCHANGE AGREEMENT (this "Agreement"),
dated as of December 15, 2004, is entered into among MainLine Sub LLC, a
Delaware limited liability company ("Holdco"), BUCKEYE GP LLC, a Delaware
limited liability company (the "General Partner"), BUCKEYE PARTNERS, L.P., a
Delaware limited partnership (the "Partnership"), BUCKEYE PIPE LINE COMPANY,
L.P., a Delaware limited partnership ("BPLCLP"), LAUREL PIPE LINE COMPANY, L.P.,
a Delaware limited partnership ("Laurel"), EVERGLADES PIPE LINE COMPANY, L.P., a
Delaware limited partnership ("Everglades"), and BUCKEYE PIPE LINE HOLDINGS,
L.P., a Delaware limited partnership (collectively with BPLCLP, Laurel, and
Everglades, the "Operating Partnerships").

                                   WITNESSETH:

      WHEREAS, Buckeye Pipe Line Company LLC, a Delaware limited liability
company (the "Former GP"), Buckeye Management Company LLC, a Delaware limited
liability company ("BMC"), Glenmoor LLC, a Delaware limited liability company
("Glenmoor"), the Partnership and the Operating Partnerships entered into the
Exchange Agreement, dated as of August 12, 1997 (the "Original Agreement"), the
transactions contemplated by which were consummated on such date effective as of
11:59 P.M.;

      WHEREAS, the Original Agreement was (i) amended and restated in its
entirety on May 2, 2002 and (ii) further amended and restated as of May 4, 2004
(as so amended and restated, the "Prior Agreement");

      WHEREAS, the Partnership is governed pursuant to an Amended and Restated
Agreement of Limited Partnership (the "Master Partnership Agreement"), dated as
of December 15, 2004, between the General Partner and the limited partners of
the Partnership (the "Limited Partners"), as amended; the Operating
Partnerships, are governed pursuant to similar Amended and Restated Agreements
of Limited Partnership, each dated as of December 15, 2004, as amended, between
the General Partner and the Partnership (collectively, the "Operating
Partnership Agreements");

      WHEREAS, in connection with the Original Agreement, the Partnership (i)
issued limited partnership units of the Partnership ("LP Units") to Buckeye Pipe
Line Services Company, a Pennsylvania corporation (the "Company") whose shares
of capital stock are owned by the Buckeye Pipe Line Services Company Employee
Stock Ownership Plan Trust, (referred to herein as the "ESOP"), in exchange for
shares of Glenmoor stock (the "Exchange Shares"), and (ii) contributed an
undivided interest in the Exchange Shares to the Operating Partnerships as of
the date of the Original Agreement;

      WHEREAS, the Operating Partnerships transferred and assigned the Exchange
Shares to the Former GP as of the date of the Original Agreement in exchange for
the release of certain obligations that the Partnership had to BMC (as the
former general partner of the Partnership) and the Former GP, and the Operating
Partnerships had to the Former GP; Glenmoor and BMC caused the Former GP to
receive the Exchange Shares and to release such obligations of the Partnership
and the Operating Partnerships; and the Exchange Shares were further transferred
by the Former GP to BMC and by BMC to Glenmoor;

<PAGE>

      WHEREAS, the Former GP was the general partner of the Partnership and the
Operating Partnerships, and pursuant to a Contribution, Assignment and
Assumption Agreement dated December 15, 2004, assigned its general partner
interests in the Partnership and the Operating Partnerships and all its right,
title and interest in the Prior Agreement, to Buckeye GP LLC (the "Assignment
Agreement");

      WHEREAS, Holdco is the successor of the merger of BMC and the Former GP
with and into Glenmoor, which simultaneously with such merger changed its name
to MainLine Sub LLC; and

      WHEREAS, the parties to the Prior Agreement desire to amend and restate
the Prior Agreement in its entirety to reflect the organizational changes
recited above.

      NOW, THEREFORE, the parties hereto, intending to be legally bound, hereby
agree as follows:

                                   ARTICLE I
                                  THE EXCHANGE

      Upon the terms and subject to the conditions of this Agreement, the
Operating Partnerships have transferred and assigned the Exchange Shares to the
Former GP in exchange for the release of certain obligations of the Partnership
to BMC (as the former general partner of the Partnership) and the Former GP, and
of the Operating Partnerships to the Former GP, as set forth in Article II
below.

                                   ARTICLE II
                             RELEASE OF OBLIGATIONS

      2.01 Obligations to Reimburse for Executive Compensation. (a) Upon the
terms and subject to the conditions of this Agreement, the General Partner, for
itself and its affiliates, successors and assigns, hereby and irrevocably
releases, relinquishes and discharges the Partnership and the Operating
Partnerships from any and all liability, obligation, claim, demand, action or
suit of any kind or nature, in law or in equity, whatsoever, known or unknown,
which may be asserted for or on account of or arising out of or in any manner
relating to the Partnership's and/or the Operating Partnerships' obligations
pursuant to Section 7.4(b) of the Master Partnership Agreement and the Operating
Partnership Agreements or otherwise to reimburse the General Partner for total
compensation, including all benefits, paid for the four highest salaried
officers performing duties for the General Partner with respect to the functions
of operations, finance, legal, marketing and business development, treasury, or
performing the function of President of the General Partner following the date
of the Original Agreement. Nothing in this Section 2.01(a) shall be deemed to
waive the obligations of the Partnership and the Operating Partnerships to
reimburse the General Partner for (i) employee fringe benefits and retirement
benefits for their executives relating to services performed prior to the date
of the Original Agreement, (ii) obligations under severance agreements with
their executives to the extent currently reimbursable under the Master
Partnership Agreement or (iii) any obligations in respect of their executives
which are not related to compensation, including, without limitation,
indemnification obligations.

                                       2

<PAGE>

            (b) Holdco and the General Partner agree, unless the General Partner
is removed as general partner of the Partnership to perform the executive level
functions referred to in Section 2.01(a) for the benefit of the Partnership and
the Operating Partnerships in a manner satisfactory to the board of directors of
the General Partner.

      2.02 ESOP Obligations Generally. As of the date of this Agreement, Holdco
acknowledges that it has received all reimbursements due to it from the
Partnership and the Operating Partnerships pursuant to the terms of the Prior
Agreement in respect of (i) cash contributions made or to be made by the Company
to the ESOP pursuant to the terms of the ESOP trust agreement, as necessary for
the ESOP to make all payments of principal, interest and premium due under the
Note Agreement, dated as of May 4, 2004, among the ESOP, The Prudential
Insurance Company of America, Pruco Life Insurance Company and Pruco Life
Insurance Company of New Jersey (the "Note Agreement"), (ii) cash deposits made
or to be made by the Company pursuant to an obligation to maintain a minimum
value of collateral pledged to secure the obligations of the ESOP or the Company
in respect of the Note Agreement, (iii) income taxes incurred by the Company on
the sale of LP Units made to satisfy the redemption obligations described in
Section 2.03 below, and (iv) routine administrative charges and expenses common
to employee stock ownership plans incurred in connection with the operation of
the ESOP. Each of Holdco and the General Partner hereby release, relinquish and
discharge the Partnership and the Operating Partnerships from any and all
further liability, obligation, claim, demand, action or suit of any kind or
nature, in law or in equity, whatsoever, known or unknown, which may be asserted
for or on account of or arising out of or in any manner relating to the
foregoing obligations under the Prior Agreement.

      2.03 No ESOP Contributions for Departing Employees. Holdco and the General
Partner acknowledge that neither the Partnership nor the Operating Partnerships
shall be obligated to reimburse Holdco or the General Partner for obligations to
redeem the ESOP accounts of departing employees upon the termination of their
employment with the Company, or for any other costs or expenses of or relating
to the operation of the ESOP other than those specified in Section 2.02(a)
above.

      2.04 Representations and Warranties. Holdco and the General Partner hereby
represent and warrant to the Partnership and the Operating Partnerships, as of
the date of the Original Agreement, that (a) neither the Company nor any entity
treated as a single employer with the Company under Sections 414(b), 414(c),
414(m), or 414(o) of the Internal Revenue Code of 1986, as amended (the "Code"),
or Section 4001(b) of the Employee Retirement Income Security Act of 1974, as
amended ("ERISA"), has incurred any liability under any provision of ERISA or
other applicable law relating to the ESOP; (b) the ESOP has been administered,
in all material respects, in compliance with its terms and complies, both in
form and operation, with the applicable provisions of ERISA (including, without
limitation, the funding and prohibited transactions provisions thereof), the
Code and other applicable laws; and (c) the ESOP has been determined by the
Internal Revenue Service to be qualified within the meaning of Section 401 of
the Code, and neither Holdco nor the General Partner is aware of any fact or
circumstances which would adversely affect the qualified status of the ESOP.

                                       3

<PAGE>

                                  ARTICLE III
                       AGREEMENT TO ACT AS GENERAL PARTNER

      3.01 Failure to Act as General Partner Over the ESOP Period. Except to the
extent this obligation is assumed by a successor general partner pursuant to
Section 3.02, the General Partner shall continue to serve as the general partner
of the Partnership and the Operating Partnerships until all principal, interest
and premium is paid in full under the Note Agreement and under any agreements or
instruments replacing the Note Agreements have been repaid, unless the
Partnership shall be sooner dissolved under Section 14.1(d) of the Master
Partnership Agreement. Each Party hereto hereby (i) consents to the transactions
set forth in the Assignment Agreement, including the assignment of all general
partner interests in the Partnership and the Operating Partnerships by the
Former GP to the General Partner, and (ii) agrees that the consummation of such
transactions did not violate any provision of the Prior Agreement.

      3.02 Assumption of Obligations by a Successor General Partner. If the
General Partner is removed as general partner of the Partnership or one or more
of the Operating Partnerships during the ESOP Period, (but not if the General
Partner voluntarily withdraws as general partner) pursuant to Section 13.1(b) of
the Master Partnership Agreement or if the General Partner transfers its general
partner interests in the Partnership or the Operating Partnerships pursuant to
Section 11.1 of the Master Partnership Agreement, the General Partner may cause
the successor general partner of the Partnership and the Operating Partnerships
to assume its respective obligations, liabilities and duties under this
Agreement.

                                   ARTICLE IV
                               GENERAL PROVISIONS

      4.01 Entire Agreement. This Agreement supersedes all prior discussions and
agreements among the parties hereto with respect to the subject matter hereof
and contains the sole and entire agreement among the parties hereto with respect
to the subject matter hereof.

      4.02 Headings. The headings used in this Agreement have been inserted for
convenience of reference only and do not define or limit the provisions hereof.

      4.03 Waiver and Amendment. No failure by any party to insist upon the
strict performance of any covenant, duty, agreement or condition of this
Agreement or to exercise any right or remedy consequent upon a breach thereof
shall constitute a waiver of any such breach or of any other covenant, duty,
agreement or condition. Any amendment to this Agreement shall be effective only
if in a writing signed by each of the parties hereto.

      4.04 Severability. If any provision of this Agreement is or becomes
invalid, illegal or unenforceable in any respect, the validity, legality and
enforceability of the remaining provisions hereof, or of such provision in other
respects, shall not be affected thereby.

      4.05 Governing Law. This Agreement shall be governed by and construed in
accordance with the laws of the State of Delaware applicable to a contract
executed and performed in such State, without giving effect to the conflicts of
laws principles thereof.

                                       4

<PAGE>

      4.06 Counterparts. This Agreement may be executed in any number of
counterparts, each of which will be deemed an original, but all of which
together will constitute one and the same instrument.

                        [signatures follow on next page]

                                       5

<PAGE>

            IN WITNESS WHEREOF, each party hereto has caused this Agreement
amending and restating the Prior Agreement to be signed by its officer duly
authorized as of the date first above written.

                      MAINLINE SUB LLC

                      By: /s/ Robert B. Wallace
                          _____________________________________
                      NAME: Robert B. Wallace
                      TITLE: Senior Vice President, Finance,
                      and Chief Financial Officer

                      BUCKEYE GP LLC

                      By: /s/ Robert B. Wallace
                          _____________________________________
                      NAME: Robert B. Wallace
                      TITLE: Senior Vice President, Finance,
                      and Chief Financial Officer

                      BUCKEYE PARTNERS, L.P.

                      By:  BUCKEYE GP LLC,
                      AS GENERAL PARTNER

                      By: /s/ Stephen C. Muther
                          _____________________________________
                      NAME: Stephen C. Muther
                      TITLE: Senior Vice President, Administration, General
                      Counsel and Secretary

                      BUCKEYE PIPE LINE COMPANY, L.P.

                      By:  BUCKEYE GP LLC,
                      AS GENERAL PARTNER

                      By: /s/ Stephen C. Muther
                          _____________________________________
                      NAME: Stephen C. Muther
                      TITLE: Senior Vice President, Administration, General
                      Counsel and Secretary

                      [SIGNATURES CONTINUE ONTO NEXT PAGE]

                 [Third Amended and Restated Exchange Agreement]

<PAGE>

                    [SIGNATURES CONTINUED FROM PREVIOUS PAGE]

                      LAUREL PIPE LINE COMPANY, L.P.

                      BY:  BUCKEYE GP LLC,
                      AS GENERAL PARTNER

                      By:  /s/ Stephen C. Muther
                           _____________________________________
                      NAME: Stephen C. Muther
                      TITLE: Senior Vice President, Administration, General
                      Counsel and Secretary

                      EVERGLADES PIPE LINE COMPANY, L.P.

                      By:  BUCKEYE GP LLC,
                      AS GENERAL PARTNER

                      By: /s/ Stephen C. Muther
                          _____________________________________
                      NAME: Stephen C. Muther
                      TITLE: Senior Vice President, Administration, General
                      Counsel and Secretary

                      BUCKEYE PIPE LINE HOLDINGS, L.P.

                      By:  BUCKEYE GP LLC,
                      AS GENERAL PARTNER

                      By: /s/ Stephen C. Muther
                          _____________________________________
                      NAME: Stephen C. Muther
                      TITLE: Senior Vice President, Administration, General
                      Counsel and Secretary

                 [Third Amended and Restated Exchange Agreement]