<PAGE>

                                                                    Exhibit 10.5

                ASSIGNMENT, ASSUMPTION AND RECOGNITION AGREEMENT

      THIS ASSIGNMENT, ASSUMPTION AND RECOGNITION AGREEMENT (this "Assignment"),
dated as of June 1, 2004, is entered into among J.P. Morgan Acceptance
Corporation I, a Delaware corporation (the "Depositor"), Wachovia Bank, National
Association, as trustee (the "Trustee") of J.P. Morgan Mortgage Trust 2004-A3
(the "Trust"), Sunset Financial Resources, Inc., a Maryland corporation
("Sunset"), Cendant Mortgage Corporation ("Cendant") and Bishop's Gate
Residential Mortgage Trust ("Bishop's Gate" and, together with Cendant, the
"Sellers"), with Cendant Mortgage Corporation, as the servicer (in such
capacity, the "Servicer").

                                    RECITALS

      WHEREAS, the Sellers and the Servicer and J.P. Morgan Mortgage Acquisition
Corp. ("JPMorgan Acquisition") entered into a certain Mortgage Loan Flow
Purchase, Sale & Servicing Agreement, dated as of April 29, 2003 (the "Purchase
Agreement"), as amended by Amendment No. 1 thereto, dated as of September 1,
2003 (the "Amendment No. 1" and, together with the Purchase Agreement, the
"Agreement"), pursuant to which JPMorgan Acquisition acquired certain mortgage
loans (the "Mortgage Loans") pursuant to the terms of the Agreement and the
Servicer agreed to service such Mortgage Loans;

      WHEREAS, pursuant to (i) the Assignment, Assumption and Recognition
Agreement dated as of March 30, 2004 relating to the Agreement, and (ii) the
Assignment, Assumption and Recognition Agreement dated as of June 1, 2004
relating to the Agreement, Sunset acquired from JPMorgan Acquisition all of
JPMorgan Acquisition's right, title and interest in certain of the Mortgage
Loans (the "Sunset Mortgage Loans") and the Servicer agreed to service such
Sunset Mortgage Loans pursuant to the Agreement;

      WHEREAS the Depositor has agreed, on the terms and conditions contained
herein, to purchase from Sunset certain of the Sunset Mortgage Loans (the
"Specified Mortgage Loans") which are subject to the provisions of the Agreement
and are listed on the mortgage loan schedule attached as Exhibit I hereto (the
"Specified Mortgage Loan Schedule"); and

      WHEREAS the Trustee, on behalf of the Trust, has agreed, on the terms and
conditions contained herein, to purchase from the Depositor the Specified
Mortgage Loans;

      NOW, THEREFORE, in consideration of the mutual promises contained herein
and other good and valuable consideration (the receipt and sufficiency of which
are hereby acknowledged), the parties agree as follows:

      1. Assignment and Assumption

      (a) On and as of the date hereof, Sunset hereby sells, assigns and
transfers to the Depositor all of its right, title and interest in the Specified
Mortgage Loans and all rights related thereto as provided under the Agreement to
the extent relating to the Specified Mortgage Loans (without recourse or
warranty except as may be otherwise specifically provided herein), the Depositor
hereby accepts such assignment from Sunset (the "First Assignment and
Assumption"), and the Sellers hereby acknowledge the First Assignment and
Assumption.

<PAGE>

            Sunset specifically reserves and does not assign to the Depositor
hereunder any and all right, title and interest in, to and under and all
obligations of Sunset with respect to any Sunset Mortgage Loans subject to the
Agreement which are not the Specified Mortgage Loans.

      (b) On and as of the date hereof, immediately after giving effect to the
First Assignment and Assumption, the Depositor hereby sells, assigns and
transfers to the Trustee, on behalf of the Trust, all of its right, title and
interest in the Specified Mortgage Loans and all rights related thereto as
provided under the Agreement to the extent relating to the Specified Mortgage
Loans, and the Trustee, on behalf of the Trust, hereby accepts such assignment
from the Depositor (the "Second Assignment and Assumption"), and the Sellers
hereby acknowledge the Second Assignment and Assumption.

      (c) On and as of the date hereof, Sunset represents and warrants to the
Depositor and the Trustee that Sunset has not taken any action that would serve
to impair or encumber the respective ownership interests of the Depositor and
the Trustee in the Specified Mortgage Loans since the date of Sunsets'
acquisition of the Specified Mortgage Loans.

      2. Recognition of Trustee

      (a) From and after the date hereof, both Sunset and the Sellers shall note
the transfer of the Specified Mortgage Loans to the Trustee, in their respective
books and records and shall recognize the Trustee, on behalf of the Trust, as of
the date hereof, as the owner of the Specified Mortgage Loans, and Servicer
shall service the Specified Mortgage Loans for the benefit of the Trust pursuant
to the Agreement, the terms of which are incorporated herein by reference. It is
the intention of the Sellers, the Servicer, the Depositor, the Trustee and
Sunset that this Assignment shall be binding upon and inure to the benefit of
the Depositor, the Trustee and Sunset and their respective successors and
assigns.

      (b) Without in any way limiting the foregoing, the parties confirm that
this Assignment includes the rights relating to amendments and waivers under the
Agreement. Accordingly, the right of Sunset to consent to any amendment of the
Agreement and its rights concerning waivers as set forth in Section 12.07 of the
Agreement shall be exercisable, to the extent any such amendment or waiver
affects the Specified Mortgage Loans or any of the rights under the Agreement
with respect thereto (including, without limitation, the servicing of the
Specified Mortgage Loans), by the Trustee as assignee of Sunset.

      3. Representations and Warranties

      (a) The Depositor represents and warrants that it is a sophisticated
investor able to evaluate the risks and merits of the transactions contemplated
hereby, and that it has not relied in connection therewith upon any statements
or representations of the Sellers or Sunset other than those contained in the
Agreement or this Assignment.

      (b) Each of the parties hereto represents and warrants that it is duly and
legally authorized to enter into this Assignment.

      (c) Each of the parties hereto represents and warrants that this
Assignment has been duly authorized, executed and delivered by it and (assuming
due authorization, execution and delivery thereof by each of the other parties
hereto) constitutes its legal, valid and binding obligation, enforceable against
it in accordance with its terms, except as such enforcement may be limited by
bankruptcy, insolvency, reorganization or other similar laws affecting the
enforcement of creditors' rights generally and by general equitable principles
(regardless of whether such enforcement is considered in a proceeding in equity
or at law).

                                       2

<PAGE>

      (d) Each Seller hereby restates, as of the Closing Date (as defined in the
Pooling and Servicing Agreement referred to below) (or, with respect to the
representations and warranties specifically mentioned in Section 3.05 of the
Agreement, as of the applicable Funding Date (as defined in the Agreement)), the
representations and warranties set forth in the Agreement, with respect to each
of the Specified Mortgage Loans that were sold by it under the Agreement, to and
for the benefit of the Depositor, the Trustee and the Trust, and by this
reference incorporates such representations and warranties herein, as of such
Closing Date. In addition, each Seller hereby represents and warrants that (i)
each Specified Mortgage Loan sold by it under the Agreement is a "qualified
mortgage" within the meaning of 860G(a)(3) of the Code and (ii) each Specified
Mortgage Loan sold by it under the Agreement that is a Cooperative Loan is
secured by stock in a "cooperative housing corporation" within the meaning of
Section 216(b) of the Code.

      4. The Servicer hereby acknowledges that Wells Fargo Bank, N.A. has been
appointed as the master servicer (the "Master Servicer") of the Specified
Mortgage Loans pursuant to the pooling and servicing agreement dated as of the
date hereof among the Depositor, the Master Servicer, the Securities
Administrator and the Trustee (the "Pooling and Servicing Agreement") for
Mortgage Pass-Through Certificates, Series 2004-A3 and, therefore, has the right
to enforce all obligations of the Servicer under the Agreement with respect to
the Specified Mortgage Loans. Such rights will include, without limitation, the
right to terminate the Servicer under the Agreement with respect to the
Specified Mortgage Loans upon the occurrence of an event of default thereunder,
the right to receive all remittances required to be made by the Servicer under
the Agreement with respect to the Specified Mortgage Loans, the right to receive
all monthly reports and other data required to be delivered by the Servicer
under the Agreement with respect to the Specified Mortgage Loans, the right to
examine the books and records of the Servicer, indemnification rights and the
right to exercise certain rights of consent and approval relating to actions
taken by Sunset with respect to the Specified Mortgage Loans. The Servicer shall
make all distributions under the Agreement to the Master Servicer by wire
transfer of immediately available funds to:

                  Wells Fargo Bank, N.A.
                  ABA Number: 121-000-248
                  Account Name: SAS Clearing
                  Account number: 3970771416
                  For further credit to: J.P. Morgan Mortgage Trust 2004-A3,
                                         Distribution Account Number: 18172300

      The Servicer shall deliver all reports required to be delivered under the
Agreement to the Master Servicer at the following address:

                  Wells Fargo Bank, N.A.
                  9062 Old Annapolis Road
                  Columbia, Maryland 21045
                  Attention: Client Manager
                  Telecopier: (410) 715-2380

      5. Establishment of Collection Account

      The Servicer shall establish and maintain a separate Collection Account in
the name of the Trustee, in trust for J.P. Morgan Mortgage Trust 2004-A3, for
all funds collected and received on the Specified Mortgage Loans.

      6. Amendments to the Agreement

                                       3

<PAGE>

      The parties to the Agreement hereby agree to amend the Agreement as
follows:

      (a) The definition of "Qualified Substitute Mortgage Loan" is hereby
amended solely with respect to the Specified Mortgage Loans by deleting clause
(x) in its entirety and replacing it as follows:

            "(x) with respect to each ARM Loan, have the same Index as that of
            the Deleted Mortgage Loan,"

      (b) The definition of "Qualified Substitute Mortgage Loan" is hereby
amended solely with respect to the Specified Mortgage Loans by inserting the
following:

            "and (xiv) with respect to each ARM Loan, not permit conversion of
            the related Note Rate to a permanent fixed Note Rate,"

      (c) With respect to the Specified Mortgage Loans, "Permitted Investments"
shall have the meaning of such term as defined in the Pooling and Servicing
Agreement.

      (d) Notwithstanding any provision in the Purchase Agreement to the
contrary, the parties to the Purchase Agreement hereby agree that the Servicer's
aggregate obligations under clause (4)(ii) of the Purchase Agreement, as amended
by Section 2(b) of Amendment No. 1, with respect to a Mortgage Pool (as defined
in the Pooling and Servicing Agreement) for any month shall be limited to the
total amount of Servicing Fees actually received for the Specified Mortgage
Loans in such Mortgage Pool by the Servicer during such month.

      (e) Section 10.01 is hereby amended by inserting the following at the end
of Section 10.01(2):

      "provided, further, that for purposes of this provision, the failure on
the part of the Seller/Servicer to comply with Sections 7.04 or 7.05 shall be
deemed material."

      7. Indemnification by Master Servicer

      The Master Servicer shall indemnify and hold harmless the Servicer and its
affiliates, and in each case, its officers, directors and agents from and
against any losses, damages, penalties, fines, forfeitures, reasonable legal
fees and related costs, judgments and other costs and expenses arising out of or
based upon a breach by the Master Servicer or any of its officers, directors,
agents or affiliates of its obligations in connection with the preparation,
filing and certification of any Form 10-K pursuant to the Pooling and Servicing
Agreement or the negligence, bad faith or willful misconduct of the Master
Servicer in connection therewith. In addition, the Master Servicer shall
indemnify and hold harmless the Servicer and its affiliates, and in each case,
its officers, directors and agents from and against any losses, damages,
penalties, fines, forfeitures, reasonable legal fees and related costs,
judgments and other costs and expenses arising out of or based upon a breach by
any Servicer (as defined in the Pooling and Servicing Agreement), other than the
Servicer (as defined herein), of its obligations in connection with any back-up
certification (or any other back-up documents) to any certification of any Form
10-K required to be provided by the Master Servicer, but solely to the extent
the Master Servicer receives amounts from such Servicer in connection with any
indemnification provided by such Servicer (in each case as defined in the
Pooling and Servicing Agreement) to the Master Servicer.

                                       4

<PAGE>

      8. Continuing Effect

      Except as contemplated hereby, the Agreement shall remain in full force
and effect in accordance with its terms.

      9. Governing Law

      This Assignment and the rights and obligations hereunder shall be governed
by and construed in accordance with the internal laws of the State of New York.

      10. Notices

      Any notices or other communications permitted or required under the
Agreement to be made to Sunset, the Depositor, the Trustee, Cendant, Bishop's
Gate and the Servicer shall be made in accordance with the terms of the
Agreement and shall be sent to the Depositor and Trustee as follows:

      In the case of Sunset:

                  Sunset Financial Resources, Inc.
                  10245 Centurion Parkway North
                  Suite 305
                  Jacksonville, Florida  32256
                  Telephone: 904-425-4351
                  Facsimile: 904-425-4755

      In the case of the Depositor:

                  J.P. Morgan Acceptance Corporation I
                  270 Park Avenue New York, New York 10017
                  Attention: J.P. Morgan Mortgage Trust 2004-A3

      In the case of the Trustee:

                  Wachovia Bank, National Association
                  401 South Tryon Street
                  Charlotte, North Carolina 28288-1179
                  Attention:  Structured Finance Trust Services,
                  J.P. Morgan Mortgage Trust 2004-A3

      In the case of Cendant:

                  Cendant Mortgage Corporation
                  3000 Leadenhall Road
                  Mt. Laurel, NJ 08054
                  Attention:  Peter A.Thomas,Vice President,Secondary Marketing

      In the case of Bishop's Gate:

                  c/o Cendant Mortgage Corporation, as Administrator
                  3000 Leadenhall Road

                                       5

<PAGE>

                  Mt. Laurel, NJ 08054
                  Attention:Peter A. Thomas, Vice President, Secondary Marketing

      In the case of the Servicer:

                  Cendant Mortgage Corporation
                  3000 Leadenhall Road
                  Mt. Laurel, NJ 08054
                  Attention:Peter A. Thomas, Vice President, Secondary Marketing

or to such other address as may hereafter be furnished by the Depositor and the
Trustee to the parties in accordance with the provisions of the Agreement.

      11. Ratification

      Except as modified and expressly amended by this Assignment, the Agreement
is in all respects ratified and confirmed, and all terms, provisions and
conditions thereof shall be and remain in full force and effect.

      12. Counterparts

      This Assignment may be executed in counterparts, each of which when so
executed shall be deemed to be an original and all of which when taken together
shall constitute one and the same instrument.

      13. Definitions

      Any capitalized term used but not defined in this Assignment has the same
meaning as in the Agreement.

                            [SIGNATURE PAGE FOLLOWS]

                                       6

<PAGE>

      IN WITNESS WHEREOF, the parties hereto have executed this Assignment the
day and year first above written.

                                        SUNSET FINANCIAL RESOURCES, INC.

                                        By: /s/ Thomas G. Manuel
                                            ------------------------------
                                            Name: Thomas G. Manuel
                                            Title: EVP

                                        J.P. MORGAN ACCEPTANCE
                                        CORPORATION I

                                        By: /s/ Jonathan Davis
                                            -----------------------------
                                            Name: Jonathan Davis
                                            Title: Vice President

                                        WACHOVIA BANK, NATIONAL
                                        ASSOCIATION, as Trustee of J.P. Morgan
                                        Mortgage Trust 2004-A3

                                        By: /s/ Stephanie J. Aronovitch
                                            -----------------------------------
                                            Name: Stephanie J. Aronovitch
                                            Title: Assistant Vice President

                                        CENDANT MORTGAGE CORPORATION

                                        By: /s/ Crissy Judge
                                            -----------------------------------
                                            Name: Crissy Judge
                                            Title: Assistant Vice President

                                        BISHOP'S GATE RESIDENTIAL
                                        MORTGAGE TRUST

                                        By: /s/ Crissy Judge
                                            ----------------------------------
                                            Name: Crissy Judge
                                            Title: Assistant Vice President

<PAGE>

                                        CENDANT MORTGAGE CORPORATION,as Servicer


                                        By: /s/ Crissy Judge
                                            ----------------------------------
                                            Name: Crissy Judge
                                            Title: Assistant Vice President

Acknowledged and Agreed:
WELLS FARGO BANK, N.A., as Master Servicer

By: /s/ Ruth Fussell
    ------------------------------
    Name: Ruth Fussell
    Title: Vice President

<PAGE>

                                    EXHIBIT I

                             MORTGAGE LOAN SCHEDULE

               [SEE SCHEDULE A TO POOLING AND SERVICING AGREEMENT]