<PAGE>

                                                                    Exhibit 10.6

                ASSIGNMENT, ASSUMPTION AND RECOGNITION AGREEMENT

      THIS ASSIGNMENT, ASSUMPTION AND RECOGNITION AGREEMENT (this "Assignment"),
dated as of June 1, 2004, is entered into among J.P. Morgan Acceptance
Corporation I, a Delaware corporation (the "Depositor"), Wachovia Bank, National
Association, as trustee (the "Trustee") of J.P. Morgan Mortgage Trust 2004-A3
(the "Trust"), Sunset Financial Resources, Inc., a Maryland corporation
("Sunset") and National City Mortgage Co. ("National City").

                                    RECITALS

      WHEREAS National City, as seller and as servicer, and J.P. Morgan Mortgage
Acquisition Corp. ("JPMorgan Acquisition") entered into a certain Flow Master
Seller's Warranties and Servicing Agreement, dated as of February 24, 2004 (the
"Agreement"), pursuant to which Sunset acquired certain mortgage loans (the
"Mortgage Loans") pursuant to the terms of the Agreement and National City
agreed to service such Mortgage Loans;

      WHEREAS, pursuant to an Assignment, Assumption and Recognition Agreement
dated as of March 30, 2004 relating to the Agreement, Sunset acquired from
JPMorgan Acquisition all of JPMorgan Acquisition's right, title and interest in
certain of the Mortgage Loans (the "Sunset Mortgage Loans") and National City
agreed to service such Sunset Mortgage Loans pursuant to the Agreement;

      WHEREAS the Depositor has agreed, on the terms and conditions contained
herein, to purchase from Sunset certain of the Sunset Mortgage Loans (the
"Specified Mortgage Loans") which are subject to the provisions of the Agreement
and are listed on the mortgage loan schedule attached as Exhibit I hereto (the
"Specified Mortgage Loan Schedule"); and

      WHEREAS the Trustee, on behalf of the Trust, has agreed, on the terms and
conditions contained herein, to purchase from the Depositor the Specified
Mortgage Loans;

      NOW, THEREFORE, in consideration of the mutual promises contained herein
and other good and valuable consideration (the receipt and sufficiency of which
are hereby acknowledged), the parties agree as follows:

      1. Assignment and Assumption

      (a) On and as of the date hereof, Sunset hereby sells, assigns and
transfers to the Depositor all of its right, title and interest in the Specified
Mortgage Loans and all rights and obligations related thereto as provided under
the Agreement to the extent relating to the Specified Mortgage Loans (without
recourse or warranty except as may be otherwise specifically provided herein),
the Depositor hereby accepts such assignment from Sunset (the "First Assignment
and Assumption"), and National City hereby acknowledges the First Assignment and
Assumption.

      Sunset specifically reserves and does not assign to the Depositor
hereunder any and all right, title and interest in, to and under and all
obligations of Sunset with respect to any Sunset Mortgage Loans subject to the
Agreement which are not the Specified Mortgage Loans.

      (b) On and as of the date hereof, immediately after giving effect to the
First Assignment and Assumption, the Depositor hereby sells, assigns and
transfers to the Trustee, on behalf of the Trust, all of its right, title and
interest in the Specified Mortgage Loans and all rights and obligations related
thereto as provided under the Agreement to the extent relating to the Specified
Mortgage Loans, and the Trustee, on

<PAGE>

behalf of the Trust, hereby accepts such assignment from the Depositor (the
"Second Assignment and Assumption"), and National City hereby acknowledges the
Second Assignment and Assumption.

      (c) On and as of the date hereof, Sunset represents and warrants to the
Depositor and the Trustee that Sunset has not taken any action that would serve
to impair or encumber the respective ownership interests of the Depositor and
the Trustee in the Specified Mortgage Loans since the date of Sunsets'
acquisition of the Specified Mortgage Loans.

      2. Recognition of Trustee

      (a) From and after the date hereof, both Sunset and National City shall
note the transfer of the Specified Mortgage Loans to the Trustee, in their
respective books and records and shall recognize the Trustee, on behalf of the
Trust, as of the date hereof, as the owner of the Specified Mortgage Loans, and
National City shall service the Specified Mortgage Loans for the benefit of the
Trust pursuant to the Agreement, the terms of which are incorporated herein by
reference. It is the intention of National City, the Depositor, the Trustee and
Sunset that this Assignment shall be binding upon and inure to the benefit of
the Depositor, the Trustee and Sunset and their respective successors and
assigns.

      (b) Without in any way limiting the foregoing, the parties confirm that
this Assignment includes the rights relating to amendments and waivers under the
Agreement. Accordingly, the right of Sunset to consent to any amendment of the
Agreement and its rights concerning waivers as set forth in Sections 12.02 and
12.12, respectively, of the Agreement shall be exercisable, to the extent any
such amendment or waiver affects the Specified Mortgage Loans or any of the
rights or obligations under the Agreement with respect thereto (including,
without limitation, the servicing of the Specified Mortgage Loans), by the
Trustee as assignee of Sunset.

      3. Representations and Warranties

      (a) The Depositor represents and warrants that it is a sophisticated
investor able to evaluate the risks and merits of the transactions contemplated
hereby, and that it has not relied in connection therewith upon any statements
or representations of National City or Sunset other than those contained in the
Agreement or this Assignment.

      (b) Each of the parties hereto represents and warrants that it is duly and
legally authorized to enter into this Assignment.

      (c) Each of the parties hereto represents and warrants that this
Assignment has been duly authorized, executed and delivered by it and (assuming
due authorization, execution and delivery thereof by each of the other parties
hereto) constitutes its legal, valid and binding obligation, enforceable against
it in accordance with its terms, except as such enforcement may be limited by
bankruptcy, insolvency, reorganization or other similar laws affecting the
enforcement of creditors' rights generally and by general equitable principles
(regardless of whether such enforcement is considered in a proceeding in equity
or at law).

      (d) National City hereby restates, as of the Closing Date (as defined in
the Pooling and Servicing Agreement referred to below), the representations and
warranties contained in Sections 3.01 and 3.02 of the Agreement, with respect to
each of the Specified Mortgage Loans that were sold by it under the Agreement,
to and for the benefit of the Depositor, the Trustee and the Trust, and by this
reference incorporates such representations and warranties herein, as of such
Closing Date.

                                       2

<PAGE>

      4. National City hereby acknowledges that Wells Fargo Bank, N.A. has been
appointed as the master servicer (the "Master Servicer") of the Specified
Mortgage Loans pursuant to the pooling and servicing agreement dated as of the
date hereof among the Depositor, the Master Servicer, the Securities
Administrator and the Trustee (the "Pooling and Servicing Agreement") for
Mortgage Pass-Through Certificates, Series 2004-A3 and, therefore, has the right
to enforce all obligations of National City under the Agreement with respect to
the Specified Mortgage Loans. Such rights will include, without limitation, the
right to terminate National City under the Agreement with respect to the
Specified Mortgage Loans upon the occurrence of an event of default thereunder,
the right to receive all remittances required to be made by National City under
the Agreement with respect to the Specified Mortgage Loans, the right to receive
all monthly reports and other data required to be delivered by National City
under the Agreement with respect to the Specified Mortgage Loans, the right to
examine the books and records of National City, indemnification rights and the
right to exercise certain rights of consent and approval relating to actions
taken by Sunset with respect to the Specified Mortgage Loans. National City
shall make all distributions under the Agreement to the Master Servicer by wire
transfer of immediately available funds to:

                  Wells Fargo Bank, N.A.
                  ABA Number: 121-000-248
                  Account Name: SAS Clearing
                  Account number: 3970771416
                  For further credit to:  J.P. Morgan Mortgage Trust 2004-A3,
                                          Distribution Account Number: 18172300

      National City shall deliver all reports required to be delivered under the
Agreement to the Master Servicer at the following address:

                  Wells Fargo Bank, N.A.
                  9062 Old Annapolis Road
                  Columbia, Maryland 21045
                  Attention: Client Manager
                  Telecopier: (410) 715-2380

      5. Establishment of Custodial Account

      The Servicer shall establish and maintain a separate Custodial Account in
the name of the Trustee, in trust for J.P. Morgan Mortgage Trust 2004-A3, for
all funds collected and received on the Specified Mortgage Loans.

      6. Amendments to the Agreement

      The parties to the Agreement hereby agree to amend the Agreement as
follows:

      (a) The definition of "Qualified Substitute Mortgage Loan" is hereby
amended solely with respect to the Specified Mortgage Loans by inserting the
following:

      "(vi) have a Loan-to-Value Ratio no higher than that of the Deleted
      Mortgage Loan at substitution; (vii) have the same Adjustment Date as that
      of the Deleted Mortgage Loan; (viii) have a Minimum Mortgage Interest Rate
      not less than that of the Deleted Mortgage Loan; (ix) have a Maximum
      Mortgage Interest Rate not less than (and not more than 2% greater than)
      the Maximum Mortgage Interest Rate of the Deleted Mortgage Loan; (x) not
      permit conversion of the related Mortgage Interest Rate to a permanent
      fixed Mortgage Interest Rate; and (xi) be

                                       3

<PAGE>

      accompanied by an Opinion of Counsel that such Qualified Substitute
      Mortgage Loan would not adversely affect the REMIC status of any REMIC
      created under the Pooling and Servicing Agreement or would not otherwise
      be prohibited by the Pooling and Servicing Agreement."

      (b) With respect to the Specified Mortgage Loans, "Eligible Investments"
shall have the meaning of "Permitting Investments" as defined in the Pooling and
Servicing Agreement.

      (c) Notwithstanding any provision in the Agreement to the contrary, the
parties to the Agreement hereby agree that, in connection with any prepayments
in full of the Specified Mortgage Loans in a Mortgage Pool (as defined in the
Pooling and Servicing Agreement), the Servicer shall contribute from its own
funds, to the extent that such contributions do not exceed the aggregate
Servicing Fee for the Specified Mortgage Loans in such Mortgage Pool, any
shortfall in the interest component thereof such that one month's interest shall
be deposited to the Custodial Account as defined in the Agreement.

      7. Indemnification by Master Servicer

      The Master Servicer shall indemnify and hold harmless National City and
its affiliates, and in each case, its officers, directors and agents from and
against any losses, damages, penalties, fines, forfeitures, reasonable legal
fees and related costs, judgments and other costs and expenses arising out of or
based upon a breach by the Master Servicer or any of its officers, directors,
agents or affiliates of its obligations in connection with the preparation,
filing and certification of any Form 10-K pursuant to the Pooling and Servicing
Agreement or the negligence, bad faith or willful misconduct of the Master
Servicer in connection therewith. In addition, the Master Servicer shall
indemnify and hold harmless National City and its affiliates, and in each case,
its officers, directors and agents from and against any losses, damages,
penalties, fines, forfeitures, reasonable legal fees and related costs,
judgments and other costs and expenses arising out of or based upon a breach by
any Servicer (as defined in the Pooling and Servicing Agreement), other than
National City, of its obligations in connection with any back-up certification
(or any other back-up documents) to any certification of any Form 10-K required
to be provided by the Master Servicer, but solely to the extent the Master
Servicer receives amounts from such Servicer in connection with any
indemnification provided by such Servicer (in each case as defined in the
Pooling and Servicing Agreement) to the Master Servicer.

      8. Continuing Effect

      Except as contemplated hereby, the Agreement shall remain in full force
and effect in accordance with its terms.

      9. Governing Law

      This Assignment and the rights and obligations hereunder shall be governed
by and construed in accordance with the internal laws of the State of New York.

      10. Notices

      Any notices or other communications permitted or required under the
Agreement to be made to Sunset, the Depositor, the Trustee and National City
shall be made in accordance with the terms of the Agreement and shall be sent to
the Depositor and Trustee as follows:

                                       4

<PAGE>

      In the case of Sunset:

                  Sunset Financial Resources, Inc.
                  10245 Centurion Parkway North
                  Suite 305
                  Jacksonville, Florida  32256
                  Attention: Byron Boston
                  Telephone: 904-425-4351
                  Facsimile: 904-425-4755

      In the case of the Depositor:

                  J.P. Morgan Acceptance Corporation I
                  270 Park Avenue
                  New York, New York 10017
                  Attention: J.P. Morgan Mortgage Trust 2004-A3

      In the case of the Trustee:

                  Wachovia Bank, National Association
                  401 South Tryon Street
                  Charlotte, North Carolina 28288-1179
                  Attention:  Structured Finance Trust Services,
                  J.P. Morgan Mortgage Trust 2004-A3

      In the case of National City:

                  National City Mortgage Company
                  Attention: Hugh Yeary
                  3232 Newmark Dr.
                  Miamisburg, Ohio 45342

      With a copy to:

                  National City Mortgage Company
                  Attention: T. Jackson Case
                  3232 Newmark Dr.
                  Miamisburg, Ohio 45342

or to such other address as may hereafter be furnished by the Depositor and the
Trustee to the parties in accordance with the provisions of the Agreement.

      11. Ratification

      Except as modified and expressly amended by this Assignment, the Agreement
is in all respects ratified and confirmed, and all terms, provisions and
conditions thereof shall be and remain in full force and effect.

                                       5

<PAGE>

      12. Counterparts

      This Assignment may be executed in counterparts, each of which when so
executed shall be deemed to be an original and all of which when taken together
shall constitute one and the same instrument.

      13. Definitions

      Any capitalized term used but not defined in this Assignment has the same
meaning as in the Agreement.

                            [SIGNATURE PAGE FOLLOWS]

                                       6

<PAGE>

      IN WITNESS WHEREOF, the parties hereto have executed this Assignment the
day and year first above written.

                                        SUNSET FINANCIAL RESOURCES, INC.

                                        By:  /s/ Thomas G. Manuel
                                             ------------------------------
                                            Name: Thomas G. Manuel
                                            Title: EVP

                                        J.P. MORGAN ACCEPTANCE
                                        CORPORATION I

                                        By:  /s/ Jonathan Davis
                                             -----------------------------------
                                            Name: Jonathan Davis
                                            Title: Vice President

                                        WACHOVIA BANK, NATIONAL
                                        ASSOCIATION, as Trustee of J.P. Morgan
                                        Mortgage Trust 2004-A3

                                        By:  /s/ Stephanie J. Aronovitch
                                             -----------------------------------
                                            Name: Stephanie J. Aronovitch
                                            Title: Assistant Vice President

                                        NATIONAL CITY MORTGAGE CO.

                                        By:  /s/ Mary Beth Criswell
                                             -----------------------------------
                                            Name: Mary Beth Criswell
                                            Title: Vice President

Acknowledged and Agreed:
WELLS FARGO BANK, N.A., as Master Servicer

By:  /s/ Ruth Fussell
     ---------------------------------------
     Name: Ruth Fussell
     Title:  Vice President

<PAGE>

                                    EXHIBIT I

                             MORTGAGE LOAN SCHEDULE

               [SEE SCHEDULE A TO POOLING AND SERVICING AGREEMENT]