<PAGE>

                                                                   EXHIBIT 10.57

                                                                  EXECUTION COPY

                                    AGREEMENT

                                  BY AND AMONG

                         NEW YORK MORTGAGE TRUST, INC.,
                             A MARYLAND CORPORATION

                       THE NEW YORK MORTGAGE COMPANY, LLC,
                      A NEW YORK LIMITED LIABILITY COMPANY

                               STEVEN B. SCHNALL,

                                       AND

                                JOSEPH V. FIERRO,

                                December 23, 2003

<PAGE>

                                TABLE OF CONTENTS

<TABLE>
<CAPTION>
                                                                                                                      Page
<S>                                                                                                                   <C>
ARTICLE I REPAYMENT OF NOTES......................................................................................      2
         1.1        Obligations of NYMT and NYMC..................................................................      2
         1.2        Payment and Delivery Instructions.............................................................      2

ARTICLE II REPRESENTATIONS AND WARRANTIES.........................................................................      2
         2.1        Representations and Warranties by NYMT........................................................      2
         2.2        Representations and Warranties by NYMC........................................................      2
         2.3        Representations and Warranties by Schnall and Fierro..........................................      3

ARTICLE III CONDITIONS PRECEDENT TO THE CLOSING...................................................................      3
         3.1        Conditions to NYMT's Obligations..............................................................      3
         3.2        Conditions to NYMC's Obligations..............................................................      4

ARTICLE IV CLOSING AND CLOSING DOCUMENTS..........................................................................      4
         4.1        Closing.......................................................................................      4
         4.2        Schnall's and Fierro's Deliveries.............................................................      4
         4.3        NYMT's Deliveries.............................................................................      4
         4.4        NYMC's Deliveries.............................................................................      5
         4.5        Fees and Expenses; Closing Costs..............................................................      5

ARTICLE V MISCELLANEOUS...........................................................................................      5
         5.1        Notices.......................................................................................      5
         5.2        Entire Agreement; Modifications and Waivers; Cumulative Remedies..............................      7
         5.3        Successors and Assigns........................................................................      7
         5.4        Article Headings..............................................................................      7
         5.5        Governing Law.................................................................................      7
         5.6        Counterparts..................................................................................      7
         5.7        Survival......................................................................................      7
         5.8        Further Acts..................................................................................      8
         5.9        Severability..................................................................................      8
         5.10       Attorneys' Fees...............................................................................      8
</TABLE>

<PAGE>

                                    AGREEMENT

         THIS AGREEMENT (this "Agreement") is made as of the 23rd day of
December, 2003, by and among NEW YORK MORTGAGE TRUST, INC., a Maryland
corporation ("NYMT"), THE NEW YORK MORTGAGE COMPANY, LLC, a New York limited
liability company ("NYMC"), STEVEN B. SCHNALL, an individual ("Schnall") and
JOSEPH V. FIERRO, an individual ("Fierro").

                                    RECITALS

         A.       Schnall is the record and beneficial owner of a 70% membership
interest in NYMC (the "Schnall Interest") and Fierro is the record and
beneficial owner of a 30% membership interest in NYMC (the "Fierro Interest").
Schnall and Fierro have entered into a Contribution Agreement with NYMT, dated
as of December 22, 2003 (the "Contribution Agreement"), pursuant to which they
have agreed to contribute their membership interests in NYMC to NYMT in exchange
for certain consideration specified in such Contribution Agreement.

         B.       NYMT intends to file a registration statement on Form S-11 in
connection with the proposed initial public offering of the common stock, par
value $0.01 per share, of NYMT ("Common Stock"), which initial public offering
(the "IPO") is expected to close simultaneously with the closing of the
transactions contemplated under the Contribution Agreement (the "Closing Date").

         C.       Schnall holds a promissory note, dated August 31, 2003, as
amended and restated on December 23, 2003, in the principal amount of
$12,132,550.00, due and payable by NYMC no later than February 27, 2004 (the
"First Schnall Note") and Fierro holds a promissory note, dated August 31, 2003,
as amended and restated on December 23, 2003, in the principal amount of
$2,574,352.00, due and payable by NYMC no later than February 27, 2004 (the
"First Fierro Note").

         D.       Schnall and Fierro intend to cause NYMC to make certain
distributions to Schnall and Fierro, including potentially by issuing to Schnall
and Fierro one or more additional promissory notes, in amounts approximately
equal to the estimated paid-in capital and retained earnings of NYMC as of the
Closing Date, subject to adjustment within 60 days after the Closing Date to
reflect the actual paid-in capital and retained earnings of NYMC as of the
Closing Date. Any such additional promissory notes (the "Additional Notes" and,
together with the First Schnall Note and the First Fierro Note, the "Notes")
shall be in substantially the same form as the First Schnall Note and the First
Fierro Note.

         E.       NYMT desires to fund the repayment by NYMC of the Notes
outstanding as of the Closing Date by contributing to NYMC funds in an amount
sufficient to effect such repayments out of the net proceeds of the IPO.

<PAGE>

                                    AGREEMENT

         NOW, THEREFORE, for and in consideration of the mutual covenants herein
contained, the parties hereto agree as follows:

                                   ARTICLE I

                               REPAYMENT OF NOTES

         1.1      Obligations of NYMT and NYMC.

         NYMT hereby agrees to contribute to NYMC on the Closing Date out of the
net proceeds received by NYMT from the IPO an amount sufficient to enable NYMC
to pay in full, and NYMC hereby agrees to pay in full promptly upon receipt of
such contribution, the entire principal and all accrued but unpaid interest
under all Notes outstanding as of the Closing Date in accordance with the terms
thereof.

         1.2      Payment and Delivery Instructions

         Schnall and Fierro shall each provide written instructions to NYMT and
NYMC at least 48 hours prior to the Closing Date setting out wire transfer
instructions for the payments specified in Section 1.1.

                                   ARTICLE II

                         REPRESENTATIONS AND WARRANTIES

         2.1      Representations and Warranties by NYMT.

         NYMT hereby represents and warrants to NYMC, Schnall and Fierro as of
the date of this Agreement and as of the Closing Date as follows:

                  (a)      Organization and Power. NYMT is duly organized,
validly existing, and in good standing under the laws of the State of Maryland,
and has full right, power, and authority to enter into this Agreement and to
assume and perform all of its obligations under this Agreement; and, the
execution and delivery of this Agreement and the performance by NYMT of its
obligations hereunder have been duly authorized by all requisite action of NYMT
and require no further action or approval of NYMT's stockholders or of any other
individuals or entities in order to constitute this Agreement as a binding and
enforceable obligation of NYMT.

                  (b)      Noncontravention. Neither the entry into nor the
performance of, or compliance with, this Agreement by NYMT has resulted, or will
result, in any violation of, or default under, or result in the acceleration of,
any obligation under any existing mortgage, indenture, lien agreement, note,
contract, permit, judgment, decree, order, restrictive covenant, statute, rule,
or regulation applicable to NYMT.

         2.2      Representations and Warranties by NYMC.

                                       2
<PAGE>

         NYMC hereby represents and warrants to NYMT, Schnall and Fierro as of
the date of this Agreement and as of the Closing Date as follows:

                  (a)      Organization and Power. NYMC is validly existing and
in good standing under the laws of the State of New York, and has full right,
power, and authority to enter into this Agreement and to assume and perform all
of its obligations under this Agreement; and, the execution and delivery of this
Agreement and the performance by NYMC of its obligations hereunder have been
duly authorized by all requisite action of NYMC and require no further action or
approval of NYMC's stockholders or of any other individuals or entities in order
to constitute this Agreement as a binding and enforceable obligation of NYMC.

                  (b)      Noncontravention. Neither the entry into nor the
performance of, or compliance with, this Agreement by NYMC has resulted, or will
result, in any violation of, or default under, or result in the acceleration of,
any obligation under any existing mortgage, indenture, lien agreement, note,
contract, permit, judgment, decree, order, restrictive covenant, statute, rule,
or regulation applicable to NYMC.

         2.3      Representations and Warranties by Schnall and Fierro.

         Each of Schnall and Fierro hereby severally, and not jointly, makes the
following representations and warranties as of the date of this Agreement as of
the Closing Date:

                  (a)      Holders in Due Course. Schnall is the holder in due
course of the First Schnall Note and any Additional Note payable to Schnall.
Fierro is the holder in due course of the First Fierro Note and any Additional
Note payable to Fierro.

                  (b)      Tax Advice. Schnall and Fierro have each obtained
from their own personal tax advisors advice regarding the transaction
contemplated by this Agreement, including, without limitation, the tax
consequences of the transactions contemplated hereby.

                                  ARTICLE III

                       CONDITIONS PRECEDENT TO THE CLOSING

         3.1      Conditions to NYMT's Obligations.

         NYMT's obligation to perform its obligations hereunder on the Closing
Date is subject to the timely satisfaction of each and every one of the
conditions and requirements set forth in this Section 3.1, all of which shall be
conditions precedent to NYMT's obligations under this Agreement.

                  (a)      Representations and Warranties of NYMC, Schnall and
Fierro. The representations and warranties of NYMC set forth in Section 2.2 and
of each of Schnall and Fierro set forth in Section 2.3 shall be true and correct
as if made again on the Closing Date.

                  (b)      Completion of Contribution Agreement. The
transactions contemplated by the Contribution Agreement shall have been
completed.

                                       3
<PAGE>

                  (c)      Completion of IPO. The IPO shall have been completed.

         3.2      Conditions to NYMC's Obligations.

         NYMC's obligation to perform its obligations hereunder on the Closing
Date is subject to the timely satisfaction of each and every one of the
conditions and requirements set forth in this Section 3.2, all of which shall be
conditions precedent to NYMC's obligations under this Agreement.

                  (a)      NYMT's Obligations. NYMT shall have performed all
covenants, agreements and other obligations of NYMT hereunder which are to be
performed on the Closing Date.

                  (b)      Representations and Warranties of NYMT, Schnall and
Fierro. The representations and warranties of NYMT set forth in Section 2.1 and
of each of Schnall and Fierro set forth in Section 2.3 shall be true and correct
as if made again on the Closing Date.

                  (c)      Completion of Contribution Agreement. The
transactions contemplated by the Contribution Agreement shall have been
completed.

                  (d)      Completion of IPO. The IPO shall have been completed.

                                   ARTICLE IV

                          CLOSING AND CLOSING DOCUMENTS

         4.1      Closing.

         The consummation and closing (the "Closing") of the transactions
contemplated under this Agreement shall take place at the offices of NYMC in New
York, New York, or such other place as is mutually agreeable to the parties, on
the Closing Date; provided, that this Agreement shall be terminated and the
parties shall have no obligations hereunder if Closing Date does not occur by
April 30, 2004. In the event this Agreement is terminated, NYMC's obligations
under the Notes shall remain in full force and effect.

         4.2      Schnall's and Fierro's Deliveries.

         At the Closing, upon confirmation of receipt of payment in full of all
amounts owing under all Notes, Schnall and Fierro shall deliver the following to
NYMC:

                  (a)      The Notes. Schnall and Fierro shall deliver all
original Notes to NYMC.

         4.3      NYMT's Deliveries.

         At the Closing, NYMT shall deliver the following:

                                       4
<PAGE>

                  (a)      Contribution to NYMC of an amount sufficient to
enable NYMC to pay in full the entire principal and all accrued but unpaid
interest under all Notes outstanding as of the Closing Date in accordance with
the terms thereof.

                  (b)      Bring-Down Certificate. A certificate signed by an
authorized officer of NYMT that all conditions to NYMC's obligations set forth
in Section 3.2 hereof have been satisfied.

         4.4      NYMC's Deliveries.

         At the Closing, NYMT shall deliver the following:

                  (a)      Payment by NYMC to Schnall and Fierro, respectively,
in accordance with the written instructions delivered by them pursuant to
Section 1.2 hereof, of the entire principal and all accrued but unpaid interest
under all Notes outstanding as of the Closing Date in accordance with the terms
thereof.

                  (b)      Bring-Down Certificate. A certificate signed by an
authorized officer of NYMC that all conditions to NYMT's obligations set forth
in Section 3.1 hereof have been satisfied.

         4.5      Fees and Expenses; Closing Costs.

         NYMT shall pay all fees, expenses and closing costs relating to the
transactions contemplated by this Agreement.

                                    ARTICLE V

                                  MISCELLANEOUS

         5.1      Notices.

         Any notice provided for by this Agreement and any other notice, demand,
or communication required hereunder shall be in writing and either delivered in
person (including by confirmed facsimile transmission) or sent by hand delivered
against receipt or sent by recognized overnight delivery service or by certified
or registered mail, postage prepaid, with return receipt requested. All notices
shall be addressed as follows:

                  NYMT:

                  New York Mortgage Trust, Inc.
                  1301 Avenue of the Americas
                  New York, New York 10019
                  Attention: President
                  Fax No.: (212) 655-6269

                                       5
<PAGE>

                  with a copy to:

                  Hunton & Williams LLP
                  Riverfront Plaza, East Tower
                  951 E. Byrd Street
                  Richmond, Virginia 23219
                  Attention: Daniel M. LeBey, Esq.
                  Fax No.: (804) 788-8218

                  NYMC:

                  New York Mortgage Trust, Inc.
                  1301 Avenue of the Americas
                  New York, New York 10019
                  Attention: President
                  Fax No.: (212) 655-6269

                  with a copy to:

                  Hunton & Williams LLP
                  Riverfront Plaza, East Tower
                  951 E. Byrd Street
                  Richmond, Virginia 23219
                  Attention: Daniel M. LeBey, Esq.
                  Fax No.: (804) 788-8218

                  Schnall:

                  Steven B. Schnall
                  c/o The New York Mortgage Company, LLC
                  1301 Avenue of Americas
                  New York, New York 10019
                  Fax No.: (212) 634-9420

                  Fierro:

                  Joseph V. Fierro
                  c/o The New York Mortgage Company, LLC
                  1301 Avenue of Americas
                  New York, New York 10019
                  Fax No.: (212) 634-9420

         Any address or name specified above may be changed by a notice given by
the addressee to the other party. Any notice, demand or other communication
shall be deemed given and
                                       6
<PAGE>

effective as of the date of delivery in person or receipt set forth on the
return receipt. The inability to deliver because of changed address of which no
notice was given, or rejection or other refusal to accept any notice, demand or
other communication, shall be deemed to be receipt of the notice, demand or
other communication as of the date of such attempt to deliver or rejection or
refusal to accept.

         5.2      Entire Agreement; Modifications and Waivers; Cumulative
Remedies.

         This Agreement supersedes any existing letter of intent between the
parties, constitutes the entire agreement among the parties hereto and may not
be modified or amended except by instrument in writing signed by the parties
hereto, and no provisions or conditions may be waived other than by a writing
signed by the party waiving such provisions or conditions. No delay or omission
in the exercise of any right or remedy accruing to any party upon any breach
under this Agreement by another party shall impair such right or remedy or be
construed as a waiver of any such breach theretofore or thereafter occurring.
The waiver by any party of any breach of any term, covenant, or condition herein
stated shall not be deemed to be a waiver of any other breach, or of a
subsequent breach of the same or any other term, covenant, or condition herein
contained. All rights, powers, options, or remedies afforded to the parties
either hereunder or by law shall be cumulative and not alternative, and the
exercise of one right, power, option, or remedy shall not bar other rights,
powers, options, or remedies allowed herein or by law, unless expressly provided
to the contrary herein.

         5.3      Successors and Assigns.

         This Agreement may not be assigned by any party hereto without the
prior written approval of the other parties hereto. This Agreement shall be
binding upon, and inure to the benefit of, each of the parties, and their
respective legal representatives, successors, and permitted assigns.

         5.4      Article Headings.

         Article headings and article and section numbers are inserted herein
only as a matter of convenience and in no way define, limit, or prescribe the
scope or intent of this Agreement or any part hereof and shall not be considered
in interpreting or construing this Agreement.

         5.5      Governing Law.

         This Agreement shall be construed and interpreted in accordance with
the laws of the State of New York, without regard to conflicts of laws
principles.

         5.6      Counterparts.

         This Agreement may be executed in any number of counterparts and by any
party hereto on a separate counterpart, each of which when so executed and
delivered shall be deemed an original and all of which taken together shall
constitute but one and the same instrument.

         5.7      Survival.

                                       7
<PAGE>

         All representations and warranties contained in this Agreement, and all
covenants and agreements contained in the Agreement which contemplate
performance after the Closing Date shall survive the Closing.

         5.8      Further Acts.

         In addition to the acts, instruments and agreements recited herein and
contemplated to be performed, executed and delivered by the parties, the parties
shall perform, execute, and deliver or cause to be performed, executed, and
delivered at the Closing or after the Closing, any and all further acts,
instruments, and agreements and provide such further assurances as the other
parties may reasonably require to consummate the transaction contemplated
hereunder.

         5.9      Severability.

         In case any one or more of the provisions contained in this Agreement
shall for any reason be held to be invalid, illegal, or unenforceable in any
respect, such invalidity, illegality, or unenforceability shall not affect any
other provision hereof, and this Agreement shall be construed as if such
invalid, illegal, or unenforceable provision had never been contained herein.

         5.10     Attorneys' Fees.

         Should a party employ an attorney or attorneys to enforce any of the
provisions hereof or to protect its interest in any manner arising under this
Agreement, or to recover damages for breach of this Agreement, any
non-prevailing party in any action pursued in a court of competent jurisdiction
(the finality of which is not legally contested) shall pay to the prevailing
party all reasonable costs, damages, and expenses, including reasonable
attorneys' fees, expended or incurred in connection therewith.

                  [REMAINDER OF PAGE INTENTIONALLY LEFT BLANK.]

                     [SIGNATURES APPEAR ON FOLLOWING PAGE.]

                                       8
<PAGE>

         IN WITNESS WHEREOF, this Agreement has been entered into effective as
of the 23rd day of December, 2003.

                                  NEW YORK MORTGAGE TRUST, INC.,
                                  a Maryland corporation


                                  By: /s/ David Akre
                                      _______________________________________
                                  Name:  David Akre
                                  Title: Co-Chief Executive Officer


                                  THE NEW YORK MORTGAGE COMPANY, LLC,
                                  a New York limited liability company


                                  By: /s/ Steven B. Schnall
                                      _______________________________________
                                  Name:  Steven B. Schnall
                                  Title: President and Chief Executive Officer


                                  STEVEN B. SCHNALL


                                  By: /s/ Steven B. Schnall
                                      _______________________________________
                                  Name:  Steven B. Schnall
                                  Title: Member of The New York
                                         Mortgage Company, LLC


                                  JOSEPH V. FIERRO


                                  By: /s/ Joseph V. Fierro
                                      _______________________________________
                                  Name:  Joseph V. Fierro
                                  Title: Member of The New York
                                         Mortgage Company, LLC


                                       9