<PAGE>


                                                                    EXHIBIT 10.4

                                                                       EXECUTION

                       SERVICING RIGHTS TRANSFER AGREEMENT

      THIS SERVICING RIGHTS TRANSFER AGREEMENT (this "Agreement"), dated as of
January 1, 2005, is by and between BAYVIEW FINANCIAL, L.P. ("Bayview") and UMT
LT TRUST ("UMT LT").

      WHEREAS, pursuant to that certain mortgage loan sale agreement dated as of
January 1, 2005 (the "Sale Agreement"), by and between UMT LT and UMT Funding
Trust ("SPE"), UMT LT will sell, transfer, assign, set over, deposit with and
otherwise convey the mortgage loans listed on Schedule A to the Sale Agreement
(the "Mortgage Loans"), excluding the servicing rights thereto;

      WHEREAS, pursuant to that certain trust agreement dated as of January 1,
2005 (the "Trust Agreement"), by and between SPE and Wachovia Bank, National
Association, as trustee (the "Trustee"), SPE will transfer all of its interest
in the Mortgage Loans to the Trustee, as trustee for the owners of Bayview
Asset-Backed Securities, Series 2005-1;

      WHEREAS, UMT LT owns the Servicing Rights (as defined below) with respect
to the Mortgage Loans and desires, on the Closing Date, to transfer, assign, and
deliver the Servicing Rights with respect to the Mortgage Loans to Bayview, in
accordance with the terms and conditions of this Agreement;

      NOW THEREFORE, in consideration of the mutual promises, covenants, and
conditions hereinafter set forth, and other good and valuable consideration, the
receipt and sufficiency of which are hereby acknowledged, the parties hereto
agree as follows:

      1. Capitalized terms used but not otherwise defined herein shall have the
meanings assigned to them in the Trust Agreement. In addition, the following
term, as used in this Agreement, shall have the meaning assigned to it below:

      "Servicing Rights:" Any and all of the following on or after the Cut-off
Date: (a) any and all rights to service the Mortgage Loans; (b) any payments to
or monies received by UMT LT for servicing the Mortgage Loans; (c) all
agreements or documents creating, defining or evidencing any such servicing
rights to the extent they relate to such servicing rights and all rights of UMT
LT thereunder; (d) any interest on escrow payments made by Mortgagors for taxes,
assessments, primary mortgage or hazard insurance premiums or comparable items
with respect to the Mortgage Loans, to the extent allowed by law to be kept by
the servicer and any amounts actually collected by UMT LT with respect thereto;
(e) all accounts and other rights to payment related to any of the property
described in this paragraph; and (f) any and all documents, files, records,
servicing files, servicing documents, servicing records, data tapes, computer
records, or other information pertaining to the Mortgage Loans or pertaining to
the past, present or prospective servicing of the Mortgage Loans.

      2. (a) UMT LT hereby represents and warrants to Bayview that immediately
prior to the sale of the Servicing Rights to Bayview, UMT LT was the sole owner
of the Servicing

<PAGE>

Rights, subject to no prior lien, mortgage, security interest, pledge, charge or
other encumbrance, and on the Closing Date, Bayview will acquire the Servicing
Rights free and clear of any prior lien, mortgage security interest, pledge,
change or other encumbrance.

      (b) It is understood and agreed that the representations and warranties
set forth herein and the obligations of UMT LT set forth herein shall survive
the sale of Servicing Rights to Bayview and shall inure to the benefit of
Bayview. Upon discovery by either UMT LT or Bayview of a breach of any of the
foregoing representations, warranties or obligations, the party discovering such
breach shall give prompt written notice to the other parties. Within 60 days of
the discovery of any such breach that materially and adversely affects the value
of the Servicing Rights or the interest of Bayview therein, UMT LT shall use all
reasonable efforts to promptly cure such breach in all material respects.

      In addition to the obligations set forth in the immediately preceding
paragraph, UMT LT indemnifies, defends and holds harmless Bayview and its
partners, directors, officers, employees, agents and affiliates against any
losses, damages, penalties, fines, forfeitures, reasonable legal fees and
related costs, judgments, and other out-of-pocket costs and expenses resulting
from any claim, demand, defense or assertion based on or grounded upon, or
resulting from: (i) a breach of the representations, warranties or obligations
contained in this Agreement, (ii) any act or omission in connection with the
servicing of any Mortgage Loan prior to the sale of the Servicing Rights to
Bayview, (iii) any act or omission by UMT LT in its performance of its
obligations under any servicing or collection agreement prior to the sale of the
Servicing Rights to Bayview and (iv) claims, litigation, proceedings,
governmental investigations, orders, injunctions or decrees filed against
Bayview, the basis for which occurred prior to the sale of the Servicing Rights
to Bayview, involving the servicing of the Mortgage Loans.

         3. (a) UMT LT hereby transfers, assigns, and delivers to Bayview, and
Bayview hereby accepts such transfer, assignment, and delivery of, all of UMT
LT's right, title and interest in and to the Servicing Rights with respect to
the Mortgage Loans, it being understood that no endorsement of mortgage notes or
assignments of mortgages shall be required in connection with this transfer
other than as provided in the Trust Agreement. From the date hereof, Bayview or
its assignee, transferee and/or designee shall be primarily responsible for
servicing the Mortgage Loans, subject, however, to the terms of the Servicing
Agreement.

      (b) In consideration for the Servicing Rights, on the Closing Date,
Bayview will purchase the Bayview Asset-Backed Securities, Series 2005-1, Class
A, and will cause Bayview Loan Servicing, LLC to enter into a subservicing
agreement with UMT LT, as subservicer.

      4. This Agreement may be executed simultaneously in any number of
counterparts, each of which counterparts shall be deemed to be an original, and
all of which counterparts shall together constitute but one and the same
instrument.

      5. This Agreement shall be fully assignable and transferable by Bayview.

      6. In the event any provision of this Agreement is inconsistent with or in
violation of any applicable state or federal law, the parties agree that such
provision shall be of no force or

126777 Bayview 2005-1
Servicing Rights Transfer Agreement

                                       2
<PAGE>

effect and that this Agreement shall continue as though said contrary provision
was deleted from this Agreement.

      7. Any notice, demand, or communication which either party desires or is
required to give the other party in connection with this Agreement shall be in
writing and shall be either served personally or sent by telecopier or prepaid
first class UMT LT States mail or by overnight courier, addressed to the
applicable parties as follows:

                  UMT LT:              UMT LT Trust
                                       5740 Prospect Ave.
                                       Suite 1000
                                       Dallas, Texas 75206
                                       Attention:  Christine Griffin
                                       Telephone Number:  (800) 955-7917
                                       Telecopier Number:  (214) 237-3309

                  Bayview:             Bayview Financial, L.P.
                                       4425 Ponce de Leon Boulevard, 4th Floor
                                       Coral Gables, Florida 33146
                                       Attention:  Stuart Waldman
                                       Telephone Number:  (305) 341-5576
                                       Telecopier Number:  (305) 854-3042

                  With a copy to:      McKee Nelson LLP
                                       1919 M Street, N.W.
                                       Washington, D.C. 20036
                                       Attention:  Edward Gainor
                                       Telephone Number:  (202) 775-4137
                                       Telecopier Number:  (202) 775-8586

      8. No addendum, supplement, modification or amendment of this Agreement
shall be binding unless executed in writing by the parties hereto.

      9. This Agreement, including all documents and schedules incorporated by
reference herein, constitutes the entire understanding between the parties
hereto with respect to the subject matter hereof and supersedes all other
agreements, covenants, representations, warranties, understandings and
communications between the parties, whether written or oral, with respect to the
transactions contemplated by this Agreement.

      10. THIS AGREEMENT SHALL BE GOVERNED BY AND CONSTRUED IN ACCORDANCE WITH
THE LAWS OF THE STATE OF NEW YORK, WITHOUT REFERENCE TO ITS CONFLICT OF LAW
PROVISIONS (OTHER THAN SECTION 5-1401 OF THE GENERAL OBLIGATIONS LAW), AND THE
OBLIGATIONS, RIGHTS AND REMEDIES OF THE PARTIES HEREUNDER SHALL BE DETERMINED IN
ACCORDANCE WITH SUCH LAWS.

126777 Bayview 2005-1
Servicing Rights Transfer Agreement

                                       3
<PAGE>

      IN WITNESS WHEREOF, the parties hereto have caused this Agreement to be
executed by their duly authorized officers as of the Closing Date.

                                       UMT LT TRUST

                                       By: ____________________________________
                                           Name:
                                           Title:

                                       BAYVIEW FINANCIAL, L.P.

                                       By: BAYVIEW FINANCIAL MANAGEMENT
                                           CORP., its General Partner

                                       By: ____________________________________
                                           Name:       Stuart Waldman
                                           Title:      Senior Vice President

126777 Bayview 2005-1
Servicing Rights Transfer Agreement