<PAGE>

                                                                     Exhibit 4.4

                  ===========================================


                       OAKWOOD MORTGAGE INVESTORS, INC.,

                        OAKWOOD ACCEPTANCE CORPORATION

                                      AND

                           THE CHASE MANHATTAN BANK,
                                  AS TRUSTEE

                                  __________


                              AMENDMENT NO. 3 TO

                 SERIES 1999-E POOLING AND SERVICING AGREEMENT

                          Dated as of August 10, 2001

                                  __________


                       OAKWOOD MORTGAGE INVESTORS, INC.,

                              SENIOR/SUBORDINATED

                   PASS-THROUGH CERTIFICATES, SERIES 1999-E

                  ===========================================
<PAGE>

     AMENDMENT NO. 3 (this "Amendment"), dated as of August 10, 2001, to the
                            ---------
Series 1999-E Pooling and Servicing Agreement, dated as of  November 1, 1999
(the "Series Agreement") among OAKWOOD MORTGAGE INVESTORS, INC., a Nevada
      ----------------
corporation (the "Company"), OAKWOOD ACCEPTANCE CORPORATION, a North Carolina
                  -------
corporation, as servicer (the "Servicer"), and THE CHASE MANHATTAN BANK, a New
                               --------
York  banking corporation, as trustee (the "Trustee").  The Series Agreement,
                                            -------
together with the Company's Standard Terms to Pooling and Servicing Agreement,
May 1999 Edition, as amended from time to time (the "Standard Terms") are
                                                     --------------
referred to herein as the "Pooling and Servicing Agreement".
                           -------------------------------

                             PRELIMINARY STATEMENT

     Section 11.01 of the Standard Terms provides, inter alia, that the Pooling
and Servicing Agreement may be amended from time to time by the Company, the
Servicer, and the Trustee, without the consent of any of the Certificateholders
to cure any ambiguity, and to correct or supplement any provisions that are
inconsistent with any other provisions; provided, however, that no such
amendment shall (i) reduce in any manner the amount of, or delay the timing of,
payments received on the Assets that are required to be distributed on any
Certificate without the consent of the Holder of such Certificate, (ii) affect
adversely in any material respect the interests of the Holders of any Class of
Certificates in a manner other than described in clause (i) of this paragraph,
without the consent of the Holders of Certificates of such Class evidencing at
least 66% of the Voting Rights with respect to such Class, or (iii) reduce the
aforesaid percentage of Certificates the Holders of which are required to
consent to any such amendment, without the consent of such Holders of all
Certificates then outstanding, as specified in section 11.01 of the Standard
Terms.

     All capitalized terms not otherwise defined herein are defined in the
Pooling and Servicing Agreement.  All Article, Section or Subsection references
herein shall mean Article, Section or Subsections of the Pooling and Servicing
Agreement, except as otherwise provided herein.

SECTION 1. Amendments to Series Agreement.

(a)  Section 2,  Defined Terms, is hereby amended by adding the following
                 -------------
     definitions:

           "Guarantee":  The guarantee by the Guarantor, dated as of August 10,
            ---------
     2001, for the benefit of the Guaranteed Holders.

           "Guarantee Excess Amount": With respect to any Distribution Date, (a)
            -----------------------
     the positive difference, if any, between (1) one-half of the distributions
     to be made with respect to any Interest Distribution Amount, Carryover
     Interest Distribution Amount, Writedown Interest Distribution Amount and
     Carryover Writedown Interest Distribution Amount pursuant to clauses (v)
     and (xi) of Section 5(b) hereof, and (2) an amount equal to interest
     accrued at the applicable Pass-Through Rate for the related Interest
     Accrual Period on the Guaranteed Class B-2 Certificate Principal Balance,
     and (b) one-half of any Principal Distribution Shortfall Carryover Amount
     distributable pursuant to clause (xi) under Section 5(b) hereof.
<PAGE>

          "Guaranteed Holders": The holders of 50% of the Class B-2
           ------------------
     Certificates, which as of August 10, 2001, are held by or on behalf of
     Oakwood Financial Corporation.

          "Guarantee Payment Amount":  With respect to any Distribution Date,
           ------------------------

          (a) the positive difference, if any, between (1) an amount equal to
          interest accrued at the applicable Pass-Through Rate for the related
          Interest Accrual Period on the Guaranteed Class B-2 Certificate
          Principal Balance, and (2) one-half of the distributions to be made
          with respect to any Interest Distribution Amount, Carryover Interest
          Distribution Amount, Writedown Interest Distribution Amount and
          Carryover Writedown Interest Distribution Amount pursuant to clauses
          (v) and (xi) of Section 5(b) hereof,

          (b) after giving effect to the allocation of the Available Amount,
          one-half of any due but unpaid principal amounts distributable
          pursuant to clause (xi) under Section 5(b) hereof, and

          (c) if, after giving effect to the distributions of principal amounts
          pursuant to clause (xi) under Section 5(b) hereof, the Adjusted
          Certificate Balance of the Class B-2 Certificates has been reduced to
          zero, an amount equal the remaining Guaranteed Class B-2 Certificate
          Balance, if any.

          "Guaranteed Class B-2 Certificate Principal Balance":  On any
           --------------------------------------------------
     Distribution Date, one-half of the Initial Certificate Principal Balance of
     the Class B-2 Certificates as set out in Section 3, reduced by all prior
     distributions of principal to the Class B-2 Certificates held by the
     Guaranteed Holders.

          "Guarantor":  Oakwood Homes Corporation, a North Carolina corporation.
           ---------

(b)  The following provisions is hereby added as Section 5.1:

     Section 5.1. Guarantee.
                  ---------

          (a)     On any Distribution Date, the Guarantor is hereby obligated to
     the pay the Guarantee Payment Amount, if any, for the benefit of Oakwood
     Financial Corporation and its successors and assigns as holders of 50% of
     the outstanding Class B-2 Certificates.

          (b)     No later than 1:00 p.m. New York City time on each Remittance
     Date, after taking into account the amounts allocated to the various
     Subaccounts in accordance with Section 5(a) hereof, the Trustee shall, in
     accordance with the related Remittance Report and in accordance with the
     terms of the Guarantee, notify the Guarantor of any Guarantee Payment
     Amount payable under the Guarantee on the related Distribution Date.  In
     addition, the Servicer shall notify the Guarantor as soon as practical (but
     no later than the related Remittance Date) after determining that a
     Guarantee Payment Amount shall be payable under the Guarantee on the
     related Distribution Date.  Upon receipt of notice as described above, the
     Guarantor shall be required to deliver the Guarantee Payment Amount, if
     any, on or prior to 10:00 a.m. on the Distribution Date for the related

                                       2
<PAGE>

     Distribution Date. Such Guarantee Payment Amount received by the Trustee
     shall be paid to the Guaranteed Holders on such Distribution Date (or such
     later date, if such amounts are received subsequent to such Distribution
     Date). In no event shall the Guarantee Payment Amount be distributed on any
     Class of Certificates or other holders of the Class B-2 Certificates other
     than the Guaranteed Holders and any such amounts received by the Trustee
     which are not distributable to the Guaranteed Holders shall be returned by
     the Trustee to the Guarantor. Any Guarantee Payment Amounts made by the
     Guarantor to the Trustee shall be made in cash and shall be considered to
     be payments made directly to the Guaranteed Holders and not payments made
     to the Issuing REMIC in the nature of a guarantee within the meaning of
     I.R.C. (S) 860G(d)(2)(B).

          (c)  On each Distribution Date, the Guarantor is entitled to receive
     any Guarantee Excess Amount.  At such time as the Guaranteed Class B-2
     Certificate Principal Balance has been reduced to zero, the Guarantor will
     be entitled to receive any subsequent distributions with respect to the
     Class B-2 Certificates held by the Guaranteed Holder as set forth in
     Section 5 (which amounts will constitute Guarantee Excess Amount).  All
     Guarantee Excess Amounts will be deemed to be reimbursements for prior
     Guarantee Payment Amounts.

(c)  Section 7, Remittance Reports, is hereby amended by (i) deleting "and" at
                ------------------
     the end of clause (7), (ii) deleting "." at the end of clause (8) and
     substituting "; and,", and (iii) inserting the following text at the end of
     Section 7:

     (9)  the amount of the Guarantee Payment Amount, if any, for such
     Distribution Date, the Aggregate amount of any unpaid Guarantee Payment
     Amounts for any previous Distribution Dates, and the Guaranteed Class B-2
     Certificate Principal Balance immediately prior to such Distribution Date.

SECTION 2.  Characterization of the Guaranty.
            --------------------------------

     The guarantee is a direct obligation of the Guarantor to the holders of the
Class B-2 Certificates.  The Guarantee shall not be an asset of the Trust or
either REMIC.

SECTION 3.  Counterparts.
            ------------

     This Amendment may be executed in any number of counterparts, each of which
so executed shall be deemed to be an original, but all of such counterparts
shall together constitute but one and the same instrument.

SECTION 3.  Governing Law.
            -------------

     THIS AMENDMENT SHALL BE CONSTRUED IN ACCORDANCE WITH THE LAWS OF THE STATE
OF NORTH CAROLINA, WITHOUT REFERENCE TO ITS CONFLICT OF LAW PROVISIONS, AND THE
OBLIGATIONS, RIGHTS AND REMEDIES OF THE PARTIES HEREUNDER SHALL BE DETERMINED IN
ACCORDANCE WITH SUCH LAWS.

                  Remainder of page intentionally left blank

                                       3
<PAGE>

     IN WITNESS WHEREOF, Oakwood Mortgage Investors, Inc., Oakwood Acceptance
Corporation and the Trustee have caused this Amendment to be duly executed by
their respective officers thereunto duly authorized as of the date first above
written.

                                   OAKWOOD MORTGAGE
                                   INVESTORS, INC.

                                   By:     /s/ Dennis W. Hazelrigg
                                      ----------------------------------
                                          Name:  Dennis W. Hazelrigg
                                          Title:  President


                                   OAKWOOD ACCEPTANCE
                                   CORPORATION, as Servicer


                                   By:     /s/ Douglas R. Muir
                                      ----------------------------------
                                          Name:  Douglas R. Muir
                                          Title:  Vice President


                                   THE CHASE MANHATTAN BANK, as
                                   Trustee

                                   By:     /s/ Craig M. Kantor
                                      ----------------------------------
                                          Name:  Craig M. Kantor
                                          Title:  Vice President


                              [Amendment No. 3 to
                Series 1999-E Pooling and Servicing Agreement]