<PAGE>


                                                                    EXHIBIT 10.6

                               FIFTH AMENDMENT TO
                           FIRST AMENDED AND RESTATED
                    WAREHOUSING CREDIT AND SECURITY AGREEMENT

FIFTH AMENDMENT TO FIRST AMENDED AND RESTATED WAREHOUSING CREDIT AND SECURITY
AGREEMENT (this "Amendment") dated as of March 15, 2004, between OAK STREET
MORTGAGE LLC, a Delaware limited liability company ("Oak Street LLC"), OAK
STREET MORTGAGE, INC., a Delaware corporation ("Oak Street Inc."), and OAK
STREET MORTGAGE OF TENNESSEE LLC, a Tennessee limited liability company ("Oak
Street TN") ("Oak Street LLC, Oak Street Inc. and Oak Street TN are collectively
referred to as "Borrower") and RESIDENTIAL FUNDING CORPORATION, a Delaware
corporation ("Lender").

A.    Borrower and Lender have entered into a revolving mortgage warehousing
      facility with a present Warehousing Commitment Amount of $200,000,000,
      which is evidenced by a First Amended and Restated Warehousing Promissory
      Note dated August 31, 2002, a Sublimit Promissory Note dated August 31,
      2002 (the "Notes"), and by a First Amended and Restated Warehousing Credit
      and Security Agreement dated as of August 31, 2002 (as the same may have
      been and may be amended or supplemented, the "Agreement").

B.    Borrower has requested that Lender amend the Agreement to provide for a
      Third Party Originated Loan Sublimit, and Lender has agreed to such
      amendment, subject to the terms and conditions of this Amendment.

NOW, THEREFORE, the parties to this Amendment agree as follows:

1.    Subject to Borrower's satisfaction of the conditions set forth in Section
      4, the effective date of the Amendment is March 10, 2004 ("Effective
      Date").

2.    Unless otherwise defined in this Amendment, all capitalized terms have the
      meanings given to those terms in the Agreement. Defined terms may be used
      in the singular or the plural, as the context requires. The words
      "include," "includes" and "including" are deemed to be followed by the
      phrase "without limitation." Unless the context in which it is used
      otherwise clearly requires, the word "or" has the inclusive meaning
      represented by the phrase "and/or," References to Sections and Exhibits
      are to Sections and Exhibits of this Amendment unless otherwise expressly
      provided.

3.    Exhibit A to the Agreement is amended and restated in its entirety as set
      forth in Exhibit A to this Amendment. All references in the Agreement and
      the other Loan Documents to Exhibit A are deemed to refer to the new
      Exhibit A.

4.    Exhibit E to the Agreement is amended and restated in its entirety as set
      forth in Exhibit E to this Amendment. All references in the Agreement and
      the other Loan Documents to Exhibit E are deemed to refer to the new
      Exhibit E.

5.    Exhibit H to the Agreement is amended and restated in its entirety as set
      forth in Exhibit H to this Amendment. All references in the Agreement and
      the other Loan Documents to Exhibit H are deemed to refer to the new
      Exhibit H.

6.    Borrower must deliver to Lender (a) two executed copies of this Amendment
      and (b) a $350 document production fee.

                                     Page 1
<PAGE>

7.    Borrower represents, warrants and agrees that (a) there exists no Default
      or Event of Default under the Loan Documents, except as set forth in
      Section 6 of this Fourth Amendment, (b) the Loan Documents continue to be
      the legal, valid and binding agreements and obligations of Borrower,
      enforceable in accordance with their terms, as modified by this Amendment,
      (c) Lender is not in default under any of the Loan Documents and Borrower
      has no offset or defense to its performance or obligations under any of
      the Loan Documents, (d) except for changes permitted by the terms of the
      Agreement, Borrower's representations and warranties contained in the Loan
      Documents are true, accurate and complete in all respects as of the
      Effective Date and (e) there has been no material adverse change in
      Borrower's financial condition from the date of the Agreement to the
      Effective Date.

8.    Except as expressly modified, the Agreement is unchanged and remains in
      full force and effect, and Borrower ratifies and reaffirms all of its
      obligations under the Agreement and the other Loan Documents.

9.    This Amendment may be executed in any number of counterparts, each of
      which will be deemed an original, but all of which shall together
      constitute but one and the same instrument.

IN WITNESS WHEREOF, Borrower and Lender have caused this Amendment to be duly
executed on their behalf by their duly authorized officers as of the day and
year above written.

                                     OAK STREET MORTGAGE LLC,
                                     a Delaware limited liability company

                                     By: /s/ Craig L. Royal
                                         -----------------------------
                                     Its: Chief Financial Officer

                                     OAK STREET MORTGAGE, INC.,
                                     a Delaware corporation

                                     By: /s/ Craig L. Royal
                                         -----------------------------
                                     Its: Chief Financial Officer

                                     OAK STREET MORTGAGE OF TENNESSEE LLC, a
                                     Tennessee limited liability company

                                     By: /s/ Craig L. Royal
                                         -----------------------------
                                     Its: Chief Financial Officer

                                     RESIDENTIAL FUNDING CORPORATION,
                                     a Delaware corporation

                                     By:  /s/ Robin L. Swanson
                                         -----------------------------

                                     Its: Director
                                          ----------------------------

                                     Page 2
<PAGE>

                                                                    EXHIBIT A-SF

                               REQUEST FOR ADVANCE

                           OAK STREET MORTGAGE LLC [ ]
                    OAK STREET MORTGAGE OF TENNESSEE LLC [ ]
                          OAK STREET MORTGAGE, INC. [ ]

Loan Number : ____________________________   Reviewed
                                             By:         _______________________

Mortgagor :   ____________________________   Warehouse
SSN:          ____________________________   Date:       _______________________

Address:      ____________________________   Effective
              ____________________________   Date:       _______________________
Zip Code:     ____________________________

Loan-to-Value Ratio:________ FICO Score:________ Debt-to-income Ratio: _________

<TABLE>
<S>     <C>                     <C>          <C>        <C>
Status: [] Committed            [] RFC       Loan Type: [] prime [] FHA [] VA
        [] Uncommitted                                  [] Subprime/Grade_______
        [] Wet Settlement       [] Received             [] High LTV [] First [] Second
        [] Repurchased                                  [] Nonperforming
        [] MERS                              Term:      [] Fixed__________ Term
        [] Open-end Second                              [] ARM ______ Adjustment
        [] Closed-end Second                               Period
        [] 3rd Party Originated                         [] Balloon__________Term
</TABLE>

                                          Interest Rate: _________

Mortgage Note Date:____________________   Maturity Date:________________________

Mortgage Note Amount:__________________   Unpaid Principal Balance:_____________

Original Lender (if applicable):_______   Acquisition Cost: (if applicable):____

Requested Warehouse Advance Amount:____   Title Company:________________________

Investor:______________________________   Title Company Contact:________________

Investor Contact:______________________   Title Company Phone:__________________

Investor Phone:________________________   Expiration Date:______________________

Committed Purchase Price:______________   Purchase Commitment No.:______________

                              FUNDING INSTRUCTIONS

[] Wire Funding

     Account to Debit:___________________     Date of Wire:_____________________

     Credit Acct. Name:__________________     Amount of Wire:___________________

     Bank Name:__________________________     Credit Acct. No.:_________________

     City and State:_____________________     ABA No.:__________________________
     Ref:________________________________     Advise:____________ Phone:________

[] Check Funding

                                    Page A-1
<PAGE>

      Check No.:__________________________ Amount:______________________________

                             REQUIRED DOCUMENTATION

The following documents in connection with the above request are enclosed:

RIGHT

[]    Original and 1 copy of Mortgage Note

[]    Certified copy of Mortgage

[]    Original or certified copy of recorded Mortgage (Repurchased Mortgage
      Loans and Nonperforming Mortgage Loans)

[]    Original ALTA Mortgage's Policy of Title Insurance or equivalent
      (Repurchased Mortgage Loans and Nonperforming Mortgage Loans)

[]    *Copy of Investor Purchase Commitment (or satisfactory evidence thereof)

[]    *Copy of HUD-1 Settlement Statement or equivalent with evidence of initial
      Advance amount (open-end Second Mortgage Loans)

[]    *Copy of AssetWise Certificate (Required for High LTV Mortgage Loans)

[]    *Copy of Borrower's closing instructions to escrow/title company (table
      funded Mortgage Loans only)

[]    *Copy of a letter (or other acceptable documentation) from the holder of
      the Mortgage Loan to Borrower documenting the purchase price for the
      Mortgage Loan, releasing the holder's ownership interest in the Mortgage
      Loan against payment of that purchase price by Borrower and containing
      wire transfer instructions for payment of that purchase price (Third Party
      Originated Loans only)

LEFT

[]    *Request for Advance (original and 1 copy)

[]    Recordable assignment of Mortgage (not required for Mortgage Loans
      registered on the MERS system after delivery of an Electronic Tracking
      Agreement, if approved by Lender)

[]    Certified copies of interim assignments of Mortgage (if applicable)

[]    *Investor repurchase demand letter (Repurchased Mortgage Loans only)

[]    *Summary of Mortgage Loan documentation or Investor problems, expected
      cure period and current payment history (Repurchased Mortgage Loans only)

[]    *Broker price opinion or Appraisal (Repurchased Mortgage Loans and
      Nonperforming Mortgage Loans)

NOTE: Items designated with the "*" are required prior to a Wet Settlement
Advance.

For the new value this day received, Oak Street Mortgage LLC, Oak Street
Mortgage of Tennessee, LLC and Oak Street Mortgage, Inc. ("Borrowers"), grants a
security interest to Residential Funding Corporation ("Lender") in all of
Borrower's right, title and interest in and to the Mortgage Loan described
above, together with all related "Collateral," as more particularly described in
the First Amended and Restated Warehousing Credit and Security Agreement (as
amended, supplemented or otherwise modified) between Borrower and Lender.

OAK STREET MORTGAGE LLC

Authorized Signature:_________________________

OAK STREET MORTGAGE OF TENNESSEE LLC

Authorized Signature:_________________________

OAK STREET MORTGAGE, INC.

Authorized Signature:_________________________

                                    Page A-2
<PAGE>

                                                                       EXHIBIT H

                         ELIGIBLE LOANS AND OTHER ASSETS

                             Oak Street Mortgage LLC
                            Oak Street Mortgage, Inc.
                      Oak Street Mortgage of Tennessee LLC

LIMITATIONS ON WAREHOUSING ADVANCES AGAINST MORTGAGE LOANS

      Lender's obligation to make Warehousing Advances under the Agreement is
      subject to the following limitations:

            1.    No Warehousing Advance will be made against any Mortgage Loan
                  that has been previously sold or pledged to obtain financing
                  (whether or not such financing constitutes Debt) under another
                  warehousing financing arrangement or a gestation agreement.

            2.    No Warehousing Advance will be made against any Mortgage Loan
                  that Lender believes may be based on untrue, incomplete or
                  inaccurate or fraudulent information or may otherwise be
                  subject to fraud.

            3.    No Warehousing Advance will be made against a Mortgage Loan if
                  any of the limitations set forth in this Exhibit H would be
                  exceeded after giving effect to the Warehousing Advance.

            4.    No Warehousing Advance will be made against a Mortgage Loan
                  with an original principal balance in excess of $2,000,000.

            5.    No Warehousing Advance will be made against a Third Party
                  Originated Loan that was closed more than 45 days prior to the
                  date on which the requested Warehousing Advance is to be made.

SUBLIMITS

      These general limitations apply to all Warehousing Advances against
      Eligible Loans:

                  1. Wet Settlement Advances:        40% of the Warehousing
                                                     Commitment Amount.

                  2. Third Party Originated Loans:   $10,000,000 (Wet Settlement
                                                     Advances not permitted)

ELIGIBLE LOANS AND TERMS OF WAREHOUSING ADVANCES

      Subject to compliance with the terms and limitations set forth below and
      the terms, representations and warranties and the covenants in the
      Agreement, each of the following Mortgage Loans is an Eligible Loan for
      purposes of the Agreement:

                                        1
<PAGE>

1.    PRIME MORTGAGE LOAN

      (a) Definition: A First Mortgage Loan or a Second Mortgage Loan with the
      following characteristics:

          (i) For a First Mortgage Loan:

                 A.   Underwritten substantially in accordance with Fannie Mae
                 or Freddie Mac underwriting standards (except as to maximum
                 amount); and

                 B.   Loan-to-Value Ratio not to exceed 80% or, if the
                 Loan-to-Value Ratio exceeds 80%, the Prime Mortgage Loan is
                 insured by or subject to a commitment for mortgage insurance
                 in an amount and on terms and conditions that satisfy the
                 underwriting standards of Fannie Mae or Freddie Mac; or

                  C.    A Government Mortgage Loan.

          (ii) For a Second Mortgage Loan:

                 A.   The credit of the obligor has been underwritten
                 substantially in accordance with Fannie Mae or Freddie Mac
                 underwriting standards; and

                 B.   Loan-to-Value Ratio not more than 100%.

      (b) Interest Rate:

          (i)  Other Mortgage Loans:     1.00% over LIBOR

          (ii) Aged Mortgage Loans:      1.50% over LIBOR

      (c) Prime Sublimit:                $24,000,000

          (i)   First Mortgage Loan:     $24,000,000

          (ii)  Second Mortgage Loan:    $12,000,000

          (iii) Aged Mortgage Loans:     $600,000

      (d) Committed/Uncommitted:

          (i) First Mortgage Loans:      Purchase Commitment required

          (ii) Second Mortgage Loans:    Purchase Commitment NOT required

      (e) Wet Settlement Advances:       Permitted

      (f) Aged Mortgage Loans:           Permitted for First Mortgage Loans only

      (g) Committed Advance Rate:

          (i)  First Mortgage Loan:      100% of the lesser of (i) the Mortgage
                                         Note Amount or (ii) the Committed
                                         Purchase Price

          (ii) RFC Mortgage Loan:        100% of the lesser of (i) of the
                                         Mortgage Note Amount or (ii) the
                                         Committed Purchase Price

                                       2
<PAGE>

      (h) Uncommitted Advance Rate:      100% of the Mortgage Note Amount

      (i) Standard Warehouse Period:     90 days

      (j) Aged Warehouse Period:         120 days

      (k) Required Prepayments:          All Mortgage Loans in warehouse 45 days
                                         will be reduced by 2% of the Mortgage
                                         Note Amount.

                                         On the day a Pledged Loan becomes an
                                         Aged Mortgage Loan, the Warehousing
                                         Advance against such Pledged Loan must
                                         be (a) repaid in full, to the extent
                                         the Aged Mortgage Loan Sublimit would
                                         be exceeded, or (b) otherwise, reduced
                                         by 5% of the Mortgage Note Amount.

2.    SUBPRIME MORTGAGE LOAN

      (a) Definition: A First Mortgage Loan or Second Mortgage Loan that has a
      risk rating of "A-," "B" or "C" (determined using underwriting standards
      that comply with industry standards in the sole judgment of Lender), and
      that is acceptable for purchase by at least two investors.

      (b) Interest Rate:

          (ii) Other Mortgage Loans:     1.00% over LIBOR

          (ii) Aged Mortgage Loans:      1.50% over LIBOR

      (c) Subprime Sublimit:             $200,000,000

          (i)   First Mortgage Loan:     $200,000,000
          (ii)  Second Mortgage Loan:    $40,000,000
          (iii) Aged Mortgage Loans:     $10,000,000

      (d) Committed/Uncommitted:         Purchase Commitment NOT required

      (e) Wet Settlement Advances:       Permitted

      (f) Aged Mortgage Loans:           Permitted for First Mortgage Loans only

      (g) Committed Advance Rate:

          (i) RFC Mortgage Loan:         100% of the lesser of (i) of the
                                         Mortgage Note Amount or (ii) the
                                         Committed Purchase Price

      (h) Uncommitted Advance Rate:

          (i)   First Mortgage Loan:     100% of the Mortgage Note Amount

          (ii)  Second Mortgage Loan:    100% of the Mortgage Note Amount

                                       3
<PAGE>

      (i) Standard Warehouse Period:     90 days

      (j) Aged Warehouse Period:         180 days

      (k) Required Prepayments:          All Mortgage Loans in warehouse 45 days
                                         will be reduced by 2% of the Mortgage
                                         Note Amount.

                                         On the day a Pledged Loan becomes an
                                         Aged Mortgage Loan, the Warehousing
                                         Advance against such Pledged Loan must
                                         be (a) repaid in full, to the extent
                                         the Aged Mortgage Loan Sublimit would
                                         be exceeded, or (b) otherwise, reduced
                                         by 5% of the Mortgage Note Amount.
                                         Thereafter, the Warehousing Advance
                                         must be reduced by 5% of the Mortgage
                                         Note Amount.

3.    HIGH LTV MORTGAGE LOAN

      (a) Definition: A Second Mortgage Loan that meets the 125 Loan Program
      eligibility criteria set forth in the GMAC-RFC Client Guide and for which
      an AssetWise Certificate has been issued, and a First Mortgage Loan that
      meets the Home Solution Program eligibility criteria set forth in the
      GMAC-RFC Client Guide and for which an AssetWise Certificate has been
      issued.

      (b) Interest Rate:                 2.00% over LIBOR

      (c) High LTV Sublimit:             $1,000,000

      (d) Committed/Uncommitted:         Purchase Commitment from Lender
                                         required

      (e) Wet Settlement Advances:       Permitted

      (f) Aged Mortgage Loans:           Not Permitted

      (g) Committed Advance Rate:

          (i)  First Mortgage Loan:      98% of the lesser of (i) of the
                                         Mortgage Note Amount or (ii) the
                                         Committed Purchase Price

          (ii) Second Mortgage Loan:     95% of the lesser of (i) of the
                                         Mortgage Note Amount or (ii) the
                                         Committed Purchase Price

      (h) Standard Warehouse Period:     45 days

4.    REPURCHASED MORTGAGE LOAN/NONPERFORMING MORTGAGE LOAN/REO PROPERTY

      (a) Definitions:

                                       4
<PAGE>

              Repurchased Mortgage Loan: A Mortgage Loan that has been
              repurchased from an Investor or a Mortgage Pool pursuant to a
              Servicing Contract.

              Nonperforming Mortgage Loan: A First Mortgage Loan or a Second
              Mortgage Loan that is not a High LTV Mortgage Loan and (i) is in
              the process of foreclosure, (ii) is 60 days or more delinquent or
              (iii) with respect to which the Warehousing Period has expired.

              REO Property: An improved real property containing a 1- to
              4-family residence, which property is owned by Borrower as the
              result of a foreclosure proceeding or the acceptance of a deed in
              lieu of foreclosure, or has been purchased from an Investor to
              satisfy a repurchase obligation of Borrower to the Investor.

      (b) Interest Rate:                 3.25% over LIBOR

      (c) Sublimit:                      $2,400,000

      (d) Committed/Uncommitted:         Purchase Commitment NOT required

      (e) Wet Settlement Advances:       Not Permitted

      (f) Aged Mortgage Loans:           Not Permitted

      (g) Advance Rate for Repurchased
          and Nonperforming Mortgage
          Loans:                         90% of the lesser of (i) Lender's
                                         initial Warehousing Advance, (ii) the
                                         unpaid principal balance, (iii) the
                                         repurchase price, or (iv) the Appraised
                                         Property Value or BPO Value

      (h) Advance Rate for REO Property: The lesser of (i) 60% Lender's initial
                                         Warehousing Advance or (ii) 60% of the
                                         Appraised Property Value or BPO Value

      (i) Standard Warehouse Period:     365 days

      (j) Required Prepayments for
          Repurchased and Nonperforming
          Mortgage Loans:                5% of the Mortgage Note Amount, paid
                                         each month occurring more than 90 days
                                         after the date of the Warehousing
                                         Advance

      (k) Required Prepayments for REO
          Property:                      5% of the initial Warehousing Advance
                                         against an REO Property paid each month
                                         occurring more than 90 days after the
                                         date of the Warehousing Advance

                                       5