<PAGE>

                                                                    Exhibit 10.4

                      SECOND AMENDMENT AND WAIVER AGREEMENT

                  SECOND AMENDMENT AND WAIVER AGREEMENT, dated as of June 19,
2003 (this "Second Amendment"), to the Amended and Restated Receivables Loan and
Security Agreement, dated as of April 30, 2002, among Silverleaf Finance I, Inc.
(the "Borrower"), Silverleaf Resorts, Inc. ("SRI"), as Servicer, Autobahn
Funding Company LLC, as lender (the "Lender"), DZ Bank AG Deutsche
Zentral-Genossenschaftsbank, Frankfurt Am Main, as agent (the "Agent"), U.S.
Bank Trust National Association, as Agent's Bank, and Wells Fargo Bank
Minnesota, National Association, as Backup Servicer (as the same may have been
amended, supplemented, modified or restated prior to the effectiveness hereof in
accordance with its terms, including by that certain First Amendment to Amended
and Restated Receivables Loan and Security Agreement, the "Existing RLSA").
Capitalized terms used herein and not otherwise defined herein shall have the
meanings attributed thereto in the Existing RLSA, as amended hereby (as so
amended, and as otherwise amended, supplemented, modified or restated from time
to time in accordance with its terms, the "Amended RLSA").

                  WHEREAS, the Borrower and SRI have requested that the Agent
and the Lender waive certain provisions of the Existing RLSA; and

                  WHEREAS, in connection with the aforementioned requested
waiver, the parties hereto have agreed to amend the Existing RLSA on the terms
and subject to the conditions herein set forth;

                  NOW, THEREFORE, for valuable consideration, the receipt and
sufficiency of which is hereby acknowledged, and subject to the fulfillment of
the conditions set forth below, the parties hereto agree as follows:

         SECTION 1. AMENDMENT TO THE EXISTING RLSA

                  1.1      The definition of "Borrowing Limit" in Section 1.01
of the Existing RLSA is hereby amended and restated to read in its entirety as
follows:

                           "Borrowing Limit" means (i) $100,000,000 prior to the
                  occurrence of the closing or closings with respect to one or
                  more Securitizations providing for financing in an aggregate
                  amount of at least $50,000,000 and (ii) $50,000,000
                  immediately upon and after the occurrence of the closing or
                  closings with respect to one or more Securitizations providing
                  for financing in an aggregate amount of at least $50,000,000;
                  provided, that at all times, on or after the Early
                  Amortization Commencement Date, the Borrowing Limit shall mean
                  the aggregate outstanding amount of the Loans.

                  1 2      The definition of "Applicable Margin" in Section 1.01
of the Existing RLSA is hereby amended and restated to read in its entirety as
follows:

                           "Applicable Margin" shall have the meaning set forth
                  in the Fee Letter.

                  1.3      The definition of "EBITDA" in Section 1.01 of the
Existing RLSA is hereby amended by inserting the following language immediately
after the words "for the corresponding period" at the end of such definition:

<PAGE>

                  provided that, with respect to SRI, EBITDA for any period
                  shall be determined without giving effect to the increase in
                  the allowance for uncollectible notes in the amount of
                  $28,711,000 recognized during the fiscal quarter of SRI ending
                  March 31, 2003.

                  1.4      Clause (yy) of the definition of "Eligible
Receivable" in Section 1.01 of the Existing RLSA is hereby amended and restated
to read in its entirety as follows:

                           (yy) If such Pledged Receivable was initially Pledged
                  hereunder at (1) any time on or after the Amendment Date, but
                  prior to June 19, 2003, the Obligor related to such Pledged
                  Receivable shall have had a FICO Score of at least 500 at the
                  time such Obligor purchased the Interval related to such
                  Pledged Receivable or (2) any time on or after June 19, 2003,
                  the Obligor related to such Pledged Receivable shall have had
                  a FICO Score of at least 600 at the time such Obligor
                  purchased the Interval related to such Pledged Receivable.

                  1.5      The definition of "Overconcentration Amount" in
Section 1.01 of the Existing RLSA is hereby amended by deleting clause (e) of
such definition and substituting in lieu thereof the following:

                           (e) the amount by which the sum of the Outstanding
                  Principal Balances of each Eligible Receivable initially
                  Pledged hereunder at any time on or after the Amendment Date
                  but prior to June 19, 2003, the Obligor of which shall have
                  had a FICO Score of at least 500 but not greater than 539 at
                  the time such Obligor purchased the Interval related to such
                  Eligible Receivable exceeds 10% of the Eligible Receivables
                  Balance at such time.

                  1.6      Section 1.01 of the Existing RLSA is hereby amended
by adding the following additional defined term in appropriate alphabetical
order:

                           "Securitization" means a financing transaction
                  undertaken by the Borrower and/or SRI and/or any Affiliate
                  thereof, which (a) is intended to be structured in a manner so
                  that the indebtedness incurred need not be recognized on the
                  balance sheet of SRI and (b) involves the direct or indirect
                  sale or other conveyance of Receivables and/or similar
                  promissory notes or receivables to a Person that shall
                  privately or publicly sell securities, notes or certificates
                  backed by such Receivables and/or similar promissory notes or
                  receivables; provided, however, that the term Securitization
                  shall not include a financing provided by one or more
                  commercial paper conduits.

                  (a) The Borrower shall pay the Lender (either directly or
                  through the Agent) certain fees (the "Fees") in the amounts
                  and on the dates set forth in an amended and restated fee
                  letter, dated as of June 19, 2003, among SRI, the Borrower,
                  the Agent, and the Lender, as such fee letter, may be amended,
                  restated, supplemented or otherwise modified from time to time
                  in accordance with its terms (as so amended, restated,
                  supplemented or modified, the "Fee Letter").

                  1.8      Section 2.12 of the Existing RLSA is hereby amended
by adding at the end thereof the following:

                                       2

<PAGE>

                  If, as a result of the decrease of the Borrowing Limit
         pursuant to the definition thereof, or for any other reason, the
         Facility Amount exceeds the Borrowing Limit at any time, the Borrower
         shall immediately prepay to the Agent, for the account of the Lender, a
         principal amount of Loans in an amount equal to such excess (together
         with all accrued Yield with respect to such principal amount of Loans).
         The Borrower shall promptly reimburse the Agent and the Lender for any
         reasonable out-of-pocket expenses incurred by the Agent and the Lender,
         respectively, in respect of any such prepayment including, without
         limitation, Liquidation Fees.

                  1.9      Clause (a) of Section 6.13 of the Existing RLSA is
hereby amended and restated to read in its entirety as follows:

                           (a) The Lender or the Agent (and their respective
                  agents or professional advisors) shall at the expense of the
                  Borrower, have the right under this Agreement, up to five
                  times during each calendar year, upon reasonable prior notice
                  to the Servicer and the Borrower, to examine and audit, during
                  business hours or at such other times as might be reasonable
                  under applicable circumstances, any and all of the books,
                  records, or other information of the Servicer and/or the
                  Borrower, or held by another for the Servicer and/or the
                  Borrower or on its behalf, concerning this Agreement and
                  compliance therewith. The Lender or the Agent agree that the
                  liability of the Borrower for the expenses incurred by the
                  Lender and/or the Agent in connection with the examinations
                  and audits described in the previous sentence shall be limited
                  to $20,000 during any calendar year. The Lender and the Agent
                  (and their respective agents and professional advisors) shall
                  treat as confidential any information obtained during such
                  examination which is not already publicly known or available;
                  provided, however, the Lender or the Agent may disclose such
                  information if required to do so by law or by any regulatory
                  authority.

                  1.10     Schedule VI to the Existing RLSA is hereby amended
and restated to read in its entirety as set forth in Schedule VI to this Second
Amendment.

                  1.11     Schedule VIII to the Existing RLSA is hereby deleted
in its entirety.

         SECTION 2. WAIVER

                  2.1      Pursuant to the terms and subject to the conditions
hereof, the Lender and the Agent hereby waive (A) the breach of the covenant
under Clause (iv) of Section 5.01(x), (B) solely to the extent related to the
increase in the allowance for uncollectible notes in the amount of $28,711,000
during the fiscal quarter of SRI ending March 31, 2003, the breach of the
covenant under Section 5.01(z), (C) Clause (iv) of Section 7.01(p) and (D)
solely to the extent related to the aforementioned breaches of the covenants of
Clause (iv) of Section 5.01(x) and Section 5.01(z), Section 7.01(d) of the
Existing RLSA.

                  2.2      The Borrower and SRI understand and agree that the
aforementioned waivers of the Lender and the Agent are conditioned upon no other
unwaived Event of Default or Early Amortization Event having occurred prior to
the date hereof.

                  2.3      The parties hereto hereby acknowledge and agree that,
except for the specific waivers and agreements set forth above, nothing in this
Second Amendment shall be deemed to be a consent to or waiver or amendment of
any covenant or agreement contained in the Existing RLSA, the Amended RLSA or
any other document executed in connection therewith, and each such party hereby
agrees that all of the covenants and agreements contained in the Amended RLSA or
any other document executed in connection with the Existing RLSA or the
Amendment RLSA, are hereby ratified and confirmed in all respects.

                                       3

<PAGE>

         SECTION 3. CONDITIONS TO EFFECTIVENESS

                  This Second Amendment shall be effective upon the delivery to
the Agent of the following items (in each case, in form and substance,
satisfactory to the Agent):

                  (i) counterparts hereof executed by each of the parties
         hereto;

                  (ii) a fully executed copy of an amended and restated fee
         letter (the "Fee Letter Amendment") in the form attached hereto as
         Exhibit A; and

                  (iii) payment of all fees to be paid as of the date hereof
         pursuant to the Fee Letter Amendment, all of which fees are
         fully-earned as of the date hereof, are non-refundable for any reason
         whatsoever, constitute compensation for services and do not constitute
         interest or a charge for the use of money.

         SECTION 4. MISCELLANEOUS

                  4.1      The Borrower and SRI each hereby certifies that the
representations and warranties set forth in Article P/ of the Amended RLSA (and
any other representations and warranties made by the Borrower or SRI in the
Amended RLSA) are true and correct on the date hereof with the same force and
effect as if made on the date hereof, except to the extent that (i) such
representations and warranties speak specifically to an earlier date in which
case they shall have been true and correct on such date or (ii) with respect to
Section 4.0 1(bb) of the Amended RLSA, any inaccuracy has arisen solely due to
the increase in the allowance for uncollectible notes in the amount of $2
8,711,000 recognized during the fiscal quarter of SRI ending March 31, 2003. In
addition, the Borrower and SRI each represents and warrants (which
representations and warranties shall survive the execution and delivery hereof)
that (a) after giving effect to this Second Amendment, no unwaived Early
Amortization Event or Event of Default (nor any event that but for notice or
lapse of time or both would constitute an unwaived Early Amortization Event or
Event of Default) shall have occurred and be continuing as of the date hereof
nor shall any unwaived Early Amortization Event or Event of Default (nor any
event that but for notice or lapse of time or both would constitute an unwaived
Early Amortization Event or Event of Default) occur due to this Second Amendment
becoming effective, (b) the Borrower and SRI each has the corporate power and
authority to execute and deliver this Second Amendment and the Fee Letter
Amendment and has taken or caused to be taken all necessary corporate actions to
authorize the execution and delivery of this Second Amendment and the Fee Letter
Amendment, and (c) no consent of any other person (including, without
limitation, shareholders or creditors of the Borrower or SRI), and no action of,
or filing with any governmental or public body or authority is required to
authorize, or is otherwise required in connection with the execution and
performance of this Second Amendment or the Fee Letter Amendment other than such
that have been obtained.

                  4.2      The Amended RLSA is hereby ratified and confirmed in
all respects and remains in full force and effect in accordance with its terms.

                  4.3      All references in the Amended RLSA to "this
Agreement" and "herein" and all references to the Existing RLSA in the documents
executed in connection with the Existing RLSA shall mean the Amended RLSA.

                                       4

<PAGE>

                  4.4      This Second Amendment may be executed by the parties
hereto individually or in combination, in one or more counterparts, each of
which shall be an original and all of which shall constitute one and the same
agreement. Delivery of an executed counterpart of a signature page to this
Second Amendment by facsimile shall be effective as delivery of a manually
executed counterpart of this Second Amendment.

                  4.5      The Borrower hereby agrees to pay all costs and
expenses incurred by the Lender and the Agent in connection with this Second
Amendment including, without limitation, the fees and expenses of Kaye Scholer
LLP, counsel to the Lender and the Agent.

                  4.6      THIS SECOND AMENDMENT SHALL, IN ACCORDANCE WITH
SECTION 5-1401 OF THE GENERAL OBLIGATIONS LAW OF THE STATE OF NEW YORK, BE
GOVERNED BY THE LAWS OF THE STATE OF NEW YORK, WITHOUT REGARD TO ANY CONFLICTS
OF LAW PRINCIPLES THEREOF THAT WOULD CALL FOR THE APPLICATION OF THE LAWS OF ANY
OTHER JURISDICTION.

         IN WITNESS WHEREOF, the parties hereto have executed this Second
Amendment as of the date first above written.

                                   SILVERLEAF FINANCE I, INC.

                                   By: /S/ HARRY J. WHITE, JR.
                                      ----------------------------------
                                   Name: Harry J. White, Jr.
                                   Title: CFO

                                   SILVERLEAF RESORTS, INC.

                                   By: /S/ ROBERT E. MEAD
                                      -----------------------------------
                                       Name: Robert E. Mead
                                       Title: CEO

                                   DZ BANK AG DEUTSCHE ZENTRAL-
                                   GENOSSENSCHAFTSBANK, FRANKFURT AM
                                   MAIN, as Agent

                                   By: /S/ PATRICK PREECE
                                      -----------------------------------
                                   Name:      Patrick Preece
                                       Title: Vice President

                                   By: /S/ DAN MARINO
                                      -----------------------------------
                                       Name:  Dan Marino
                                       Title: Vice President

                                       5

<PAGE>

                                   AUTOBAHN FUNDING COMPANY LLC

                                   By: DZ Bank AG Deutsche Zentral-
                                       Genossenschaftsbank, Frankfurt Am Main,
                                       its Attorney-in-fact

                                   By: /S/ PATRICK PREECE
                                      -----------------------------------
                                   Name:      Patrick Preece
                                       Title: Vice President

                                   By:  /S/ DAN MARINO
                                      -----------------------------------
                                        Name:  Dan Marino
                                        Title: Vice President

                                   U.S. BANK TRUST NATIONAL ASSOCIATION

                                   By:  /S/ IGNAZIO TAMBURELLO
                                      -----------------------------------
                                        Name:  Ignazio Tamburello
                                        Title: Assistant Vice President

                                   WELLS FARGO BANK MINNESOTA,
                                   NATIONAL ASSOCIATION

                                   By: /S/ SUE DIGNAN
                                      -----------------------------------
                                       Name:  Sue Dignan
                                       Title: Assistant Vice President

                                       6

<PAGE>

                                   SCHEDULE VI
                    TO SECOND AMENDMENT AND WAIVER AGREEMENT

                                                             SCHEDULE VI

                        NET ELIGIBLE RECEIVABLES BALANCE

                  "Net Eligible Receivables Balance" means, at any time, (X)
without duplication, the sum of (a) a percentage of the aggregate Outstanding
Principal Balance of each Eligible Receivable with respect to which the related
Obligor has made at least but no more than 17 consecutive monthly payments in
accordance with the terms of such Eligible Receivable (such consecutive monthly
payments being deemed to include the final consecutive monthly payments made
with respect to, and in accordance with the terms of, a prior Eligible
Receivable payable by such Obligor which was repaid in fill or terminated
concurrently with the execution of such Eligible Receivable) at such time, such
percentage to be equal to the Tier III Advance Rate in effect at such time,
plus, without duplication, (b) a percentage of the aggregate Outstanding
Principal Balance of each Eligible Receivable with respect to which the related
Obligor has made at least but no more than 18 consecutive monthly payments in
accordance with the terms of such Eligible Receivable (such consecutive monthly
payments being deemed to include the final consecutive monthly payments made
with respect to, and in accordance with the terms of, a prior Eligible
Receivable payable by such Obligor which was repaid in full or terminated
concurrently with the execution of such Eligible Receivable) at such time, such
percentage to be equal to the Tier II Advance Rate in effect at such time, plus,
without duplication, (c) a percentage of the aggregate Outstanding Principal
Balance of all Eligible Receivables with respect to which the related Obligor
has made (i) 19 or more consecutive monthly payments in accordance with the
terms of such Eligible Receivable (such consecutive monthly payments being
deemed to include the final consecutive monthly payments made with respect to,
and in accordance with the terms of, a prior Eligible Receivable payable by such
Obligor which was repaid in full or terminated concurrently with the execution
of such Eligible Receivable) at such time or (ii) if such Obligor shall have had
a FICO Score of at least 610 at the time such Obligor purchased the Interval
related to such Eligible Receivable, 6 or more consecutive monthly payments in
accordance with the terms of such Eligible Receivable (without giving effect to
any payments made with respect to any prior Eligible Receivable payable by such
Obligor) at such time, such percentage to be equal to the Tier I Advance Rate in
effect at such time, plus, without duplication, (d) 0% of the aggregate
Outstanding Principal Balance of any Receivables which are not Eligible
Receivables or are Eligible Receivables not described in clauses (a) through (c)
above, minus (Y) the Overconcentration Amount at such time and minus (Z) without
duplication, the FICO Score Adjustment Amount at such time; provided, however,
that at any time that SRI shall have a Tangible Net Worth in an amount which
shall be less than an amount equal to (A) the greater of (1) $100,000,000 or (2)
an amount equal to 90% of the Tangible Net Worth of SRI as of September 30, 2001
plus (B) seventy-five percent (75%) of the aggregate amount of proceeds received
by SRI after January 1, 2002 in connection with (1) each issuance by SRI of any
class or classes of capital stock after January 1, 2002 and (2) the incurrence
of Debt after January 1, 2002, other than Debt which shall be the most senior
Debt of SRI plus (C) fifty percent (50%) of the aggregate amount of net income
(calculated in accordance with GAAP) of SRI after January 1, 2002, each of the
percentages referenced in subparts (a), (b) and (c) of this definition shall be
reduced by 10%; and provided, further, however, that for purposes of determining
the number of consecutive monthly payments made by an Obligor in respect of an
Eligible Receivable in (a), (b) and (c) of this definition, the number of
consecutive monthly payments made (in accordance with the terms of such Eligible

                                      A-1

<PAGE>

Receivable) by such Obligor in respect of a prior Receivable, the terms of which
were modified in accordance with the Credit and Collection Policy to permit the
substitution of the Interval relating to such Eligible Receivable in replacement
for the Interval relating to such prior Receivable and to require the Obligor to
make different monthly payments in respect of such replacement Interval, shall
be included in determining the number of consecutive monthly payments made by an
Obligor in respect of such Eligible Receivable.

                  For the purposes of this Schedule VI, monthly payments made
with respect to an Eligible Receivable shall be deemed to be consecutive monthly
payments even though there is a one to four month gap in such payments due to
such Eligible Receivable having been Modified; provided, that such Eligible
Receivable shall have been Modified only once since the date of its origination.

                  For the purposes of this Schedule VI:

                           "Tier I Advance Rate" means, with respect to each
period set forth below, the percentage set forth below to the right of such
period;

<TABLE>
<CAPTION>
------------------------------------------------------------------------
                                                      FULLY-SEASONED
                      PERIOD                           ADVANCE RATE
------------------------------------------------------------------------
<S>                                                   <C>
October 30, 2000 to and including April 29, 2002           85.0%
------------------------------------------------------------------------
April 30, 2002 to and including June 18, 2003              82.5%
------------------------------------------------------------------------
June 19, 2003 to and including September 14, 2003          81.0%
------------------------------------------------------------------------
September 15, 2003 to and including December 14,           80.0%
2003
------------------------------------------------------------------------
December 15, 2003 to and including February 14,            79.0%
2004
------------------------------------------------------------------------
February 15, 2004 and thereafter                           78.0%
------------------------------------------------------------------------
</TABLE>

                           "Tier II Advance Rate" means, with respect to each
period set forth below, the percentage set forth below to the right of such
period:

<TABLE>
<CAPTION>
------------------------------------------------------------------------
                                                      FULLY-SEASONED
                      PERIOD                           ADVANCE RATE
------------------------------------------------------------------------
<S>                                                   <C>
October 30, 2000 to and including April 29, 2002           82.5%
------------------------------------------------------------------------
April 30, 2002 to and including June 18, 2003              80.0%
------------------------------------------------------------------------
June 19, 2003 to and including September 14, 2003          78.5%
------------------------------------------------------------------------
September 15, 2003 to and including December 14,           77.5%
2003
------------------------------------------------------------------------
December 15, 2003 to and including February 14,            76.5%
2004
------------------------------------------------------------------------
February 15, 2004 and thereafter                           75.5%
------------------------------------------------------------------------
</TABLE>

                                      A-1

<PAGE>

                           "Tier III Advance Rate" means, with respect to each
period set forth below, the percentage set forth below to the right of such
period:

<TABLE>
<CAPTION>
------------------------------------------------------------------------
                                                      FULLY-SEASONED
                      PERIOD                           ADVANCE RATE
------------------------------------------------------------------------
<S>                                                   <C>
October 30, 2000 to and including April 29, 2002           80.0%
------------------------------------------------------------------------
April 30, 2002 to and including June 18, 2003              77.5%
------------------------------------------------------------------------
June 19, 2003 to and including September 14, 2003          76.0%
------------------------------------------------------------------------
September 15, 2003 to and including December 14,           75.0%
2003
------------------------------------------------------------------------
December 15, 2003 to and including February 14,            74.0%
2004
------------------------------------------------------------------------
February 15, 2004 and thereafter                           73.0%
------------------------------------------------------------------------
</TABLE>

List of Exhibits:

         Exhibit A: Form of Amended and Restated Fee Letter

                                      A-1